Michel's blog

Today all of a sudden Platform Builder 5.0 didn't want to load my workspace anymore. Just before I updated the machine using Windows Update so I figured it must have had something to do with that. Platform Builder would start loading the workspace, show "Refreshing the Catalog" in the status bar, then show "Getting variables that correspond with catalog items" and then it would just sit there consuming 50% CPU and not responding to anything else. Luckily I'm using VMWare for all my CE development work so it was easy to revert back to an earlier snapshot before I updated XP. Unfortunately, this didn't solve the problem! I continued my search and after some frustrating hours I found the cause:

A bit earlier in the day I added a whole lot of header files (121 to be precise) to one of my subprojects sources file (using the FILE_VIEW_INCLUDES_FOLDER macro), and this proved to be the problem. After I reverted the sources file back to its previous revision in SVN the project loaded again.

Another one of those Platform Builder 5.0 mysteries solved...

If sysgen_capture doesn't work for some reason you can always manually clone the component you want to modify. By following the instructions on the blog post "Cloning Public Code: An Example" you should be able to get this working, but to help you a bit more here are some step by step instructions for manually cloning public code.

In this blogpost we'll be cloning TELNETD because for some reason sysgen_capture doesn't work for this component. TELNETD is located under the WINCE project tree "servers". I know the exact name of the WINCEPROJ by opening up the makefile in \PUBLIC\SERVERS\CESYSGEN and searching for WINCEPROJ. The correct sysgen_capture command for TELNETD would be:

sysgen_capture -p servers telnetd

The -p parameter indicates the WINCEPROJ, so in this case it's of course not "common", but "servers". Unfortunately for some reason this command doesn't output a sources.telnetd file (in fact it doesn't output any file!). No stress, let's just clone manually:

1. Copy the TELNETD folder to your OS Design (\WINCE500\PBWorkspaces\<MyOSDesign>\TELNETD or \WINCE600\OSDesigns\<MyOSDesign>\<MyOSDesign>\TELNETD)
2. Open the sources. file in the folder you just copied
3. Set TARGETTYPE to DYNLINK
4. Set RELEASETYPE to LOCAL
5. Delete WINCETARGETFILE0=$(_RELEASELIBDIR)\$(TARGETDEFNAME).def
6. Open the makefile. in <WINCEROOT>\PUBLIC\SERVERS\CESYSGEN and search for telnetd
7. Copy the line @set TARGETLIBS... and @set DLLENTRY...
8. Paste into your sources. file
9. Change @set TARGETLIBS... in your sources. file to:
   
   TARGETLIBS=$(_PROJECTROOT)\cesysgen\sdk\lib\$(_CPUINDPATH)\ws2.lib \
              $(_PROJECTROOT)\cesysgen\sdk\lib\$(_CPUINDPATH)\coredll.lib \
              $(_PROJECTROOT)\cesysgen\sdk\lib\$(_CPUINDPATH)\ceosutil.lib  \
              $(_PROJECTROOT)\cesysgen\sdk\lib\$(_CPUINDPATH)\authhlp.lib
10. Change @set DLLENTRY... in your sources. file to:
    
    DLLENTRY=DllEntry

11. Add the include paths:
    
    _OEMINCPATH=$(_WINCEROOT)\public\common\oak\inc; $(_WINCEROOT)\public\common\sdk\inc; $(_WINCEROOT)\public\common\ddk\inc
    NOTE: DO NOT PUT SPACES BETWEEN THE PATHS! I know it is confusing because I do it above, but I have to to support line breaking (otherwise this page will look ugly...).
12. And build...

Easy as that! Now that you have the TELNETD component successfully cloned you can continue following the instructions from step 7 in the blog post "Cloning Public Code: An Example" (of course replacing "netui" with "telnetd" in the text).

Here's the complete sources. file for reference (don't forget to delete the spaces in the _ISVINCPATH, _OEMINCPATH and INCLUDES macros! I need those spaces here to format the HTML nicely, but you will get compiler error "/I" requires an argument if you leave those spaces):

For some strange reason, people are still changing public code and doing build and sysgen's on their Windows CE tree (read this to learn why this is a bad thing). Changing code in the PUBLIC and PRIVATE trees might seem like a shortcut but actually it is everything but a shortcut. When you clone the code before changing it in place you will save yourself hours and hours of build time. A targeted build of a cloned driver takes seconds, a clean build and sysgen takes hours.

Cloning is not difficult at all, so there is really no reason to change PUBLIC or PRIVATE code. To prove that it is not difficult (once you know what you are doing) I will show you how to clone PUBLIC code in this blog post. Note that there is an article by Steve Maillet and Mike Hall in MSDN about cloning public code too. They split the clone into a "LIB" part and a "DLL" part. I don't really like that method; I just merge the sources file into one that creates the DLL in one go. Have a look at the "split" method here.

I chose to clone NETUI, since this component is responsible for a lot of dialog boxes that you may want to change to match your systems specifications.

I am cloning NETUI on a Windows CE 6.0 tree, but it works exactly the same on a Windows CE 5.0 tree (apart from the OSDesign folder which is called PBWorkspaces in CE 5.0).

Prerequisites: A sysgenned OSDesign with the "Network UI" component included.

Let's start!

1. Copy the entire \WINCE600\PUBLIC\COMMON\OAK\DRIVERS\NETUI folder to your OSDesign, for instance \WINCE600\OSDesigns\MyOS\MyOS
2. Open a build window (in Platform Builder / VS2005 click "Open Release Directory in Build Window" from the menu "Build"
3. In the build window, type:
   
   cd..
   cd..
   cd netui
4. Now type:
   
   sysgen_capture -p common netui
   This command will output a set of files in the current directory.
5. Since we only want to build netui, you can delete:
   
   sources.btdrt
   sources.ceddk
   sources.iphlpapi
   sources.k.iphlpapi
   sources.ws2
   sources.ws2k
6. Now intelligently merge sources. with sources.netui (we make the changes in the sources. file):
• Make a backup of the sources. file (copy to sources.org) so you can always go back if needed
• We are trying to build a DLL, so change TARGETTYPE=LIBRARY to TARGETTYPE=DYNLINK
• We are building NETUI as a project in our workspace, so set the RELEASETYPE to LOCAL (RELEASETYPE=LOCAL)
• We are not building a LIB anymore, so there's no need to preprocess the def file. Remove the following lines:
  
  WINCETARGETFILE0=$(_RELEASELIBDIR)\$(TARGETDEFNAME).def
  PREPROCESSDEFFILE=1

And add this line to tell the build system to use the local def file for the exports:


DEFFILE=$(TARGETDEFNAME).def
• We don't need the resource file later (because we are not creating a lib that has to be transformed into a dll later), so remove the COPYRES=1 line and the WINCETARGETFILES line pointing to the res.
• OSDesign SubProjects get build last, so we don't need to copy SYNCHRONIZE_DRAIN=1 from sources.netui to our sources. file
• Since we are now building a DLL, copy the DLLENTRY line from sources.netui to sources.
• Now move the entire SOURCELIBS and TARGETLIBS from sources.netui to sources.
• and delete any references to netui (netui.lib/netui.res) itself (because we are building that now)
• Change the paths to the target libraries from using the PUBLIC sysgened libraries to the libraries sysgened to your workspace, so change $(_SYSGENSDKROOT)\ to $(_PROJECTROOT)\cesysgen\sdk\ and $(_SYSGENOAKROOT)\ to $(_PROJECTROOT)\cesysgen\oak\
• You can also delete the #xref lines, they are remains of older days (CE 2.11 era).
• Now try to build (by typing "build" in the build window you still have open)
• oops! Doesn't build. It can't find the standard include files... Add the following to the top of the sources file:
  
  _ISVINCPATH=$(_WINCEROOT)\public\common\sdk\inc;
  _OEMINCPATH=$(_WINCEROOT)\public\common\ddk\inc; $(_WINCEROOT)\public\common\oak\inc; $(_WINCEROOT)\public\common\sdk\inc;
  NOTE: DO NOT PUT SPACES BETWEEN THE PATHS! I know it is confusing because I do it above, but I have to to support line breaking (otherwise this page will look ugly...).
• And since this project is building OEM code (a kernel component) set WINCEOEM=1 to indicate we want to use the _OEMINCPATH (so if you really want to you can delete the _ISVINCPATH since we don't use that one).
• Try to build again:
  
  BUILD: [01:0000000048:ERRORE] \WINCE600\OSDesigns\MyOS\MyOS\NETUI\.\btmgmtui.cpp(39) : fatal error C1083: Cannot open include file: '../bluetooth/sample/btenum/btenum.hxx': No such file or directory
  
  So, one of the sources files is trying to include a header on a relative path. Since NETUI came from \WINCE600\PUBLIC\COMMON\OAK\DRIVERS\NETUI this path must be \WINCE600\PUBLIC\COMMON\OAK\DRIVERS\BLUETOOTH

  Now there are 3 ways to solving this: Either add the \WINCE600\PUBLIC\COMMON\OAK\DRIVERS\NETUI folder to the include path (add INCLUDES=$(INCLUDES);\WINCE600\PUBLIC\COMMON\OAK\DRIVERS\NETUI) so that the header file can be found off the relative path, or change the source file including the header to use a full path to the header file, or copy the btenum.hxx file into the NETUI folder and change the source file including the header to use the local header file. I think the last option is the cleanest, so let's copy \WINCE600\PUBLIC\COMMON\OAK\DRIVERS\BLUETOOTH\SAMPLE\BTENUM\btenum.hxx to \WINCE600\OSDesigns\MyOS\MyOS\NETUI\btenum.hxx and change btmgmtui.cpp to #include "btenum.hxx"

• Build again: 0 Warnings,  0 Errors. Whuuhooo!
• Now there is one final thing we have to add to the NETUI sources. file. We have to tell the build system we want to copy the final binary to the _FLATRELEASEDIR so our netui.dll will overwrite the default one from the PUBLIC\COMMON folder. Add WINCEREL=1 to the top of the sources. file.
7. Now add the NETUI project to your workspace. In the Platform Builder VS IDE click "Add Existing Subproject..." from the "Project" menu. Browse to the \WINCE600\OSDesigns\MyOS\MyOS\NETUI folder, select "Sources/Dirs Files" from the "Files of type" dropdown list, select the sources. file, and click "Open"
8. In the Solution Explorer window you can now find your NETUI component under the "SubProjects" node. Right click NETUI and click "Rebuild" to see if you can build the project from the IDE. It should all build fine.
9. If all builds fine, you can delete the sources.netui and the sources.org files since you no longer need them now.
10. The "Insert existing project" action created a couple of files for you. One of those files is a bib file. BIB files are used to indicate which files need to be included in the image. Since we cloned NETUI, but did not remove it from the OSDesign features, we now have 2 references to NETUI in the combined BIB files (one is in COMMON.BIB and the other in our newly generated NETUI.BIB). We don't need an extra reference in the BIB files, all we want is overwrite the default NETUI.DLL in the _FLATRELEASEDIR, so you can remove this reference from the newly generated NETUI.bib.

Now you can change whatever you need to change in NETUI without worrying about messing up the PUBLIC portion of the WINCE tree or having to do a "Clean Build and Sysgen".

If you have followed all the steps above your final NETUI sources. file should look like this (except the spaces in the _ISVINCPATH, _OEMINCPATH and INCLUDES macros!):

If you read this blog or the CE newsgroups you should by now know you should never change any files in the PUBLIC or PRIVATE folders of your WINCE installation tree. So what if you want to change a setting in common.reg (which is located in the PUBLIC folder)? Well, add the same entry to your platform.reg (in your PLATFORM\BSP folder) and set it to whatever you like. So, what if you want to delete a setting from common.reg? Again, add the same entry to your platform.reg but precede the key name with a "-". This works because platform.reg is merged into reginit.ini after common.reg, so any duplicate entries or deletion commands (the "-") will override the settings in common.reg.

So, what about dat files? Unfortunately, deleting dat entries with the "-" tag is not supported. Dat files are responsible for the initial copy of files and creation of folders during boot. For instance, dat files are used to copy shortcuts from the \Windows folder to the desktop of your CE device. Now what if you don't want to have icons on your desktop? The only way that seems possible is to actually change the public dat files in place, but you know you should never change any files in the PUBLIC tree! We just have to come up with a different solution...

The solution in this case is to filter the dat files and remove any unwanted lines before creating the final image but after the sysgen phase and without touching the original files in PUBLIC. Windows CE calls out to a number of batch files during the makeimg process to let you write your own scripts to affect the image at different points during the image building process. One of those batch files is "PostFmergeObj.bat" which, as the name suggests, is called just after fmerge merged the "Obj" files (which are in fact "Dat" files... confused already? ;o) into your FLATRELEASEDIR.

By using a bit of scripting magic we can use this batch file to filter out unwanted lines from the final merged InitObj.dat file:

The core of the script is from line 15 onwards. In line 15 we delete any previous initobj.org file this batch file may have left behind. Then, in line 16, we rename initobj.tmp (all dat files are merged into this file by fmerge) to initobj.org and in line 17 we call the dos utility Findstr:

By calling findstr with options /I and /V we effectively filter out the lines we specify in PostFmergeObj.txt and we pipe the output of findstr into InitObj.tmp. After this batch file returns the build system will call the "Text to Unicode" tool (Txt2ucde.exe) that converts any ASCII text strings in Initobj.tmp to Unicode text strings and creates the file Initobj.dat which then contains the Unicode conversion of InitObj.tmp (see http://msdn.microsoft.com/en-us/library/ms930234.aspx).

Note that you can also use regular expressions to find strings with findstr. Using this you have a very powerful tool to filter whatever file you want filtered and of course this trick doesn't only work on dat files, it works on any ascii text file (like bib, reg, etc).

All you have to do for all this to start working in your build is create PostFmergeObj.bat in your PLATFORM\BSP\FILES folder and copy'n'paste the above script. The Windows CE build system will automatically search for this batch file and if it finds it execute it just after it has merged the various configuration files. Don't forget to create a "PostFmergeObj.txt" file with the lines you want to delete from the final InitObj.dat file in the same folder as the batch file, like this:

When you now perform a "make image" this batch file should automatically execute and its output (the "echo" commands in the batch file) should appear in the build output window.

The first public beta of version 3.0 of the .NET Micro Framework SDK just got released through the Microsoft Connect website, check out the new features!

In all the drivers and applications I develop, I always take the time to add plenty of logging because I know good logging will save me loads of time debugging my code (as if I ever have to "debug" my perfect code, ha! ;o).

Windows CE has a really great mechanism for debug logging called Debug Zones. This mechanism allows you to divide your debug messages into 16 zones and each of these zones can be turned on or off at run-time. This allows you to control the verbosity of your debug output and helps prevent cluttering of messages. Read this excellent blog post by Travis first before reading on: http://blogs.msdn.com/ce_base/archive/2006/12/18/debug-messages-and-debu...

To add debug logging using Debug Zones to your application or driver do the following:

1. Define your zones using the DEBUGZONE macro
1. Define a DBGPARAM structure and fill it
2. Call either DEBUGREGISTER or RETAILREGISTERZONES from your DllMain (driver) or WinMain (application)
3. Log messages using either DEBUGMSG or RETAILMSG

Sounds easy (and actually it is) but to make it more easy (and because I found myself doing the same thing over and over) I created a template I now use in all my driver and application code, even on projects targeted for Desktop Windows. Even though "Big" Windows doesn't have a "Debug Zone" mechanism you can still use the template. The big benefit of course being your code is completely portable between desktop PC's and CE devices (and also MBS/UNICODE)! The only thing you can't do is switch debug zones at run-time, but you can still select which zones should be on or off at compile-time.

Let me explain the template by walking through it; I'll start with DebugZones.cpp (the line numbers reflect the actual line numbers in the template files):

The first and most important thing to note is that you should not change anything in this file. You configure the template all through the header file, which I will be discussing later.

In this first part of DebugZones.cpp I simply define a global variable dpCurSettings of type DBGPARAM and fill the structure with defines from DebugZones.h. The __TMODULE__ corresponds to the "lpszName" member, the ZONEx_TEXT defines fill the "rglpszZones" array and finally RETAILZONES or DEBUGZONES (depending on whether you build a "debug" or "release" version of the code) corresponds to the "ulZoneMask" member of the DBGPARAM structure. The CE debug system uses this structure to get Debug Zone information from the module and to set or clear debug zones during execution of the module. When you use the template for code targeted at Desktop Windows this structure will only be used by the DEBUGMSG/RETAILMSG macros (they use the ulZoneMask member as a display condition).

This code is only used when you target Desktop Windows. Only Windows CE defines NKDbgPrintf so for Desktop Windows I had to create my own implementation of this function to make the Debug Zones system portable. This function simply takes a format string followed by a variable number of parameters and transforms that into a formatted string and outputs it to the debug output window of Visual Studio. Note that the maximum length of the formatted string is BUFFER_SIZE characters.

This next bit of code is a helper function that solves a problem when you want your code to be portable between MBS and UNICODE. MBS is multi-byte string, also called ANSI string, and UNICODE is, well, UNICODE strings. Let's say you are compiling for UNICODE but want to display a multi-byte string using a function that uses a format string. The way to do that is to use %S instead of %s in the format string. When you use %s in code compiled for UNICODE the variable indicated by %s should be in UNICODE. When you compile for MBS the variable indicated by %s should be MBS. In other words, %s indicates the "native" string format. If you want to display a multi-byte string in code compiled for UNICODE you'd use %S, and if you want to display a UNICODE string in code compiled for MBS you'd use %S as well. In other words, %S indicates the "opposite" string format. And here lies the problem: What if you want to display a multi-byte string regardless of whether the code is compiled for MBS or UNICODE?

Take a look at the following code:

Note that the _T macro is defined as L in UNICODE (making the following string a UNICODE string) and as nothing in MBS (leaving the following string a multi-byte string).

When compiled for UNICODE the above code works perfectly:

But now see what happens when you compile the above code for MBS:

You only get the output of the first line! So what just happened? Well, the _vstprintf_s function in NKDbgPrintf expected the first argument to be of the opposite string format. Since you compiled for MBS this means it expects a UNICODE string. The _vstprintf_s function detects it is not and bails out, setting szBuffer to an empty string.

The to_T function solves this problem:

When compiled for MBS the output is:

I can here you say: So why don't you just use _T() in that 2nd call to NkDbgPrintf? And yes, you are right, in the above code that would work, but what if it's a variable pointing to a multi-byte string? You can't use _T() around a variable:

Line 64 defines the maximum number of times you can call this function before it will start overwriting its internal buffers. Increase this value if you want to use more than 4 parameters that need to be converted in one call. To understand this, see the following code and its output:

Output:

This is the part of the template you should change to match your driver or application. First, in line 30, change the __MODULE__ define to the name of your driver or application. If you are creating a flash driver you'd set this to "FLASH" for instance.
Next, in lines 33 to 48, you define the friendly names of the debug zones you want. These names will show up when you use the Debug Zones dialog or the "zo" command from Platform Builder. The predefined ones are always useful:

• Init

Use this zone to display debug messages during the initialization phase of your driver. I use this zone in the Init function of drivers.

• Function

This zone is used to display function entry and exit. Used by the FUNC_ENTRY and FUNC_EXIT macros (I'll discuss those later)

• Memory

This zone is used to display memory allocation and de-allocation. Having a separate zone for memory really helps in identifying memory leaks or memory related problems in your driver or application (besides CeDebugX)

• Info

This zone is used to display general information. I use this zone a lot through-out my code and enabling this zone usually means I get heaps of debug messages, but also heaps of information about the flow of my code

• Warning

Use this zone for non-fatal errors (also known as warnings ;o)

• Error

Use this zone for fatal errors; situations where you had to abort a certain operation

Another zone I use often is "IOCTL". If my driver handles custom IOCTLs I use that zone to display information when those IOCTLs are handled. If you want to add the IOCTL zone you'd change line 36 to:

Lines 51 to 66 define the actual zone IDs. You use these defines as the first parameter of LOG_MSG (that I'll discuss later) to indicate the zone this message belongs to. After you added the IOCTL zone to the friendly names list, you'd change line 54 to:

Lines 69 to 84 define the zone masks. You use these defines to indicate which zones should be enabled by default. After you added the IOCTL zone to the zone ID list, you'd change line 72 to:

Line 86 and 87 define the default zones to use in a release build and a debug build. I enable zones INIT, WARNING and ERROR in a release build and add FUNCTION, MEMORY and INFO to debug builds. Note that in a "ship" build (on Windows CE, WINCESHIP=1) debug/retail messages are defined as "nothing" (so all the logging code is effectively removed from the final binary).

If you define FULL_CONTEXT (line 91) the LOGMSG macro will output the full path to the source code filename in front of the message, instead of just the module name. I don't enable this by default because it makes the debug output cluttered and less readable (but it can be very handy in some cases).

This concludes the part of DebugZones.h that you should change. The remainder of the code mostly deals with the differences between Desktop Windows and Windows CE:

The interesting part is from line 165 to 173 where I define three macros that allow you to log messages in a good format (in my humble opinion):

• FUNC_ENTRY

Usage:


FUNC_ENTRY(format_string [, arg1, arg2, ...]);

Examples:

FUNC_ENTRY(_T("void"));
FUNC_ENTRY(_T("lpParameter = 0x%08X"), lpParameter);

Output:
(Note the '+' characters that indicate function entry)

ModuleName: +FunctionName(void)+
ModuleName: +FunctionName(lpParameter = 0xA5A5A5A5)+
• FUNC_EXIT

Usage:


FUNC_EXIT(format_string [, arg1, arg2, ...]);

Examples:

FUNC_EXIT(_T(""));
FUNC_EXIT(_T("return %s"), bRet ? _T("true") : _T("false"));

Output:
(Note the '-' characters that indicate function exit)

ModuleName: -FunctionName()-
ModuleName: -FunctionName(return false)-
• LOGMSG

Usage:


LOGMSG(ZONE_ID, format_string [, arg1, arg2, ...]);

Examples:

LOGMSG(ZONE_INIT, _T("Started successfully!\r\n"));
LOGMSG(ZONE_ERROR, _T("ERROR: Invalid parameter (dwParam = %d)\r\n"), dwParam);
LOGMSG(ZONE_WARNING, _T("WARNING: Could not reach IP %s\r\n"), to_T(inet_ntoa(IPAddress)));

Output:

ModuleName:  FunctionName> Started successfully!
ModuleName:  FunctionName> ERROR: Invalid parameter (dwParam = 1234)
ModuleName:  FunctionName> WARNING: Could not reach IP 192.168.1.1

To use the Debug Zones template in your projects you simply add DebugZones.cpp and DebugZones.h to your solution and call RETAILREGISTERZONES(hMod) from your entry point function: