What to build when...

[An updated version of this post for WEC2013 can be found here]

A question that keeps coming back on the newsgroups is "I changed some code, but it does not end up in my image", or "I changed some registry values in platform.reg, but if I look at the device registry, it's not there!".

The source of these problems is build-related. You've got to understand the build system in order to know exactly what to do. This blog post aims to give you a clear handle on "What to build when"!

The first and most important thing to do is to delete two options from the Platform Builder Build menu: Build and Sysgen and Rebuild and Clean Sysgen. These commands will build the entire tree (so including any source in PUBLIC and PRIVATE). For us normal developers these commands do not make any sense. If you're not in the CE development team working at Microsoft you simply don't have all the source code. Rebuilding the entire tree may work, but it will override any installed QFE (QFE's contain updated binaries and sometimes, but not always, updated source code). After rebuilding the entire tree you will sooner or later encounter strange errors that are either very hard to fix, or just impossible to fix. There is no way to revert back to a clean tree once you've done a (Re)Build and (Clean) Sysgen. The only way is to completely remove Windows CE and reinstall. Don't forget to reinstall all QFE's after that as well (download the QFEInstaller tool to help with that).

Now you know why you often see me shouting NEVER EVER DO A BUILD AND SYSGEN! in the newsgroups...

UPDATE: The instructions for removing the bad build commands from the menu are for Windows CE 5.0/6.0. For instructions on Windows Embedded Compact 7 read this. Note that the below "What to build when" instructions are STILL VALID for WEC7 AND WEC2013!

The original menu
The original Advanced Build Commands menu

Step 1.
Customize toolbar
Right click on the toolbar and click Customize...


Step 2.
Delete Build and Sysgen
Click on menu Build, submenu Advanced Build Commands, then right click on Build and Sysgen and click Delete


Step 3.
Delete Rebuild and Clean Sysgen
Do the same for Rebuild and Clean Sysgen


Step 4.
The original menu
Close the Customize window. The Advanced Build Commands menu should now have only 4 options: [Sysgen], [Clean Sysgen], [Build Current BSP and Subprojects] and [Rebuild Current BSP and Subprojects].



If you are still using Platform Builder for Windows CE 5.0 the instructions stay the same except the Build menu is Build OS and there's no submenu (everything is under the Build OS menu).

I know that some developers working with CE change code in the PUBLIC and PRIVATE trees. If you do that you need to do a build and sysgen to include your changes in your kernels BUT: You will end up with a corrupted CE installation. If you need to change PUBLIC or PRIVATE code you have to clone the code. Cloning code is in most cases not much more than a simple copy. Copy the sources you want to change to a folder in your BSP, change the sources file so that it builds a DLL instead of a lib and voila, you're done. I know I make it sound easier than it is, but taking a shortcut and modifying code in the PUBLIC or PRIVATE trees will prove to be a long way around in the long run.

Phew! Now that we got that out of the way let's see what build command we've got to use in what situation:

  • Create a new OS Design (or add a new build configuration to an existing OS Design): Sysgen (Build Solution)
  • Since a new OS Design doesn't have anything in its build configuration's output folders (same with a new build configuration) you'll have to sysgen the entire solution (for that build configuration). The longest process, but luckily you don't have to do this many times (see below).

  • Change Platform Settings: Make image
  • Platform Settings
    If you change any Platform Settings (like IMGNOKITL, IMGNODEBUGGER, IMGPROFILER) all you have to do is a Make Image.

  • Change driver source code in your BSP: Build the driver and Make Image
  • If you change driver source code, all you have to do is just build the driver (WINCEREL must be set to 1 but it is set by default so unless you changed it there's no need to worry) and do a makeimg. If you only want to debug the driver you can also add the DLL to the Release Directory Modules list (menu Tools) and just restart the device (or reload the device driver on the device) without having to do a makeimg and download to device. Building just the driver is a simple right-click on the driver and Build in the IDE or "build" in the driver's folder on the command line.

  • Change multiple source code files in your BSP: Build the BSP and Make Image
  • The safe option, this way you can't forget to rebuild anything. Building the BSP is a simple right-click on the PLATFORM\BSP folder and Build in the IDE or "build" in the BSP's root folder on the command line.

  • Change platform.reg, bib, dat or db files in your BSP: Sysgen BSP, Copy Files to Release Directory, Build All Subprojects and Make Image
  • A lot of steps, but this will still not take longer than a couple of minutes. If you change any of the platform.* files we need to re-filter (Sysgen) those files and make sure the filtered files are copied into the FLATRELEASEDIR (Copy Files to Release Directory). That last action did however clear the project binaries from the FLATRELEASEDIR so we need to make sure those binaries and settings are getting copied into the FLATRELEASEDIR again (Build All Subprojects) and finally we are ready to Make Image. Now your registry changes will be in the image (check reginit.ini to make sure, last entry wins).

  • Change some source files and platform.reg, bib, dat or db files in your BSP: Build and Sysgen the BSP, Copy Files to Release Directory, Build All Subprojects and Make Image
  • Only difference with previous situation is that you now have to build the BSP to include the source code changes.

    Please note that "Build and Sysgen the BSP" is not the evil "Build and Sysgen" command. The evil command is only in the Build menu (and should not be anymore if you followed the instructions above). The "Build and Sysgen BSP" command can be executed by right clicking your BSP root node in the Solution Explorer and choosing "Build and Sysgen (cebuild -qbsp)". Notice the -q in that command? That shows you it's a good command...

  • Change the workspace configuration (add or delete a component): Sysgen the entire workspace (Build solution)
  • For most components a simple Sysgen is enough. For some components (like when changing from RAM based registry to Hive based Registry) a Clean Sysgen is needed. This action takes the longest (anywhere from 5 minutes for a small workspace configuration on a very fast machine to a couple of hours for a really big configuration and a very slow machine). A Sysgen is a right-click on the workspace, Advanced Build Commands->Sysgen in the IDE or "blddemo -q" on the command line.

  • After installing one or multiple QFEs: Clean Sysgen the entire workspace (Rebuild solution)
  • To make sure all components are re-linked with the (possibly updated) libraries a Clean Sysgen is needed. This action takes the longest (anywhere from 5 minutes for a small workspace configuration on a very fast machine to a couple of hours for a really big configuration and a very slow machine). A Clean Sysgen can be performed by choosing Rebuild Solution from the build menu, or choosing Advanced Build Commands->Clean Sysgen in the IDE or "blddemo clean -q" on the command line.

Sometimes it's easier to build from the command line. If you are unsure what command to type you can always perform the action in the IDE first and watch the 3rd line in the build output window starting with "Starting Build:". Behind the colon is the exact command line for that action, eg Sysgen on the BSP: "Starting Build: SysgenPlatform %_TARGETPLATROOT% preproc&&SysgenPlatform %_TARGETPLATROOT% postproc", so on the command line you would type "SysgenPlatform %_TARGETPLATROOT% preproc" followed by enter and the 2nd command "SysgenPlatform %_TARGETPLATROOT% postproc" followed by enter.

If you use the commandline, make sure you never forget "-q" when running "blddemo" as "blddemo" without "-q" is a Build and Sysgen!

I hope this blog post will help you speed up your builds and lower your frustration with the build system!

Good luck!

Comments

Thanks Michel. That helped a lot.

Hi Michel,

When building a BSP, why all the SOC folders (OMAP2420,PXA270, etc..) as well as platform\common folder and some other common folders are building. This is taking lot of time.

When building a particular BSP, the particular SOC ( like OMAP2420_MS_V1 for H4sample BSP) alone can be built right. Any options are there to do that.

Regards,
Kumar

Hi Kumar,

NOTE: In the text below, when I talk about Build command BSP, I mean right-click on your BSP and choose that command from the dropdown menu (in other words: a Targeted Build). I'm NOT talking about commands from the Build Menu in Platform Builder!

The COMMON and SOC folders should only build if you changed something there. If I do a Rebuild BSP only my BSP rebuilds, not the common folders. Are you sure you don't change anything in the common folders?

Only if I do a Rebuild and Clean Sysgen BSP I can see that those common folders are rebuilding (as expected and wanted!). Now, if you'd want to change that behavior (but I don't think that is such a good idea) you can change the DIRS statement in the dirs file in \WINCE600\PLATFORM\COMMON\SRC\SOC to OPTIONAL_DIRS and add set BUILD_OPTIONS=OMAP2420_MS_V1 to your bsp batch file (in the root of your BSP). That will only build the OMAP2420_MS_V1 SOC folder instead of all. Of course the folders in COMMON should be build if you do a Rebuild and Clean Sysgen Build of the BSP since they could be used by your BSP.

Let me know if this works for you.

Hi,

How can I clean a BSP, while releasing only source code to customer.

Regards,
GSR

Delete *.bif, build.*, all obj folders, the target and lib folders and the folder cesysgen/files (but not the cesysgen folder itself!).

This is a really useful article, thanks. I'm new to Platform Builder and have found building an image from my osdesign to be very painful (taking 45 minutes). I'm using Platform Builder 6.0 service pack 1. As I understood it blddemo -q which runs when selecting Build/Build Solution or Build Project from within the IDE should only build the changes. So first time can take a while, but next time can be quicker if the sysgen doesn't need to run again - is that right? Anyways, if I don't make any changes and then blddemo -q it's as though a full rebuild is happening. I've copied some output from the build window below. Is this what's expected? Am I running the wrong build command? What part of the build process is the longest part?

Thanks in advance, Steve

-----------------
Starting Build: blddemo -q
==============
BLDDEMO: Generating OS Design Folders
BLDDEMO: Done Generating OS Design Folders
BLDDEMO: Generating OS Design Files to C:\BIT_nj\code\nj2\Wince600\DEVNAME_BSP_ARMV4I\oak
BLDDEMO: Done Generating OS Design Files
CEBUILD: Deleting old build logs
CEBUILD: Skipping directly to SYSGEN phase
Building dep trees: winceos dcom gdiex ie script servers shellsdk shell rdp wceshellfe wceappsfe directx voip datasync netcfv2 netcfv35 sqlcompact SQLCE cellcore ostest mediaapps speech FP_VOIP nj2
CEBUILD: Running sysgen preproc (for COMMON)
Starting sysgen phase for project ( common )
......
Starting sysgen phase for project ( FP_VOIP )
CEBUILD: Running sysgen postproc (for COMMON)
Starting sysgen phase for project ( common )
......
----------------------

Yes, a blddemo -q is a sysgen. The only thing taking longer is a clean sysgen. Please re-read the article, it's all there. Don't do a sysgen when you don't need to.

thank you, it's very very useful. And i think it's a basic knowledge for developing wince but we always ignore it which caused the developers wasted too much time on the build system.
Now I've a question.
I've installed vs2005+sp1+pb6+sp1+r2+r3, and no QFE.
But I've already run "rebuild and sysgen".
As u said "it will override any installed QFE ". So I've not installed any QFE. Is it OK for me?(I'll never do it again,-_-). Or R3 is a QFE too? So maybe I'll reinstall vs2005+sp1+pb6+sp1+r2+r3.
thank you.

R3 is just another set of QFE's... Reinstall it all is your only safe option... Sorry!

thanks

Hi Michel,
Can you add to do a Sysgen after creating a new OSDesing please.

Thanks,
Shai

Done.

Thanks for this very helpful post. One question, though. I'm a strong proponent of frequent "make clean" operations on my development platforms, one of the reasons being to ease VCS operations. What command would you recommend for the clean, and what would be the steps for a full rebuild once done ?

I am with you. Whenever I do a "release to customer" (or testing a release candidate) I use a clean VM, check out my OS Design from SVN (and of course only the non-generated and only source files are in SVN!) and do a simple "Sysgen" on the OS Design's shipbuild.

I am using a tool to hardlink the Windows CE root from a clean WINCE600 root so I always start from a completely clean WINCE600 tree, with a completely fresh OS Design. This way if I get any build errors I know I forgot to check in something, and if it all works fine I know I'm testing with a qualified build binary.

Thanks again!

If you ever want to add more details about which files should go in SVN (build.dat is one that puzzles me) as well as what hardlink tool you use, that'd be helpful - but we're getting way off-topic. Ideas for new blog entries ?

Yes, those are long standing blog posts on my "to-do" list. I may do a combined (will be long) post on this explaining exactly how I work with SVN, hardlinks and building qualified releases.

I just need some time... ;o)

Great article!

I'm not sure abou the way working with CE change code in the PUBLIC and PRIVATE trees. How to make a dll only for my changed code, then how will it be loaded? I'm a little confused. Could you explain more details?

You should NEVER change any code that does not belong to you. If you want to change Microsoft code (in PUBLIC or PRIVATE, or even PLATFORM COMMON or SOC) you will have to clone the component you want to change. You can clone it to a subproject in your OS Design or to your BSP, whatever makes more sense.

More information on how to clone can be found here:

http://guruce.com/blogpost/cloningpubliccodeanexample
http://guruce.com/blogpost/manualcloneofpubliccode

Hi -
Thanks for the article. I did the same mistake as others: "Build and Sysgen", after installing a QFE thinking that all the changes get incorporated. And you know the results :)
I request you to post/reply steps that need to be taken after installing a QFE.
During normal scenerio, build the project, copy to release dir and makeimage may be sufficient. But after installing QFE, it is difficult to know which project to build individually and eventually end up calling "Build and Sysgen" thinking that it will take care of everything.
The notes from QFE says "rebuild the platform", but does not explain how or a link that explains how.
Please provide information and help to solve the most common problem among users.

thanks,
chandra

The instructions are correct; rebuild the platform. In tech-speak this is a "clean sysgen", or a "blddemo clean -q". From the IDE a "Rebuild Solution" is the same, or you can choose "Clean Sysgen" from the advanced build menu.

Anyway, instructions now also added in the blog item. Thanks for the request!

I am very new to CE. The first part of your post says to never ever performa build and sysgen. Even as far to delete it from the menu. Then, later you say if I change BSP source to perform a build and sysgen. Can you pleasee explain these conflicting statemets?

The manufacturer of our hardware provided us with a sample project, and step by step guide. One of the first steps is to change two files' source depending on configuration of hardware purchased, then to build and sysgen. I simply cannot get it to work but the manufacturer is telling me the problem must be in my installation. I have reinstalled VS, VS SP1, PB, PB SP1, R2, R3, Rollup 2010, QFE 2011 about 4-5 times with the same result. After reading your post it seems that Build and Sysgen is the wrong option, but then I see you say if I change BSP files to Build and Sysgen. Can you see why I am confused?

They are not conflicting... There is a difference between "Build and Sysgen" and "Build and Sysgen the BSP" (I even emphasized that in the blog text by making BSP bold). Never choose "Build and Sysgen" from the build menu, but you can (and often have to) select it from the context menu when you right click on your BSP root node in the Solution Explorer.

The manufacturers instructions ARE WRONG. You should never do anything else but a "Sysgen" or "Clean Sysgen" as our blog post explains.

Update: I've updated the blog post to make the distinction even clearer.

Thank you for the help. So the directions should be to right click on the bsp and choose build and sysgen? As a very new person, this minor detail is extremely huge. I had seen the bold text but again, it is not spelled out anyplace and I thought they were the same. Perhaps this should get more attention in the post. I am appreciative.

I had already updated the blog text. If it is still unclear please let me know.

I'm new and my colleagues weren't able to tell me the right way to build for specific changes, your article has been extremely helpful.

Thank you for a concise explanation of what to build when. But I'm confused... I can't find the option "Build All Projects" in Platform Builder 6.0. Was this renamed? Will "Build Current BSP and Subprojects" do?

The "Build All Subprojects" menu option will only appear in the Build menu if you have any subprojects in your OS Design. If not, then you can skip that step and just do a makeimg straight away.

PS. I updated the blog text to say "Build all Subprojects" instead of "Build all Projects", my bad!

Thanks, it makes sense now. Even to a n00b :)

I have cloned the BSP and after that while trying commands in it, I accidentally gave blddemo without knowing it is Build and Sysgen (I came to know it now only, after reading this blog). What do I do now?

In that case all that is left is to uninstall, clean and reinstall your CE version... That is the only way to guarantee your tree is working properly. Building a kernel might still work, but you just never know what binaries are linked now as some will have been built from different source than what was shipped by MS (on either the official DVDs or in update packages later installed). Reinstall if you want to build qualified, known-to-be-good kernels.

I actually didn't let the build to complete fully. I closed the command window immediately after 1 min (as I was just taking notes). Does that make any difference?

That depends on the speed of your computer. The first phase usually takes some time. Do you still have the build.log of that particular session? If yes; you can verify no binaries had been deleted yet and no source code had been built yet. In you don't have the build log of that session then there is no way to know for sure. In that case the only way to know if your tree is not corrupt is a full clean and reinstall.

I checked the log. Initially, some scanning was there. Then, Building WCEFiles0 Pass completed and Sysgen started and it was in progress when I closed the window.

Clean & reinstall... Sorry!

I'm a beginner. I'm using Visual Studio 2005 and getting trained in building WinCE 6.0 OS Image. I cloned an existing BSP and was trying blddemo related commands on it. One among them was "blddemo". I didn't know that it is the dangerous "Build and Sysgen"! I came to know it only now, after reading this blog. What should I do now? How do I check if the previous binaries of QFE has lost? Kindly guide me.

I've replied to you on your previous comment already. See below.

How much of these commands are still valid in Compact 2013? I'm new to 2013 still and I've found that there is not a Sysgen BSP command, only build.

Everything in this blog post is still valid for WEC2013.

Luckily MS has finally removed the dangerous build commands (blddemo without -q) in WEC2013, so you don't have to worry about those anymore.

Build solution = sysgen (blddemo -q)
Rebuild solution = clean sysgen (blddemo clean -q)

The build commands with their proper names can be found in the Build -> Advanced Build Commands submenu.

If you right click on your BSP in the Solution Explorer you get build commands that affect your BSP ONLY:

Build = This builds the BSP (blddemo -qbsp)
Rebuild = This rebuilds the BSP (blddemo clean -qbsp)
Sysgen = This sysgens the BSP (platform.reg, .bib, etc)
Build and Sysgen = combination of build and sysgen (doh!)
Rebuild and clean sysgen = combination of rebuild and sysgen (doh!)

Note that the above "Build and Sysgen" commands are completely safe commands (they only affect your BSP tree for which you have all the source).

I've updated and reposted this blog item for WEC2013 here: http://guruce.com/blogpost/what-to-build-when-in-wec2013

Thanks Michel! I was quite elated to see that they removed "Build and Sysgen"! I have a question about the new blog post. I'll comment on that article.