Creating a Command line build environment

Long time no see... have been awfully tied up with work, and have been equally lazy too!

Many a times, people have asked me how to create a command-line build enviroment. A complete documentation for it is available in PB documentation, here is an easy summary for lazy people like me:

1. From command line, browse to: Windows\Program Files\Platform Builder for Windows Mobile\5.00\CEPB\BIN
2. Type the following commmand:

pbxmlutils /getbuildenv /workspace "\PBWorkspaces\MyWorkspace\MyWorkspace.pbxml" /config " MyBSP:TargetDevice"> SetMyBuildEnv.bat

Volah, the batch file (SetMyBuildEnv.bat) now has the build environment setup!!

Here's an example:

To set up build environment for an OS design called MyWorkspace, based on an x86 CPU and targeted for the Emulator, type the following command:

pbxmlutils /getbuildenv /workspace "C:\WINCE500\PBWorkspaces\MyWorkspace\MyWorkspace.pbxml" /config "Emulator: x86_Debug" > SetEnv.bat

Do I hear: "Okay, so the documentation already tells me that... what's new?"

Value addition is here: Create a shortcut on Desktop that takes you there....

1. Copy the SetMyBuildEnv.bat to a convenient location. Lets say C:\MyEnvironment
2. Browse to \WINDOWS\system32, right click on cmd.exe and create its shortcut to Desktop.
3. Browse to Desktop, rename Shortcut to cmd.exe to MyBuild.exe, or any name you like.
4. Right-Click, select properties. In Shortcut tab, observe Target : C:\WINDOWS\system32\cmd.exe.
5. Change this to:

C:\WINDOWS\system32\cmd.exe /k C:\MyEnvironment\SetMyBuildEnv.bat

Click Apply, and you are done!!

No more need to use PB for building small changes..... Command line build would do it faster, in a more controlled fashion.

CE6: User and Kernel Mode Drivers

There have been quite an umpteen number of changes to the architecture of CE in version 6.0. One of the changes, not one having stole the limelight, is a new model for loading device drivers said to provide for increased security, robustness or performance.

Prior to CE6, device drivers primarily resided in the device.exe process. Device.exe was a process like any other application, subject to the same memory limitations and performance issues resulting from interprocess calls back and forth between the application, driver, filesystem and kernel.

In addition, since all drivers were located in the same process space a corrupt driver could potentially fault the entire process, shutting down all drivers.

The new kernel architecture provides more flexibility by providing two different driver models. The old device.exe functionality has been moved into the kernel, and a new user device process (udevice.exe) has been introduced. This change results in two different driver models, kernel mode drivers and user mode drivers

Kernel Mode drivers

Kernel mode drivers are loaded by device.dll inside of the kernel. The big win here is performance – the expensive interprocess calls have been eliminated and there is no need for the user mode process initiating a driver access to be switched out, because the user mode process coexists with the kernel. The kernel includes the file system and device drivers so everything is resident in memory at the same time.

However, this performance benefit comes at a price. Kernel mode drivers are now a part of the kernel, and they have full privileges for the entire kernel address space. A driver failure that results in memory corruption could easily bring down the kernel and with it the entire system. Therefore it’s critical that kernel mode drivers be robust.

This looks very much CE5 behaviour... lets just stop and see the user mode drivers before going forward.

User Mode Drivers

CE6 provides a new mechanism to load drivers into user mode processes (called udevice.exe) instead of into the kernel.

A driver that loads in this fashion is called a user mode driver, which gives up a level of performance in return for increased system robustness and security.

In addition, user mode drivers are restricted in their use of certain APIs such as VirtualCopy, which restricts them from arbitrarily accessing any hardware resource in the system.

Microsoft has designed the user mode driver model to have a very high level of compatibility with kernel mode drivers so musch so that a driver written for user mode can be loaded into kernel mode with no changes.

A new kernel component called the user mode driver reflector handles the interface between user mode applications and the user mode driver.It’s a simple matter to load a driver into user mode, through new bit defined in the Flags registry key which when set tells kernel to load the corresponding driver into user mode. That’s it!

The user mode driver will run in its own process isolated from the kernel and the rest of the system. If the driver fails for some reason only its copy of udevice.exe is affected, the rest of the system should remain intact.

Coming back, Yes, a flaw in a kernel driver can bring down the entire kernel!While this failure mode sounds bad (and is) the end result isn’t really any different from the old model. The analog in CE5 is to bring down the device.exe process and with it all the remaining device drivers.

This failure is likely to have the same overall effect on most platforms!!

However BSP developers now have the option of choosing between a high performance kernel mode driver and a more protected user mode driver. Untrusted third party drivers can be loaded into user mode to increase system security and robustness. Unstable or otherwise questionable drivers can start life in user mode and then be moved to kernel mode as they become proven...

Remote Application Verifier

Remote Application Verifier: This tool is part of the Windows CE test kit, that monitors :

• Heap allocations
• Handle allocations
• GDI Resources
• More… (If you knew how to write a shim)

The output is a log file that is placed on the device, pointing to various leaks (potential or otherwise) Here's a li'l example to settle the initial dust. Consider I have written this harmless code in an application:

#include "stdafx.h"
void ThreadRoutine(void);
int _tmain(int argc, TCHAR *argv[], TCHAR *envp[])
{
// Create the thread
CreateThread(NULL,NULL, (LPTHREAD_START_ROUTINE)ThreadRoutine,NULL, NULL, NULL);
return 0;
}
void ThreadRoutine(void)
{
MessageBox(NULL, TEXT("The thread was created!!"),NULL,MB_OK);
return;
}

See the problem already?.....

Run the App verifier tool on this code (I'll leave the "how to" as an exercise), ad export the log.
The log shows a potential leak at 30h.

We need to generate the code file now, in order to know where our code is leaking.

Right click on the subproject name and choose Properties, click on the General tab and locate the setting next to Generate Code File. Select Yes from the drop down box. Click OK and Rebuild the subproject.

When PB is finished rebuilding the project right click on the project’s name again and select Explore. Navigate to the \obj\ARMV4I\debug and locate the LeakingMemory.cod file.
Click and drag that file from the explorer window to VS document editor. Navigate to the line that starts with 00030. It looks like the problem occurs in the CreateThread method.
See the code now: Looks like the handle created by the CreateThread function in the heap is never released, essentially resulting in a Heap memory leak!!

The good approach to code would have been:

#include "stdafx.h"
void ThreadRoutine(void);
int _tmain(int argc, TCHAR *argv[], TCHAR *envp[])
{
DWORD Base_Address;
DWORD Page_Size=1024;
HANDLE HDl;
HDl = CreateThread(NULL,NULL, (LPTHREAD_START_ROUTINE)ThreadRoutine, NULL, NULL, NULL);
// Close the Handle before exiting
CloseHandle(HDl);
return 0;
}
void ThreadRoutine(void)
{
MessageBox(NULL, TEXT("The thread was created!!"),NULL,MB_OK);
return;
}

The above example shows the capabilities of App Verifier with a simple application. Such callous mistakes are abundant in a large, multithreaded environment, leading to a unexpected system failure. The App Verifier may be used in 3 ways:

• From the PC : Easier to control from PC, Automatically brings data to PC, however needs Activesync or ethernet connection
• On the device : No need for a connection with a PC, however Requires a device U/I, Dialog needs a 800x600 screen
• Using Shell: No device U/I required, Can run from Device,PC using Platform Builder, PC using a serial port, however this is not as polished as other options.

The App verifier is not foolproff, nor is it exhaustive... in fact, it can become quite daunting. Another downside is somewhat flaky connectivity at times.

All said and done, the App verifier is a powerful tool to use for detecting per process level leaks.

Detecting Memory leaks in CE

Memory leaks dont need an introduction here. Those looking for a definition may bypass the rest of this discussion.

Even a novice programmer would have seen couple of them every now and then, slowing overall system performance, and in some cases causing catastrophic fatalities like process and or system hangup!

I chanced to work on the CE6 lab(MS curriculum) on tools for detecting memory leaks in CE based systems, and wondered if ppl around were aware of these comprehensive tools? .... I simply was'nt, till the time I actually had to write about them.

Why is Memory Management so important in CE?
To start with, the simplest reason is Embedded systems are designed with memory constraints, having just enough memory to do the job........... essentially implying we have li'l memory to fool around with!

Another reason is that Embedded Systems, unlike their desktop siblings, run continuously for longer durations of time... some are designed to be kept powered up for months. Even small memory leaks in applications, in a recurrect fashion, lead to complete memory fallout in some unpredictable time.

Bottom line: Memory Leaks in embedded systems...... are deadly!!

Over the next couple of days, I would write on some of the tools I have used to detect and fix memory leaks in my applications. To start with, the first is on the Remote Application Verifier.

CE 6 Emulator

Hi,

I have been working on CE6 Beta after setup(VS2005 and CE6 Beta plugin) over the last couple of days.

The new ARMV4I device emulator raised some queries on my mind. It was intriguing to me how a ARM based emulator(as the name suggests) worked on an x86 machine.

A li'l spadework on the Microsoft team... and pop came the answer: The new device emulator is basically "just an application" that converts ARM instructions to win32 API's and x86 system calls..... precisely the reason emulator works on virtual PC, used in MSDN virtual labs.

This feature is definitely a boon..... considering the learning fom first CE6 virtual lab..... and this is not the end, there are more labs coming up from the author in due time.

BTW, for those struggling to get CE6 OS images run on device emulator(DMA) on their own machines, there is a small change you need to make.

In the Paltform directory of CE6, browse to DEVICEEMULATOR folder, open deviceemulator.bat for edit, and change:

set BSP_KITL_ETHERNET=1
set BSP_KITL_DMA=

to

set BSP_KITL_ETHERNET=
set BSP_KITL_DMA=1

Now build the code using Build Solution option in Build menu. This should configure the KITL to transport over DMA, handy for debug images.

The CE6 virtual lab already had this setting in place!

CE 6 Virtual Lab

Hi,

For all waiting to get their hands on CE6 asap, but cud'nt get the VS2005 or CE6 Beta DVD yet, here is a great virtual lab in MSDN from Mike Hall at Microsoft:

http://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?EventID=1032304834&EventCategory=3&culture=en-US&CountryCode=US

It does'nt require you to spend hours installing VS2005 or CE6.... it has a virtual PC with both installed!

I tried it some time back... I must commend on the carefully designed contents. It covers configuring, building, and debugging an operating system image as well as new features of Platform Builder, including the new registry editor. It also uses the new ARMV4I based Device Emulator.

Just log in and make the most it.

Welcome

Hi All,

Thanks for visiting my "under construction" page.

Over the next couple of days, I'll be posting more on myself.... personal and professional.

.... watch this space for more!

-Tarun Nigam