Testing programs on Pocket PC

Shamil Kerimov (shamil@softspb.com), January 17, 2002.

Introduction

If a program is not tested it does not work. Good developers understand that testing is needed. Experienced developers demand testing. What is specific about testing Pocket PC programs? This article describes the experience of Pocket PC Testing Center.

Background

As soon as the first computer was born, the programs written for it had many bugs. But time passed, and now we have a great experience of testing programs for many different hardware platforms. Pocket PC is a very young platform, and there is no much experience of testing software written for Pocket PC.

Pocket PC Specific features

Testing software applications for Pocket PC is slightly different from testing those written for desktops. The basic difference is that different Pocket PCs are really different - there are 3 different architectures. To ensure that your program works right on all Pocket PCs you must compile your code into 3 different executables and test each one on a corresponding device. As we have already noticed, the most problematic device is the HP Jornada 540. It is the slowest Pocket PC device ever, and the bugs we catch on it can even disappear when testing the same application on other platforms.

Here is a list of things that are specific to Pocket PC and affect the testing process,

• SIP
  SIP is a Soft Input Panel used to enter text information to Pocket PC. Due to its ability of hiding/showing itself depending on the type of input field, it may produce bugs if you switch it manually from your application.
• Full Screen
  Using Pocket PC fullscreen mode is much different from the way it is done on desktop PC and may have many reefs. Fullscreen-related bugs usually appear when another window is being activated. It happens because the task bar is not being hidden but placed just under your program. To test your application, try at least switching to another application and then back, and see what happens with your application when some popup window appears. Also, check if showing/hiding SIP does not bring to bugs in your full-screen program.
• Memory management
  Pocket PC memory management is based on the following rule: "Application should not be closed before system memory runs low". If the memory is low, system forces the application to close, and some applications do not close correctly. Also, memory division between storage and program areas can lead to unexpected program behaviour.
• CF card
  Applications stored on a memory card can be run as if they were in main memory. But removing CF card from the slot may cause a crash if the program is exchanging data with CF card without tracking its presence.
• Today screen
  Today screen is something like active desktop in Windows 98. But there are lots of differences.
• Built-in PIM database
  Pocket PC has a built-in PIM database that is used for storing contacts, emails, tasks and appointments. It is a nice thing because many applications can share the same data.
• Help
  All Pocket PC applications should provide help through Help item in the start menu. There is a standard for help files and their usage in your applications.
• Synchronization
  Many Pocket PC applications communicate with the desktop computer. This feature has no analogue in desktop Windows.
• More difficult to automate
  There are lots of programs for Windows that help automate testing process. For Pocket PC you can use Pocket PC TestSuite. But there are many procedures like power off/on, synchronization, using IrDA, inserting/removing CF that are cannot be automated.

The most wide-spread bugs

• Today screen
  If you have created a program using dialog based MFC application template, it probably contains a bug. If you start Today program when your application is running your application will be shown instead. You can read how to fix it here.
• Full screen bugs
  These bugs usually appear when you switch to another application and back or when a popup window or SIP is shown. You can read about creating correct full screen applications here.
• Help
  Just check that your application does not contain help button and shows help when your choose Help item in the Start menu.
• Memory leaks
  It is difficult to check that your application does not contains memory leaks. Nevertheless it is normal when Pocket PC applications work for weeks or even months (because Pocket PC applications should not contain exit functionality). That is why it is important to check that there are no memory leaks.
• Different language versions
  There are more differences between English iPaq PC and German one than between English iPAQ and English Cassiopeia. I realy suggest testing applications on at least one non-English Pocket PC. Most bugs appear because folders have different names in different languages. You should not use folder names in your program but get them using special API functions ( SHGetSpecialFolderPath() and SHGetSpecialFolderLocation() ).

Logo Requirements

Microsoft has the "Ready for Windows" Pocket PC logo program. You can download Pocket PC Technical Handbook document that contains requirements for getting this logo sertification. Even if you are not going to certify your program for being a logo-application I recommend to fulfill the logo requirements because they describe standard style and will help you create a prorgam that looks like standard built-in Pocket PC applications.

Automation

Good testers know that everything that can be automated in testing should be automated. On Pocket PC you can use Pocket PC TestSuite application for writing test suites that simulate clicks, keyboard events, etc. It can also be used for testing your application in low memory and low CPU conditions.

Related resources:

• http://www.pocketpcdn.com/articles/testsuite.html
  Article: Pocket PC TestSuite
• http://www.pocketpcdn.com/test/documents/testplan.doc
  Article: Pocket PC Testing Center Standard Test Paln
• http://www.microsoft.com/mobile/assets/PocketPC_SoftApp_Handbook.doc
  Article: "Ready for Windows" Pocket PC logo requirements