Pocket PC Developer Network / Articles /

Pocket PC TestSuite

Vassili Philippov (vasja@spbteam.com), May 28, 2002.

Summary

This article describes automation of the testing process on Pocket PC using Pocket PC TestSuite 2.0. You will find how to record test scripts, how to check results in the scripts and how to test programs in stress conditions.

Products Mentioned in the Article

Introduction

Quality assurance is one of the most time consuming phase of the product cycles. There are several types of testing that cannot exist without automatic testing: regression testing and stress testing.

Regression testing is the selective retesting of a software system that has been modified to ensure that any bugs have been fixed and that no other previously-working functions have failed as a result of the reparations and that newly added features have not created problems with previous versions of the software. Regression test sessions are executed daily, therefore, testing should be automated.

Another important aspect of quality assurance is the stress testing. There are several testing scenarios that can be used only with automatic testing. For example, you can perform a big number of similar actions in the script. Or you can automatically track memory leaks between certain testing cycles.

The Pocket PC TestSuite is the only tool for automatic testing on Pocket PC. The second version of it introduces 2 key features: recording taps and possibility to use the powerful scripting language, Tcl. First feature allows quick creating of the script base. Tcl scripts allow cycles, conditions and other custom scenarios.

It will take less than an hour to create the powerful script for your application that you can use from version to version on the daily basic without additional efforts.

You can download demo movie about using Pocket PC TestSuite 2.0 testsuite.zip - 250 Kb.

Getting started

Sample script Using Pocket PC TestSuite is simple. Let's say we want to test copy/paste in the Calculator when it is repeated 100 times. We do the following:

  1. Start Pocket PC TestSuite
  2. Create new test script
  3. Choose "Record Taps" menu item
  4. Start Calculator application, do what we want to test
  5. Return to the TestSuite
  6. Press OK to insert recorded commands to the script
  7. Add loop command around the recorded commands to repeat this action 100 times: for {set i 0} {$i<100} {incr i 1} { #recorded commands here }

Your script shoule look like:

#TCL Script source "My Documents\\testsuite.tcl" for {set i 0} {$i<100} {incr i 1} { ts_run "\\Windows\\calc.exe" ts_wait 1913 ts_click 123 132 ts_wait 1462 ts_click 21 308 ts_wait 1121 ts_click 45 271 ts_wait 1632 ts_click 30 270 ts_wait 1035 ts_click 18 306 ts_wait 977 ts_click 48 286 }

Now we run this script and see how Pocket PC TestSuite simulates copy/paste in the Calculator 100 times. If we need to test with 10 000 times we can change 100 to 10000 in the test suite script, start it do another job or go to drink coffee. Just imaging how you can perform such test without TestSuite.

Features

Here are features of Pocket PC TestSuite 2.0 with brief description:

My personal experience of using TestSuite and experience of our testers say that the most important features are test scripts, recording and memory stress.

Tcl

As you have already seen test scripts can contain loops. There are also variables, if conditions, procedures, expressions, file access functions and many other. Pocket PC TestSuite 2.0 uses Tcl as a base language for test scripts. Tcl is a simple but powerful programming language. I have seen programs like Visual Studio written in Tcl. But in 95% of cases you do not need power of this language. Test scripts are rather simple and usually you do not need anything except variables, loops and sometimes functions. You just know that if you need you have this power to create really advanced test scripts.

Here just some commented sample that give you enough knowledge about Tcl to write test scripts.

Variables

#this is a comment line because it starts with # #create 2 variables set x 40 set y 50 #click to point (x, y) ts_click $x $y

Loops

set x 40 set y 50 #a loop, here we click 10 times increasing y coordinate by 1 for {set i 0} {$i<10} {incr i 1} { ts_click $x [expr $y+$i] }

Functions

#this function clicks 10 times to the given point proc click10times {x y} { for {set i 0 } {$i<10} {incr i 1} { ts_click $x $y ts_wait 500 #wait 0.5 second } } click10times 78 45 click10times 210 200 click10times 34 100 click10times 80 80

You can read more about Tcl in a Tcl tutorial or in FAQs: Tcl FAQ 1, Tcl FAQ 2.

Checking

It is important when you test a program to check results. Pocket PC TestSuite 2.0 includes a set of functions for that. Here are main use cases for checking:

  1. You get free memory (using ts_getfreemem function) than simulate some actions that should bring the program to the initial state and check that free memory os not changed.
  2. You have a script and know what should be the final state of this script. You create a bitmap picture (using Remote Zooming or ts_savescreenrect function) and check in the end of your script that the screen is equal to this bitmap file (using ts_checkscreenrect function).
  3. You can check results of your script by checking color of some points (using ts_getpixel function).
  4. You can check text in windows controls (using ts_gettext function).

Test functions

There 22 special functions (except standard Tcl functions) that you can use in your test scripts. You can create your own functions that will perform batch operations if needed. Here is a description of basic test functions:

Simulating

Simulating functions simulates user activity like mouse pressing, key pressing, etc.
ts_click x y
Simulates tap (click) to the given point
ts_mousedown x y
Simulates mouse down event in the given point
ts_mouseup x y
Simulates mouse up event in the given point
ts_mousemove x y
Simulates mouse move event to the given point
ts_wait mills
Delays script for mills milliseconds
ts_key keycode
Simulates pressing of the key with the given code
ts_keydown keycode
Simulates key down even of the key with the given code
ts_keyup keycode
Simulates key up even of the key with the given code
ts_text yourstring
Simulates entering of the given string
ts_run program_path arguments
Starts program with the given path and the given arguments (arguments are optional)
ts_cpapplet applet_id page_id
Opens Control Panel applet with the given id (page id is optional)

Checking

These functions are used to check results. For example you know that in the end of your script the program must be in a certain state. You can check this state using the following functions:
ts_getpixel x y
Returns RGB color of the given point (for example "230 4 34")
ts_gettext x y
Returns text of the control under the given point. You can use this function to check text in edit control, comboboxes, etc.
ts_getfreemem
 Returns available memory in bytes
ts_checkscreenrect x1 y1 x2 y2 bmp_file_name
Checks if the screen rectangle of [(x1,y1) - (x2,y2)] is equal to the given bitmap file

Reporting

These functions are used to check results. For example you know that in the end of your script the program must be in a certain state. You can check this state using the following functions:

ts_log message_string
Writes the given string to the log file. When the test script is finished this log file is shown to the user.
ts_messagebox message_string
Shows message box with the given string. Use this function for debug purposes only because it can affect the tested process.

Misc

Other functions:

ts_savescreenrect x1 y1 x2 y2 bmp_file_name
Saves screen rectangle to the given bitmap file. You can use this function to capture screen rectangle and than check that it is not changed after some actions.
ts_getos
 Returns device platform. Can be "pocketpc" or "pocketpc2002".
ts_getcputype
 Returns device CPU type. Can be "arm" or "mips".
ts_reg_read root_key key value
Returns string from registry. root_key myst be one of: HKEY_LOCAL_MACHINE, HKEY_CURRENT_USER, HKEY_CLASSES_ROOT.

Entrek TOOLBOX

Pocket PC TestSuite 2.0 is optimized for using with Entrek TOOLBOX. Entrek CODESNITCH detects leaks of allocated memory and other objects. Using Entrek CODESNITCH together with Pocket PC TestSuite 2.0 allows you creating scripts and find all leaks during execution of these scripts. You can find more about Entrek products at www.entrek.com.

Conclusion

Testing is a very important stage of products development. Pocket PC TestSuite 2.0 is the only tool that automates testing process on Pocket PC and supports recording, checking, and testing in stress conditions. With script recording testing becomes as simple as clicking in the tested program but with power of Tcl you can create very advanced scripts that will using loops and procedures, check results and write them to the log.

Although Pocket PC TestSuite 2.0 is not free it will be suitable for all companies that do Pocket PC software development and care about quality.

Related resources: