Pocket PC TestSuite
By Vassili Philippov, May 28, 2002.
Print version
Summary
This article describes automation of the testing process on Pocket PC using
Pocket PC TestSuite 2.0. You will find how to record test scripts, how
to check results in the scripts and how to test programs in stress conditions.
Products Mentioned in the Article
Introduction
Quality assurance is one of the most time consuming phase of the product cycles.
There are several types of testing that cannot exist without automatic
testing: regression testing and stress testing.
Regression testing is the selective retesting of a software system that has
been modified to ensure that any bugs have been fixed and that no other
previously-working functions have failed as a result of the reparations
and that newly added features have not created problems with previous
versions of the software. Regression test sessions are executed daily,
therefore, testing should be automated.
Another important aspect of quality assurance is the stress testing. There
are several testing scenarios that can be used only with automatic testing.
For example, you can perform a big number of similar actions in the script.
Or you can automatically track memory leaks between certain testing cycles.
The Pocket PC TestSuite is the only tool for automatic testing on Pocket PC. The second
version of it introduces 2 key features: recording taps and possibility to
use the powerful scripting language, Tcl. First feature allows quick
creating of the script base. Tcl scripts allow cycles, conditions and
other custom scenarios.
It will take less than an hour to create the powerful script for your
application that you can use from version to version on the daily basic
without additional efforts.
You can download demo movie about using Pocket PC TestSuite 2.0 testsuite.zip - 250 Kb.
Getting started
Using Pocket PC TestSuite is simple. Let's say we want to test copy/paste in the Calculator when it is repeated 100 times. We do the following:
- Start Pocket PC TestSuite
- Create new test script
- Choose "Record Taps" menu item
- Start Calculator application, do what we want to test
- Return to the TestSuite
- Press OK to insert recorded commands to the script
- Add loop command around the recorded commands to repeat this action 100 times:
<![CDATA[
for {set i 0} {$i<100} {incr i 1} {
#recorded commands here
}
]]>
Your script shoule look like:
<![CDATA[
#TCL Script
source "My Documents\\testsuite.tcl"
for {set i 0} {$i<100} {incr i 1} {
ts_run "\\Windows\\calc.exe"
ts_wait 1913
ts_click 123 132
ts_wait 1462
ts_click 21 308
ts_wait 1121
ts_click 45 271
ts_wait 1632
ts_click 30 270
ts_wait 1035
ts_click 18 306
ts_wait 977
ts_click 48 286
}
]]>
Now we run this script and see how Pocket PC TestSuite simulates copy/paste in the Calculator
100 times. If we need to test with 10 000 times we can change 100 to 10000 in the test suite script, start it
do another job or go to drink coffee. Just imaging how you can perform such test without
TestSuite.
Features
Here are features of Pocket PC TestSuite 2.0 with brief description:
-
Test scripts - you can create test scripts that simulate user activity.
Test script language is based on Tcl plus 22 additional testing specific functions.
-
Simulating user activity - there are functions for simulating taps (clicks),
mouse down, mouse up, keys and delays. You can also simulate tap and hold action.
-
Recording - you can start record mode than do something and all your taps
are saved in a script file.
-
Checking functions - you can not only simulate user activity
but also check results. You can check: color of any point, text in any windows control,
compare part of screen with a bitmap, get free memory available.
-
Reporting - you can create and write to any file using Tcl functions.
We also provide functions for working with logs that are shown in
the TestSuite program after the script is finished.
-
Edit scripts on Pocket PC - you can edit scripts directly on your Pocket PC.
-
Memory stress - you can test your program in limited free memory conditions.
-
CPU stress - you can test your program stress CPU conditions (different level of CPU stressing).
My personal experience of using TestSuite and experience of our testers say that the most
important features are test scripts, recording and memory stress.
Tcl
As you have already seen test scripts can contain loops. There are also variables, if conditions, procedures,
expressions, file access functions and many other. Pocket PC TestSuite 2.0 uses Tcl as a base
language for test scripts. Tcl is a simple but powerful programming language. I have seen
programs like Visual Studio written in Tcl. But in 95% of cases you do not need power of
this language. Test scripts are rather simple and usually you do not need anything except
variables, loops and sometimes functions. You just know that if you need you have this power
to create really advanced test scripts.
Here just some commented sample that give you enough knowledge about Tcl to write test
scripts.
Variables
<![CDATA[#this is a comment line because it starts with #
#create 2 variables
set x 40
set y 50
#click to point (x, y)
ts_click $x $y
]]>
Loops
<![CDATA[set x 40
set y 50
#a loop, here we click 10 times increasing y coordinate by 1
for {set i 0} {$i<10} {incr i 1} {
ts_click $x [expr $y+$i]
}
]]>
Functions
<![CDATA[#this function clicks 10 times to the given point
proc click10times {x y} {
for {set i 0 } {$i<10} {incr i 1} {
ts_click $x $y
ts_wait 500 #wait 0.5 second
}
}
click10times 78 45
click10times 210 200
click10times 34 100
click10times 80 80
]]>
You can read more about Tcl in a Tcl tutorial
or in FAQs: Tcl FAQ 1,
Tcl FAQ 2.
Checking
It is important when you test a program to check results. Pocket PC TestSuite 2.0 includes
a set of functions for that. Here are main use cases for checking:
-
You get free memory (using ts_getfreemem function) than simulate some actions
that should bring the program to the
initial state and check that free memory os not changed.
-
You have a script and know what should be the final state of this script. You create
a bitmap picture (using Remote Zooming or ts_savescreenrect function) and
check in the end of your script that the screen is equal to this bitmap file
(using ts_checkscreenrect function).
-
You can check results of your script by checking color of some points (using ts_getpixel
function).
-
You can check text in windows controls (using ts_gettext function).
Test functions
There 22 special functions (except standard Tcl functions) that you can use in your test scripts.
You can create your own functions that will perform batch operations if needed.
Here is a description of basic test functions:
Simulating
Simulating functions simulates user activity like mouse pressing, key pressing, etc.
|
|
ts_click x y
|
Simulates tap (click) to the given point
|
|
ts_mousedown x y
|
Simulates mouse down event in the given point
|
|
ts_mouseup x y
|
Simulates mouse up event in the given point
|
|
ts_mousemove x y
|
Simulates mouse move event to the given point
|
|
ts_wait mills
|
Delays script for mills milliseconds
|
|
ts_key keycode
|
Simulates pressing of the key with the given code
|
|
ts_keydown keycode
|
Simulates key down even of the key with the given code
|
|
ts_keyup keycode
|
Simulates key up even of the key with the given code
|
|
ts_text yourstring
|
Simulates entering of the given string
|
|
ts_run program_path arguments
|
Starts program with the given path and the given arguments (arguments are optional)
|
|
ts_cpapplet applet_id page_id
|
Opens Control Panel applet with the given id (page id is optional)
|
Checking
These functions are used to check results. For example you know that in the end of
your script the program must be in a certain state. You can check this state using
the following functions:
|
|
ts_getpixel x y
|
Returns RGB color of the given point (for example "230 4 34")
|
|
ts_gettext x y
|
Returns text of the control under the given point. You can use this function
to check text in edit control, comboboxes, etc.
|
|
ts_getfreemem
|
Returns available memory in bytes
|
|
ts_checkscreenrect x1 y1 x2 y2 bmp_file_name
|
Checks if the screen rectangle of [(x1,y1) - (x2,y2)] is equal to the given bitmap file
|
Reporting
These functions are used to check results. For example you know that in the end of
your script the program must be in a certain state. You can check this state using
the following functions:
|
|
ts_log message_string
|
Writes the given string to the log file. When the test script is finished this log file
is shown to the user.
|
|
ts_messagebox message_string
|
Shows message box with the given string. Use this function for debug purposes only because
it can affect the tested process.
|
Misc
Other functions:
|
|
ts_savescreenrect x1 y1 x2 y2 bmp_file_name
|
Saves screen rectangle to the given bitmap file. You can use this function to capture
screen rectangle and than check that it is not changed after some actions.
|
|
ts_getos
|
Returns device platform. Can be "pocketpc" or "pocketpc2002".
|
|
ts_getcputype
|
Returns device CPU type. Can be "arm" or "mips".
|
|
ts_reg_read root_key key value
|
Returns string from registry. root_key myst be one of: HKEY_LOCAL_MACHINE,
HKEY_CURRENT_USER, HKEY_CLASSES_ROOT.
|
Entrek TOOLBOX
Pocket PC TestSuite 2.0 is optimized for using with Entrek TOOLBOX. Entrek CODESNITCH
detects leaks of allocated memory and other objects. Using Entrek CODESNITCH together with
Pocket PC TestSuite 2.0 allows you creating scripts and find all leaks during execution
of these scripts. You can find more about Entrek products at www.entrek.com.
Conclusion
Testing is a very important stage of products development. Pocket PC TestSuite 2.0 is the
only tool that automates testing process on Pocket PC and supports recording, checking,
and testing in stress conditions. With script recording testing becomes as simple as
clicking in the tested program but with power of Tcl you can create very advanced scripts
that will using loops and procedures, check results and write them to the log.
Although Pocket PC TestSuite 2.0 is not free it will be suitable for all companies that do
Pocket PC software development and care about quality.
Related resources:
Discuss
Discuss this article.
Here you can write your comments and read comments of other developers.
|
