Automated Testing

Table of Contents

1. Introduction
2. Installation
3. Configuring and Running Test Scenarios
   1. gnu.xml.template
   2. gnu-ipp.xml.template
   3. iphone.xml.template
   4. msvc.xml.template
   5. symbian.xml.template
   6. symbian-aps.xml.template
   7. symbian-vas.xml.template

PJSIP has been equipped with an automated testing script (called test scenarios) since version 1.6. This article explains what it is and how to configure it.

Introduction

The automated testing script/scenario updates, configures, builds, and performs various unit tests, and submit the result to ​CDash test dashboard. The scenario uses ​CCDash framework to automate the tasks, and supports Windows/Visual Studio build system, GNU Makefile based build system (for Linux, Mac OS X, and possibly iPhone), as well as Symbian targets.

Supported operations:

• SVN update
• configure
• build
• running unit tests
• upload the result to CDash

Supported targets:

• GNU/Makefile based (Linux, Mac OS X, iPhone, etc.)
• Windows or Visual Studio including Windows Mobile
• Symbian S60

Installation

1. Create a CDash project, either by registering your project ​here or by installing CDash locally.
2. Checkout PJSIP source code from SVN
3. Download and install ​CCDash (click the link for the info)
4. Download and install any features to be used/tested, e.g. Intel IPP, Symbian APS/VAS plugin, etc.

Configuring and Running Test Scenarios

The general steps to perform a test:

1. Choose the appropriate scenario template for your target/test objective. The templates are listed below:

Target platform	Template file name	Description	Requirements
Makefile based (Linux, Mac OS X, iPhone)	gnu.xml.template	Default settings	-
Linux, Mac OS X	gnu-ipp.xml.template	Intel IPP	Intel IPP and samples
MSVC/Win32	msvc.xml.template	Default settings	-
Symbian/S60 3rd Ed	symbian.xml.template	Default settings	-
Symbian/S60 3rd Ed	symbian-aps.xml.template	Nokia APS and APS-Direct	Nokia APS plugin
Symbian/S60 3rd Ed	symbian-vas.xml.template	Nokia VAS and VAS-Direct	Nokia VAS plugin

2. Create a scenario from a template by running:

    $ cd tests/automated
    $ ./configure.py file.xml.template -o file.xml
   

3. Run the scenario by running:

    $ ./run_scenario.py file.xml
   

The detailed steps for each template will be explained below.

gnu.xml.template

This configures, build, and tests with default settings. To perform this test:

 $ ./configure.py -t gnu -o gnu.xml gnu.xml.template
 $ ./run_scenario.py gnu.xml

gnu-ipp.xml.template

Template for testing IPP.

Requirements:

• Intel IPP and samples
• Environment variables:
  • IPPROOT
  • IPPSAMPLES
  • IPPARCH (optional)
• IPP path added to LD_LIBRARY_PATH

Sample session:

 $ export IPPROOT=/opt/intel/ipp/6.1.2.051/em64t/
 $ export IPPSAMPLES=/home/bennylp/Desktop/opt/ipp-samples
 $ (optional) export IPPARCH=em64t
 $ export LD_LIBRARY_PATH='$LD_LIBRARY_PATH:/opt/intel/ipp/6.1.2.051/em64t/sharedlib'
 $ ./configure.py -t gnu -o gnu-ipp.xml gnu-ipp.xml.template
 $ ./run_scenario.py gnu-ipp.xml

iphone.xml.template

Template for testing [Getting-Started/iPhone iPhone].

Requirement(s):

• on Mac OS X
• at least one iPhone SDK must be installed

Sample session:

 $ ./configure.py -o iphone.xml iphone.xml.template
 $ ./run_scenario.py iphone.xml

Note:

• currently there's no way to select which iPhone SDK to use, as the configure.py automatically picks up the latest version of SDK installed.

msvc.xml.template

Visual Studio targets.

TBD.

symbian.xml.template

Symbian S60 targets.

TBD.

symbian-aps.xml.template

Symbian S60 with APS target.

TBD.

symbian-vas.xml.template

Symbian S60 with VAS target.

TBD.