Changes between Version 8 and Version 9 of AutomatedTesting

Timestamp:

Aug 25, 2010 4:07:56 PM (14 years ago)

Author:

bennylp

Comment:

--

AutomatedTesting

@@ -61 +61 @@
     }}}
     This will perform the automated tests and submit the result under "Experimental" group on CDash. Depending on which test is being run, it may take between two minutes to two hours to complete. You may press Ctrl-C to stop the execution.
- 4. Once you're satisfied that the scenario(s) will work correctly, run '''{{{run_continuous.py}}}''' to perform both "Continuous" and "Nightly" builds on the specified scenario(s):
-    {{{
-     $ ./run_continuous.py scenario1.xml [scenario2.xml ...]
-    }}}
-    The script will run the automated tests in the scenario(s) everytime changes occur in the repository, or at every 00:00:00 GMT for the "Nightly" build.
- 5. Go to [http://my.cdash.org/index.php?project=PJSIP PJSIP's CDash] test dashboard to check the test results. Note that test results are only submitted once all tests in a scenario are completed.
- 6. Now the automated test will run everytime there is a new checkin in the repository, or at every 00:00:00 GMT for the "Nightly" build.
-
-
-[[BR]]
-
-== The Templates ==
+
+== Performing Automated Continuous and Nightly Builds and Tests ==
+
+Once you're satisfied that the scenario(s) will work correctly, the next step is to run a script to automatically do the Continuous and Nightly builds. The '''{{{run_continuous.py}}}''' script continuously monitors the repository for changes, and perform the builds and tests when it does. It also performs Nightly builds at every 00:00:00 GMT.
+
+The '''{{{run_continuous.py}}}''' script takes the scenario file(s) as argument:
+
+{{{
+  $ ./run_continuous.py scenario1.xml [scenario2.xml ...]
+}}}
+
+However there is a problem with the above invocation, namely if the scenario template changes, it won't be picked up by {{{run_continuous.py}}}, since we need to re-run {{{configure.py}}} for that. This will be discussed below.
+
+
+== Automatically Regenerating The Scenario ==
+
+Use {{{prepare.xml.template}}} template scenario to generate a scenario that can be run to run {{{configure.py}}} for other scenarios. To use this:
+ 1. Instantiate {{{prepare.xml}}} from {{{prepare.xml.template}}}:
+ {{{
+$ ./configure.py -t gnu -o prepare.xml prepare.xml.template
+ }}}
+ 1. Edit the {{{prepare.xml}}} file, and remove scenarios (each scenario is represented as a <Test> item) which you don't want to run.
+ 1. For each scenario that you '''do''' want to run, make sure the requirements are met. Typically the scenario want some environment variables to be set. You'd need to make sure that they are set before running the scenario.
+ 1. Run the {{{prepare.xml}}} scenario to regenerate other scenarios:
+ {{{
+$ ./run_scenario.py prepare.xml
+ }}}
+ 1. Now you can run {{{prepare.xml}}} scenario before other scenarios in {{{run_continuous.py}}}:
+{{{
+  $ ./run_continuous.py prepare.xml scenario1.xml [scenario2.xml ...]
+}}}
+
+
+
+[[BR]]
+
+== Appendix: The Templates ==
 
 The detailed steps to configure each template will be explained below.