1. Neeraj Jain Cavisson System Inc
Automation Test Suite
Neeraj Jain Cavisson System Inc

2. Agenda
Features
Directory Structure
Commands
Reports
Demo
Q & A

3. Features Run several tests (test set)
Tests to run may be selected by user
On Failure of a test
Test Set progress may continue
Abort the test set
Jump to certain test in test harness*
Skip certain tests*
For each test set run
Test summary results are saved
Detail logs for each test are saved

4. Features (Continued) Reporting
Results summary as text, html, XL and pdf *
Results may be ed to subscriber on completion *
Results may be ed to subscriber on each failure *
Compare summary results against past test-set run(s)
Test Run Management
Show available test runs
Manage test run data

5. Directory Structure
Automation is installed in /home/netstorm/work directory which is called home directory.
Bin Directory
All programs are kept in bin directory under home directory of netstorm. All programs under this directory are replaced when upgrade is done.
Test case Directory
All test cases are kept in testcases directory under home directory of netstorm. There is one directory per test case. This directory is NOT replaced when upgrade is done.
All test cases have four files which are in the test case directory:
testcase.conf: This file have two keywords
SCENARIO_NAME- To give scenario name. Scenario name should be without .conf extension
DESCRIPTION – To give description of test case.
pre_test_setup (Optional) - This script for making any pre test setup.
post_test_setup (Optional) - This script for making any post test setup.
check_status (Optional) - This script is for checking the status.

6. Directory Structure (Continued)
Test suite Directory
All testsuites configuration files are kept in testsuites directory under home directory of netstorm. Name of testsuite configuration file will be with conf extension. This configuration file will have following keywords:
TEST_CASE_NAME: This keyword is for given the test case name, which we want to run. This keyword also have second field Continue or Abort. This is for to make decision for continuing or aborting test cases on any failure (By default it is Continue). This keyword can multiple times in configuration file.
_ADDRESS: This keyword is for giving the address.
REPORT_FORMAT: This keyword is for giving format of report. Presently we are supporting only text format.
This directory is NOT replaced when upgrade is done.
Logs Directory
All logs are kept in tsr directory under logs directory in home directory of netstorm. In this directory there will be directory named on test suite run number. In this directory there will be summary.report and progress.report file. tsr directory is NOT replaced when upgrade is done.

7. How To Run Test Suite? login as netstorm. Go into work directory
Run the ts_run command as following
bin/ts_run –n testsuites/<test suite name>
For Example:
bin/ts_run –n testsuites/Sample_test_suite1
On running above command it will show the test suite run number (TSR).
After completion Test case there will be directory as TSR under tsr directory in logs.

8. Commands
ts_run
This command is used to run test suite. Command runs in following manner:
bin/ts_run –n testsuites/<test suite name> 
ts_show_test_suite
This command is used to get the information about the test suite. Command runs in following manner
bin/ts_show_test_suite –n <test suite name> 
Sample output:
Test Suite Name: Sample_test_suite
Test Case Name Description
Sample_test_case Test Sample
Sample_test_case Test Sample2

9. Commands (Continued) ts_show_logs
This command is used to get the logs of a specfic or all tsr. Command runs in following manner
bin/ts_show_logs –n <tsr | ALL>
Sample output:
Test Suite Name: Sample_test_suite
Test Suite Run Number: 1086
Start Date/Time: 09/14/07 13:59:57
Test Case Name Test-Run# Start Date/Time End Date/Time Status
Sample_test_case /14/07 13:59: /14/07 14:00:19 Pass
Sample_test_case /14/07 14:00: /14/07 14:00:41 Pass

10. Commands (Continued) ts_show_test_cases
This command for showing information about the test cases which are in testcase directory. Command runs in following manner
bin/ts_show_test_case
Sample output:
Test Case Name Pre Test Script Post Test Script Check Status Scenario
Sample_test_case1 Yes Yes Yes test_casesuits
Sample_test_case2 Yes Yes Yes test_casesuits

11. Commands (Continued) ts_compare_tsr
This command is for comparing results of two or more test suite runs. Command runs in following manner
bin/ ts_compare_tsr <tsr1> <tsr2> …….
Sample output:
./ts_compare_tsr
Test Suite Compare Report
Test Case Name
tx_am_tiny_1req Pass Pass Pass

12. Reports
There are two reports summary.report and progress.report in tsr log file.
Sample output of summary.report:
Test Suite Name: Sample_test_suite
Test Suite Run Number: 1086
Start Date/Time: 09/14/07 13:59:57
Test Case Name Test-Run# Start Date/Time End Date/Time Status
Sample_test_case /14/07 13:59: /14/07 14:00:19 Pass
Sample_test_case /14/07 14:00: /14/07 14:00:41 Pass

13. Reports (Continued) Sample output of progress.report:
Starting test suite Sample_test_suite, TSR Number = 1086
Starting execution of test case - Sample_test_case1
Starting pre_test_setup script
spawn su -c /home/netstorm/work/bin/nsu_assign_ip ' /24| :7|-|-|eth0'^M
Password: ^M
IP assignment completed!^M
spawn su -c /home/netstorm/work/bin/nsu_assign_ip -s ' /24| :1|-|-|eth0'^M
spawn echo Calling pre_dut_setup^M
Starting scenario
Checking status
TRIED = 1 SUCC = 1 FAIL = 0
Test 1 Passed...
Starting post_test_setup script

14. Demo
Live Demo

15. Q & A
????