1. State of Testing
Jason Kenny – ATS committer

2. Overview
What is New
Running tests
Making a test
Plans for the future

3. What is new in Testing since last summate
The new tests have been committed into 7.1 and master!
New tests have been added
Fixes have been made to correct some false positive failures
Better port selection code for Linux
Some race conditions with start up of the micro server
Other fixes in ATS to make it more testable
Options to control timeout/kill delays
MakeOriginServer api as been improved!
New ssl=True|[False] to make it easier to start a https server
New option={} to allow passing various options to the microserver

4. Microserver supports new observers
New –load = <python file> options
Will auto import needed API to file
Call Hooks.register(hook_type,callback)
Current hooks are limited but will grow
Example:
def observe(headers):
if 'via' in headers: …
Hooks.register(Hooks.ReadRequestHook, observe)

5. Running Tests is Easy!
It does not bite! … so run them!

6. Layout in Source
Tests/ autest.sh – “training wheel” wrapper script to run the tests bootstrap.py – script to setup a virtualenv with testing engine (called by autest.sh) tools/ - contains micro server, traffic replay gold_tests/ - contains the tests autest-site/ contains extension for ATS testing <some folder> - contains some set of tests for some concept

7. Running AuTest - As easy as 1-2-3 (almost)
Build ats and install to a directory
Recommend to add a –prefix /home/<user>/ats/<version>
Cd tests
./autest.sh --ats-bin=<path/to/bin> --sandbox /tmp/sb
The bootstrap script should work … but it is not perfect
More advance users:
Install python 3.5 or higher
Pip install autest hyper
Use autest directly (no wrapper script)
Needs to run in gold_tests directory or with a –D option to point to this location.

8. Making tests is easy(er) to do!!
So why have you not made one!

9. Structure of a basic test
Config Files
Some Client:
Curl
Traffic-replay
Server:
uServer
ATS

10. Basic Test template (Part 1)
Test.Summary = '''Tell us what this test does''' ts=Test.MakeATSProcess("ts") # Define default ATS server=Test.MakeOriginServer("server") #define a server #add response to the server dictionary server.addResponse("sessionfile.log", request_header, response_header) #update a config files ts.Disk.records_config.update({ 'proxy.config.diags.debug.enabled': 1, 'proxy.config.diags.debug.tags': 'url.*', }) ts.Disk.remap_config.AddLine( 'map )

11. Basic Test template (Part 2)
# create a new test run
tr=Test.AddTestRun()
tr.Processes.Default.Command='curl " --verbose'.format(ts.Variables.port)
tr.Processes.Default.ReturnCode=0
# make sure we start the server and ATS process before this Curl command
tr.Processes.Default.StartBefore(server)
tr.Processes.Default.StartBefore(Test.Processes.ts)
# test the response is what we expect
tr.Processes.Default.Streams.stderr="gold/remap-hitATS-404.gold"
# are the server and ATS still running
tr.StillRunningAfter=server
tr.StillRunningAfter=ts

12. Plans for the future Add a condition feature test
Will allow for testing for if a program ( such as curl or ATS) support a certain feature as a requirement for a test to run
Add extension to define a test with traffic-replay
Should allow auto setup of micro-server for
Should be easier to setup for common send-response tests
Should make it easier to make a test based on issue seen in production
Update/add new reporters
Add a junit format output options
Add API for defining new Tester objects
Implement better ways to control if a test fails or warns
Implement –j option

13. Thank you! Question? AuTest web site:
Autest documentation: 
Test extension documentation for ATS: