Functional Testing Montpellier October 2006 perfSONAR Meeting Michalis Michael (Cynet)

Functional Testing Service Oriented TestingService Oriented Testing Black Box TestingBlack Box Testing Behavioural TestingBehavioural Testing Automated TestingAutomated Testing

Functional Testing is necessary to: Make sure that the service actually works Make sure that the service actually works This is accomplished by a wide range of tests This is accomplished by a wide range of tests If the tests are successful the service is functioning inside the specified parameters If the tests are successful the service is functioning inside the specified parameters

Functional Testing process A wide range of requests is fed to the serviceA wide range of requests is fed to the service Requests that are well formatted containing valid information Requests that are well formatted containing valid information Requests that are malformed Requests that are malformed Requests containing invalid or non sufficient information Requests containing invalid or non sufficient information Responses are collected and checked for validityResponses are collected and checked for validity Responses information are cross-checked with information inside the service databaseResponses information are cross-checked with information inside the service database

ProcessProcess Create Requests Feed requests into Service Log responses Check for validity/Correlate responses against actual DB data Test Passed Test Failed

LS Functional Tests Focus on checking LS response to :Focus on checking LS response to : Well formatted requests containing valid information Well formatted requests containing valid information Requests that are not well formatted with or with out valid information Requests that are not well formatted with or with out valid information Requests that do not contain enough information Requests that do not contain enough information Requests with excess information also maybe fedRequests with excess information also maybe fed Suggestions:Suggestions: Check LSCleanup (will do if time is enough) Check LSCleanup (will do if time is enough) Check the RRD MA against the LS Check the RRD MA against the LS

A LSRegisterRequest example The request is missing the Data element. The LS should return an error response

LS Functional Test – Parts needed A request building script-class.A request building script-class. A script-class that feeds the requests to the LS and stores the responses from the LS.A script-class that feeds the requests to the LS and stores the responses from the LS. (Requests are send via the SOAPClient class that is already available in the release) A script-class that processes the responses (parser) and correlates the results with the actual data stored in the database (eXist). Here is decided whether the test has succeeded or failed.A script-class that processes the responses (parser) and correlates the results with the actual data stored in the database (eXist). Here is decided whether the test has succeeded or failed.

What has been done so far A document was send to the perf-dev mailing list, listing the tests that are going to be done, in order to get feedback from the developersA document was send to the perf-dev mailing list, listing the tests that are going to be done, in order to get feedback from the developers Started working on the request generating class. Class for the LSRegisterRequest has been developedStarted working on the request generating class. Class for the LSRegisterRequest has been developed

TO DO Finish the developing of the classes for generating requests for the other types of requestsFinish the developing of the classes for generating requests for the other types of requests Make sure that the communication with the LS via the SOAPClient class is workingMake sure that the communication with the LS via the SOAPClient class is working Last ( but trickier ) the development of the script-class for parsing and checking the responses from the LSLast ( but trickier ) the development of the script-class for parsing and checking the responses from the LS

Help Needed More like RFC documentation ( An accurate and detailed documentation is needed for the tester to understand the way a service works and responds).More like RFC documentation ( An accurate and detailed documentation is needed for the tester to understand the way a service works and responds). Maybe some information on how independent components of the services work, like the databases i.e.Maybe some information on how independent components of the services work, like the databases i.e. Suggestions for enriching and improving the scheduled tests.Suggestions for enriching and improving the scheduled tests.

An example – RRD MA Metadata Key RequestMetadata Key Request <nmwg:message type="MetadataKeyRequest" id="mdrq1" … PIONIER-Public unix ge-2/1/0.102 out utilization Test categories Message Format Message Values Combination Both correct and incorrect messages are fed

An example – RRD MA Tested messagesTested messages MetaData Key RequestMetaData Key Request SetupData RequestSetupData Request StoreRequestStoreRequest All examples found inAll examples found in Helped produce a number of testsHelped produce a number of tests Interaction with the LS will also be tested.Interaction with the LS will also be tested.

Current Status RRDMA Tests status 1.Infrastructure complete The main script - Coordinator The main script - CoordinatorExecutes The creator script The creator script Creates the requests in xml files Creates the requests in xml files Notes the validity of the requests Notes the validity of the requests The request script The request script Actually used the SoapClient Class Actually used the SoapClient Class Feeds the xml requests to the service and outputs the responses to xml files Feeds the xml requests to the service and outputs the responses to xml files The parser script The parser script Checks the validity of the responses Checks the validity of the responses Double-checks the responses with the actual data in the database Double-checks the responses with the actual data in the database

Current Status 2 2.Creator Script The creator script is almost done. Actually different creator scripts were created for different message types. Minor changes are to be made when everything is completed 3.Request Script Done 4.Parser Script Work in progress. So far 2 test types are recognised. Work is needed also on the double-checking mechanism.

Current Status 3 RRDMA service has changed since perfSONAR 1.0. Major changes: –1.0 Uses Error Messages –1.0 will use Error Codes An RRDMA 1.1 server should be able to interact natively with a 1.0 client The functional testing team will check 1.1 version against 1.1 and 1.0 tailored tests.

Q & A QuestionsQuestions CommentsComments