Realizing Vision 2020: A Proposal Aditya P. Mathur Professor, Associate Head, and Director Department of Computer Sciences and Software Engineering Research Center Purdue University, West Lafayette, IN, USA Pundi Narasimham President STS Worldwide, Atlanta, USA Tuesday August 8, 2000 Last updated: July 13, 2000

Realizing Vision 2020: A Proposal “The sixth is the challenge of establishing a scientific and progressive society, a society that is innovative and forward-looking, one that is not only a consumer of technology but also a contributor to the scientific and technological civilization of the future.“: ….His Excellency YAB Dato' Seri Dr Mahathir Mohamad. Realizing Vision 2020: A Proposal

Scientific and progressive society Purdue-STS Contribution: Establishment of procedures to transform current mode of business practices of the Government of Malaysia into digital-mode. Transformation of key selected governmental procedures into the digital domain. Realizing Vision 2020: A Proposal

Realizing Vision 2020: A Proposal Contributor to the scientific and technological civilization of the future Purdue-STS Contribution: Development of the most advanced tools for test and management of Internet Services. Research and Prototype development at Purdue. Productization and commercialization by STS Malaysia. Realizing Vision 2020: A Proposal

The Digital Government Project [1] Long term goal: Execute all essential tasks in a secure digital domain. Provide the citizens of Malaysia the opportunity to complete all transactions with the government in a timely manner and in a secure digital domain. Realizing Vision 2020: A Proposal

The Digital Government Project [2] Collaborators: Department of Computer Sciences at Purdue University. STS Offshore Services (M) SDN BHD Malaysia. Selected University in Malaysia. Senior Personnel: Professor Aditya Mathur, Purdue University Professor Michael Stohl, Dean International Programs, Purdue University. Pundi Narasimham, President, STS. Realizing Vision 2020: A Proposal

The Digital Government Project [3] Project Phases: Phase I: Feasibility study and project planning. Purdue/STS. January-July 2001. Phase II: Prototyping: Purdue. September-March 2002. Phase III: Digitization I: Purdue/STS. June 2002-May 2003. Phase IV: Digitization II: STS: Period to be determined. Realizing Vision 2020: A Proposal

The Test Tools Project [1] Phase I: Transfer Internet Services Test technology developed at Purdue to STS. Phase II: STS productizes the technology in collaboration with faculty from a University in Malaysia. Phase III: STS commercializes the product as a product from Malaysia. Purdue continues research in advanced test tools in collaboration with a University in Malaysia. Realizing Vision 2020: A Proposal

The Test Tools Project [2] Timeline: Phase I: December 2000. Phase II: January-July 2001. Phase III: August-December 2001. Realizing Vision 2020: A Proposal

The Test Tools Project [3] Development of the test tools will continue once the first version has been marketed and has been adopted by customers. Realizing Vision 2020: A Proposal

The Test Tools Project [4] What to test and manage? Internet services: e-commerce, web-sites, CORBA applications, XML applications, Java applications. What will the tool(s) allow a tester to do? Visual Test capture and replay. Visual Test assessment and enhancement. Dynamic testing (future). Monitoring and control of CORBA, XML, and Jini applications. Performance testing. Realizing Vision 2020: A Proposal

The Test Tools Project [5] Uniqueness of the Purdue technology: Integrated test and management. Support for test assessment via Interface Mutation. Support for heterogeneous environment. Multiple platforms. Multiple software technologies (e.g. CORBA and XML) Strength of Purdue: World leader in research in software testing. Research group at Purdue has published over 100 research papers in world class journals and conference proceedings since 1987. Eight Ph.D. theses have been produced in this area. Realizing Vision 2020: A Proposal

The Test Tools Project [6] Details of the testing techniques and the tool follow. Realizing Vision 2020: A Proposal

Structure of an Internet Service Component Component Client/Server Client/Server Request/data Request/data Stub/Skeleton Stub/Skeleton ORB ORB Component ORB: Object Request Broker Communication . Realizing Vision 2020: A Proposal

Interface Testing 100% Method Coverage Methods: m1, m2, …,mk # methods executed # methods defined Methods: m1, m2, …,mk 100% Exception Coverage # exceptions raised # exceptions defined Exceptions: e1, e2, …,ek 100% iMutation Score # distinguished mutants total # imutants - #equivalent imutants Interface Component Realizing Vision 2020: A Proposal

What is Interface Mutation ? Test Suite T contains Request-A. Client Server Interface Request-A Response-A Mutated Interface Client Server Request-A Response-B Realizing Vision 2020: A Proposal

Realizing Vision 2020: A Proposal Observations [1] Interface mutation leads to fewer tests that reveal almost as many errors as revealed by statement and decision coverage. Interface mutation is a scalable alternative to using code coverage. Realizing Vision 2020: A Proposal

Realizing Vision 2020: A Proposal Observations [2] Reveals programming errors in components. errors in the use of component interfaces Reveals certain types of deadlocks 4 1 2 5 3 Client Server Client Request Server callback Realizing Vision 2020: A Proposal

Testing for fault tolerance Problem: Often error recovery code is not executed by test inputs How do we know if the fault recovery code adequately meets the requirements? Solution: Simulate the occurrence of faults by injecting them Fault injection testing at the interface Increases coverage of fault recovery code Reveals inadequacies in fault recovery code Realizing Vision 2020: A Proposal

Realizing Vision 2020: A Proposal Load Testing On: Avg. Latency Effect of: Avg. Load C1 network Clients Server C2 Realizing Vision 2020: A Proposal

Realizing Vision 2020: A Proposal Dynamic Testing Question: How to test an Internet Service while it is in use? Answer: Use the dynamic testing procedure. What is the dynamic testing procedure? Realizing Vision 2020: A Proposal

Realizing Vision 2020: A Proposal Dynamic Testing Client Faulty Server group Ra Client 1 Isolated server 3 Faulty server Test Client Ra 2 Realizing Vision 2020: A Proposal

Limitations of Dynamic Testing Test client might generate undesirable actions: Persistent data modification. Irreversible actions. Application limited to: Closed and well understood domains. Simulated or isolated service environments. Realizing Vision 2020: A Proposal

Organization of the Service Domain Why organize ? Efficient and scalable management Personalized management Assignment of individual responsibilities Realizing Vision 2020: A Proposal

Dimensions of Organization Component Types Geographical regions Client categories Realizing Vision 2020: A Proposal

Managing XML Applications specification WABASH GUI MC specification XML Parser Application Events Object state information WABASH MC Module Object control Commands (allow/deny) Subscribe Publish Realizing Vision 2020: A Proposal

Realizing Vision 2020: A Proposal Architecture of Wabash 3.0 Realizing Vision 2020: A Proposal

Realizing Vision 2020: A Proposal Ongoing Research [1] API development. Non-intrusive procedures for dynamic testing. Generalized event-control model and its implementation. Implementation of the unified architecture to assist with the management of JMX, JINI, CORBA objects, and XML applications. Light-version of Wabash for SmartHome management. Realizing Vision 2020: A Proposal

Realizing Vision 2020: A Proposal Ongoing Research [2] Automatic generation of test inputs. Test capture and replay. Dynamic data collection and analysis. Realizing Vision 2020: A Proposal

The Wabash Project: History Progress: August 1998: Wabash project launched. August 1999 TDS 1.1 available to SERC affiliates. August 2000 Wabash 2.0 available to SERC affiliates. Experiments to assess goodness of proposed interface testing criteria completed. December 2000 Uniform interface for Jini/JMX/CORBA objects and XML applications. Realizing Vision 2020: A Proposal

Realizing Vision 2020: A Proposal Architecture of Wabash 2.0 [1] LLI MC AR db Zonal Manager L CS LOG Host 1 GUI requests data from the Zonal Manager Wabash GUI Zonal Manager returns data collected from LL to GUI LL returns data to Zonal Manager Zonal Manager requests data from corresponding LL LL determines whether request can be passed or not Client sends a request to a managed object Request C If the request is allowed, LL forwards it to the CORBA Server after time-stamping it If the request is not allowed, LL throws exception and does not forward request to the CORBA Server Response LL gets the response from the CORBA Server LL stores information about the request, records it in a log, sends the response back to client Realizing Vision 2020: A Proposal