Testing, Monitoring, and Controlling CORBA-based Applications Sudipto Ghosh Priya Govindarajan Aditya P. Mathur Baskar Sridharan Software Engineering Research Center Purdue British Telecom, UK Monday, July 12, 1999

July 12, 1999British Telecom2 Client/Server Stub/Skeleton Request/data Client/Server Stub/Skeleton Request/data Component Orb Communication over network or within a process.. Structure of a CORBA Application

July 12, 1999British Telecom3 Test Adequacy: Goodness of system test. Parts of the system that need additional testing. Needs of a tester

July 12, 1999British Telecom4 System behavior: Visualization of system state transition in the presence of a given input sequence. Visualization of system state transition in the presence of injected faults. Needs of a tester

July 12, 1999British Telecom5 System performance: Measurement of instantaneous and average load distribution (Hardware and Software). Measurement of instantaneous and average response time. Needs of a tester

July 12, 1999British Telecom6 Needs of a tester System control: Temporary or permanent shut down of selected services. Injection of an arbitrary stream of requests directed at selected servers.

July 12, 1999British Telecom7 How TDS meets a tester’s needs? Test adequacy: By measuring interface-based coverage: –Method coverage –Exception coverage –iMutation Coverage

July 12, 1999British Telecom8 State related: –What is the current state of component C? Performance related: –What is the average round-trip time of a message sent to component C ? Access related: –Did client C’ access component C during the past 24 hours ? Monitoring: Sample Questions

July 12, 1999British Telecom9 Component related: –Deny a set of services to some or all clients of component C upon the occurrence of event E –start component C –replicate component C –send message m to component C Control: Sample Commands

July 12, 1999British Telecom10 Component Interface Methods: m 1, m 2, …,m k Exceptions: e 1, e 2, …,e k Test Adequacy Criteria 100% Method Coverage # methods executed # methods defined 100% Exception Coverage # exceptions raised # exceptions defined 100% iMutation Score # distinguished mutants total # imutants - #equivalent imutants Proposed Test Adequacy Criteria

July 12, 1999British Telecom11 Replace inout by out and vice versa Swap parameters of the same type Increment/decrement parameter by 1 Set parameter to a value (boundary value) Nullify an object reference Proposed iMutant Operators

July 12, 1999British Telecom12 Are e C and e A statistically the same? Component: C i Errors present: E i Adequacy Criterion: AC i Test Set: T i Revealed errors: e i e C =  e i Adequacy Criterion AC =  AC i Test set: T Errors revealed = e A Application Test Adequacy Criteria during Integration

July 12, 1999British Telecom13 1. Test each component separately Client 2. Test the entire system What adequacy criteria to use? Only AC 1 OR Both AC 1 and AC 2 Traditional Test Adequacy Criteria for Client Interface-based Test Adequacy Criteria for each server component AC 1 AC 2 Methodology for Testing Applications

July 12, 1999British Telecom14 How do the proposed test adequacy criteria compare with the traditional criteria? Magellan Component Interface Seeded Errors Traditional Criteria Test set T 1 adequate w.r.t. block, decision, c-use and p-use. Errors revealed = E 1 Proposed Criteria Test set T 2 adequate w.r.t. method,exception and interface mutation Errors revealed = E 2 Hypothesis: E 1 = E 2 Experiments: Testing Components

July 12, 1999British Telecom15 How do the two criteria compare? Only Traditional Criteria for Client(s) Test set T 1 adequate wrt block, decision, c-use and p-use in the client(s). Total #errors revealed = E 1 Traditional Criteria for client and pro- posed criteria for other components Test set T 2 adequate wrt AC int and AC client Total #errors revealed = E 2 (E 1  E 2 ) :: (  T 1   T 2  ) Client Magellan System Experiments: Testing Applications

July 12, 1999British Telecom16 Application: –Multi-layer infrastructure for building “help- desk” applications. Statistics for one layer: –Approx. Lines of code 80K –Number of IDL files16 –Number of methods150 –Number of exceptions:7 –Number of.exe files: 37 –Number of interfaces: 24 Experiments: System to be Tested

July 12, 1999British Telecom17 Monitoring: Strategy UserEvent(s) specifies State MonitorApplication State monitors Event detectorUser specified events detects Event notification Waiting user notifies

July 12, 1999British Telecom18 Control: Strategy UserControl action(s), trigger event(s) specifies Event notifierController signals ControllerControl action(s) executes

July 12, 1999British Telecom19 States Local: S : an arbitrary state. S could be: Component C processing request R Component C is inactive Component C is active

July 12, 1999British Telecom20 Events Local: E : an arbitrary event. E could be: –Component C assumes state S –E after time T (T: local time) –E during interval [T1, T2] –E1 or E2 –E1, E2 (sequence)

July 12, 1999British Telecom21 How TDS meets a tester’s needs? System test: By allowing the tester to combine features for (V2.0) –test adequacy measurement –system performance measurement –system behavior monitoring –system control

July 12, 1999British Telecom22 An Example Two Servers - Bank_Srv and TM_Srv Bank_Srv interface account { readonly attribute float balance; void makeLodgement (in float f); void makeWithdrawal (in float f); }; interface bank { exception reject {string reason;}; account newAccount (in string name) raises (reject); void deleteAccount (in account a); }; TM_Srv interface transaction { int verifyTransaction (in account); };

July 12, 1999British Telecom23 Goal To Monitor and Control object B1. –B1 implements interface Bank –Construct event E: “Arrival of request createAccount() to object B1” –Monitor E –Specify control operation: “Ignore all incoming requests on object B1”

July 12, 1999British Telecom24 TDS 2.0 Architecture for Monitoring and Control

July 12, 1999British Telecom25 Local Listener Controller Server MonitorClient Monitor DI B1 B2 DATABASE Host 1 Local Listener Controller Server MonitorClient Monitor DI TM Host 2 Local Listener Client Monitor Client Host 3 Request: Response: Global Listener deleteAccount (acc)

July 12, 1999British Telecom26 TDS 2.0 Event Specification and Registration

July 12, 1999British Telecom27 Local Listener Controller Server MonitorClient Monitor DI B1 B2 DATABASE Host 1 Local Listener Controller Server MonitorClient Monitor DI TM Host 2 Local Listener Client Monitor Client Host 3 TDS SoftPanel Control Panel Data Retriever SR Host1 Bank B2 B1 Interface Bank - B1 1. account newAccount(in string name) 2. currentAccount newCurrentAccount(in string name, in float amount) 3. void deleteAccount(in account a) Construct Event: (Host, Server, Comp, Method, {Param,Rel,Value}*) (Host1, Bank_Srv, B1, newAccount, {name, = =,” Joe”}) Global Listener

July 12, 1999British Telecom28 TDS 2.0 Event Monitoring and Notification

July 12, 1999British Telecom29 Global Listener Local Listener Controller Server MonitorClient Monitor DI B1 B2 DATABASE Host 1 Local Listener Controller Server MonitorClient Monitor DI TM Host 2 Local Listener Client Monitor Client Host 3 TDS SoftPanel Control Panel Data Retriever SR1 Bank_Srv, Bank, B1, createAccount(name = “Joe”) SoftRover1 SoftRover2 Arch Extr Host: Host1 Server:Bank_Srv, Interface:Bank, Object: B1 Method: createAccount(name = “Joe”) Time: dd:mm:yy Host 1 Host: Host1 Server:Bank_Srv B1.createAccount(name = “Joe”) Client issues a request:

July 12, 1999British Telecom30 TDS 2.0 Specification and Application of a Control Operation

July 12, 1999British Telecom31 Local Listener Controller Server MonitorClient Monitor DI B1 B2 DATABASE Host 1 Local Listener Controller Server MonitorClient Monitor DI TM Host 2 Local Listener Client Monitor Client Host 3 TDS SoftPanel Control Panel Data Retriever SR1 Host: Host1 Server:Bank_Srv B1.createAccount(name = “Matt”) Issue request: Host: Host1 Server:Bank_Srv, Interface:Bank, Object: B1 Specify Control Operation Object: B1, Control Operation: (Ignore all incoming requests) Host1, Bank_Srv Object: B1, Control Operation: (Ignore all incoming requests) Apply Control Operation Send Exception to Client Global Listener

July 12, 1999British Telecom32 TDS 1.1 demonstrated in May 99. Experiments with a system from Tivoli begun in May 99. Implementation of the Monitoring and Control component begun in June 99. Implementation of the Architecture- Extraction component begins in August 99.. Summary

July 12, 1999British Telecom33 Operation Sequence in the Example Listener and Controller are instantiated on Host1 Listener sends information about its presence to Global Listener Global Listener pushes information to database Server Bank_Srv is activated on Host1 Data Initializer (DI) sends information about Bank_Srv to Local Listener (Host1) Listener Sends information to database Global Listener pushes information to database Listener and Controller are instantiated on Host2 Listener(Host2) sends information about its presence to Global Listener Global Listener pushes information to database

July 12, 1999British Telecom34 Operation Sequence in the Example Listener and Controller are instantiated on Host2 Listener sends information about its presence to Global Listener Global Listener pushes information to database Server TM_Srv is activated on Host2 Data Initializer (DI) sends information about TM_Srv to Local Listener (Host2) Listener Sends information to database Global Listener pushes information to database Listener instantiated on Host3 Listener(Host3) sends information about its presence to Global Listener Global Listener pushes information to database

July 12, 1999British Telecom35 Operation Sequence in the Example Client instantiated on Host3 Client issues deleteAccount(acc) on object B1 Request reaches Client Monitor Client Monitor sends request-information to Listener (Host3) Listener sends request-information to Global Listener Global Listener pushes data to database Listener returns ‘success’ value to Client Monitor Client Monitor sends request to Bank_Srv Request reaches Server Monitor Server Monitor sends request-information to Listener (Host1) Listener sends information to Global Listener Listener sends “success” return value to Server Monitor

July 12, 1999British Telecom36 Operation Sequence in the Example Server Monitor sends request-information to Controller Controller processes request-information and sends “success” return value Server Monitor passes the request to object B1 (which implements interface Bank) Object B1 issues verifyAccount(acc) request on object TM Request traverses identical path (as from client) Object B1 receives response from TM The response from B1 reaches Client