Alexandre Duarte Gustavo Wagner Francisco Brasileiro Walfredo Cirne Multi-Environment Software Testing on the Grid Universidade Federal de Campina Grande - Brasil

ISSTA 2006 There are applications with very time consuming test phases –Rothermel, Untch, and Chu (7 weeks) – Smart Alarms (36 hours) Configuration mistakes in software installation are the major reasons for computer system errors –Pointed by Brown and Patterson in 2001 –Observed in two surveys we have conducted with the grid community along the last years Motivation

ISSTA 2006 Motivation Grid applications are supposed to be correctly executed on highly heterogeneous and dynamic environment Traditional ways to qualify test processes are based on code coverage metrics

ISSTA 2006 Speeding up Test Execution Distribute Select Prioritize

ISSTA 2006 Dealing with Configuration Mistakes Build several different test environments –Expensive –Time consuming –Does not cover different user and administrative policies

ISSTA 2006 Agenda Motivation To present a solution to test grid applications How to describe a test of a grid application Conclusions and Future Work

ISSTA 2006 Environmental Coverage Development environments are controlled Test applications designed to run on heterogeneous environments in a set of different production environments

ISSTA 2006 GridUnit Speeds up test execution by exploring the high levels of parallelism provided by the grid Provides a way to detect configuration mistakes by exploring the heterogeneous nature of the grid: –Shared by multiple users –Without a centralized control unit –Composed by multiple administrative domains (virtual organizations)

ISSTA 2006 GridUnit Architecture

ISSTA 2006 InGriD Builds environments in grid machines Manages all environments deployed in the grid It is an extension of Smart Framework for Work Group (SmartFrog), developed by HP Labs –Deal with grid machines

ISSTA 2006 InGriD Architecture

ISSTA 2006 InGriD and a Grid: How it works... Requirements: environment = seghidro... Job Description File Broker Peer Site 1Site 2Site N Machines with seghidro environment

ISSTA 2006 Testing a Real Application The application creates a web album from a set of digital images –Time consuming Photo editing is independent from each other Grid nodes have: –Sun JVM 5.0 and JRockit 5.0 –Intel Pentium 4 and Intel Itanium 2 –4 environments

ISSTA 2006 Running the Tests Environment Description InGriD JUnit Automated Tests + Test Session Spec. GridUnit Grid

ISSTA Environment Specifications

ISSTA Test Session Specification

ISSTA 2006 OurGrid Largest computation grid in production in Brazil today –About 30 organizations (UFCG, USP, PUC-RS, UFPE, PUC- Minas, UFF, UFPB, UFBA, UCB, LNCC, UNISANTOS,...)  300 machines

ISSTA 2006 GridUnit + OurGrid Does not require any source code modification Open Source Free to use and free to join

ISSTA 2006 Screenshot – Green Bar

ISSTA 2006 Screenshot – Red Bar

ISSTA 2006 Screenshot – Gray Bar

ISSTA 2006 Conclusions and Future Work We presented a proposal of a solution to improve the confidence of heterogeneous environment applications –InGriD: to describe and deploy test environments –GridUnit: to coordinate and monitor execution of test sets Environmental coverage metric is complementary and orthogonal to test coverage metrics Formalization of the environmental coverage metric for software testing

谢谢 - Thank you – Obrigado Merci – Danke – Arigato - Gracias Additional Information: Alexandre Duarte – Gustavo Wagner – Francisco Brasileiro – Walfredo Cirne –