1. Distributed Hybrid Earthquake Engineering Experiments: Experiences with a Ground-Shaking Grid Application Laura Pearlman, Carl Kesselman, Sridhar Gullapalli USC Information Sciences Institute B.F. Spencer, UIUC Ian Foster, Paul Hubbard, ANL Chuck Severance, University of Michigan Joe Futrelle, Kathleen Ricker, NCSA laura@isi.edu

2. NEESgrid NEES: NSF-funded project in support of earthquake engineering. NEESgrid: National earthquake engineering collaboratory –Deployment complete September 30, 2004 –Operational through 2014 This talk is an overview of one application of NEESgrid.

3. Outline The Application: Distributed Hybrid Experiments Architecture Experiences: The MOST Experiment Security Considerations Related Work Conclusions

4. Traditional Methods Computational Simulations Physical Experiments –Build and instrument a specimen. –Subject it to forces. –Record sensor measurements throughout the experiment. Physica l System Control / Data Acquisition System

5. Physical Testing Apparatus

6. Characteristics of Physical Experiments Specimens can be large-scale (50 tons or more) Take months to construct Some steps can’t be “undone” Require specialized equipment and facilities

7. Hybrid Experiments Computational Simulation Control / Data Acquisition System Physica l System

8. Pre-NEESgrid Hybrid Experiments Physical and computational simulations occur at same site, usually with communication via a shared memory backbone such as SCRAMnet. Proprietary communication protocol between the computational and physical system

9. Structural Experiment Geotechnical Experiment Structural Experiment Geotechnical Computation Structural Computation Soil-Structure Test

10. Distributed Hybrid Experiment Characteristics Multiple physical experiments –At different geographic sites –Resources owned by different organizations –Heterogeneous control platforms Multiple computational simulations –At different geographic sites –Resources owned by different organizations –Heterogeneous simulation platforms –Sometimes used in place of physical experiments

11. Experiment Variations 1/5 th -scale LBCB Full-scale LBCB LBCB simulator (Computer Model)

12. Equipment Site Central Services NEESgrid Services NEES-POP Control System DAQ System telecontrol streaming data Files Data/ metadata Repository, Credential cache Web portal Ingestion tool Control client Files Observers Physica l System

13. Telecontrol Service Requirements Uniform interface for heterogeneous systems Impose minimal requirements on control systems. Support for recovery from transient errors Performance requirements vary widely (1 ms for some experiments; 10 seconds for others).

14. NEESgrid Teleoperation Control Protocol (NTCP) Transaction-based State model –provides for at-most-once execution –guarantees that a control point is doing at most one thing at a time –guarantees that requests involving any control point are executed in the order received Protocol allows for negotiation of request parameters prior to execution.

15. NTCP Plugins Control System B Control System A Simulation NTCP ClientPlugin

16. Multi-site Online Simulation Test (MOST) First distributed hybrid experiment in NEESgrid (and, we believe, in the US). Combined physical experiments at the University of Illinois (UIUC) and University of Colorado (CU) with a simulation at NCSA. Simulations and experiments created by earthquake engineers at UIUC and CU.

17. The MOST Structure SAC Consortium Benchmark Structure Pinned Connection Moment Connections

18. NCSA Computational Model m1m1 f1f1 UIUC Experimental Model f1f1 m1m1 f2f2 f2f2 U. Colorado Experimental Model The MOST Substructures Slide courtesy of Bill Spencer and Narutoshi Nakata, UIUC

19. Computation-Only Test Simulation Coordinator NTCP Server NTCP Server NTCP Server Mplugin UIUC Simulation NCSA Simulation CU Simulation Colorado NCSAUIUC

20. MOST Components Coordinating Simulation NTCP Server NTCP Server NTCP Server Mplugin Shore-Western Plugin NCSA Simulation Matlab (xPC host) Colorado NCSAUIUC Matlab (xPC target)

21. Control Components in MOST Globus Toolkit version 3 NTCP Server Mplugin Shore-Western Plugin NTCP Java API Mplugin Java API NTCP Plugin Interface Mplugin Matlab toolbox NTCP Matlab toolbox

22. The MOST Event

23. Mini-MOST

24. Security Considerations

25. NTCP Security Features Authentication via GSI Authorization via gridmap lookups and a policy plugin. Control plugin architecture allows for running control system on a separate host, isolated behind a firewall Control plugin also allows for setting of local limits.

26. Ongoing NEES Experiments Fast-MOST: Berkeley, Buffalo, CU, and UIUC are performing a MOST- like experiment with stricter performance requirements. MISST: RPI, UIUC, Lehigh: soil- structure interaction. UC-Davis: Soil study with remote control of a robot arm.

27. Other Related Work Multi-Site Pseudo-Dynamic Substructure Testing: –Method for dividing structures into substructures and testing separately –Developed in Japan in 1999 –MOST simulations are based on this method TeleScience Portal – using NTCP to control electron microscopes. NMI Common Instrument Middleware Architecture (CIMA) project

28. Conclusions Distributed hybrid earthquake engineering experiments are a “real” grid application. NTCP works: –Earthquake engineers are developing and using their own clients and plugins. –Its failure-recovery features have allowed experiments to continue after transient network failures But there’s still work to do… –NTCP requires some shared knowledge between developers of clients and plugins. Better use of service data could fix this. –Some aspects of the protocol need to be “cleaned up” –There’s not much support for recovering from backend failures.

29. Acknowledgements Dan Abrams, Cristina Beldica, Ian Buckle, Ben Clifford, Mike D’Arcy, Amr Elnashai, Tom Finholt, David Gehrig, Tomasz Haupt, Dan Horn, Erik Johnson, Young Suk Kim, Dan Kuchma, Lee Liming, Ravi Madduri, Doru Marcusiu, Gilberto Mosqueda, Narutoshi Nakata, Gokhan Pekcan, Pawel Plaszczak, Tom Prudhomme, Chase Phillips, Andrei Reinhorn, Hatem A Seliem, Benson Shing, Eric Stauffer, Bozidar Stojadinovic, Guangqiang Yang, and Nestor Zaluzec.

30. For More Information NEESgrid web site: http://www.neesgrid.org