1. July 21, 2005AESE 20051 Fast Hybrid Simulation with Geographically Distributed Substructures Gilberto Mosqueda Boza Stojadinovic Jason P. Hanley (Presenter) Andrei M. Reinhorn Mettupalayam Sivaselvan

2. July 21, 2005AESE 20052 Distributed Testing in NEES NEES is the Network for Earthquake Engineering Simulation –Utilize capabilities of geographically distributed experimental facilities –Provide remote access to these facilities MOST was the first full scale test of the NEESgrid infrastructure Goal of research is to expand NEES infrastructure to: –Improve reliability of results –Increase speed of test

3. July 21, 2005AESE 20053 Communications Protocol NTCP  NEESgrid Teleoperations Control Protocol A standardized protocol for exchange of control information Provides integrated authentication and authorization services Robust handling of network faults –Re-transmission after timeout –Ability to pause or restart an experiment

4. July 21, 2005AESE 20054 Faster Continuous Testing Parallel communication with remote sites –Implement multi-threaded Simulation Coordinator Streamlined communications protocol (NTCP) –Minimize network communication –Reduce overhead caused by security protocols Event-driven controller –Reduce the time it takes to load experimental substructures –Implement algorithms for continuous loading of experimental substructures

5. July 21, 2005AESE 20055 Multi-threaded Communication Threads running in parallel In linear execution step time is the sum of communication times with each site In parallel execution step time is the maximum of the communication times with each site

6. July 21, 2005AESE 20056 NTCP Improvements The original design of the protocol has too much overhead A simulation time step could involve up to 3 round-trip communications Propose > Execute > Query A new command (ProposeAndExecute) was created to do all this in one round-trip communication Improvements are backward compatible with any control system interfaced to NTCP

7. July 21, 2005AESE 20057 Event-driven Controller Continuous loading of experimental substructures Used polynomial extrapolation to predict load path and corrects using interpolation when target becomes available Time out’s calibrated to network round-trip communication time

8. July 21, 2005AESE 20058 Structural Model 6-span bridge model –Span and two column are numerical models –Other three columns are experimental models Experimental Sites: Berkeley Boulder Buffalo Computational Sites: Buffalo (Span) UIUC Lehigh

9. July 21, 2005AESE 20059 Remote Substructures Berkeley Stiffness 0.49kN/mm Yield disp. 16mm Max. disp. 100mm Buffalo Stiffness 0.32kN/mm Yield disp. 8mm Max. disp. 25mm The displacement and forces were scaled to match the initial stiffness of the numerical full-scale column.

10. July 21, 2005AESE 200510 Numerical Simulation Bridge model was subjected to 15 seconds of ground shaking –Ground motion is an artificial record created from spectral density Operator-splitting integration algorithm –Integration time step of 0.01 seconds –1500 simulation steps Numerical simulation integrated with the Simulation Coordinator –Remove one round-trip communication per step

11. July 21, 2005AESE 200511 Site Architecture

12. July 21, 2005AESE 200512 Network Architecture All sites connected via Internet2 backbone Local NEESpop’s as close to network uplink as possible (1-2 hops) Internet2 (Abilene) Network Backbone (10Gbps)

13. July 21, 2005AESE 200513 Simulation Protocol 1.Local simulation of all sites 2.Local test of specimen with remote sites simulated 3.Distributed simulation of specimens 4.Distributed test with single specimen, others simulated 5.Distributed test

14. July 21, 2005AESE 200514 Simulation Results Non-linear behavior is due to the actual measured behavior of the physical specimens (Buffalo & Berkeley) Remaining (3) column substructures were simulated numerically

15. July 21, 2005AESE 200515 Timing Results Simulation took 1074 sec. (about 17 minutes) Average step time 0.66 sec. Four substantial delays –Four ~ 23 sec. –One ~ 3 sec. Multiple recovery from network timeouts in the TCP and Application layers

16. July 21, 2005AESE 200516 Lessons Learned Scheduling multi-site tests is hard –Getting 5 different sites equipment and personnel free at the same time is not easy Usage of some communication channel is a must –Audio/Video conference –Chat room To address all problems computer scientists and structural engineers must work together –They don’t always speak the same language

17. July 21, 2005AESE 200517 Conclusions A distributed control strategy was implemented into NEESgrid to support relatively fast and continuous hybrid simulation testing methods with geographically distributed substructures Considerable improvements to overall testing time and network robustness Reliability of the results was improved by the minimization of force-relaxation and other strain- rate related errors in the experimental substructures

18. July 21, 2005AESE 200518 Thank you