July 21, 2005AESE Fast Hybrid Simulation with Geographically Distributed Substructures Gilberto Mosqueda Boza Stojadinovic Jason P. Hanley (Presenter) Andrei M. Reinhorn Mettupalayam Sivaselvan

July 21, 2005AESE Distributed Testing in NEES NEES is the Network for Earthquake Engineering Simulation –Utilize capabilities of geographically distributed experimental facilities –Provide remote access to these facilities MOST was the first full scale test of the NEESgrid infrastructure Goal of research is to expand NEES infrastructure to: –Improve reliability of results –Increase speed of test

July 21, 2005AESE Communications Protocol NTCP  NEESgrid Teleoperations Control Protocol A standardized protocol for exchange of control information Provides integrated authentication and authorization services Robust handling of network faults –Re-transmission after timeout –Ability to pause or restart an experiment

July 21, 2005AESE Faster Continuous Testing Parallel communication with remote sites –Implement multi-threaded Simulation Coordinator Streamlined communications protocol (NTCP) –Minimize network communication –Reduce overhead caused by security protocols Event-driven controller –Reduce the time it takes to load experimental substructures –Implement algorithms for continuous loading of experimental substructures

July 21, 2005AESE Multi-threaded Communication Threads running in parallel In linear execution step time is the sum of communication times with each site In parallel execution step time is the maximum of the communication times with each site

July 21, 2005AESE NTCP Improvements The original design of the protocol has too much overhead A simulation time step could involve up to 3 round-trip communications Propose > Execute > Query A new command (ProposeAndExecute) was created to do all this in one round-trip communication Improvements are backward compatible with any control system interfaced to NTCP

July 21, 2005AESE Event-driven Controller Continuous loading of experimental substructures Used polynomial extrapolation to predict load path and corrects using interpolation when target becomes available Time out’s calibrated to network round-trip communication time

July 21, 2005AESE Structural Model 6-span bridge model –Span and two column are numerical models –Other three columns are experimental models Experimental Sites: Berkeley Boulder Buffalo Computational Sites: Buffalo (Span) UIUC Lehigh

July 21, 2005AESE Remote Substructures Berkeley Stiffness 0.49kN/mm Yield disp. 16mm Max. disp. 100mm Buffalo Stiffness 0.32kN/mm Yield disp. 8mm Max. disp. 25mm The displacement and forces were scaled to match the initial stiffness of the numerical full-scale column.

July 21, 2005AESE Numerical Simulation Bridge model was subjected to 15 seconds of ground shaking –Ground motion is an artificial record created from spectral density Operator-splitting integration algorithm –Integration time step of 0.01 seconds –1500 simulation steps Numerical simulation integrated with the Simulation Coordinator –Remove one round-trip communication per step

July 21, 2005AESE Site Architecture

July 21, 2005AESE Network Architecture All sites connected via Internet2 backbone Local NEESpop’s as close to network uplink as possible (1-2 hops) Internet2 (Abilene) Network Backbone (10Gbps)

July 21, 2005AESE Simulation Protocol 1.Local simulation of all sites 2.Local test of specimen with remote sites simulated 3.Distributed simulation of specimens 4.Distributed test with single specimen, others simulated 5.Distributed test

July 21, 2005AESE Simulation Results Non-linear behavior is due to the actual measured behavior of the physical specimens (Buffalo & Berkeley) Remaining (3) column substructures were simulated numerically

July 21, 2005AESE Timing Results Simulation took 1074 sec. (about 17 minutes) Average step time 0.66 sec. Four substantial delays –Four ~ 23 sec. –One ~ 3 sec. Multiple recovery from network timeouts in the TCP and Application layers

July 21, 2005AESE Lessons Learned Scheduling multi-site tests is hard –Getting 5 different sites equipment and personnel free at the same time is not easy Usage of some communication channel is a must –Audio/Video conference –Chat room To address all problems computer scientists and structural engineers must work together –They don’t always speak the same language

July 21, 2005AESE Conclusions A distributed control strategy was implemented into NEESgrid to support relatively fast and continuous hybrid simulation testing methods with geographically distributed substructures Considerable improvements to overall testing time and network robustness Reliability of the results was improved by the minimization of force-relaxation and other strain- rate related errors in the experimental substructures

July 21, 2005AESE Thank you