The NEESgrid Experience: 2000 - 2003 Tom Finholt School of Information University of Michigan

Outline SOC functions Description Primary Secondary System History Organization Funding Incentives Collaboration readiness

Outline (continued) Access Resource diagram Technology involved Successes and challenges Usage Analysis of prototype tools Action logs Chat logs Analysis of community expectations Conclusions

SOC functions Primary Secondary Shared instrument Distributed research center Virtual community of practice Community data system

Description: System Functional Technical Support real time remote control of instruments access to archival and real time data communication with remote colleagues Technical Grid-based, thin client model CHEF Support 3 FTEs at Michigan

What’s A Grid? Web: Uniform access to documents Grid: http:// Web: Uniform access to documents http://                Software catalogs Grid: Flexible, high-performance access to resources for distributed communities Computers Sensors and instruments Data archives Colleagues Source: Harvey Newman, Caltech

NEESgrid High-Level Architecture Portal, Web and 3rd Party End User Interfaces APIs, Tools and Libraries Supporting End User Interfaces Collaboration Services Data & Information Services Telepresence Services Simulation & Analysis Services APIs Supporting Higher Level Information Services Grid Resource Management Middleware System Resources: Compute, Network, Data Storage, Testing Sites Capabilities data, simulation, collaboration, visualization, telepresence validated and improved via EA demonstration scenarios and delivered via APIs and tools for users services and interfaces at equipment and resource sites management services for operation

Instrumented Structures and Sites Remote Users Simulation Tools Repository High-Performance Network(s) Laboratory Equipment Field Equipment Curated Data Repository Leading Edge Computation Global Connections (FY 2005 – FY 2014) Remote Users: (K-12 Faculty and Students) Laboratory Equipment

Grid Operations Center The Grid in NEESgrid Experimental Component Grid Operations Center Grid Data Repository Campus Net Component NEESgrid Component Internet Fabric and Operations Hub C Hub A Hub B NEESpop A Teleobservation Equipment Experimental Equipment Telepresence Equipment This simple animation is designed to show where the grid components lie in a typical collaborative experiment. It is self-explanatory, but it should be noted that one or more of the NEESpops (e.g., at Site B and C) are optional… they are included here just for consistency. One important point in this slide is that campus networks and Internet operations/fabric are NOT part of the SI award’s scope. If they were, the resulting competition for project financial and team technical resources could readily constrain any progress by the SI awardee. Passive co-PI Video I/O Active PI Data Cache Audio I/O Data Cache Site A: Experimental Data Producer Site B: Remote Lead Investigator Site C: Passive Collaborator

Science and Engineering Grids

Bhuj, India. One of the towers of this apartment complex totally collapsed,and the central stairway leaned on another building of the complex. Photo courtesy of Dr. J.P. Bardet, University of Southern California http://geoinfo.usc.edu/gees/RecentEQ/India_Gujarat/Report/Damage/Bhuj/Bardet_Feb18.html

Description: History Scoping study Alpha 1.0 Alpha 1.1 MOST experiment NCSA, ISI, ANL, UM Alpha 1.0 demonstrated at UNR, November 2002 released February 2003 Alpha 1.1 released May 2003 MOST experiment real-time control of reaction wall from numerical simulation UIUC and Colorado, July 2003

NEESgrid interface

NEESgrid interface

NEESgrid interface

Operational System Prototype (7/03) NEESgrid demonstration in the context of a real EE test moving from demonstration to usability enabling discovery and research Demonstration goals test functionality and usability of the current implementation identify and deploy new features Demonstration attributes synchronized physical experiments at two NEES sites integrated physical experiments and numerical simulation pseudo-dynamic testing and control data analysis, repositories, and visualization multi-site teleobservation

Multi-Site, On-Line Simulation Test (MOST) July 2003 UIUC Experimental Model f1 m1 f2 Colorado Experimental Model NCSA Computational Model m1 f1 f2 SAC Consortium Benchmark Structure

Multi-Site, On-Line Simulation Test (MOST) UIUC Experimental Model Colorado Experimental Model m1, q1 F2 F1 e = f1, x1 f2 NEESpop NEESpop SIMULATION COORDINATOR UIUC MUST-SIM Dan Abrams Amr Elnashai Dan Kuchma Bill Spencer and others Colorado FHT Benson Shing NEESpop m1 f1 f2 NCSA Computational Model

MOST Column Test Specimens Illinois Test Specimen Colorado Test Specimen

July Demonstration Experiment UIUC MUST-SIM NEES POP Colorado NEES POP Colorado Experimental Model f2 F2 F1 e UIUC Experimental Model f1 m1 = NCSA NEES POP m1 f1 f2 NCSA Computational Model

Description: Organization NEES program (2000-2004) NEESgrid, system integration NEES Consortium Development 15 equipment sites NEES Consortium (2003-2014) Board and Executive Committee Standing Committees

BYU/UCSB/USC Les Youd

Description: Funding NEESgrid scoping study NEESgrid cooperative agreement between NSF and NCSA approximately $300 K, 2000-01 NEESgrid $10 million, 2001-04

Description: Incentives Funding scientists recruited through the NEES equipment site program and through the NEES grand challenge program Recognition innovative system highly visible to earthquake engineering and computer science communities (e.g., SC 2002 demo) Novel capabilities first operational use of globus/OGSA technology “hybrid” operations -- combining numerical and physical simulations

Description: Collaboration readiness Experiments are collaborative but frequently defined by collocation Some use of the Web shared databases UCSD “webshaker” site Ubiquitous email use Equipment sites are all Internet2 participants

Access: People Earthquake engineers In the United States only about 1500 practice and academic geotechnical, structural, and tsunami sub-fields Affiliated with the Earthquake Engineering Research Institute and CUREE

The field of earthquake engineering Research University-based Funded by NSF and industry Focus on simulation Physical models (e.g., reduced scale specimens) Numerical models (e.g., finite element analysis) Practice Professional firms Structural engineering (e.g., earthquake remediation) Formulation of uniform building codes Lifelines (e.g., ensure survival of roads, gaslines, power distribution)

Access: Instruments Structural Geotechnical Tsunami reaction walls shake tables field test Geotechnical centrifugews Tsunami wave basins

Shake table: Nevada, Reno

Reaction wall: Minnesota

Centrifuge: UC Davis

Wave basin: Oregon State

George E. Brown, Jr. Network for Earthquake Engineering Simulation NSF Major Research Equipment and Facility Construction award (MRE) $80 million, 2001-04 $10 million for system integration (NCSA, ANL, USC-ISI, Michigan, Oklahoma) $2 million for consortium development (CUREE) $78 million for new equipment sites 3 shake tables (Buffalo, Nevada-Reno, UCSD) 2 centrifuges (RPI, UC Davis) 5 reaction walls (Berkeley, Buffalo, Colorado, Illinois, Lehigh, Minnesota) 3 field test (Texas, UCSB/USC/BYU, UCLA) 1 lifeline (Cornell) 1 tsunami (Oregon State)

                                                                                                                        

Technology involved Globus (USC ISI and ANL) CHEF (Michigan) GSI and Gridftp CHEF (Michigan) Telepresence systems (ANL) Data repository (NCSA) Deployment, operations, and support (NCSA)

NEESgrid Software Features Alpha 1.0 Release February 2003 packaged components Basic Data Streaming Services (NSDS) Data capture through LabView Collaborative environment (CHEF-based) Data viewer (CHEF-based) CHEF

NEESgrid Software Features Alpha 1.0 Release Tele-Observation but not Tele-Operation E-Notebook services Grid services Tele-Observation E-Notebook

NEESgrid Software Evolution Alpha 1.0 release (February 2003) Alpha 1.1 release (June 2003) Release 2.0 (Fall 2003) tele-operation via NEESgrid Tele-Control Protocol (NTCP) faster sampling rates for DAQ simplified certificate request tools improved NEESgrid metadata service (NMDS) Oracle and Sybase support with linked repositories improved CHEF interface

NEESgrid Software Evolution Release 3.0 (Spring 2004) audio for telepresence auto configuration of telepresence sites telecontrol request queuing data mirroring and redundancy management

Successes Scientific Community Future First teleobservation of shake table (November 2002) First data saved to repository (November 2002) Community NEES Consortium incorporated (January 2003) First NEES Consortium meeting (May 2003) Future MOST experiment, July 2003 Operational collaboratory October 1, 2004

NEESgrid November 2002 Demonstration Earthquake simulation at UNR early adopter site biaxial shake table with cameras and instrumentation 40% scale model of a two span bridge concrete slab over steel girders Bridge model instrumented with sensors strain gauges, load cells, displacement, acceleration

16 Channel Instrumentation Placement Data Repository

NEESgrid November 2002 Demonstration Browser-based collaborative framework Electronic notebook for data recording Experiment management tools data and metadata Streaming data and video tele-observation of experiment data channels from sensors Data analysis and visualization Tele-observation SAP2000 model

Earthquake engineers – in Hofstede’s scheme Power distance Hierarchical Bias toward seniority Individualist “My lab is my empire” Solo PI model Masculine Adversarial Competitive Uncertainty avoidance Highly skeptical of new technologies Extremely risk adverse

Challenges Cultural differences Building it so they will come Different jargon Different world views Building it so they will come Objects to think with

Grid specialists – in Hofstede’s scheme Power distance Egalitarian Bias toward talent Collectivist Use the Internet to create worldwide communities Project model Masculine Adversarial Competitive Uncertainty avoidance Extremely open to new technologies Extremely risk seeking

Agreeing on terms

How earthquake engineers think Customer Need Customer Requirements Deployment and Operations Requirements Analysis Progress Feedback Structure Design Structure Construction Structure Acceptance Customer Needs Assessment Design, Engineering, and Development Structure Operations

How grid specialists think System Design System Analysis Prelim Analysis START Implementation More Iterations Prelim Design Proto Evaluation Prototyping Evaluation

Prototype usage H.323 videoconferencing Worktools

User expectations Community survey Workshop survey

Survey methods Community survey National, random stratified proportional sample Stratified on region (six regions – CA, NW, SW, MW, SE, NE) and sub-field (e.g., structural, geotechnical etc.) EERI membership roll and key tsunami mailing lists as sampling frame n = 361 Web and mail administration, March – May, 2002 Two waves of email follow-up; three waves of postal follow-up 99 responses (27% response rate) Response profile reflects the sample proportions

Survey methods Workshop survey Opportunity sample Registered participants in nineteen of the twenty regional workshops The survey administration was not ready for the first workshop held in Charleston, SC n = 287 Web and paper-and-pencil, pre- and post-workshop administration, January – April, 2002 260 pre-workshop responses (91% response rate) 15 matched pre- and post-workshop responses (5% matched response rate) 60% matched response rate for Houston, TX workshop 53% matched response rate for Lawrence, KS workshop 5% matched response rate for San Jose, CA workshop 0% matched response for the other fourteen workshops

Survey methods Technical surveys Practice survey (2002) Response from fourteen of fifteen sites Practice survey (2002) Administered to 444 engineers 187 responses (42%) 11% non-NEES equipment sites 9% women 56% students 39% from Year 1 (26% repeat response rate)

Practice survey: Data use

Practice survey: Collaboration 2001 2002 Item Mean SD N Number of collaborations you are currently involved with 2.5 6.1 66 2.3 4.7 72 Number of collaborations with remote participants 1.4 3.2 62 1.3 3.6 71 Number of collaborators on your primary collaboration 5.7 7.1 60 7.3 Number of collaborators from prior collaborations in primary collaboration 1.6 3.1 57 1.7 3.0 61

Expectations about NEES Questionnaire item Percent in agreement Workshop samplea Community sampleb 1. Provide me with access to experimental data that are not currently available. 83% 63% 2. Expand opportunities for lifelong learning. 81% 50% 3. Improve my capacity to practice earthquake engineering. 82% 54% Notes: a n = 167; b n = 76

Expectations about NEES Questionnaire item Percent in agreement Workshop samplea Community sampleb 4. Provide me with new opportunities for collaboration with colleagues at other institutions. 78% 57% 5. Provide me with access to experimental facilities that are not available at my institution. 75% 55% 6. Provide me with access to computational resources that are not currently available. 70% 59% Notes: a n = 167; b n = 76

Expectations about NEES Questionnaire item Percent in agreement Workshop samplea Community sampleb 7. Improve my ability to design codes and provisions for reducing seismic losses. 69% 51% 8. Expand your opportunities for data management and visualization. 67% 53% 9. Provide me with new opportunities for incorporating measured data in the classroom. 54% 32% Notes: a n = 167; b n = 76

Response profiles 19% 35% 70% 52% 3% 2% 8% 11% Community sampleb Sub-field affiliation Workshop samplea Community sampleb Geotechnical 19% 35% Structural 70% 52% Tsunami 3% 2% Other 8% 11% Notes: a n = 250; b n = 89

Response profiles 41% 73% 59% 27% Community sampleb Workshop samplea Earthquake engineering orientation Workshop samplea Community sampleb Practitioner 41% 73% Academic researcher/education 59% 27% Notes: a n = 250; b n = 89

Top concerns among community respondents Item for concern Percent concerned or very concerned 1. Determining standard community-wide data formats 60% 2. Determining policies and procedures for sharing of equipment 44% 3. Network quality, monitoring and upgrade 43% 4. Determining standard meta-data formats 41% 5. Intellectual property rights for shared data 39%

Summary of survey data analysis Workshop and community respondents share consensus about NEES expectations …but level of agreement is much higher among workshop respondents Access to experimental data is paramount …and is more important than access to experimental facilities for both workshop and community respondents

Summary of survey data analysis Workshop respondents differed from community respondents Skewed more heavily toward structural earthquake engineers (70% vs. 52%) Skewed more heavily toward academic earthquake engineers (59% vs. 27%) Producing standard data formats is a critical goal

The field of dreams I was sitting on the verandah of my farm house in eastern Iowa when a voice clearly said to me, “If you build it, he will come.” – Ray Kinsella in Shoeless Joe, by W.P. Kinsella Image source: http://www.fieldofdreamsmoviesite.com/

Building it so they will come… Balance contributions (pp. 50-51 in the Atkins report) Weight domain science too heavily? Overemphasize procurement of existing technologies Computer scientists become viewed as “merely” consultants and implementers Weight computer science too heavily? End user needs insufficiently addressed Emphasis on novelty at the expense of usability and stability Source: Atkins report --http://www.communitytechnology.org/nsf_ci_report/

Innovation vs. extrapolation computational grids e-science Social &Technological Forces collaboratories distance learning community networks electronic commerce digital libraries electronic journals video conferencing Extrapolation

Building it so they will come… Give users objects to think with (scenarios, mock-ups, prototypes) Be patient…let users convince themselves Know where you’ve been (collect baseline data) and what’s changed (collect data as you go along)

Building it so they will come… Remove institutional barriers to collaboration David and Spence draft report p. 13 Arzberger and Finholt report OECD report on access to publicly funded research data NIH statement on data sharing Pritzker covenant Sources: Arzberger and Finholt report -- http://nbcr.sdsc.edu/Collaboratories/CollaboratoryFinal2.doc; OECD report -- http://dataaccess.ucsd.edu/FINAL_Interim_Report_20_Oct2002.doc; NIH draft statement -- http://grants2.nih.gov/grants/policy/data_sharing/

Conclusions Data access dominates teleoperation Cultural schism NEES Consortium not IT savvy Fit to grid capabilities is imperfect Mixed goals bulk of award to build new facilities for physical simulation long-term desire is for numerical simulation