Establishing a User-Driven, World- Class Oceanographic Data Center by the Right People, in the Right Place, and at the Right Time L. Charles Sun National Center for Ocean Research June, 2005, Taipei, Taiwan

2 Outline 1. Time, Place, and People 2. Steps in Establishing an NODC 3. Mission and Role of an NODC 4. QC and QA 5. Products and Services 6. Information Technology 7. Organizational Considerations and Chart 8. “Collaboratory” 9. IDARS, Argo & GTSPP: Three examples of “Collaboratories” 10. Data Portal: “Gateway” to Ocean Data 11. Climate Data Portal: The Proven Prototype 12. Other Technologies for the Collaboratory 13. The Future

3 Time, Place, and People Time: Since 1975 ~ Place: The Center of the world People: We are the right people

4 Steps in Establishing an NODC - I 1 Recruit a team of interested parties to propose a mission and organizational model for the center. 2 Construct a draft mission. 3 Conduct negotiations with the potential partners.

5 Steps in Establishing an NODC - II 4 Prepare a draft administrative organization. 5 Prepare a final version of the mission and information on partnerships for final approval.

6 Organization Chart Office of the Director Director Deputy Director Associate Director Ocean Dynamics Chief Data Processing Research Data and Product Development Data Base Management Chief Data Archival Database Development and Maintenance Information Technology Chief Networking Operating System Maintenance Hardware/software purchase and Maintenance Library Chief Service Staff

7 Mission of an NODC To safeguard versions of oceanographic data and information. To provide high quality data to a wide variety of users in a timely and useful manner.

8 Roles of an NODC Conventional role – as a minimum Contemporary role – in response to advances in data collection and information technology

9 Conventional Role - I Receive data, perform quality control, archive and disseminate it on request. Keep copies of all or part of its data holdings in the format in which the data were received. Developing and protecting national archives of oceanographic data

10 Conventional Role - II Produce and provide inventories of its holdings on request. Referral of the users to sources of additional data and information not stored in the NODC. Participate in international oceanographic data and information exchange.

11 Contemporary Role - I Receive data via electronic networks on a daily basis, process the data immediately, and provide outputs to the user or to the data collectors for data in question. Report the results of quality control directly to data collectors as part of the quality assurance module for the system.

12 Contemporary Role - II Process and publish data on the Internet and on CD/DVD-ROMs. Publish statistical studies and atlases of oceanographic variables. Performing a level of quality control on its data holdings

13 Quality Control and Assurance Data can be detected easily by a data center Obvious errors such as an impossible date and time and location Data cannot usually be detected by a data center Subtle errors such as an instrument may be off calibration

14 Information Technologies - I Data Storage/Archive Data Processing Local Area Networking Wide Area Networking – the Internet (and the GTS)

15 Information Technologies - II Publishing DVD/CD–ROMs Graphics Capability (Graphical Information System) Software Development & Implementation Hardware procurement & Maintenance

16 Products Development - I Work with the client to determine what the real need. Examples of data products include atlases, datasets of ocean observations filtered by area, time and variables observed

17 Products Development - II Review the world wide web sites of existing NODCs for ideas and examples of data and Information products.

18 Services Providing directory and inventory information Acting as a referral center Receiving data for specific processing followed by delivery of the processed data

19 Organizational Considerations A centralized data center A distributed data center Centers of Data : “Data Portals” or “Virtual Collaboratories” Data Center Center of Data A Center of Data B Center of Data C

20 What is a Collaboratory? The fusion of computers and electronic communications has the potential to dramatically enhance the output and productivity of researchers. A major step toward realizing that potential can come from combining the interests of the scientific community at large with those of the computer science and engineering community to create integrated, tool- oriented computing and communication systems to support scientific collaboration. Such systems can be called "collaboratories." From "National Collaboratories - Applying Information Technology for Scientific Research," Committee on a National Collaboratory, National Research Council. National Academy Press, Washington, D. C., 1993.

21 Acknowledgement Soreide, N. N. and L. C. Sun, 1999: Virtual Collaboratory: How Climate Research can be done Collaboratively using the Internet. U.S. – China Symposium and Workshop on Climate variability, September 21-24, 1999, Beijing, China Presented by Len Pietrafesa, North Carolina State University.

22 Collaboratory Infrastructure Data Portal –Computer and networking hardware and software –Increased network bandwidth/speed –Next Generation Internet (NGI) connection Visualization –Interactive Java graphics –3D, Virtual Reality, collaborative virtual environments –immersion technology CAVE, ImmersaDesk... Relationships: –Observing System Project Offices –Research community, Academia... –Other Collaboratory nodes –Steering Committee

23 International Steering Committee Collaboratory Partner Collaboratory Partner Collaboratory Partner Collaboratory Partners & Customers Providers of Data & Information Users of Data & Information Observations & Satellite Groups Modeling & Forecasting Groups Research Groups New Users Educational Administrators General Public Structure of the Collaboratory for Ocean Research

24 IDARS * as an example... Real-Time Coastal Water Temperature Data Real-Time Argo Profile Data Real-Time Global Temperature and Salinity Profile Data Time Series Data NOAA CoastWatch AVHRR SST Images * Interactive Data Access and Retrieval System

25 Argo as an example... Argo as an example...

26 GTSPP * as an example... GTSPP * as an example... * * Global Temperature-Salinity Profile Program

27 Argo and GTSPP Argo and GTSPP set a standard in the international ocean data management community Data dissemination in near-real time –Researcher involvement has assured data quality Benefits of data dissemination –Wide use of Argo and GTSPP data –Traditional research, modeling, forecasting groups –Related disciplines, educational, administrative, public With recent advances in technology, we can do much more...

28 Distributed Object Technology Data servers and datasets are objects – software packages of procedures and data that contain their own context Solid, commercial underpinning for distributed object technology in the ocean sciences

29 Adaptability and Scalability of distributed object systems Distributed object systems in the commercial arena –Are robust, reducing system maintenance and upkeep costs –Supported by Object Management Group (OMB) standards body for Internet Inter-ORB Protocol (IIOP) distributed object protocols CORBA/IIOP and Java RMI/IIOP consortium of large (Fortune 500) companies Cross platform independence, compliance with standards

30 The Data Portal: a “gateway” to ocean data Why do we need a Data Portal? –Each center of data provides a highly customized Web sites for their data but different datasets have different navigation and interface characteristics so the user faces a bewildering spectrum of data access interfaces and locations Data Portal is single, uniform, consistent “gateway” to ocean data in a common format User goes to a single location and sees a consistent interface Complements the customized data access

31 Data Portal/Visualization/Collaboration Traditional users: Modelers Forecasters Researchers New users: Educators Students General Public Data & Information Users Distributed data Observed data Satellite data Data and information products Model outputs Visualization Uniform network access

32 Web Browser Java Application User NetworkNetwork CORBA* Client Support Java Servlet Graphics One or more Web Servers TAO data support CORBA* Data Observing System Server Data Common Object Request Broker Architecture (CORBA) is an industry standard Middleware. CORBA is used in the NOAAServer software from which this effort will leverage. Based on performance indicators, Java Remote Method Invocation (RMI), an alternative middleware, could easily be substituted for CORBA. CORBA* Network Data Server Data Portal

33 Web Browser Java Application User NetworkNetwork CORBA* Client Support Java Servlet Graphics One or more Web Servers Drifter Data support CORBA* Data TAO data support CORBA* Data Observing System Servers Data Common Object Request Broker Architecture (CORBA) is an industry standard Middleware. CORBA is used in the NOAAServer software from which this effort will leverage. Based on performance indicators, Java Remote Method Invocation (RMI), an alternative middleware, could easily be substituted for CORBA. CORBA* Network Data Servers Data Portal

34 Web Browser Java Application User NetworkNetwork CORBA* Client Support Java Servlet Graphics One or more Web Servers Drifter Data support CORBA* Data TAO data support CORBA* Data Observing System Servers In-Situ/Satellite data support CORBA* Data In-Situ/Satellite Data Servers Model data support CORBA* Data Model Output Servers Data Gridded data support CORBA* Data Gridded Data Servers Common Object Request Broker Architecture (CORBA) is an industry standard Middleware. CORBA is used in the NOAAServer software from which this effort will leverage. Based on performance indicators, Java Remote Method Invocation (RMI), an alternative middleware, could easily be substituted for CORBA. CORBA* Network Data Servers Data Portal

35 How do we build a Data Portal? Build on a proven prototype –connects 5 geographically distributed data servers in Silver Spring, Boulder, Seattle –CORBA for network connections –unified interactive Java graphics –data from distributed servers are co-plotted together on the same axis on the users desktop

36 Boulder CO Prototype Data Portal: CDP * Seattle WA Silver Spring MD Honolulu HI * Climate Data Portal

37 Climate Data Portal Sample Plots

38 Data Selection : Web Interface Utilizes CORBA for network connections. Utilizes EPIC Web Technology: –Java Applets –JavaScript –Java Servlets Searches data by keywords, location and time ranges.

39 Web Interface s creen Shots

40 Other Technologies for the Collaboratory: Networks (100 Megabits/sec today, 10 Gigabits/sec in future) –Next Generation Internet (NGI) and Internet 2 Visualization –Interactive Java graphics –3D, Virtual reality –Immersion technology Collaboration tools –high-speed telecommunications systems for advanced collaboration applications –tele-immersion systems allow individuals at different locations to share a single virtual environment –Use networks not airplanes for collaboration

41 Virtual Reality Virtual Reality lets the scientist touch the data, move into it, and see it from different viewpoints –The realism of virtual reality enables the scientist and the lay person to understand complex ideas more easily –Scientists using virtual reality affirm this new technology discloses features of their data and model outputs which were undiscovered with standard visualization techniques Virtual reality can be approachable and affordable Widens audience for scientific data and information –Government administrators and decision makers –Educators and students –General public Some examples follow… Courtesy of Nancy N. Soreides, PMEL

42 Why use Virtual Reality? Virtual reality modeling language (VRML) rendering of temperatures and sea surface topography along the equator in the tropical Pacific, viewed from South America, showing the dynamics of El Nino and La Nina. Using an inexpensive PC and a web browser with a free plug-in, the images can be rotated, animated, and zoomed. Changes in the equatorial Pacific during El Nino and La Nina are clearly understood by scientist and layman. or El Nino La Nina Courtesy of Nancy N. Soreides, PMEL

43 Stereographic Virtual Reality 3D, interactive virtual reality visualizations are not difficult for a scientist to create or to view, from the web or from the desktop, and the effect can be enhanced dramatically by including the capability of stereographic viewing. With a PC and a 99-cent pair of red/green sci-fi glasses, the spheres and vectors will pop out of the page in stereo, revealing the true 3D location of the fish, the steep slopes of the bathymetry, and the vertical motions near the submarine canyon. The images can be rotated, animated and zoomed. n/vrml/stereo.html Fish larvae and velocity vectors in a submarine canyon, from a circulation model of Pribolof Canyon in the Bering Sea. Use red/green glasses to see images on the right in stereo. Stereo Courtesy of Nancy N. Soreides, PMEL

44 Immersive devices provide the graphical illusion of being in a three- dimensional space by displaying visual output in 3D and stereo, and by allowing navigation through the space. Navigating through our virtual environments and viewing the data from different vantage points greatly increases our ability to perform analysis of scientific data. The impact of such visualizations in person is stunning, and must be experienced by the scientist to be fully comprehended. Users of these advanced immersion technologies affirm that no other techniques provide a similar sense of presence and insight into their datasets. Immersive Virtual Reality Courtesy of Nancy N. Soreides, PMEL

45 The CAVE The CAVE is a multi-person, high resolution, 3D graphics video and audio virtual environment. The size of a small room (10x10x10 foot), it consists of rear-projected screen walls and a front-projected floor. Using special "stereoscopic" glasses inside a CAVE, scientists are fully immersed in their data. Images appear to float in space, with the user free to "walk" around them, yet maintain a proper perspective. The CAVE was the first virtual reality technology to allow multiple users to immerse themselves fully in the same virtual environment at the same time. View of the CAVE Scientist inside the CAVE CAVES have been deployed in academia, government, and industry, including NASA, NCAR, NCSA, Argon National Laboratory, Caterpillar Corp., General Motors, among others. Courtesy of Nancy N. Soreides, PMEL

46 The ImmersaDesk Courtesy of Nancy N. Soreides, PMEL

47 The Future The Future “The development of scientific data manipulation and visualization capabilities requires an integrated systems approach … [including] the end- to-end flow of data from generation to storage to interactive visualization, and must support data retrieval, data mining, and sophisticated interactive presentation and navigation capabilities.” “Data Exploration of petabyte databases will required both technology development and altered work patterns for research scientists and engineers.”* * Data and Visualization Corridors, Report on the 1998 DVC Workshop Series, Edited by Paul H. Smith and John van Rosendale, Sponsored by the Department of Energy and the National Science Foundation, Courtesy of Nancy N. Soreides, PMEL

48