CyberInfrastructure and GIS at the National Science Foundation Dr. Jennifer M. Schopf Office of CyberInfrastructure National Science Foundation April 16, 2010

2 abstract Cyberinfrastructure is emerging as an infrastructure to transform the way we conduct research, development, and education in the 21st century. Multiple agencies have spent significant resources to advance different aspects of Cyberinfrastructure. This panel is organized to bring the latest development of Cyberinfrastructure within agencies to the AAG audience. 15 minutes presentation each followed by interactive q&a Service Oriented Architecture Earth observations and sensor webs Distributed geographic information processing Web computing (Web 2.0 and Web 3.0) Open and interoperable access technologies High Performance Computing (HPC) Spatial Data Infrastructure Open-source software and middleware Visualization and application needs Cross-domain sharing and collaborations Knowledge capture and utilization

3 National Science Foundation  Vision: Advancing discovery, innovation and education beyond the frontiers of current knowledge, and empowering future generations in science and engineering workforce  Independent agency within Executive Branch  Established 1950  Annual budget of over $7 billion  Funds basic research and education  Peer-review grant mechanism

4 NSF Strategic Goals  Discovery – Advancing frontiers of knowledge  Learning – S&E workforce and scientific literacy  Research Infrastructure – Advanced instrumentation and facilities  Stewardship – Supporting excellence in S&E research and education

Directorate for Biological Sciences (BIO) Office of Inspector General (OIG) Director Deputy Director National Science Board (NSB) Office of Equal Opportunity Programs (OEOP) Office of the General Counsel (OGC) Office of Legislative & Public Affairs (OLPA) Budget, Finance & Award Management (BFA) Office of Information & Resource Management (IRM) NSF Organization Directorate for Computer & Information Science & Engineering (CISE) Directorate for Education & Human Resources (EHR) Directorate for Engineering (ENG) Directorate for Geosciences (GEO) Directorate for Mathematical & Physical Sciences (MPS) Directorate for Social, Behavioral & Economic Sciences (SBE) Office of Cyberinfrastructure (OCI) Office of Integrative Activities (OIA) Office of International Science & Engineering (OISE) Office of Polar Programs (OPP)

6 Office of Cyberinfrastructure (OCI)  NSF Cyberinfrastructure includes  Supercomputers  Data management systems  High capacity networks  Digitally-enabled observatories and scientific instruments  An interoperable suite of software and middleware services and tools  Education programs to support future computational science  Foundation for CS&E across disciplines  Pragmatic, sustainable infrastructure

7  Common co-funding with:  BIO,CISE, ENG, EPSCoR, GEO, OISE and SBE

8 Co-Funding Partnerships within NSF  BIO  CISE  ENG  EPSCoR  GEO  OISE and  SBE

9 Cyberinfrastructure Framework for 21 st Century Science and Engineering (CF21)  High-end computation, data, visualization for transformative science  Facilities/centers as hubs of innovation  MREFCs and collaborations including large-scale NSF collaborative facilities, international partners  Software, tools, science applications, and VOs critical to science, integrally connected to instruments  Campuses fundamentally linked end-to-end; grids, clouds, loosely coupled campus services, policy to support  People Comprehensive approach workforce development for 21st century science and engineering 9

10 Expertise Research and Scholarship Education Learning and Workforce Development Interoperability and ops Cyberscience Organizations Universities, schools Government labs, agencies Research and Med Centers Libraries, Museums Virtual Organizations Communities Networking Campus, national, international networks Research and exp networks End-to-end throughput Cybersecurity Computational Resources Supercomputers Clouds, Grids, Clusters Visualization Compute services Data Centers Data Databases, Data reps, Collections and Libs Data Access; stor., nav mgmt, mining tools, curation Scientific Instruments Large Facilities, MREFCs,telescopes Colliders, shake Tables Sensor Arrays - Ocean, env’t, weather, buildings, climate. etc Software Applications, middleware Software dev’t & support Cybersecurity: access, authorization, authen. Sustain, Advance, Experiment NSF CyberInfrastructure Ecosystem Discovery Collaboration Education

11 CF21 Software Infrastructure for Sustained Innovation (SI2)  Significant multiscale, long-term software program  Perhaps $ M over a decade $10M identified in FY10 ($4M OCI/$6M Dirs) $14M annual in OCI in future years –Catalyze significant funds from Dirs  Sustain: Connected institutes, teams, investigators  Integrated into CF21 framework w/Dirs  3-6 centers, 5+5 years, for critical mass, sustainability  Advance: Numerous teams of scientists and computational and computer scientists with longer term grants  Experiment: Many individuals w/short term grants, funded by OCI and directorates 11

12 Software, continued  Ongoing discussions to build this program across NSF  Some of the institutes will be discipline specific  Some may be algorithm/tool themed (e.g., data, provenance, viz)  All should be fundamental to other programs (e.g., SEES)  Education, science applications, industrial partners linked deeply  MREFC’s, other large facilities need to participate  iPlant, NEON, LSST, etc… 12

13 SDCI: Software Development for Cyberinfrastructure (FY07)  HPC, Data, and Middleware target areas defined  2 types of proposals  New dev’t – show compelling case for new software dev  Improvement and Support - original software must have a track record of use and impact  Required characteristics for proposals  Multiple application areas and expected usage  Awareness/distinction among alternatives  Project plan with proof-of-concept and metrics  Open source and use of NMI Build and Test facility  Demonstration in first 2 years

14 SDCI: Software Development for Cyberinfrastructure (FY07)  HPC, Data, and Middleware target areas defined  2 types of proposals  New dev’t – show compelling case for new software dev  Improvement and Support - original software must have a track record of use and impact  Required characteristics for proposals  Multiple application areas and expected usage  Awareness/distinction among alternatives  Project plan with proof-of-concept and metrics  Open source and use of NMI Build and Test facility  Demonstration in first 2 years

15 Open Source  Requirement for all current OCI programs  And many others across NSF  Strongly encourages reuse  Some people think simply open source is enough – it’s not  Necessary but not sufficient for sustainable software 15

16 Open Source software is free… Free as in speech…free as in beer, or…

17 v  Seems like a great bargain  Easy to access  Can catch you eye at a weak moment  …but sometimes more than you expected Open Source Software is Like a Free Puppy

18  Long term costs  Needs love and attention  May lose charm after growing up  Occasional clean-ups required  Many left abandoned by their owners  May not be quite what you think

19 Strategic Technologies for Cyberinfrastructure: STCI  Support work leading to the development and/or demonstration of innovative cyberinfrastructure services  Fill gaps left by more targeted funding opportunities  Projects outside the scope of targeted solicitations  Can be used for sustainability funding

20 STCI  Very broad-reaching program  Emphasize the broad applicability of the work  Identify current and prospective end users  Include supporting material workshop reports, NSB recommendations, etc  Explain why your proposed work cannot be funded under other current NSF programs  Target dates twice a year  August and February

21 Enormous, Irreplaceable Data Sets ~ 150 TB/Year ~ 30 TB/Night ~ 15 PB/Year ~ 64 TB/Year ~ 40 TB/Year Long tail of small science ?? TB/Year

22 DataNet: Sustainable Digital Data Preservation and Access Network Partners Program  Four primary goals:  Provide reliable digital preservation, access, integration, and analysis capabilities for science/engineering data over decades-long timeline  Achieve long-term preservation and access capability in an environment of rapid technology advances  Create systems and services that are economically and technologically sustainable  Empower science-driven information integration capability on the foundation of a reliable data preservation network  Each project needed to develop a model for shared governance and the standards and protocols to enable interoperability

23 Data Programs  DataNet: OCI Flagship Data Program  Focus on data-level interoperability and data preservation  Sustain: 5 Centers, $20M, 5years (+5)  Advance: eg. SDCI awards  ~3-4 year, $1-2M, support of data tools for broad set of applications and disciplines  Experiment: eg. InterOp awards  Smaller scale, innovative use of data for new communities 23

DataNet Awards  DataNet Observation Network for Earth (PI: Michener)  Facilitates research on climate change and biodiversity, integrating earth observing networks  Emphasis on user community engagement, promote data deposition and re-use  Science question: What are the relationships among population density, atmospheric nitrogen, CO2, energy consumption and global temps?  Data Conservancy (PI: Choudhury)  Integrates observational data to enable scientists to identify causal and critical relationships in physical, biological, ecological, and social systems  User centered design paradigm, ethnographic studies  Science question: How do land and energy use in mega-cities impact the carbon cycle and climate change?

25 Planned CF21 HPC Program  Sustain: Petascale-to-Exascale  1-2 Sustainable facilities (~$200M+)  Likely NSF-DOE cooperation  10 years (5+5)  Advance  4-5 hubs of Excellence/Innovation, people, expertise  Mixture of data and compute-intensive centers, supporting broader array of services  Experiment  Explore new architectures, couple with application/software dev 25 UIUC Petascale Facility: $60M building!

26 HPC Will Also Need  Discipline specific connections  MRI, Divisional, Directorate programs can be aligned to connect in to this NSF-wide structure Recommended common software, identity management, policy Data, software sharing 26

27 HPC Will Also Need  Discipline specific connections  MRI, Divisional, Directorate programs can be aligned to connect in to this NSF-wide structure Recommended common software, identity management, policy Data, software sharing  How does eXtreme Digital (XD), TeraGrid Phase 3 fit in?  Competition underway now  Foundation to build broader CF21 services in future 27

28 Education, Learning, Workforce Development  Postdoc program: CITracs  Emphasis on helping computational scientists learn about CI or vice versa   CI-TEAM: Training, Education, Advancement, and Mentoring for Our 21st Century WF  Prepare current and future generations of scientists, engineers, and educators  Design, develop, adopt and deploy cyber-based tools and environments for research and learning, both formal and informal  28

29 GIS at NSF: A simple search reveals awards in  Social, Behavioral and Economic Sciences  Geography and Spatial Sciences  Science, Technology, & Society  Mapping endangered languages  Archaeology  Biology  Office of Polar Programs  Geology (Oceanography)  CISE/OCI  Education and Human Resources  EPSCoR

30 PI David Schimel NEON: National Ecological Observatory Network New horizons for large-scale biology

31 NEON  Integrated sensing system to detect, understand, and forecast  Examining consequences of climate and landuse change and the effects of invasive species  Both regional and continental scales  Neon CI:  Design, implement and support CI to adapts to and enhances NEON’s evolving scientific objectives and operational needs

32 Suites of Integrated-Colocated-Standardized observations and experiments to understand responses of ecosystems to climate change  20 core wildland sites  40 relocatable sites (mostly on mngd land)  36 aquatic sites (all but 2 co-located)  Including 10 aquatic experiment sites  3 Airborne remote sensing systems  542 Level 1 (primary) observations: raw data calibrated into physical, biological or chemical units  118 Level 4 (algorithmic) continental-scale data products  178 Terabytes of data/year, total

33 NEON CI Components  Management system to acquire, store, and manage samples  Heavily instrumented sensor platforms and arrays  External data feeds from airborne systems  External data integration of non-NEON observational data  Integration of chemical, isotopic and genetic analysis data

34 Data Volumes  1,000+ sensors per domain actively monitored, reported and recorded  20+ GBytes/year acquired and transmitted by In situ Sensors representing over 600 BILLION data points  Airborne Remote Sensing data volumes 36 TB per year and complex processing  70,000 discrete measurement streams  200 sensor measurement types (ex: PRT:AirTemp)  Highly Heterogeneous  Data Volumes per year  640 x 109 In situ Sensors: data points per year  22 x 109 In situ Sensors: sensor raw data  113 x 109 In situ Sensors: LiDAR raw data  36 x 109 Airborne Remote Sensing: raw data  ~ 106 Biological Monitoring and Measurements: observations

35 Expertise Research and Scholarship Education Learning and Workforce Development Interoperability and ops Cyberscience Organizations Universities, schools Government labs, agencies Research and Med Centers Libraries, Museums Virtual Organizations Communities Networking Campus, national, international networks Research and exp networks End-to-end throughput Cybersecurity Computational Resources Supercomputers Clouds, Grids, Clusters Visualization Compute services Data Centers Data Databases, Data reps, Collections and Libs Data Access; stor., nav mgmt, mining tools, curation Scientific Instruments Large Facilities, MREFCs,telescopes Colliders, shake Tables Sensor Arrays - Ocean, env’t, weather, buildings, climate. etc Software Applications, middleware Software dev’t & support Cybersecurity: access, authorization, authen. Sustain, Advance, Experiment CyberInfrastructure Ecosystem Discovery Collaboration Education

36 More Information  Jennifer M. Schopf    Dear Colleague letter for CF21