What the Data Products Program Offers Users Todd Boroson Dick Shaw Presentation to NOAO Users Committee October 23, 2003

Data Products Program – Mission/Vision Shift away from low value-added software to reduce data Shift towards providing calibrated data products, catalogs, services Full commitment to enabling public access to NOAO data; place great value on legacy data products for new science Full involvement with international efforts to standardize data access, products, and services

Data Products Program – Overview Resources –Evolved from IRAF group – currently 12 programmers, 5 scientists –Developing relationships with supercomputer centers –Outside projects fund activities not directly supporting core mission Structure/process –Group split between Tucson and La Serena –Project scientist/lead programmer –Formal management w/ schedules, plans, reviews Guidelines –Priorities set by scientific oversight, community interaction –Integrated program, external commitments also affect priorities –Ultimate goal of supporting access to LSST data and tools has stepping stones of other large f-p arrays (NEWFIRM, ODI)

Components of DPP Archive –“User Interface” for community to access NOAO data products –Must be extensible to LSST era –Vision is workspace/tools/NVO-compliance Data Reduction Pipelines –Gain experience with pipeline for Mosaic imagers –Incorporate time-domain capabilities –Develop further for NEWFIRM Data Reduction Tools (incl. Gemini) –Collaborative project with Gemini –These serve as modules for pipelines IRAF System –Improvements evaluated in terms of supporting DPP infrastructure, providing support for pipelines, and making community more self- sufficient –Movement toward more “open” model –System improvements for better scripting environment

More components of DPP Data Transport System –“glue” that makes system approach possible NVO –Participate in development of international standards for data discovery and dissemination. –Provide tools to support data discovery and visualization with emphasis on data we are providing New analysis tools –Link with NSA, NVO Work on LSST –Most interested in ensuring community access to data and the services necessary to support community research

Evolution of DPP Capabilities Obtain survey data products through NSA Pipelines available to assist observers All raw data available through archive –Immediately to observers (replace STB) –After proprietary period to community Reduced data available through archive Data access and tools for other/new big surveys & instruments –MACHO/SuperMACHO –NEWFIRM/ODI/? –Pan-STARRS –LSST

Issues for Users Difficulties of calibration and data quality for ground-based O/IR data –Driven by science objectives for archived data –One strategy is to motivate through promise of archive, pipeline –Another is instrument support for prescribed protocols only –Should ultimate usefulness of data in archive be a criterion for TAC consideration? Management of proprietary periods –18 month default; exceptions? Ongoing community interaction/guidance for DPP