DCS Overview MCS/DCS Technical Interchange Meeting August, 2000

RIT Center for Imaging Science SOFIA Data Cycle System: Overview The “Mission” “The DCS shall provide the infrastructure to support the operation of facility instruments on SOFIA using best current knowledge and tools, and supporting continuous improvement in an efficient, extensible, and modular architecture.” -- DCS CoDR, June 1999.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Requirements “The recommendations of the CoDR committee are the top level requirements for the DCS.” -- DCS PDR, March 2000.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Requirements The DCS shall record and archive all data necessary for complete analysis, calibration and future access to observational data obtained with the FSIs. The DCS shall support a select number of observing modes with pipeline reduction, including quicklook analysis, for each of the FSIs.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Requirements The DCS shall provide on-line documentation necessary for the GI to propose and carry out observations with the FSIs. The DCS shall be implemented as a system of distributed modules, integrated by an extensible hardware and software infrastructure.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Requirements Functionality mandated by SOFIA leaders covers two functional domains: SOFIA: Data acquisition from live instrument, fast in-flight reduction (“quicklook”). SSMOC: Data acquisition from simulators and other external sources, full reduction of FSI data.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Requirements Goal: Avoid building two separate systems if at all possible. Observation: The functional role is the same, only the algorithms and equipment varies. Solution: Two DCS systems, one in SOFIA and one at SSMOC. Each instance of the DCS has the pipelines and interfaces needed for its environment. The core DCS is identical and invariant.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Design Goals “The DCS must support and benefit from continuous improvement, both in itself and within the SOFIA program, over a twenty year lifetime.” -- DCS PDR, March 2000.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Design Goals The DCS design must be: –Modular –Extensible –Maintainable This is the heart of “continuous improvement.” -- DCS PDR, March 2000.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Design Goals: Modular The DCS must not be a monolithic program. The DCS must not be limited to a single system. The DCS shall be a collection of small independent services, residing on one or more machines, providing functionality “as needed”.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Design Goals: Extensible The DCS must support the easy incorporation of new procedures and techniques to its repertoire. The DCS will provide for the safe test and evaluation of new components, while maintaining access to established “proven components”.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Design Goals: Maintainable The DCS must not be tied to any specific vendor or platform. The DCS must be based on open, widely available standards and technologies. The DCS must be well documented, both in design and implementation.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Base Technologies Technologies on which the DCS system is based: Distribution of and communication between objects across a system. (CORBA) Extensible and flexible exchange of information, both in data and documentation. (XML)

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Subsystems User Interaction Layer Data Acquisition Data Reduction DCS Storage Task Library

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Subsystems: User Interaction Common interface to all DCS resources The “DCS experience” is customizable on a per user basis without affecting the rest of the DCS Leverages off the web and related tools –Access regardless of geographic location –Relies only on commonly available technology at user’s end (e.g. web browser)

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Subsystems: User Interaction User Interaction Layer Data Acquisition Data Reduction DCS Storage Task Library User profile User Proxy Requests and Responses

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Subsystems: Task Library Home to all core functionality on user’s behalf. Can build on simple tasks to provide sophisticated sequences of actions. Easily extended with new activities and procedures. “Once you know how to do it, we can automate it.”

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Subsystems: Task Library User Interaction Layer Data Acquisition Data Reduction DCS Storage Task Library User Proxy Requests and Responses Storage access is unique to task’s needs Initiate and monitor data reduction Initiate, monitor, and control data acquisition

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Subsystems: Data Store Includes retrieval and storage of raw science data, but also “much more”. Captures “everything” (e.g. raw FSI data, reduced data, OBS plans, instrument modes, pipeline parameters, science personnel). Enables easy import & export in variety of formats (e.g. general investigators, IPAC) by being platform neutral.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Subsystems: Data Store User Interaction Layer Data Acquisition Data Reduction DCS Storage Task Library Storage access is unique to task’s needs User profile Obtain OBS plan, store raw FSI data Obtain raw FSI data, store reduced data

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Subsystems: Data Acquisition Data acquisition, like the rest of the DCS, is composed of small modules working in concert together. Modularity insulates the DCS from instrument specifics, and encourages re- use between similar instruments. Effectively, the DCS “translates” an experiment to instrument specific commands and needs.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Subsystems: Data Acquisition User Interaction Layer Data Acquisition Data Reduction DCS Storage Task Library Obtain OBS plan, store raw FSI data Initiate, monitor, and control data acquisition quick look

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Subsystems: Data Reduction Instrument science teams focus on algorithm development, not the DCS. Reduction is automatic, distributed, pipelined, and relentless. DCS Data Reduction removes the need for every GI to have their own compute engines, but does not prevent GI from doing their own reduction.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Subsystems: Data Reduction User Interaction Layer Data Acquisition Data Reduction DCS Storage Task Library Obtain raw FSI data, store reduced data Initiate and monitor data reduction

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Subsystems User Interaction Layer Data Acquisition Data Reduction DCS Storage Task Library

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Technologies Built on top of the COTS technologies CORBA and XML, the DCS team is developing four new technologies that enable the overall DCS: –DCS Registry –Data Typing –Data Connection Association Object (DCAO) –XML Parser Generator

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Technologies: DCS Registry Enables various modules in the DCS to locate each other based on combinations of names, versions, and other criteria. Allows simultaneous, non-interfering deployment of modules with the same functionality: –Modules that are proven and trusted –Modules undergoing evaluation and test

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Technologies: Data Typing Provides a “live” dictionary for use in the DCS that defines various data types and formats. Best explained by example: data typing allows objects to interact with each other using “rawFrameAIRES” instead of “IEEEsingle[1024][1024]”. More expressive than IDL; addresses more portability and platform issues.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Technologies: DCAO Provides common language, interface, and linkages between groups of objects that must locate and communicate with each other at runtime. Directly enables the Data Acquisition and Data Reduction subsystems, both of which rely heavily on dynamic collections of objects.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Technologies: Parser Generator Extends existing event-driven XML parser (expat) by generating the support code normally written by engineers to use a particular DTD. “Software writing software”, that ultimately yields a parsed DOM-like object for any given combination of XML document and DTD.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview DCS Rundown Subsystems –User Interaction –Task Layer –Data Store –Data Acquisition –Data Reduction Technologies –Registry –Data Typing –DCAO –XML Parser Generator