FASR-A 6 Gb/s x 2 x 100 0.1 GB/s 400 GB/day 50 GB/day ~1 GB/day X xxx FASR Data Analysis Plan 22-May-2005 Simplified Data Pipeline.

Slides:

Advertisements

Similar presentations

ASKAP Status Update CSIRO ASTRONOMY AND SPACE SCIENCE David Brodrick EPICS [Spring] Collaboration Meeting 22nd October 2014.

Advertisements

Title: ICON L0 from MOC to SOC automation Author: MOC/SOC Resides: SOC Description: This is a script that copies the ICON L0 files from the MOC to the.

OVSA Expansion Software Overview Gordon Hurford Kickoff Meeting NJIT 25-Oct-2010.

Dale E. Gary Professor, Physics, Center for Solar-Terrestrial Research New Jersey Institute of Technology 1 3/16/2012OVSA Preliminary Design Review.

For Mapping Biodiversity Data Data Management Options.

EOVSA Data Volume / Data Rates Gordon Hurford Nov 8, 2011 EOVSA Technical Design Meeting - NJIT.

Page 1JSOC Peer Review – AIA Science Center – March 17, 2005 AIA Science Data Processing Infrastructure Neal Hurlburt AIA Data Scientist

FASR Calibration and Data Management Issues Gordon Hurford Space Sciences Lab University of California, Berkeley 9 Feb 2006 FASR Development Meeting.

GLAST LAT ProjectISOC Peer Review - March 2, 2004 Document: LAT-PR Section 2.1 Requirements 1 Gamma-ray Large Area Space Telescope GLAST Large.

GLAST LAT ProjectISOC CDR, 4 August 2004 Document: LAT-PR-04500Section 21 GLAST Large Area Telescope: Instrument Science Operations Center CDR Section.

OVSA Expansion Pipeline Imaging. Log-spiral array: “uv” distribution Left: sampling of Right: inner region of spatial Fourier plane sampled spatial Fourier.

Dissemination of Haze Data, Data Products and Information Bret Schichtel, Rodger Ames, Shawn McClure and Doug Fox.

Structure Commander Technical Presentation. Copyright (C) MCS 2013, All rights reserved. 2 STRUCTURE COMMANDER Introduction Product Overview.

Advantages of using R.PC.R Software Now that you know some of the advantage let’s take a look at the software … Reliable and Comprehensive Reporting Recording.

Obtaining MISR Data and Information Jeff Walter Atmospheric Science Data Center April 17, 2009.

LIGO-G E ITR 2003 DMT Sub-Project John G. Zweizig LIGO/Caltech Argonne, May 10, 2004.

Hunt for Molecules, Paris, 2005-Sep-20 Software Development for ALMA Robert LUCAS IRAM Grenoble France.

 Interactive programs with a User Interface  Available 24-7, on the national FIA Web site  Easy access to the national FIA database Generate Reports.

Dale E. Gary Professor, Physics, Center for Solar-Terrestrial Research New Jersey Institute of Technology 1 03/15/2012Preliminary Design Review.

Some Thoughts on HPC in Natural Language Engineering Steven Bird University of Melbourne & University of Pennsylvania.

U.S. Department of the Interior U.S. Geological Survey Diving into the Data Pool with DAAC2Disk Kelly Lemig ERT, Inc., contractor to the U.S. Geological.

Pi In The Sky (Web Interface) Gaston Seneza Philander Smith College, Little Rock, AR SIParCS Intern Mentors: Dr. Richard Loft & Dr. Raghu Raj Kumar 1.

Web GIS Commander Technical Presentation. Copyright (C) MCS 2013, All rights reserved. 2 MCS Web GIS Commander is Web-based GIS (Geographical.

IVC : a simulation and model-fitting tool for optical-IR interferometry Abstract I present a new software tool, called “Interferometry Visibility Computations”

Astronomical Spectroscopy and the Virtual Observatory ESAC, March 2007 VO tools and cross-calibration Pedro García-Lario European Space Astronomy.

ALMA Software B.E. Glendenning (NRAO). 2 ALMA “High Frequency VLA” in Chile Presently a European/North American Project –Japan is almost certainly joining.

Rings Node Mitch Gordon Mark Showalter MC Face-to-Face Meeting Westwood, California 26,27 August 2014.

A Web-Enabled Aircraft Scheduler Michael Wallette 20 Nov

Low Latency Data Visualisation Solar Orbiter SOC SOWG #5, ESAC, 8-10 July 2014.

EÖTVÖS UNIVERSITY BUDAPEST Department of Physics of Complex Systems VO Spectroscopy Workshop, ESAC Spectrum Services 2007 László Dobos (ELTE)

EOVSA Data and Database System Gordon Hurford and Jim McTiernan EOVSA Prototype Readiness Review 24-Sept-2012.

F. Levrier (LERMA - ENS) J. Pety (LERMA - IRAM Grenoble) P. Salomé (LERMA - Observatoire de Paris) STAR FORMAT meeting, Heidelberg, 4-5 may 2010 Making.

Microgravity Analysis Software System (MASS) 2002Page 1 Microgravity Analysis Software System (MASS)

Jos de Bruijne Relativity and Reference Frame Working Group Meeting ESTEC, June 2004 Gaia Parameter Database

SH43A-16 The Green Bank Solar Radio Burst Spectrometer T. S. Bastian National Radio Astronomy Observatory 520 Edgemont.

This dataset contains calibrated, to 4.8-micron spectral images of comet 103/P Hartley 2 acquired by the High Resolution Infrared Spectrometer (HRII)

The GONG Farside Project (Status) Irene González Hernández and the GONG Farside team.

● Radio telescope arrays – km diameter – Resolution arcmin to micro-arcsec at radio wavelengths ● Similar (baseline/ wavelength) for other regimes.

FASR Software Considerations Gordon Hurford SSL AUI – August 2007.

EOVSA Data and Database System J. McTiernan EOVSA CoDR 25-April-2011.

Sea Ice Mapping Systems Archive Browser Interface Distribution IngestProduction Ice Analyst Application Database Henrik Steen Anderson DMI Virtual System.

DataMaster 3000 Data Acquisition and Control Software.

14 June, 2004 EVLA Overall Design Subsystems II Tom Morgan 1 EVLA Overall Software Design Final Internal Review Subsystems II by Tom Morgan.

06-1L ASTRO-E2 ASTRO-E2 User Group - 14 February, 2005 Astro-E2 Archive Lorella Angelini/HEASARC.

ADMIT: ALMA Data Mining Toolkit  Developed by University of Maryland, University of Illinois, and NRAO (PI: L. Mundy)  Goal: First-view science data.

Pipeline Basics Jared Crossley NRAO NRAO. What is a data pipeline?  One or more programs that perform a task with reduced user interaction.  May be.

LaHave House Project 1 LaHave House Project Automated Architectural Design BML + ARC.

Digital Packaging Processor - Overview Gordon Hurford Nov 7, 2011 EOVSA Technical Design Meeting - NJIT.

PDS Geosciences Node Page 1 Archiving LCROSS Ground Observation Data in the Planetary Data System Edward Guinness and Susan Slavney PDS Geosciences Node.

EPICS Development for the ASKAP Design Enhancements Program ASTRONOMY AND SPACE SCIENCE Craig Haskins 18 th October 2015 EPICS User Meeting – Melbourne.

INFSOM-RI The Reporting Engine Paolo Fabriani Engineering Ingegneria Informatica S.p.A. 2 nd ETICS All Hands Meeting Bologna –

EOVSA Pipeline Processing System J. McTiernan EOVSA Prototype Review 24-Sep-2012.

ADMIT: ALMA Data Mining Toolkit  Developed by University of Maryland, University of Illinois, and NRAO (PI: L. Mundy)  Goal: First-view science data.

From Missions to Measurements: an Ocean Discipline Experience.

EOVSA Data and Database System J. McTiernan EOVSA Technical DesignMeeting 7-Nov-2011.

ECHO Technical Interchange Meeting 2013 Timothy Goff 1 Raytheon EED Program | ECHO Technical Interchange 2013.

Transform® for Advanced Document Process Automation Andy Barnett-Picking K3BS AX Development Manager.

Science Gateway- 13 th May Science Gateway Use Cases/Interfaces D. Sanchez, N. Neyroud.

CCSAS V2 Impacts on Business and Legal Processes October 4, 2006.

Metadata for the SKA - Niruj Mohan Ramanujam, NCRA.

PLASTIC Post Launch Data Flow Lynn Kistler & Lorna Ellis May 2006.

Digital Light Sources First introduced in 2001.

NRAO VLA Archive Survey

Archive, Data Products, & Operations for the Murchison Widefield Array

Intermountain West Data Warehouse

Virtual Ice Charting System

Gustaaf van Moorsel September 9, 2003

Dynamics 365 Platform Advantages for IT Hubdrive.com.

ABB Ability™ Data collection for safety management – Safety App

Data search & distribution

Presentation transcript:

FASR-A 6 Gb/s x 2 x GB/s 400 GB/day 50 GB/day ~1 GB/day X xxx FASR Data Analysis Plan 22-May-2005 Simplified Data Pipeline

FASR-A 6 Gb/s x 2 x GB/s 400 GB/day 50 GB/day ~1 GB/day X xxx Simplified Data Pipeline Digital Data Processing Real-time conversion of broadband RF from 100 antennas to semi-calibrated visibilities Implemented with DSPs No scientific compromise in time- or frequency-resolution Output data rate is 0.1 Gbytes/sec

FASR-A 6 Gb/s x 2 x GB/s 400 GB/day 50 GB/day ~1 GB/day X xxx Simplified Data Pipeline Interim Database Semi-calibrated visibilities, housekeeping data, etc Time and frequency resolution are scientifically uncompromised 400 Gbytes / day Transient – designed to be overwritten every hrs. TBD -- offline archiving ?

FASR-A 6 Gb/s x 2 x GB/s 400 GB/day 50 GB/day ~1 GB/day X xxx Simplified Data Pipeline Interim  Application Databases Magnetography bursts general purpose etc FREQUENCY TIME

FASR-A 6 Gb/s x 2 x GB/s 400 GB/day 50 GB/day ~1 GB/day X xxx Application Databases Set of parallel databases for magnetography, bursts, etc Time and frequency resolution tailored to specific applications Fully calibrated Starting point for generation of high level data products ~50 Gbytes / day Simplified Data Pipeline

FASR-A 6 Gb/s x 2 x GB/s 400 GB/day 50 GB/day ~1 GB/day X xxx High Level Data Products Includes light curves, dynamic spectra, maps, etc Also includes parametric maps (eg coronal magnetograms, temperature maps, etc derived from imaging spectroscopy) Common set of software tools for: Quick look (latency of minutes) Summary data products with automatically-determined parameters (at end of day) Custom products In response to user-specified parameters OR analysis at user’s site Simplified Data Pipeline

FASR-A 6 Gb/s x 2 x GB/s 400 GB/day 50 GB/day ~1 GB/day X xxx User Interface Details TBD Will support: Browsing of high level data products Automated and interactive downloading of high level data products Generation of user-specified high level data products Downloading of user-selected subsets of application databases No user software requirements (except when user chooses to create custom high level data products offsite) Simplified Data Pipeline