1 The WFC3 Quicklook Project 17 January 2013. 2 What is Quicklook? The WFC3 Quicklook project is a complete archive of all ~90k in-flight WFC3 images.

Slides:



Advertisements
Similar presentations
Implementing Tableau Server in an Enterprise Environment
Advertisements

Contribution to MD9 Viktor Pusztai Ministry For Environment and Water GRID-Budapest CEOS WGISS meeting 17 September 2003 Thailand - Chiang Mai.
© 2012 Entrinsik, Inc. Informer Administration Exploring the system menu and functions PRESENTER: Jason Vorenkamp| Informer Software Engineer| March 2012.
Organize Workplace Information
WEEK THREE MANAGING, FINDING AND SELECTING INFORMATION- PART 2 Functional ICT.
Dec 2, 2014 The HST Online Cache: Changes to the User Experience Karen Levay Faith Abney Mark Kyprianou.
11/08/2001 FSW/GS Splinter Meeting 3 rd FSW/GS Splinter November 8, Ball COS WFC3 (morning session, auditorium) O. Lupie, WFC3 STScI Review of Goals.
SOFTWARE PRESENTATION ODMS (OPEN SOURCE DOCUMENT MANAGEMENT SYSTEM)
Software to Manage EEP Vegetation Plot Data A design proposal Michael Lee January 31, 2011.
Copyright © 2006 Software Quality Research Laboratory DANSE Software Quality Assurance Tom Swain Software Quality Research Laboratory University of Tennessee.
Explore the Dreamweaver Workspace View a Web page and use Help Plan and Define a Web site Add a Folder and Pages, and set the Home page Create and View.
Groupware to Support Distributed & Collocated Software Engineering Student Group Projects Sarah Drummond RISE Dept. Computer Science University of Durham.
Presented by Mina Haratiannezhadi 1.  publishing, editing and modifying content  maintenance  central interface  manage workflows 2.
Introduction to the Java Stack Michael Youngstrom.
Simple Web SQLite Manager/Form/Report
© 2013 Jones and Bartlett Learning, LLC, an Ascend Learning Company All rights reserved. Security Strategies in Linux Platforms and.
By Jeerarat Boonyanit. As you can see I have chosen Cpanel for my server management tool. cPanel is a Linux based web hosting control panel that provides.
October 30, 2008 Extensible Workflow Management for Simmod ESUG32, Frankfurt, Oct 30, 2008 Alexander Scharnweber (DLR) October 30, 2008 Slide 1 > Extensible.
Digitized Sky Survey Update Brian McLean : Archive Sciences Branch / Operations and Engineering Division.
Data Management Subsystem: Data Processing, Calibration and Archive Systems for JWST with implications for HST Gretchen Greene & Perry Greenfield.
Proposalz Generator Plus July Proposalz Generator Plus
Promoting Open Source Software Through Cloud Deployment: Library à la Carte, Heroku, and OSU Michael B. Klein Digital Applications Librarian
How to Use the Trilinos Infrastructure for Your Own Project James M. Willenbring Michael N. Phenow Trilinos User Group Wednesday, November 2nd, :00.
CAA/CFA Review | Andrea Laruelo | ESTEC | May CFA Development Status CAA/CFA Review ESTEC, May 19 th 2011 European Space AgencyAndrea Laruelo.
Subversion, an Open Source Version Control System An Introduction.
Data management in the field Ari Haukijärvi 2nd EHES training seminar.
The Lifecycle of Embedded Image Metadata within Digital Photographs: Challenges and Best Practices. - or - The Secret Life of Photo Metadata To promote,
1 Wide Field Camera 3 TIPS 18 September WFC3 Inserted in SM4 (2009)
1 XML as a preservation strategy Experiences with the DiVA document format Eva Müller, Uwe Klosa Electronic Publishing Centre Uppsala University Library,
Geospatial Technical Support Module 2 California Department of Water Resources Geospatial Technical Support Module 2 Architecture overview and Data Promotion.
1 STScI Pipeline Thermal-vacuum data handling I. Neill Reid for WFC3 Instrument CALibration team ICAL.
Configuration Management (CM)
1 TenStep Project Management Process ™ PM00.8 PM00.8 Project Management Preparation for Success * Manage Documents *
MICHAEL P. FINN & HOLLY K. CARO The CEGIS Online Bibliography: A Resource for the Working GIScientist U. S. Department of Interior U. S. Geological Survey.
We have developed a GUI-based user interface for Chandra data processing automation, data quality evaluation, and control of the system. This system, known.
Automation Automation & Drives Business Unit Smart Platform.
CERN-PH-SFT-SPI August Ernesto Rivera Contents Context Automation Results To Do…
Tweets Metadata May 4, 2015 CS Multimedia, Hypertext and Information Access Department of Computer Science Virginia Polytechnic Institute and State.
Near Real-Time Verification At The Forecast Systems Laboratory: An Operational Perspective Michael P. Kay (CIRES/FSL/NOAA) Jennifer L. Mahoney (FSL/NOAA)
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 4 Slide 1 Slide 1 Use Case Packets.
Unit 17: SDLC. Systems Development Life Cycle Five Major Phases Plus Documentation throughout Plus Evaluation…
Module 6: Configuring User Environments Using Group Policies.
A Tripal based Arthropod genome portal The i5k A Tripal based Arthropod genome portal Christopher Childers USDA/ARS/NAL i5k.nal.usda.gov.
Improved ACS Geometrical Distortion Correction Richard Hook TIPS Meeting, STScI, 18th December 2003.
CHAPTER 9 File Storage Shared Preferences SQLite.
 Project Team: Suzana Vaserman David Fleish Moran Zafir Tzvika Stein  Academic adviser: Dr. Mayer Goldberg  Technical adviser: Mr. Guy Wiener.
 systemD  FirewallD  Network manager (NMCLI)  Target CLI (iscsi targets)  GRUB 2 (Booting process)  Network teamnig & bridging.
Sea Ice Mapping Systems Archive Browser Interface Distribution IngestProduction Ice Analyst Application Database Henrik Steen AndersonDMI Paul SeymourNIC.
WELCOME TO THE ARE YOU IN? HI-TECH REVOLUTION ARE YOU IN?
Adaptive Software Development Process Framework. Version / 21 / 2001Page Project Initiation 2.0 Adaptive Cycle Planning 5.0 Final Q/A and.
MirrorManager: The Fedora Mirror System Matt Domsch Fedora Mirror Wrangler Linux Technology Strategist Office of the CTO Dell, Inc.
LCLS Commissioning & Operations High Level Software
Architecture Review 10/11/2004
Development Environment
Building A Web-based University Archive
ODF, LibreOffice and Red Hat
Maintaining software solutions
Software Version System Part1: Subversion at CERN
LCLS Commissioning & Operations High Level Software
X in [Integration, Delivery, Deployment]
ARTIFICIAL INTELLIGENCE IN SOFTWARE TESTING
What's New in eCognition 9
Helping a friend out Guidelines for better software
Background Prepared by: Mr. Mahmoud Rafeek Alfarra.
Automation of Control System Configuration TAC 18
September 12-14, 2018 Raleigh, NC.
CAD DESK PRIMAVERA PRESENTATION.
What's New in eCognition 9
Overview of Computer system
Bethesda Cybersecurity Club
Presentation transcript:

1 The WFC3 Quicklook Project 17 January 2013

2 What is Quicklook? The WFC3 Quicklook project is a complete archive of all ~90k in-flight WFC3 images. It's includes a file system of FITS files and JPEG preview images all mapped to a database. QL is maintained by a pure-Python code base.

3 What Does QL Give WFC3? Daily human inspection of all new WFC3 data. Immediate access to the entire WFC3 in-flight dataset. An interface to build highly customized datasets. Tools and data for exciting new projects.

4 How do we use Quicklook? Large datasets: Build the persistence correction products (Knox Long) Monitor the WFC3 “blobs” (Nor Pirzkal) IR flat fields (Nor Pirzkal) UVIS backgrounds (Sylvia Baggett, Jay Anderson)

5 How do we use Quicklook? Immediate Accessibility: Every WFC3 image is inspected by a human. Automation: The QL project is always getting the latest data.

6 The File System Every FITS extension including engineering and drizzled products. GO data is protected by group permissions. Updated daily with new observations. Contains a JPEG preview of each flt FITS image. >15TB of data on STScI central storage.

7 The Database The database information is stored in a SQLite relational database on STScI central store. Contains all the header keywords for all extensions of the FLT files. Contains information on creation date, file location, etc.

8 The Code Base ~4k lines of pure-Python code. File system operations, database operations, plotting, downloading. Maintained with software tools: SVN version control, Trac project management, nosetest, pylint, SQLAlchemy, Sphinx documentation.

9 QL Development QL has evolved to meet the needs of the team. Version 0: scripts + 6 page word document Version 1: Automated basic daily inspection tasks Version 2: Focused on file system completeness and database indexing

10 Future Work Move the QL project to a Linux Red Hat virtual machine. Automate many of the WFC3 monitoring tasks by using QL and using the lessons learned from building QL.

11 Contributors Supported and developed by Abhijith Rajan, Alex Viana, Matt Bourque, Michael Dulude, and Heather Gunning. Based on SyBase interface Python modules by Bryan York and Sami Nemi. The entire WFC3 team provides essential feedback for the QL project.

12 Thank You! Questions?