1 ALMA CPM3 2014 cominf (archive) Alisdair Manning 12/10/2015 1.

Slides:

Advertisements

Similar presentations

Development Life Cycle using Visual Studio Team Edition for DB Professionals Richard Fennell Engineering Director SQLBits II Birmingham 1 st March 2008.

Advertisements

Introduction to Maven 2.0 An open source build tool for Enterprise Java projects Mahen Goonewardene.

Status and plans for the H3 release NetarchiveSuite 5.0.

Software Delivery. Software Delivery Management  Managing Requirements and Changes  Managing Resources  Managing Configuration  Managing Defects 

NOAO/Gemini Data workshop – Tucson,  Hosted by CADC in Victoria, Canada.  Released September 2004  Gemini North data from May 2000  Gemini.

Production Programming in the Classroom Eric Allen, Robert Cartwright, and Charles Reis Rice University {eallen, cork,

U of R eXtensible Catalog Team MetaCat. Problem Domain.

User Group 2015 Version 5 Features & Infrastructure Enhancements.

System Design/Implementation and Support for Build 2 PDS Management Council Face-to-Face Mountain View, CA Nov 30 - Dec 1, 2011 Sean Hardman.

SEG Software Maintenance1 Software Maintenance “The modification of a software product after delivery to correct faults, to improve performance or.

Automating Drupal Deployment Dominique De Cooman.

Benefits of Using AllFusion ERwin and Advantage Gen in the Same Project Lifecycle Steve Smith Jumar Solutions 28 th March 2007.

Workflow Management Chris A. Mattmann OODT Component Working Group.

Resource Systems.  The need for agility  History of Product Development  Delivery of EPCOT  Future Challenges & Recommendations  Reflection  Questions?

ALMA Integrated Computing Team Coordination & Planning Meeting #3 Socorro, June 2014 Deployment to ARC Masao Saito (JAO) Morgan Griffith (NRAO)

Creating a Maintainable Software Ecosystem Jeremy D. Miller November 27th, 2007.

ALMA Integrated Computing Team Coordination & Planning Meeting #3 Socorro, June 2014 New AM organization and improving the acceptance procedure Masao.

TOPIC R Software Maintenance, Evolution, Program Comprehension, and Reverse Engineering SEG4110 Advanced Software Design and Reengineering.

Understand Application Lifecycle Management

COLD FUSION Deepak Sethi. What is it…. Cold fusion is a complete web application server mainly used for developing e-business applications. It allows.

S. Veseli - SAM Project Status SAMGrid Developments – Part I Siniša Veseli CD/D0CA.

1 ANASAC Meeting – May 20, 2015 ALMA Pipeline Brian Glendenning (for Jeff Kern)

ALMA Integrated Computing Team Coordination & Planning Meeting #2 Santiago, January 2014 ASDM relational database Rafael Hiriart / Jorge Avarias.

Maintenance and Support Week 15 CMIS570. User Training Need to consider the same 2 groups: End users Use the system to achieve the business purpose Creating,

Maintenance and Support Week 15 CMIS570. User Training Need to consider the same 2 groups: End users Use the system to achieve the business purpose Creating,

ALMA Integrated Computing Team Coordination & Planning Meeting #1 Santiago, April 2013 Relational APDM & Relational ASDM models effort done in online.

Rick Lopez, Director of Quality Engineering Rainya Mosher, Dev Manager, Deploy Infrastructure Deploying from OpenStack Trunk into a Production Environment.

LexBIG Release Overview Aug 21, LexBIG Context Project Goals for Sept –Incremental point release of LexBIG infrastructure to support EVS activities.

08/30/05GDM Project Presentation Lower Storage Summary of activity on 8/30/2005.

September 8, 2008 TPTF Nodal Core Projects Updates Nodal Project Managers.

1 © 1999 Microsoft Corp.. Microsoft Repository Phil Bernstein Microsoft Corp.

September Interface Kickoff Sunflower Project Statewide Management and Reporting Tool Update September 02, 2009.

Archive Access Tool Review of SSS Readiness for EVLA Shared Risk Observing, June 5, 2009 John Benson Scientist.

Continuous DB integration testing with RAT „RATCOIN”

Effort.vs. Software Product “Quality” Effort Product “Quality” Which curve? - linear? - logarithmic? - exponential?

Google Refine for Data Quality / Integrity. Context BioVeL Data Refinement Workflow Synonym Expansion / Occurrence Retrieval Data Selection Data Quality.

Rainbow: XML and Relational Database Design, Implementation, Test, and Evaluation Project Members: Tien Vu, Mirek Cymer, John Lee Advisor:

Software Maintenance Speaker: Jerry Gao Ph.D. San Jose State University URL: Sept., 2001.

Grid Deployment Enabling Grids for E-sciencE BDII 2171 LDAP 2172 LDAP 2173 LDAP 2170 Port Fwd Update DB & Modify DB 2170 Port.

ADMIT: ALMA Data Mining Toolkit  Developed by University of Maryland, University of Illinois, and NRAO (PI: L. Mundy)  Goal: First-view science data.

TopCAT Use Cases Priorities User Interface 1 ICAT developer workshop, August 2009 Laurent Lerusse – STFC

Getting Ready for STEVE Mapping Tools for STEVE William R. Bolton, Jr. State Registrar and Director Division of Vital Records Administration New Hampshire.

7 Strategies for Extracting, Transforming, and Loading.

ALMA Integrated Computing Team Coordination & Planning Meeting #3 Socorro, June 2014 Review of ALMA User Interfaces Erich Schmid ESO.

Adxstudio Portals Training

Lightning talk about Project Tracker (protrack) IRM meeting on Kyoko Nakamura.

System Maintenance Modifications or corrections made to an information system after it has been released to its customers Changing an information system.

ALMA Integrated Computing Team Coordination & Planning Meeting #1 Santiago, April 2013 ICT Group planning: Scheduling Jorge Avarias ICT Scheduling.

1 Software Maintenance and Evolution CSSE 575: Session 4, Part 2 Software Maintenance Process Steve Chenoweth Office Phone: (812) Cell: (937)

1 ECHO ECHO 9.0 for Data Partners Rob Baker January 23, 2007.

UNICOS LHCLoggingDB Josef Hofer EN/ICE/SCD. Agenda The LHC Logging Database Purpose of the LHCLogging component Basic concepts Advanced concepts Logging.

T EST T OOLS U NIT VI This unit contains the overview of the test tools. Also prerequisites for applying these tools, tools selection and implementation.

Integrating and Extending Workflow 8 AA301 Carl Sykes Ed Heaney.

CS 440 Database Management Systems Stored procedures & OR mapping 1.

1 Punishment Through Continuous Delivery If it hurts, do it more often…

16 Copyright © 2004, Oracle. All rights reserved. Testing the Migrated Oracle Database.

1 Middle East Users Group 2008 Self-Service Engine & Process Rules Engine Presented by: Ryan Flemming Friday 11th at 9am - 9:45 am.

AIRS Meeting GSFC, February 1, 2002 ECS Data Pool Gregory Leptoukh.

B.6 Roadmap 2013 – 2014 SDMX RI User Group Luxembourg, September 2013.

Archiving System News Tango Meeting (Elettra, April 2008)

From delivery to acceptance

Recent DBS Developments & Works Ahead

AppDB current status and proposed extensions

Maintaining software solutions

Delivering Business Value Faster

SSDT and Database Project Basics

Delivering Business Value Faster

Reportnet 3.0 Database Feasibility Study – Approach

Chapter 2: Building a System

Building a “System” Moving from writing a program to building a system. What’s the difference?! Complexity, size, complexity, size complexity Breadth.

Presentation transcript:

1 ALMA CPM cominf (archive) Alisdair Manning 12/10/2015 1

The Plan AQ incremental release every release Easy performance improvements for RH DataPacker NGAS Pipeline Ingestion and meta-data extraction Relational Asdm (no delivery) DataPacker: re-assemble RelAsdm delivery Harvester APDM XML (no delivery) 2015 SourceCatalogue Adapt 12/10/20152

Priorities and Resources 3 meChristopheCADCHeiko 1.Schemas 2.DataTracker 3.ALMA schema 4.Harvester 5.SourceCat 6.ASDM Updates 7.NGAS 1.Pipeline Int. 2.RH - DtD 3.RH - OODT 4.Harvester 5.DataPacker 6.NGAS 7.Download Manager 1.AQ1.Monitoring 1.ALMA schema 2.NGAS 3.DB Docs 4.APDM 5.SourceCat 1.AQ1.Monitoring1.Pipeline Int. 2.RH - DtD 3.RH - OODT 4.Harvester 5.Download Manager 12/10/20153

Priorities Update I 1.ASA – close loop on deliveries with DataTracker accepted CYCLE2-OFF SCO ARC DT 1, 8: data transfer 2: poll 3: PI from ARC? 4 :all there? 5: 6: start PP 7: write DB 12/10/20154

Priorities Update II 5 2.Pipeline Integration (including OODT into the RH) stalled – missing XML, RH tasks. To resume ASAP. 3.Monitoring Database end of ASA incremental updates Harvester extensions fixes, +spatial resolution, -gal, -abstract, -bib AQ features auto-complete, freqSupport, export, API, state RH features cycle2, GOUS, hierarchical display Performance -aq, +rh, -rh 12/10/20155

Priorities Update III 6 5.SourceCat schema restructuring CYCLE2-OFF deployed 6.SourceCat refactoring / VO re-architecture 7.NGAS maintenance / performance / merge with ESO codebase meeting with Andreas at ESO 8.AsdmUpdate 612/10/2015

Relational ALMA 7 DelayHarvester and Sourcecat work (2015) ASDM required for AQUA / ERMA create insert / retrieve performance tests Control’s schema in Garching dev env. Cost / benefit. Suitable? Performant? Iterate. 16/03 Review by DBAs Modify module build order ARCHIVE / ICD31/03 Generate hibernate mappings from model18/05 Migrate online code, harvester, data packer20/07 APDM for ComInf tools (or use Oracle 11gr2 features?) Feasible? Performance tests30/09 Hibernate mappings from model.09/11 Migrate DP, Harvester, online code.31/ /10/2015

Plan from CPM Harvester corrections, galactic longitude / latitude, spatial resolution DataTracker 1 st release at ARCs Detect ingestion of pipeline product from JAO Does products PI belong to ARC? Have all pieces made it to the ARC? to PI Another one month before end of proprietary period Pipeline ingestion into Archive SourceCatalogue bugs, refactoring, data model changes, history AsdmUpdate DataPacker expansion to re-assemble pipeline products for delivery 12/10/20158

Lost Time CYCLE2-OFF cost a lot of effort Need to verify against base configuration – better instructions. Scripting. DBA work: importing data for debugging / testing Scripted Firefighting – performance of RH, NGAS issues in production Broken unit tests in ONLINE system in particular – needs refactoring VLT RH issues needed investigation – unanticipated usage 12/10/20159