CRISP & SKA WP19 Status. Overview Staffing SKA Preconstruction phase Tiered Data Delivery Infrastructure Prototype deployment.

Slides:



Advertisements
Similar presentations
Fabric and Storage Management GridPP Fabric and Storage Management GridPP 24/24 May 2001.
Advertisements

Square Kilometre Array (SKA) Project PRE-CONSTRUCTION PHASE Briefing for Industry & Science Organisations Tuesday 14 th August Wellington.
SADC HPC Workshop, 2 Dec 2013, Cape Town
Paul Alexander DS3 & DS3-T3 SKADS Review 2006 DS3 The Network and its Output Data Paul Alexander.
ASCR Data Science Centers Infrastructure Demonstration S. Canon, N. Desai, M. Ernst, K. Kleese-Van Dam, G. Shipman, B. Tierney.
CHEPREO Tier-3 Center Achievements. FIU Tier-3 Center Tier-3 Centers in the CMS computing model –Primarily employed in support of local CMS physics community.
Sixth Hour Lecture 10:30 – 11:20 am, September 9 Framework for a Software Management Process – Artifacts of the Process (Part II, Chapter 6 of Royce’ book)
ASKAP Central Processor: Design and Implementation Calibration and Imaging Workshop 2014 ASTRONOMY AND SPACE SCIENCE Ben Humphreys | ASKAP Software and.
Summary Role of Software (1 slide) ARCS Software Architecture (4 slides) SNS -- Caltech Interactions (3 slides)
EU-GRID Work Program Massimo Sgaravatto – INFN Padova Cristina Vistoli – INFN Cnaf as INFN members of the EU-GRID technical team.
SOA Architecture Delivery Process by Dr. Robert Marcus SRI International 1100 Wilson Boulevard Arlington, VA
Office of Science U.S. Department of Energy Grids and Portals at NERSC Presented by Steve Chan.
Task 3.5 Tests and Integration ( Wp3 kick-off meeting, Poznan, 29 th -30 th January 2002 Santiago González de la.
Tools and Services for the Long Term Preservation and Access of Digital Archives Joseph JaJa, Mike Smorul, and Sangchul Song Institute for Advanced Computer.
Chapter 9: Moving to Design
LSU 01/18/2005Project Life Cycle1 The Project Life Cycle Project Management Unit, Lecture 2.
SKA in WP19 Iain Emsley Rahim ‘Raz’ Lakhoo David Wallom 3 rd Annual Meeting.
Chapter 9 Elements of Systems Design
Work Package 6: Performance Specification Ian McCrea.
LCG Milestones for Deployment, Fabric, & Grid Technology Ian Bird LCG Deployment Area Manager PEB 3-Dec-2002.
IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Collaborating with iPlant.
Hunt for Molecules, Paris, 2005-Sep-20 Software Development for ALMA Robert LUCAS IRAM Grenoble France.
OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep Ocean Observatories Initiative OOI CI Release 2 Life Cycle Objectives Review CyberPoPs & Network.
Chapter 2 The process Process, Methods, and Tools
The Preparatory Phase Proposal a first draft to be discussed.
1 | CSIRO ASKAP Science Data Archive (CASDA) – Stage 0 Project Intent Statement Confirm the necessary requirements, use cases, workflows, business processes,
Mantychore Oct 2010 WP 7 Andrew Mackarel. Agenda 1. Scope of the WP 2. Mm distribution 3. The WP plan 4. Objectives 5. Deliverables 6. Deadlines 7. Partners.
Copyright 2009 Fujitsu America, Inc. 0 Fujitsu PRIMERGY Servers “Next Generation HPC and Cloud Architecture” PRIMERGY CX1000 Tom Donnelly April
Transition from SKADS to PrepSKA Richard Schilizzi Lisbon, 3 October 2008.
INFSO-RI Enabling Grids for E-sciencE SA1: Cookbook (DSA1.7) Ian Bird CERN 18 January 2006.
Business Intelligence Appliance Powerful pay as you grow BI solutions with Engineered Systems.
GENIUS. WP600 Support Activities Participants: University of Barcelona: 6PM CESCA (Center for Scientific and Academic Services of Catalonia): 24PM University.
OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative OOI Cyberinfrastructure Architecture Overview Michael Meisinger Life Cycle Architecture Review.
Ocean Observatories Initiative OOI Cyberinfrastructure Architecture Overview Michael Meisinger September 29, 2009.
Looking Ahead: A New PSU Research Cloud Architecture Chuck Gilbert - Systems Architect and Systems Team Lead Research CI Coordinating Committee Meeting.
AAs in PEP Jan Geralt Bij de Vaate. PEP Project Execution Plan for the pre construction phase of the SKA Consortium to be formed AA-low ready for construction.
SKA System Design Kobus Cloete 9 December 2010 AAVP Workshop "Exploring the Universe with the world's largest radio telescope"
CASS, Marsfield Phil Diamond 4 October 2011 SKA Pre-Construction Phase: Where we are, and what we’re trying to achieve ?
EGEE is a project funded by the European Union under contract IST HellasGrid Hardware Tender Christos Aposkitis GRNET EGEE 3 rd parties Advanced.
1 Computing Challenges for the Square Kilometre Array Mathai Joseph & Harrick Vin Tata Research Development & Design Centre Pune, India CHEP Mumbai 16.
9 Systems Analysis and Design in a Changing World, Fourth Edition.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
THE PROJECT LIFE CYCLE PROJECT MANAGEMENT LIFE CYCLE LSU 01/18/2005 PROJECT LIFE CYCLE 1.
9 Systems Analysis and Design in a Changing World, Fourth Edition.
Grid User Interface for ATLAS & LHCb A more recent UK mini production used input data stored on RAL’s tape server, the requirements in JDL and the IC Resource.
6/23/2005 R. GARDNER OSG Baseline Services 1 OSG Baseline Services In my talk I’d like to discuss two questions:  What capabilities are we aiming for.
[IT INFRASTRUCTURE OVERVIEW] Paul Ashurst Executive Director, EIS Global Network Services IT Corporate.
Assembly, Integration & Verification 2015 SKA Engineering Meeting Richard Lord 11 November 2015.
Slide David Britton, University of Glasgow IET, Oct 09 1 Prof. David Britton GridPP Project leader University of Glasgow UK-T0 Meeting 21 st Oct 2015 GridPP.
An organizational structure is a mostly hierarchical concept of subordination of entities that collaborate and contribute to serve one common aim... Organizational.
EC Review – 01/03/2002 – WP9 – Earth Observation Applications – n° 1 WP9 Earth Observation Applications 1st Annual Review Report to the EU ESA, KNMI, IPSL,
Paul Alexander 2 nd SKADS Workshop October 2007 SKA and SKADS Costing The Future Paul Alexander Andrew Faulkner, Rosie Bolton.
1 1 SKA - CRISP WP19 17 th March 2013 Dr. Rahim Lakhoo (a.k.a Raz) P.I – Dr. David Wallom.
1 Integrated Modeling Stephen Merkowitz. 2 May 13, 2003 Integrated Modeling LISA has intricate interactions between subsystems that require an integrated.
Cloud-based e-science drivers for ESAs Sentinel Collaborative Ground Segment Kostas Koumandaros Greek Research & Technology Network Open Science retreat.
Grid Execution Management for Legacy Code Architecture Exposing legacy applications as Grid services: the GEMLCA approach Centre.
Paul Alexander1 DS3 Deliverable Status 4 th SKADS Workshop, Lisbon, 2-3 October 2008 DS3 Deliverables Review.
Planning Server Deployments Chapter 1. Server Deployment When planning a server deployment for a large enterprise network, the operating system edition.
Instituto de Biocomputación y Física de Sistemas Complejos Cloud resources and BIFI activities in JRA2 Reunión JRU Española.
Architecture of a platform for innovation and research Erik Deumens – University of Florida SC15 – Austin – Nov 17, 2015.
EGI-InSPIRE EGI-InSPIRE RI The European Grid Infrastructure Steven Newhouse Director, EGI.eu Project Director, EGI-InSPIRE 29/06/2016CoreGrid.
TeraGrid Software Integration: Area Overview (detailed in 2007 Annual Report Section 3) Lee Liming, JP Navarro TeraGrid Annual Project Review April, 2008.
Computing models, facilities, distributed computing
Software Defined Storage
UK Grid: Moving from Research to Production
Fabric and Storage Management
Software Requirements
PM view of SAFe SAFe Construction Planning Workshop A J Casson
Windows Server 2016 Software Defined Storage
Future EU Grid Projects
Presentation transcript:

CRISP & SKA WP19 Status

Overview Staffing SKA Preconstruction phase Tiered Data Delivery Infrastructure Prototype deployment

Staffing Rahim (Raz) Lakhoo Iain Emsley

SKA Preconstruction Baseline design published Consortia being appointed for major components preconstruction phase –Dish –Low Frequency Aperture Arrays –Signal and Data Transport –Central Signal Processing –Science Data Processor –Telescope Management –… Full Concept of Operations still to be released Preconstruction in 2 phases –74 weeks – Element PDR –82 weeks – Element CDR CRISP participants part of SKA Science Data Processor Consortia Phase starting 1 st Nov

Preconstruction Generation of documentation and input materials to support future Construction Phase Some prototyping but majority of effort towards inputs to reviews before start of Construction Phase No guarantee that ‘Preconstruction’ members will be in ‘Construction’

Science Data Processor 4. SKA.TEL.SDP.COMP science and data processor computing platform –The SDP computing platform hosts the processing pipelines and data layer. The platform includes hardware, operating system, SKA-originated software, and third party software. There may be multiple configurations tuned to the different telescopes. 4. SKA.TEL.SDP.DATA science and data processor data layer –The SDP data layer includes all access to data within the SDP Element, responsibility for persistence of data objects. 4. SKA.TEL.SDP.PIP science and data processor pipelines –The processing pipelines include all software for ingesting, calibrating, editing, imaging, cataloguing, and searching to produce science data products. The processing pipelines execute on the SDP computing platform or platforms. 4. SKA.TEL.SDP.DELIV Data delivery and tiered data model –Covers the delivery of data to the end users. It includes items associated with users gaining access to data and processing resources from locations away from the two observatory sites and their associated data centres. 4. SKA.TEL.SDP.PROT science and data processor prototypes –Prototypes include all activities necessary to design, procure, assemble, integrate prototype systems for the purpose of verification of particular design solutions. Prototypes includes the verification and testing of prototypes. Prototyping is supported in the field by SKA.OPS.PROT if it is required.

SKA.TEL.SDP.DELIV Data delivery and tiered data model - Activities Led by Canadian SKA Stage 1 –Level 4 contribution to the Science Data Processor SRR –Data delivery chapter to SDP element PDR documentation –Initial recommendation on common middleware stack –Report on status of and estimated requirement in R&D, evaluate risks and mitigation strategies Stage 2 –Data Delivery component of element system design –Interfacing with SADT –Fully costed SKA1 middleware design –Full common middleware stack, including maintenance plan

SKA.TEL.SDP.DELIV Data delivery and tiered data model SKA.TEL.SDP.DELIV.PI –This element considers the public interface to SKA data. It includes the provision of web portals / scientific gateways and common software/middleware tools for enabling access to data and processing by users that are not at the data centres associated with the two observatory sites. –Stage 1 Document use cases for public interfaces Catalogue existing portal solutions Select solutions to be explored by prototyping group. Create initial PI architecture document –Stage 2 Explore options for federating portals. Finalize requirements for testbed deployment Create final PI architecture document.

SKA.TEL.SDP.DELIV Data delivery and tiered data model SKA.TEL.SDP.DELIV.TDDM –This element considers tools that enable access to data at and the movement of data between regional HPC centres. This includes considering the tools for searching for existing data products and for the resources needed to create new products –Stage 1 Start to document use cases for data distribution. Catalogue existing data management tools. Select tools that should be explored by prototyping group. Create initial data distribution architecture document –Stage 2 Work with scientists using existing radio telescopes to refine use cases. Create final data distribution architecture document.

SKA.TEL.SDP.PROT.TDDL This WBS element will create a distributed test environment that will explore technologies that will be recommended as part of the SKA.TEL.SDP.DELIV WBS element. Stage 1 –Identify computing resources at site in Canada and the UK. –Setup data distribution environments based on recommendations from DELIV. –UK site will configure network emulation tools for testing wide area data distribution in controlled environment. –Federate sites in Canada and UK. –Report progress to DELIV. Stage 2 –Identify additional sites in additional geographic region. –Deploy software at new sites –Federate all sites –Create final reports for DELIV

Prototype 5 node system –attached to existing 6 node GPU enabled development HPC cluster –Shared FDR Infiniband high perf network –Node spec Dual 8-core Intel Xeon Processor 64GB RAM 2TB SSD Mellanox MCB194A-FCAT Connect-IB Dual Port FDR –Reconfigurable at the node level to create system as required for each test/experiment Delivered last week!

Prototype FDR Infiniband switch Jade Node SKA Test Node Jade Head Node ARC GPU Development (K20) Standalone Data testing

Prototype SKA SDP - Tiered Data Delivery Laboratory –“Infrastructure aims to emulate a network, its end points and resources –Provides a controllable and configurable environment for testing: Different types of data Resilience Reliability Performance” Whole ‘SKA’ analysis chain simulation