Methodology: Aspects: cost models, modelling of system, understanding of behaviour & performance, technology evolution, prototyping  Develop prototypes.

Slides:



Advertisements
Similar presentations
Whos the Architect? Credential Provisioning Network Access Directory Services Authentication, Authorization and Accounting Federation Single.
Advertisements

ASCR Data Science Centers Infrastructure Demonstration S. Canon, N. Desai, M. Ernst, K. Kleese-Van Dam, G. Shipman, B. Tierney.
Software Project Management
. Smart Cities and the Ageing Population Sustainable smart cities: from vision to reality 13 October ITU, Geneva Knud Erik Skouby, CMI/ Aalborg University-Cph.
Supply Chain Network Simulator Using Cloud Computing Project # CDP10-SCNS Project Team Principal InvestigatorManuel D. Rossetti, PhD, P.E. Graduate AssistantYaohua.
Enterprise Business Processes and Reporting (IS 6214) MBS MIMAS 19 th Jan 2011 Fergal Carton Business Information Systems.
NGNS Program Managers Richard Carlson Thomas Ndousse ASCAC meeting 11/21/2014 Next Generation Networking for Science Program Update.
Session 8. GEOSS requirements, functions, architecture Session Feedback Presenter: Max Craglia.
David Besemer, CTO On Demand Data Integration with Data Virtualization.
CERN IT Department CH-1211 Genève 23 Switzerland t Integrating Lemon Monitoring and Alarming System with the new CERN Agile Infrastructure.
A Research Agenda for Accelerating Adoption of Emerging Technologies in Complex Edge-to-Enterprise Systems Jay Ramanathan Rajiv Ramnath Co-Directors,
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 1 Slide 1 An Introduction to Software Engineering.
Liam Newcombe BCS Data Centre Specialist Group Secretary Modelling Data Centre Energy Efficiency and Cost.
Fares Zekri Account Technology Strategist Microsoft Tunisia ITU Workshop on “Cloud Computing” (Tunis, Tunisia, June 2012) Microsoft Clouds.
PhysX CoE: LHC Data-intensive workflows and data- management Wahid Bhimji, Pete Clarke, Andrew Washbrook – Edinburgh And other CoE WP4 people…
Cloud Computing By: Carley Paxton. What is Cloud Computing? CloudCloud computing is the next stage in the Internet's evolution, providing the means through.
PanDA A New Paradigm for Computing in HEP Kaushik De Univ. of Texas at Arlington NRC KI, Moscow January 29, 2015.
Geneva, Switzerland, April 2012 Introduction to session 7 - “Advancing e-health standards: Roles and responsibilities of stakeholders” ​ Marco Carugi.
Towards Constraint-based High Performance Cloud System in the Process of Cloud Computing Adoption in an Organization Speaker : 吳靖緯 MA0G0101.
© Copyright IBM Corporation 2013 June 2013 IBM Integrated System Test Page 1 IBM Integrated Solutions Test Enterprise Test Series: Ideal Stack Testing.
HPC Centres and Strategies for Advancing Computational Science in Academic Institutions Organisers: Dan Katz – University of Chicago Gabrielle Allen –
J. Scott Hawker p. 1Some material © Rational Corp. Rational Unified Process Overview See and use the RUP Browser on lab machines.
ICN Baseline Scenarios draft-pentikousis-icn-scenarios-04 K. Pentikousis (Ed.), B. Ohlman, D. Corujo, G. Boggia, G. Tyson, E. Davies, P. Mahadevan, S.
Chapter 8 Process Implementation Reference: Tan, A. (2007). Business Process Reengineering in Asia: A Practical Approach, Pearson Education, Singapore.
David Foster LCG Project 12-March-02 Fabric Automation The Challenge of LHC Scale Fabrics LHC Computing Grid Workshop David Foster 12 th March 2002.
Summary of HEP SW workshop Ian Bird MB 15 th April 2014.
Ian Collier, STFC, Romain Wartel, CERN Maintaining Traceability in an Evolving Distributed Computing Environment Introduction Security.
Mobile Analyzer A Distributed Computing Platform Juho Karppinen Helsinki Institute of Physics Technology Program May 23th, 2002 Mobile.
Meeting with University of Malta| CERN, May 18, 2015 | Predrag Buncic ALICE Computing in Run 2+ P. Buncic 1.
Breaking the frontiers of the Grid R. Graciani EGI TF 2012.
1 Open Science Grid: Project Statement & Vision Transform compute and data intensive science through a cross- domain self-managed national distributed.
Resource Optimization for Publisher/Subscriber-based Avionics Systems Institute for Software Integrated Systems Vanderbilt University Nashville, Tennessee.
Issues in Cloud Computing. Agenda Issues in Inter-cloud, environments  QoS, Monitoirng Load balancing  Dynamic configuration  Resource optimization.
Ian Bird, CERN WLCG Project Leader Amsterdam, 24 th January 2012.
Evolution of storage and data management
CLOUD ARCHITECTURE Many organizations and researchers have defined the architecture for cloud computing. Basically the whole system can be divided into.
Review of the WLCG experiments compute plans
Latency and Communication Challenges in Automated Manufacturing
Service Assurance in the Age of Virtualization
2nd GEO Data Providers workshop (20-21 April 2017, Florence, Italy)
Organizations Are Embracing New Opportunities
ONAP Multi-VIM/Cloud Long Term Architecture and Use Cases (Under Community Discussion across Use Case, Optimization Framework, OOM,
Report from WLCG Workshop 2017: WLCG Network Requirements GDB - CERN 12th of July 2017
EOSC MODEL Pasquale Pagano CNR - ISTI
Computing models, facilities, distributed computing
The “Understanding Performance!” team in CERN IT
Cisco Data Virtualization
Sergei V. Gleyzer University of Florida
September 20, 2017 Agile Techniques Workshop Susan Futey
WLCG: TDR for HL-LHC Ian Bird LHCC Referees’ meting CERN, 9th May 2017.
Daniel Murphy-Olson Ryan Aydelott1
Software Processes (a)
Thoughts on Computing Upgrade Activities
System performance and cost model working group
R&D for HL-LHC from the CWP
Strategy document for LHCC – Run 4
Introduction to Cloud Computing
Cloud Computing.
SQL Server BI on Windows Azure Virtual Machines
Cloud Computing Dr. Sharad Saxena.
SDM workshop Strawman report History and Progress and Goal.
Database Environment Transparencies
Network Services Benchmarking - NSB
DATS International Portfolio.
Gordon Erlebacher Florida State University
On the Role of Burst Buffers in Leadership-Class Storage Systems
Chapter 6: Architectural Design
What's New in eCognition 9
Presentation transcript:

Methodology: Aspects: cost models, modelling of system, understanding of behaviour & performance, technology evolution, prototyping  Develop prototypes that help evolve the running system; understand the relative costs, understand the performance, gradually learn where the next bottlenecks, issues are, prototype solutions Iterate towards the HL-LHC computing model, with feedback from real measurements and experience, develop the costs as we progress HSF CWP workshop 28 June 2017

Prototyping Overarching goal: Prototype of “data-lake” style model understand the relative costs of different workflows Federated storage with different latencies - look at impact Specific aspects:  HSF CWP workshop 28 June 2017

Potential R&D/prototyping Resource Provisioning Common infrastructure for provisioning traditional, cloud, HPC and specialized hardware.    Demonstrate capability of scheduling for peak loads in clouds Use of specialized or exotic hardware/facilities Specification & execution of workflows rather than jobs Common workload management system, integrated with data management, to manage the execution of workflows on diverse resources, with high level of automation and AI Data Management and Access Integration of data placement and data delivery by combining data management and data federation solutions Smart caching Integration of networking information and testing of advanced networking infrastructure  Data Reduction Common workflow management system, to manage the complex workflows of the future which may involve hybrid resource usage, integration with framework, and deep integration with metadata   Define common selection criteria.    Demonstrate the ability to support hundreds of users reducing multi-petabyte samples to multi-terabyte samples and export them. Data Transformation Demonstrate the broad applicability of the event server model to all LHC experiments Demonstrate breaking the event loop into series of discrete data transformations HSF CWP workshop 28 June 2017

White paper Agreed: 5-10 page white paper contribution for the CWP, high level messages, accessible - main ideas; 50 page document with the details – needs clean up of existing document HSF CWP workshop 28 June 2017

5-page document Introduction Explain the HL-LHC problem, requirements, etc. Cost, metrics, model development High-level view of potential computing models Various aspects and ideas Infrastructure & technology How this may evolve, constraints, concerns Facilities What they may look like Common data management and data access layer Networking Considerations, how to manage traffic Common resource provisioning layer Including all types of resources; common pilot factory etc Common workload management layer Different workflows, granularity Roadmap & prototyping work HSF CWP workshop 28 June 2017