CERN Openlab: my transition from science to business

Slides:



Advertisements
Similar presentations
Steve Lewis J.D. Edwards & Company
Advertisements

Module 13: Performance Tuning. Overview Performance tuning methodologies Instance level Database level Application level Overview of tools and techniques.
ESafe Reporter V3.0 eSafe Learning and Certification Program February 2007.
Module 17 Tracing Access to SQL Server 2008 R2. Module Overview Capturing Activity using SQL Server Profiler Improving Performance with the Database Engine.
A Java Architecture for the Internet of Things Noel Poore, Architect Pete St. Pierre, Product Manager Java Platform Group, Internet of Things September.
Nada Abdulla Ahmed.  SmoothWall Express is an open source firewall distribution based on the GNU/Linux operating system. Designed for ease of use, SmoothWall.
1 Databases in ALICE L.Betev LCG Database Deployment and Persistency Workshop Geneva, October 17, 2005.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment Chapter 11: Monitoring Server Performance.
3-1 Chapter 3 Data and Knowledge Management
Data Warehousing - 3 ISYS 650. Snowflake Schema one or more dimension tables do not join directly to the fact table but must join through other dimension.
Information Systems Development and Acquisition Chapter 8 Jessup & Valacich Instructor: Ramesh Sankaranarayanan.
Performance Management (Best Practices) REF: Document ID
® IBM Software Group © IBM Corporation IBM Information Server Deliver – Federation Server.
BMC Software confidential. BMC Performance Manager Will Brown.
Cloud Computing for the Enterprise November 18th, This work is licensed under a Creative Commons.
Hands-On Microsoft Windows Server 2008 Chapter 5 Configuring, Managing, and Troubleshooting Resource Access.
NetFort Customer Webinar Getting back to basics – Using LANGuardian Aisling Brennan 26 th Feb 2015.
1.
Net Optics Confidential and Proprietary Net Optics appTap Intelligent Access and Monitoring Architecture Solutions.
CERN - IT Department CH-1211 Genève 23 Switzerland t Monitoring the ATLAS Distributed Data Management System Ricardo Rocha (CERN) on behalf.
1 Overview of Databases. 2 Content Databases Example: Access Structure Query language (SQL)
Module 7: Fundamentals of Administering Windows Server 2008.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment, Enhanced Chapter 11: Monitoring Server Performance.
Informix IDS Administration with the New Server Studio 4.0 By Lester Knutsen My experience with the beta of Server Studio and the new Informix database.
Module 10: Monitoring ISA Server Overview Monitoring Overview Configuring Alerts Configuring Session Monitoring Configuring Logging Configuring.
Time lag between discovering issue and resolving Difficult to find solutions and patches that can help resolve issue Service outages expensive and.
Faster and Smarter Data Warehouses with Oracle OLAP 11g.
Access 2013 Microsoft Access 2013 is a database application that is ideal for gathering and understanding data that’s been collected on just about anything.
C6 Databases. 2 Traditional file environment Data Redundancy and Inconsistency: –Data redundancy: The presence of duplicate data in multiple data files.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment, Enhanced Chapter 11: Monitoring Server Performance.
Event Log View and Sentry Event Log Management Copyright 2002 Engagent, Inc.
And Tier 3 monitoring Tier 3 Ivan Kadochnikov LIT JINR
Microsoft Management Seminar Series SMS 2003 Change Management.
LHC Physics Analysis and Databases or: “How to discover the Higgs Boson inside a database” Maaike Limper.
Database Competence Centre openlab Major Review Meeting nd February 2012 Maaike Limper Zbigniew Baranowski Luigi Gallerani Mariusz Piorkowski Anton.
verifone HQtm Estate Management Solution
CERN IT Department CH-1211 Genève 23 Switzerland t CERN IT Monitoring and Data Analytics Pedro Andrade (IT-GT) Openlab Workshop on Data Analytics.
MND review. Main directions of work  Development and support of the Experiment Dashboard Applications - Data management monitoring - Job processing monitoring.
INFSO-RI Enabling Grids for E-sciencE File Transfer Software and Service SC3 Gavin McCance – JRA1 Data Management Cluster Service.
CHAPTER 9 File Storage Shared Preferences SQLite.
Configuring SQL Server for a successful SharePoint Server Deployment Haaron Gonzalez Solution Architect & Consultant Microsoft MVP SharePoint Server
PART1 Data collection methodology and NM paradigms 1.
The Ultimate SharePoint Admin Tool
SQL Database Management
EView/390z Management for IBM Mainframe for HPE Operations Manager i (OMi) Extending the cross-platform capabilities of Hewlett Packard Enterprise Software.
SQL Server Statistics and its relationship with Query Optimizer
Understanding the New PTC System Monitor (PSM/Dynatrace) Application’s Capabilities and Advanced Usage Stephen Vaillancourt PTC Technical Support –Technical.
Business System Development
Success Stories.
OptiView™ XG Network Analysis Tablet
Oracle Database In-Memory feature at CERN
Collecting heterogeneous data into a central repository
On the road: Test automation in practice for a BMW map update service
Ruslan Fomkin and Tore Risch Uppsala DataBase Laboratory
LHCOPN Operations: Yearly review
Objectives Differentiate between the different editions of Windows Server 2003 Explain Windows Server 2003 network models and server roles Identify concepts.
SysKit Insights SharePoint Monitoring & Troubleshooting.
Monitoring of the infrastructure from the VO perspective
Challenges in Network Troubleshooting In big scale networks, when an issue like latency or packet drops occur its very hard sometimes to pinpoint.
Oracle Architecture Overview
Accelerate Your Self-Service Data Analytics
Time Gathering Systems Secure Data Collection for IBM System i Server
Cloud computing mechanisms
Enterprise Program Management Office
Technical Capabilities
About Thetus Thetus develops knowledge discovery and modeling infrastructure software for customers who: Have high value data that does not neatly fit.
Backup Monitoring – EMC NetWorker
Backup Monitoring – EMC NetWorker
Features Overview.
Presentation transcript:

CERN Openlab: my transition from science to business CERN Openlab alumni talk CERN Openlab: my transition from science to business Maaike Limper 21/09/2017

Outline (why I’m not at work right now…) Before Openlab My CERN Openlab fellow My career since Openlab CERN Openlab: lessons learnt

Before Openlab One of the many physicists at CERN… Master in Particle Physics at University of Amsterdam 1999-2004 First time at CERN as Summer Student in 2003! PhD in Particle Physics 2004-2009 Building the ATLAS SemiConductor Tracker (SCT) Testing and installation of SCT at CERN Software development for track and vertex reconstruction Post-doctoral researcher for University of Iowa based at CERN First physics analysis with LHC data in ATLAS Prompt Reconstruction Coordinator for ATLAS “Co-discovered the Higgs-boson” along with others thousands of physicists…

My CERN Openlab fellow SQL-based approached to physics analysis: How to discover the Higgs boson in an Oracle database SQL-based approached to physics analysis: Databases allow us to do much more than store calibration constants and conditions Complex SQL with analytics-functions or precompiled procedures allow to do event filtering and calculations directly in the database SQL-analysis of basic physics data potentially faster than ROOT-based analysis Physics data has many attributes per row, making it more suitable to be kept in column-storage and partitioned in files across the grid for easy parallel processing

My CERN Openlab fellow A + B + C = D D What I actually discovered ROOT/PROOF is optimized for physics: Hard to improve using database analysis! Though users don’t always make optimum use of its features But great test-case for complex DB-analysis: Sequential scan vs index look-up Column vs row-based storage Love-hate relationship with SQL optimizer Use of partitions/distributed processing Hadoop vs PROOF vs Oracle RAC: Any distributed data processing ultimately comes down to the same thing, optimize your partitioning to push down your joins! A6 B6 C6 D6 E6 A6 B6 C6 D6 E6 A + B + C = D A1 B1 C1 A2 B2 C2 D A3 B3 C3 A4 B4 C4

My career since Openlab The final frontier of connectivity SITA OnAir: Business data analyst Inmarsat: Head of Aviation Service Performance

Aviation Service Performance Data-driven approach to performance monitoring Inmarsat Aviation has a dedicated team for Service Performance Our mission: ‘to turn data into information’ Data collected from all IFC systems is analysed using advanced custom data- processing solution and dedicated tools provide monitoring, customer analytics and enable data-sharing with our customer

The all devouring data-engine “Any sufficiently advanced technology is indistinguishable from magic” Automatic processes extract, consolidate and aggregate data from every possible data source Correlating data from different sources allow unique opportunities to study performance dependencies between our systems … pre-processed data is presented through a variety of internal and external reporting tools. On-Board portal-logs Portal content usage User-agent: Device/OS System events The all-devouring data-engine Allows Inmarsat support teams to proactively troubleshoot problems Inmarsat Internal Performance Monitoring ISP server-logs ISP portal usage User-agent: Device/OS Inmarsat network data Terminal status Terminal statistics Terminal messages ISP sessions data Payment method Session type User id Unique device ID Remote Maintenance View Allows external maintenance teams to monitor IFC status in real-time Airline: movement data OOOI timestamps Customer Monthly Reports SNMP Polling Discrete system status ARINC MIBs Flexible solutions to share pre-processed data with our Customer Ping utility Customer Analytics Portal Latency Outage timestamps Netflow Bytes/packets per IP Unique device IP Customer API Sandvine DPI Usage by category Whitelisted sites usage

CERN Openlab: lessons learnt What working at CERN has taught me No Higgs boson without IT Multi-culturalism Lateral thinking DIY Don’t trust technology Love, trust & respect your data