CEDAR Combined E-science Data Analysis Resource cedar. ac

Slides:



Advertisements
Similar presentations
EvtGen in ATLAS/LHC Roger W.L. Jones James R. Catmore Maria Smizanska Lancaster University, UK.
Advertisements

MCnet Marie Curie Research Training Network for Monte Carlo event generator –development –validation and tuning Approved for four years from 1 st Jan 2007.
10th May 2007SLAC-PPA Summit1 Mike Whalley Durham University
Interjet Energy Flow. Patrick Ryan, Univ. of Wisconsin Collaboration Meeting, June 6, Patrick Ryan University of Wisconsin Claire Gwenlan Oxford.
Software for Science Support Systems EVLA Advisory Committee Meeting, March 19-20, 2009 David M. Harland & Bryan Butler.
Simulation Project Major achievements (past 6 months 2007)
Herwig++ Particle Data1 Particle Data for Herwig++ Peter Richardson Durham University.
Introduction to the workshop LHCb Generators Tuning Mini Workshop Bucharest 22 nd & 23 rd November 2012 LHCb Generators Tuning Mini Workshop Bucharest.
Tev4LHC Workshop, QCD, Emily Nurse, UCL for the CEDAR collaboration (Andy Buckley, Jon Butterworth, James Monk, Ben Waugh, Mike Whalley,
JetWeb on the Grid Ben Waugh (UCL), GridPP6, What is JetWeb? How can JetWeb use the Grid? Progress report The Future Conclusions.
Shuei MEG review meeting, 2 July MEG Software Status MEG Software Group Framework Large Prototype software updates Database ROME Monte Carlo.
Monte Carlo event generators for LHC physics
Measurements, Model Independence & Monte Carlo Jon Butterworth University College London ICTP/MCnet school São Paulo 27/4/2015.
QCDGrid Progress James Perry, Andrew Jackson, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.
HERA/LHC Workshop, MC Tools working group, HzTool, JetWeb and CEDAR Tools for validating and tuning MC models Ben Waugh, UCL Workshop on.
4 November Development, validation and maintenance of Monte Carlo event generators & generator services in the LHC era Development, validation and.
DATABASE MANAGEMENT SYSTEMS IN DATA INTENSIVE ENVIRONMENNTS Leon Guzenda Chief Technology Officer.
Bookkeeping Tutorial. Bookkeeping & Monitoring Tutorial2 Bookkeeping content  Contains records of all “jobs” and all “files” that are created by production.
Databases E. Leonardi, P. Valente. Conditions DB Conditions=Dynamic parameters non-event time-varying Conditions database (CondDB) General definition:
The huge amount of resources available in the Grids, and the necessity to have the most up-to-date experimental software deployed in all the sites within.
The european ITM Task Force data structure F. Imbeaux.
Status of the LHCb MC production system Andrei Tsaregorodtsev, CPPM, Marseille DataGRID France workshop, Marseille, 24 September 2002.
Development, validation and maintenance of Monte Carlo event generators & generator services in the LHC era Dmitri Konstantinov 26 March
4/5/2007Data handling and transfer in the LHCb experiment1 Data handling and transfer in the LHCb experiment RT NPSS Real Time 2007 FNAL - 4 th May 2007.
K.Furukawa, Nov Database and Simulation Codes 1 Simple thoughts Around Information Repository and Around Simulation Codes K. Furukawa, KEK Nov.
The CERA2 Data Base Data input – Data output Hans Luthardt Model & Data/MPI-M, Hamburg Services and Facilities of DKRZ and Model & Data Hamburg,
LCG Generator Meeting, December 11 th 2003 Introduction to the LCG Generator Monthly Meeting.
DØ Data Handling & Access The DØ Meta-Data Browser Pushpa Bhat Fermilab June 4, 2001.
NOVA A Networked Object-Based EnVironment for Analysis “Framework Components for Distributed Computing” Pavel Nevski, Sasha Vanyashin, Torre Wenaus US.
RIVET Introduction By Mehar Ali Shah PhD Student National Centre for Physics Quaid-I-Azam University Pakistan 1.
Experience with CalcHEP H. S. Goh Univ. of Arizona very little West Coast LHC Theory Network -- UC Irvine May
M. Ellis - MICE Video Conference - 30th August Software Report Recent progress:Recent progress: –Start of code to read DATE format (two parts) One.
The GridPP DIRAC project DIRAC for non-LHC communities.
Projects, Tools and Engineering Patricia McBride Computing Division Fermilab March 17, 2004.
Using jet substructure and boosted objects: Measurements, searches, coping with pileup And something on measurements in general Jonathan Butterworth UCL.
1 Proton Structure Functions and HERA QCD Fit HERA+Experiments F 2 Charged Current+xF 3 HERA QCD Fit for the H1 and ZEUS Collaborations Andrew Mehta (Liverpool.
LCLS Commissioning & Operations High Level Software
Progress Apama Fundamentals
BESIII data processing
ANDROID APP FOR HIVETRACKS.COM SERVICE
Why Create a PGDB? Perform pathway analyses as part of a genome project Analyze omics data Create a central public information resource for the organism,
Overview of IPPP Monte Carlo Tools
Selected topic in computer science (1)
Peter Richardson IPPP, Durham University
Constraining BSM (Simplified) models with SM measurements
Computer Software Lecture 5.
Particle Properties: A Proposal from Herwig
Data Management and Database Framework for the MICE Experiment
Data Management Agenda
EVLA Archive The EVLA Archive is the E2E Archive
LCG Generator Services project
POOL persistency framework for LHC
A study on AlpGen and Sherpa in Z+jets events
The DESY ATLAS Group The Group and ALFA Physics SLHC upgrade
the Need for Data Integration
LCLS Commissioning & Operations High Level Software
Copyright © 2011 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 2 Database System Concepts and Architecture.
Database Driven Websites
Unfolding Problem: A Machine Learning Approach
Linear Collider Simulation Tools
What's New in eCognition 9
Serpil TOK, Zeki BAYRAM. Eastern MediterraneanUniversity Famagusta
XML Based Learning Environment
Gridifying the LHCb Monte Carlo production system
Simulation and Physics
Building an open library without walls : Archiving of particle physics data and results for long-term access and use Joanne Yeomans CERN Scientific Information.
Linear Collider Simulation Tools
Status and plans for bookkeeping system and production tools
b-Quark Production at the Tevatron
SDMX IT Tools SDMX Registry
Presentation transcript:

CEDAR Combined E-science Data Analysis Resource http://www. cedar. ac JetWeb HepData A collaborative project between UCL (London) and IPPP (Durham) Funded by PPARC E-Science call in 2003 (independent of the IPPP) – basically providing funding for 2 RAs (2004-2008). Main purpose is to integrate and modernize: JetWeb (UCL) and HepData (Durham) to provide data archival and presentation with validation of MCs to experimental data. Plus: development tools/environment : HepForge Extra: CEDAR tools being used to tune MCs` People involved: UCL: James Monk, Jon Butterworth, Ben Waugh, Emily Nurse…… Durham: Andy Buckley, Mike Whalley, James Stirling 27/07/2007 CEDAR

Why do we need CEDAR/Problems? Analysis of the complex events from LHC will require: MC simulations to be precisely tuned and validated over the full kinematic range JetWeb (UCL –developed at HERA) allows validation of MCs against a wide range of experimental data. ‘number crunching’ part - HZTool – written in Fortran contains individual routines for each data set/analysis, but modern analyses and MCs use C++ ! HepData (Durham – Reaction database) ~30 year archive of published ‘cross section’ data could directly provide the input real data to JetWeb but its Data Base Management System is not suitable 27/07/2007 CEDAR

HepData upgrade Old HepData database - Hierarchical -DBMS – Fortran!. (actually HDBMS suits the data well) Most modern DBMS are Relational – eq MySQL,Oracle,… New HepData version – to use MySQL Features: Handles data with a Java object model Object-relational mapping via Hibernate(DB) and Castor(external HepML/XML). New front-end via Java servlets/Tapestry. Data plotting/export via (J)AIDA. User input (more direct): HepML/Web Form – authentication? Migration of old data is underway: Hdbms <Fortran> flat files <Python> hepml <Castor/Java/Hibernate> New DB Objects – paper, datasets, axes, bins points,errors, etc. 27/07/2007 CEDAR

New HepData Structure MySQL hibernate xml castor Java Java Servlets 27/07/2007 CEDAR

JetWeb upgrade distributions + comparisons with real data. System for running MC generators + database of calculated distributions + comparisons with real data. Work on the upgrade primarily at UCL: Replace HZTool/HZSteer with Rivet/RivetGun Interface with HepData and the new Java data model Modernize the front end web interface 27/07/2007 CEDAR

JetWeb screenshots OLD 27/07/2007 CEDAR

Hztool upgrade: Rivet/RivetGun Robust Independent Validation of Experiment and Theory Rivet is essentially a C++ replacement for the Fortran HZTool. Performs an analysis on a set of particles from a simulated collision. Combination of tools, analysis handler and analyses. Outputs histograms for comparison to data from HepData and for inclusion in JetWeb. Release v0.9 on 29/6/2007 RivetGun interfaces Rivet to the MCs, thereby isolating the generator steering from Rivet and allowing selection of the parameters and running the Rivet analyses. At present supports FHerwig, FPythia, AlpGen, Sherpa, Herwig++ and Pythia8. $rivetgun-static –g FPythia –n 5000 –a HEPEX0409040 -beam1 PROTON –mom1 980 -beam2 ANTIPROTON –mom2 980 -P fpythia.params –l RivetGun:WARN 27/07/2007 CEDAR

HepForge Online development environment for free HEP projects (mainly for CEDAR, but supported) For those who want to provide quality multi-use software for HEP. HepForge provides the user with: Subversion version control (SVN) Trac issue tracker/wiki/SVN browser Mailman mailing lists Downloads management….. We don’t provide large volume storage or CPU resources though! Currently about 40 projects, 80 users. 27/07/2007 CEDAR

HepForge 27/07/2007 CEDAR

Tuning with Rivet (next step from CEDAR- Professor) As so often happens, MC parameters are highly correlated, so no point in tuning one parameter at a time… High-dimensional parameter space, n>~10 Data from LEP, RHIC, HERA, Tevatron. Delphi tuning: fit MC results to quadratic in n variables: Being reimplemented with Rivet machinery. Andy Buckley + Hendrick Hoeth (Wuppertal) + Frank Krauss & Dresden group 27/07/2007 CEDAR

Tuning with Rivet n-dimensional hypercube specifying sample ranges in each parameter generate N random n-vectors in the hypercube run RivetGun/Rivet on each set of vectors to produce N Rivet output files, each of which describes B bins. for each bin b fit polynomial function to the N generated values, using SVD Use real data from HepData to compute GoF such as Minimise individual functions as: 27/07/2007 CEDAR

CEDAR - Summary The HZTool C++ replacement, the Rivet/RivetGun system, now provides a unified mechanism to re-generate experimental analysis distributions with various MC generators. HepData archive DBMS is being updated and is now used as the data source for Rivet and JetWeb. JetWeb generates and archives Rivet distributions and compares with data from HepData through a user web interface. HepForge accounts are available for suitable projects (current list include, Herwig++, Sherpa, LHAPDF… ~40 in total Rivet will be combined with the Professor tuning system to automatically tune generators based on Rivet/Hepata mechanisms. 27/07/2007 CEDAR