Data aggregation and tools to monitor the status of Monte Carlo samples production in CMS Author: Ramūnas Peleckas University supervisor: Algimantas Juozapavičius Organization supervisors: Jean-Roch Vlimant, Giovanni Franzoni

2 What is MCM? MCM – Monte Carlo Management MCM is the replacement for PREP, with more recent technology and added functionalities MCM is used for Monte Carlo dataset production requests management

3 Glossary Campaign - the campaigns are holders for requests of the "same era" (release, energy, …). Flow - The flow objects characterize how to go from one campaign to another.

4 Glossary(2) Chained request – request that is part of the chain. A request needs to be part of a chain in order to be processed in MCM. Chained campaign - campaigns that are connected with a flow are gathered in chained campaign.

5 Stuff I've been working on Requests scheduler and its translator Chained campaign statistics page Chained request statistics graph Chained campaign visualization graph

6 Requests scheduler Goals: Use data from the MCM production site Add a possibility to use different databases Solutions: Plug-in scheduler to the database Create a database translator

7

8 MCM website Goals for chained campaign statistics page: Remove dependencies of google services Update statistics page Adapt existing code

9 Statistics graph Purpose: Chained request's life-time representation Features: Multiple chained requests can be selected Show / hide chained request feature Each chained request has different color Requests have different shapes

10 Statistics graph(2)

11 Visualization graph Purpose: Represent existing chained campaigns Nodes represent chained campaigns Arrows represent flows

12 Visualization graph(2) Features: Draggable nodes Campaigns have different colors Ability to expand nodes so that one can see chains separated from each other

13 Visualization graph(3)

14 Used technologies Python programming language CouchDB database JavaScript programming language (including AngularJS framework and D3 visualization plugin) Git version control

15 Results Requests scheduler now uses real data Updated chained campaign statistics page Chained requests life-time graph Chained campaigns visualization graph

16 Questions?