Start-Up of FEST Niko Neufeld, CERN-PH Tuesday, Jan. 27 th 2009 FEST Start-Up - Niko Neufeld 1

Acknowledgements FEST is an endeavor of the whole collaboration, in particular, the online, offline and physics (HLT) teams, so better not to thank anybody in particular: nobody mentioned == nobody forgotten FEST Start-Up - Niko Neufeld 2

What is the F ull E xperiment S ystem T est Run simulated data through the whole Online system and the whole Offline chain because: we had no data in we should be ready to process data hope to get several kHz with the first collisions 1 hour is equivalent to ~ 7 million events … Where do we start? Inject events at into the High Level Trigger (the farm), as if they were coming from TEL/UKL1s Run the HLT, to exercise control/configuration, trigger configuration, propagation of conditions and alignment constants, express stream, etc… Several FEST sessions: 26/01 – 30/01; 02/03 – 06/03 FEST Start-Up - Niko Neufeld 3

Replacing a 68 Million detector …by a lot of simulated events and one powerful computer FEST Start-Up - Niko Neufeld 4

Design Principles FEST should not only exercise the Online and Offline systems, the trigger, the Monitoring, Quality Checking and the prompt (Online) reconstruction FEST should help answer operational questions: e.g. “What is the best way to update alignment / calibration constants?” FEST should be operationally as close as possible to the “real thing” (like Pepsi Cola not like Virgin): Same run-control, use the TFC to set trigger rate, self-regulate rate This allows re-using FEST infrastructure later for dry- run tests of various components of the system FEST Start-Up - Niko Neufeld 5

(Extra) Input New software for DAQ and control of injector Some adaptations (network, TFC) 100 Million min. bias events 5 TeV, neg. polarity, VeLo closed) ~ 240 input files FEST Start-Up - Niko Neufeld 6

The FEST menu of the day GALA WEDDING MENU (Minimum of 75 people) RECEPTION Host Beverage Package Includes 5 Hours Open Bar Service HOT & COLD HORS D'OEVURE TRAYED AND SERVED Hearts of Palm Strudel with Remoulade Sauce Ragout of Marinated Grilled Vegetables on Toasted Baguette Mini Beef Filets with Horseradish Cream on Crostini Tamarind Glazed Lamb Kebobs Cumin Coconut Skewered Chicken Ginger Tuna on Rice Cracker with Wasabi Aioli DINNER Domestic House Wine with Dinner (1 bottle of red wine, 1 bottle of white wine per table) P.E.I. Gold & Sweet Potato Soup with Smoked Bacon and Coriander Cress Antipasto Platter with Grilled Peppers, Spiced Olives Artichokes, Air Dried Beef and Asagio Cheese Agnolotti with Quattro Formaggio Roasted Garlic and Basil Infused Cream sauce Individual Filet of Beef with Assorted Baby Vegetables Rosemary and Pancetta Potato Rosti Black Truffle Sauce (All meals served with Assorted Rolls and Butter) Baby Greens with Sundried Tomato and Oregano Vinaigrette Grilled Shrimp and Scallop Brochette with Citrus Butter Sauce Bailey's Parfait with Burnt Almond Brittle Bourbon Chocolate Sauce Coffee or Tea Midnight Coffee Station From the Niagara Falls Hilton Commission the injector Pass data through the HLT on a sufficient number of servers (8 – 12) Start with a low rate, periodic trigger of ~ 100 Hz Check the event building and writing of data to local storage Establish the maximum trigger rate at which the injector works (self- regulating!) Fix all problems, but don’t spend time to increase performance above 100 Hz Do not yet send to the Offline Commission the main HLT feature: Express line, plus other Get the express stream working: second stream in storage, events properly routed Verify rejection on the normal stream? Verify the output data. They should contain an HLT result bank FEST Start-Up - Niko Neufeld 7

FEST tests 2) Online Monitoring Monitoring of the HLT: Counters, rates, rejection Monitoring of sub-detectors with ‘normal’ events Are plots / histograms filled? Are they useful to identify SD problems? This requires SD experts around -> Advertise at the Commissioning meeting Reconstruction farm (“prompt reconstruction”) Sample events and run Brunel Monitor the performance: Rate, data flow. Monitor output, histograms? Send to Offline Start with the same low rate: A file every 10 minutes at 100 Hz Verify file-transfers, update of LFC entries and Bookkeeping Database Identify (additional) tools needed to get good control of the system Handle the Express line. FEST Start-Up - Niko Neufeld 8

Fest tests 3) Data quality checks How to give the green light for processing? Tools? Processing of the express line, alignment and calibration checks Verify DQ procedures. How do they feed back? Run Brunel + streaming (Offline) Setup a workflows Distribute the processing of the files, setup the streaming Test first at low rate, with runs of a few files... Test at higher rate (last but not least) Once things are OK at low rate, try to run at 1 kHz for some time FEST Start-Up - Niko Neufeld 9

Some FEST facts Software versions: Online v4r19, Moore v6r0, Brunel v34r1 Events: 100 Million min. 5 TeV, neg polarity FEST Start-Up - Niko Neufeld 10

FEST operation Like a global commissioning week: Online & Offline teams present (physically or remotely) Subdetector experts for Monitoring and Data Quality Control-Room packed Unlike a GCW: FEST interferes only mildly with sub-detector work: other partitions can run in parallel FEST Start-Up - Niko Neufeld 11

First FEST results FEST Start-Up - Niko Neufeld 12

Conclusion FEST is the last in a time-honored tradition of data-flow & processing exercises: RTTC (mostly Online), CCRC (mostly Offline), FEST (Online, Offline & HLT) By far the most ambitions & most useful of these It is very close to the “real thing”:  it needs a good amount of effort and work, but it will make us ready for the “real thing” in July! FEST’s biggest success will be if after FEST we never have to go back to it! FEST Start-Up - Niko Neufeld 13