1. ALICE Roadmap for 2009/2010 Patricia Méndez Lorenzo (IT/GS) Patricia Méndez Lorenzo (IT/GS) On behalf of the ALICE Offline team Slides prepared by Latchezar Betev (ALICE) WLCG Collaboration Workshop 21 March 2009

2. 2 Short summary of Offline tasks RAW data registration (Grid) and replication T0- >T1 RAW data registration (Grid) and replication T0- >T1 Conditions data gathering and publication on Grid Conditions data gathering and publication on Grid Quasi-online reconstruction T0 (Pass 1), ESD/AOD replication Quasi-online reconstruction T0 (Pass 1), ESD/AOD replication Delayed reconstruction at T1 (Pass 2), ESD/AOD replication Delayed reconstruction at T1 (Pass 2), ESD/AOD replication Prompt online reconstruction (subset of data @P2) Prompt online reconstruction (subset of data @P2) Batch @Grid and prompt @CAF user analysis Batch @Grid and prompt @CAF user analysis

3. 3 Registration/replication of RAW Data volumes (2008) Data volumes (2008) Total 310 TB at CERN + 200 TB replicated at T1s Total 310 TB at CERN + 200 TB replicated at T1s Includes cosmic data, detector calibration Includes cosmic data, detector calibration Tools and procedures Tools and procedures T0 registration -> CASTOR2 and AliEn file catalogue T0 registration -> CASTOR2 and AliEn file catalogue Routine operation, high redundancy of servers and software Routine operation, high redundancy of servers and software Problem-free operation Problem-free operation All updates carried out transparently by IT/FIO All updates carried out transparently by IT/FIO Replication FTD/FTS Replication FTD/FTS Tested extensively@CCRC08 Tested extensively@CCRC08 All goals (rate/stability) met and exceeded All goals (rate/stability) met and exceeded Routine operation during data taking (cosmics, calibration) Routine operation during data taking (cosmics, calibration)

4. 4 Registration/replication of RAW (2) No major issues No major issues Confidence in the storage and Grid tools is high Confidence in the storage and Grid tools is high Middleware and computing centres storage (T0 and T1s) have been fully certified Middleware and computing centres storage (T0 and T1s) have been fully certified The RAW registration at T0 will be tested at full p+p and Pb+Pb rates (up to 1.5GB/sec) in May-June 2009 The RAW registration at T0 will be tested at full p+p and Pb+Pb rates (up to 1.5GB/sec) in May-June 2009 Replication at p+p rate (100 MB/sec) in the same period Replication at p+p rate (100 MB/sec) in the same period

5. 5 Conditions data - Shuttle Shuttle (subsystem DBs to Grid conditions data publisher) system is in operation since 2 years Shuttle (subsystem DBs to Grid conditions data publisher) system is in operation since 2 years In production regime for the whole 2008 In production regime for the whole 2008 Detector algorithms (DAs) within Shuttle have evolved significantly, ready for standard data taking Detector algorithms (DAs) within Shuttle have evolved significantly, ready for standard data taking High stability of primary sources of conditions data: DCS, DAQ DBs and configuration servers High stability of primary sources of conditions data: DCS, DAQ DBs and configuration servers Toward the end of last cosmics data taking period (August) – all pieces, including DAs fully operational Toward the end of last cosmics data taking period (August) – all pieces, including DAs fully operational

6. 6 Conditions data – Shuttle (2) Major efforts concentrated to provide new conditions data Major efforts concentrated to provide new conditions data Critical LHC parameters Critical LHC parameters Newly installed detectors and control hardware Newly installed detectors and control hardware Some improvements on replication of conditions data on the Grid (file replicas) Some improvements on replication of conditions data on the Grid (file replicas) So far, ALICE maintains 2 full replicas So far, ALICE maintains 2 full replicas Conditions data access is the area with least problems on the Grid Conditions data access is the area with least problems on the Grid Both in terms of publication and client access for processing and analysis Both in terms of publication and client access for processing and analysis System in production, no special test foreseen System in production, no special test foreseen

7. 7 Offline reconstruction Reconstruction parameters Reconstruction parameters Detector dependent beam/multiplicity/luminosity conditions fully introduced in the AliRoot and conditions framework Detector dependent beam/multiplicity/luminosity conditions fully introduced in the AliRoot and conditions framework Quasi-online reconstruction status Quasi-online reconstruction status All runs from 2008 cosmics data processed All runs from 2008 cosmics data processed Emphasis on ‘First physics’ detectors Emphasis on ‘First physics’ detectors Selected runs already re-processed as ‘Pass 2’ and ‘Pass 3’ Selected runs already re-processed as ‘Pass 2’ and ‘Pass 3’ Re-processing of all cosmics data – general ‘Pass 2’ Re-processing of all cosmics data – general ‘Pass 2’ After completion of alignment and calibration studies by detectors After completion of alignment and calibration studies by detectors By April 2009 By April 2009

8. 8 Offline reconstruction (2) Development of quasi-online processing framework Development of quasi-online processing framework Further refinement of Online QA Further refinement of Online QA Speed up the launch of reconstruction jobs to assure ‘hot copy’ of the RAW data Speed up the launch of reconstruction jobs to assure ‘hot copy’ of the RAW data January 2009 – detector code readiness review and new set of milestones adapted to the run plan January 2009 – detector code readiness review and new set of milestones adapted to the run plan The middleware and fabric are fully tested for ‘pass 1’ (T0) RAW data processing The middleware and fabric are fully tested for ‘pass 1’ (T0) RAW data processing To a lesser extent at T1s – limited replication of RAW to save tapes To a lesser extent at T1s – limited replication of RAW to save tapes

9. 9 Grid batch analysis (1) Highest priority task Predominantly MC data analysis Each new MC production cycle accelerates the trend and user presence RAW (cosmics) data is still to be re-processed to qualify for massive analysis First pass has been extensively analyzed already by detector experts Ever increasing number of users on the Grid 435 registered, ~120 active

10. 10 Grid batch analysis (2) Internal ALICE prioritization within the common task queue works well Production user is ‘demoted’ in favor of normal users in the queue Generic pilot jobs assure fast execution of user jobs at the sites Introductrion of Analysis Train Grouping many analysis tasks in a common data set Allows for better CPU/Wall and reduces load on the storage servers Advanced tests – in production by end of March 2009

11. 11 Prompt analysis (1) PROOF enabled facilities currently available at two sites CAF@CERN GSIAF@GSI Darmstad Project starting: LAF@CCIN2P3 Extremely popular with users ~150 active Fast processing of MC ESDs – critical for first physics, tested extensively in 2008 Calibration and alignment iterative tasks, tested with cosmics and detector calibration data

12. 12 Prompt analysis (2) CAF@CERN was recently upgraded with higher CPU and local disk capacity ALICE encourages centres to install PROOF-enabled analysis facilities Open to all ALICE users Good integration with the Grid on the level of data exchange Data can be reconstructed/analysed directly from Grid storage Results can be published on the Grid

13. 13 Middleware updates Fully migrated to WMS submission Fully migrated to WMS submission Parallel installation at all sites of CREAM CE Parallel installation at all sites of CREAM CE Deployment (following recommendations of GD) - CREAM-CE instance in parallel with gLite CE Deployment (following recommendations of GD) - CREAM-CE instance in parallel with gLite CE Impressive service stability – test instance was maintenance-free for 4 month period Impressive service stability – test instance was maintenance-free for 4 month period New CREAM-CE installations – stable after initial debugging period New CREAM-CE installations – stable after initial debugging period All sites with CREAM-CE installation by June 2009 All sites with CREAM-CE installation by June 2009

14. SitequeuesStatus of the queues2 nd VOBOXVOBOX with clientsGeneral Status 18/03/09ALICE Offline Week -- CREAM-CE Use and Status for ALICE14 SiteCREAM- CEs CREAM Status 2 nd VOBOX Clients in VOBOX General Status FZK1 (4 queues) OKYES OK Kolkata2OKYES OK Athens1OKNO NOT OK KISTI1OKYES OK GSI1OKNOYESNOT OK* IHEP RAL1OKNOYESOK* CNAF1OKYES OK CERN2 (3 queues each) OKYES OK Torino1OKYES OK SARA1OKIn preparation YESIn testing

15. 15 Storage New storage at T2s New storage at T2s Gradually increasing the number of sites with xrootd-enabled SEs Gradually increasing the number of sites with xrootd-enabled SEs Every SE is validated and tuned independently Every SE is validated and tuned independently Storage status monitored with MonALISA Storage status monitored with MonALISA Emphasis on disk-based SEs for analysis Emphasis on disk-based SEs for analysis Including a capacity at T0/T1s Including a capacity at T0/T1s Storage types remain unchanged Storage types remain unchanged T1D0 for RAW, production, custodial ESD/AOD storage T1D0 for RAW, production, custodial ESD/AOD storage T0D1 for analysis: ESD/AOD replicas T0D1 for analysis: ESD/AOD replicas

16. 16 Storage – MSS use optimization File size increase File size increase RAW 1GB->10GB - presently at 3GB due to event size/processing time needed RAW 1GB->10GB - presently at 3GB due to event size/processing time needed Secondary files 100MB->1GB Secondary files 100MB->1GB Aggressive use of archives, containing multiple files Aggressive use of archives, containing multiple files File access patterns File access patterns Emphasis on prompt replication and reconstruction – files are still in the MSS disk buffer Emphasis on prompt replication and reconstruction – files are still in the MSS disk buffer Analysis and chaotic file access moved away from MSS SEs Analysis and chaotic file access moved away from MSS SEs

17. Data taking scenario, resources Data taking programme, rates and resources Data taking programme, rates and resources Based on discussion of LHC Chamonix meeting Based on discussion of LHC Chamonix meeting Resources requirements will be approved next week by the ALICE MB Resources requirements will be approved next week by the ALICE MB

18. Data taking scenario Cosmics Cosmics Resume data taking in July 2009, ~300TB of RAW Resume data taking in July 2009, ~300TB of RAW p+p runs p+p runs Running at maximum DAQ bandwidth Running at maximum DAQ bandwidth Few days @ 0.9 GeV (October 2009) Few days @ 0.9 GeV (October 2009) 11 months @ 10 TeV 11 months @ 10 TeV Machine parameters at P2 - optimum data taking conditions for ALICE Machine parameters at P2 - optimum data taking conditions for ALICE Computing resources must be sufficient for quasi online processing Computing resources must be sufficient for quasi online processing Address the ALICE genuine p+p physics program and provide baseline measurements for AA Address the ALICE genuine p+p physics program and provide baseline measurements for AA

19. Data taking scenario (2) A+A run A+A run Fall 2010 - a standard period of Pb+Pb running Fall 2010 - a standard period of Pb+Pb running Computing resources must be sufficient to process these data within 4 months after data taking (as foreseen in the Computing Model) Computing resources must be sufficient to process these data within 4 months after data taking (as foreseen in the Computing Model) Results to be presented at QM@Annecy (the LHC QM) in Spring 2011 Results to be presented at QM@Annecy (the LHC QM) in Spring 2011 Monte Carlo Monte Carlo 2009-2010 are standard years for Monte Carlo production 2009-2010 are standard years for Monte Carlo production

20. ALICE Data Taking Scenario Year/OperationCosmicppAAMC Data taking Data processin g Data taking Data processin g Data taking Data processin g 2009 Q1 Q2 Q3 Q4 2010 Q1 Q2 Q3 Q4 2011 Q1

21. Resources requirements CPU power required per event as in the Computing TDR and validated by cosmic data processing CPU power required per event as in the Computing TDR and validated by cosmic data processing Event (RAW/ESD/AOD) sizes as in the Computing TDR and validated by cosmic data processing Event (RAW/ESD/AOD) sizes as in the Computing TDR and validated by cosmic data processing Any Tier category (T0,1,2,3) can perform any task (reconstruction, simulation, analysis) Any Tier category (T0,1,2,3) can perform any task (reconstruction, simulation, analysis) CPU resources dynamically allocated depending on current experiment activity CPU resources dynamically allocated depending on current experiment activity

22. 22 Activities calendar 12 01 02 03 04 05 06 07 08 09 10 11 12 2008 2009 Replication of RAW Reprocessing Pass 2 Analysis train WMS and CREAM CE Cosmics data taking Data taking