1. Enabling Grids for E- sciencE www.eu-egee.org EGEE and gLite are registered trademarks EGEE-III INFSO-RI-222667 Analysis of Overhead and waiting times in the EGEE Production Grids Max Berger Thomas Zangerl Thomas Fahringer University of Innsbruck

2. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Overview EGEE Definitions Scheduling Latency Information Service Latency Conclusions

3. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 EGEE EGEE: Enabling Grids for E-SciencE Largest Grid Infrastructure in the World 140 Institutions, 300 Sites, 50 Countries, 10.000 users, 80.000 CPU cores Production Grid Infrastructure Uses the gLite middleware Organized in Virtual Organizations (VO)

4. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 VOCE VOCE: VO for Central Europe Part of the EGEE Project 18 Sites participate “Liberal” Usage Policy –Users must be from the CE Region –Any Research can be done

5. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Definitions Scheduling Latency Delay between Job Submission and actual execution in seconds Information Service (IS) Latency Delay between actual occurrence of an event and its notification for the user

6. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Experiment Description Test jobs where submitted to VOCE VO Between Aug 08 and Oct 08 Approx. every 30 minutes Measured status change notifications Real status changes through callbacks Jobs where canceled after 45 mins scheduling time

7. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Scheduling Latency / Week

8. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Scheduling Latency / Day

9. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Scheduling Latency (cont.) Mean: 121 seconds Median: 91 seconds Most of the time short, but exceptions can take a very long time No significant changes over the week –Suggested “Weekend-Effect” was not provable No significant changes over the day –The Grid is in use all the time Clustering of values This value is much lower than values shown in related work! –Real execution start vs. notified execution start

10. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Scheduling Latency Histogram

11. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Scheduling Latency / Site

12. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Information Service Latency

13. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 IS Latency (cont.) Mean: 208 seconds Median: 198 seconds IS is organized in layers Each layer polls the underlying layer Polling interval defines time needed

14. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 IS Latency (cont.)

15. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Conclusions Production Grid are different from Research Grids! Scheduling Latency is not predictable Depends on the Site Additional overhead in the Information Service IS Overhead > Scheduling Latency Information is relevant for deciding Size of workload Scheduling of activities in workflows