1. 4 december, 2006 1 The Distributed ASCI Supercomputer The third generation Dick Epema (TUD) (with many slides from Henri Bal) Parallel and Distributed Group

2. 4 december, 20062 Distributed ASCI Supercomputer Joint infrastructure of the ASCI research school Clusters integrated in a single distributed testbed Long history and continuity DAS-1 (1997) DAS-2 (2002) DAS-3 (2006)

3. 4 december, 20063 DAS is a Computer Science grid Motivation: CS needs its own infrastructure for Systems research and experimentation Application experiments DAS is simpler and more homogeneous than most production grids Single operating system “A simple grid that works’’

4. 4 december, 20064 VU (85 nodes) TU Delft (68)Leiden (32) UvA/MultimediaN (46) UvA/VL-e (40) DAS-3: overall structure SURFnet6 10 Gb/s lambdas -272 AMD Opteron nodes -792 cores, 1TB memory -more heterogeneous: 2.2-2.6 GHz single/dual core nodes -Myrinet-10G (not in Delft) -Gigabit Ethernet Operational: oct. 2006

5. 4 december, 20065 Cluster configuration LUTUDUvA-VLeUvA-MNVU TOTALS Head * storage10TB5TB2TB 10TB 29TB * CPU2x2.4GHz DC 2x2.2GHz DC 2x2.4GHz DC * memory16GB 8GB16GB8GB 64GB * Myri 10G1111 * 10GE11111 Compute326840 (1)4685 271 * storage400GB250GB 2x250GB250GB 84 TB * CPU2x2.6GHz2x2.4GHz2x2.2GHz DC2x2.4GHz2x2.4GHz DC 1.9 THz * memory4GB 1048 GB * Myri 10G1111 Myrinet * 10G ports33 (7)414786 (2) * 10GE ports8888 320 Gb/s Nortel * 1GE ports32 (16)136 (8)40 (8)46 (2)85 (11) 339 Gb/s * 10GE ports1 (1)9 (3)221 (1)

6. 4 december, 20066 Projects using DAS-3 Virtual Lab for e-Science Grid computing, scheduling, workflow, PSE, visualization MultimediaN Searching, classifying multimedia data NWO projects, e.g., StarPlane and GUARD-G NCF projects (off-peak hours) And many more (P2P, …)

7. 4 december, 20067 Projects using DAS: StarPlane Key idea: Applications can dynamically allocate light paths Applications can change the topology of the wide-area network, possibly even at the sub-second timescale VU (Bal, Bos, Maassen) UvA (de Laat, Grosso, Xu, Velders) CPU’s R R R R R NOC

8. 4 december, 20068 Projects using DAS: GUARD-G How to turn grids into a predictable utility for computing (much like the telephone system) Problems: Predictability of workloads Predictability of system availability (grids are faulty!) Allocation of light paths very useful here TU Delft (Epema) + Leiden (Wolters)

9. 4 december, 20069 Projects using DAS: KOALA, a co-allocating grid scheduler Main goals: 1.processor co-allocation: (non-)fixed/flexible jobs 2.data co-allocation: move large input files to the locations where the job components will run prior to execution 3.load sharing: in the absence of co-allocation 4.run alongside local schedulers KOALA is written in Java uses Globus components (e.g., GRAM, RSL and GridFTP) has been deployed on the DAS2 in september 2005

10. 4 december, 200610 Status DAS3 clusters Delft cluster: accepted, up and running VU, UvA-MM: acceptance this week UvA, Leiden: acceptance this year