1. Millennium Overview and Status David Culler and Jim Demmel Computer Science Division culler,demmel@cs.berkeley.edu http://www.millennium.berkeley.edu

2. Intel Visit 8/28/98Millennium2 MM Proposed Cluster of Clusters Gigabit Ethernet SIMS C.S. E.E. M.E. BMRC N.E. IEOR C. E. MSME NERSC Transport Business Chemistry Astro Physics Economy Math

3. Intel Visit 8/28/98Millennium3 Physical Connectivity

4. Intel Visit 8/28/98Millennium4 Associated Commitments Intel provides 6 M$ in equipment Sun provides all Solaris x86 software Microsoft provides all NT software Campus provides staff to support the core infrastructure, networking CS will try to raise funds for the network and campus cluster infrastructure Departments provide 1/2 of system admin for own side and 20 K$ for group cluster –racks, network, software,...

5. Intel Visit 8/28/98Millennium5 Where are we? Evolving Vision => SimMillennium Hardware deployment Software availability Cluster Environment Grants Networking

6. Intel Visit 8/28/98Millennium6 The Vision To work, think, and study in a computationally rich environment with deep information stores and powerful services –test ideas through simulation –explore and investigate data and information –share, manipulate, and interact through natural actions Organized in a manner consistent with the University setting

7. Intel Visit 8/28/98Millennium7 SimMillennium Project Goals Enable major advances in Computational Science and Engineering –Simulation, Modeling, and Information Processing becoming ubiquitous Explore novel design techniques for large, complex systems –Fundamental Computer Science problems ahead are problems of scale Develop fundamentally better ways of assimilating and interacting with large volumes of information –and with each other Explore emerging technologies –networking, OS, devices

8. Intel Visit 8/28/98Millennium8 Components of the Effort Community Cluster-based Resources Connectivity User Interaction Computational Economics

9. Intel Visit 8/28/98Millennium9 NSF investment: Cluster Network Transforms large collection of individual resources into a powerful system –can be focus on a problem High Bandwidth –scales with the number of processors (Gb/s per proc) Low Latency Low Overhead Low Cost Simple and Flexible Almost no errors Low Risk Today: Myrinet

10. Intel Visit 8/28/98Millennium10 NSF Investment: Inter-cluster network Gigabit Ethernet connecting group clusters and campus cluster Bay Networks provides 70% discount Campus provides fiber plant, maintenance, and staff

11. Intel Visit 8/28/98Millennium11 NSF Investment: UI Technology Two Projection Table –large field of view in horizontal (or vertical) orientation Phantom Haptic Interface –3D force feedback Motion Tracker – untethered position 3D Shutter Glasses –low cost visualization

12. Intel Visit 8/28/98Millennium12 User Interaction Research Agenda Expand access to 3D visualization –Explore any data anywhere –Ease development Develop lab-bench metaphor for Viz –two hands, physical icons Fast prototyping and exchange through Informal Interfaces –sketching Dealing with large volumes of information –lenses, brushing and linking 3D collaboration and interaction

13. Intel Visit 8/28/98Millennium13 Computational Economy How is this vast, integrated pool of resources managed? Traditional system approach: empower global OS to provide “optimal” allocation to blind applications –predefined metric, tuned to fixed workload –ignores the inherent adaptation of demand Computer Center –charge => director-to-user feedback according to cost Economic view: decentralized allocation according to perceived value –pricing => user-to-user feedback –compatible niches,sense of control, cooperation –idea has been around, why now?

14. Intel Visit 8/28/98Millennium14 NSF Investment: Staff Support Provide enabling technology and let it evolve –monitoring, enforcement –exchange –negotiation tools Integrate it into users enviroment Tools and measurements to determine effectiveness

15. Intel Visit 8/28/98Millennium15 Integrated Research Agenda Advance the State of Computational Science and Engineering –immerse a community in a computationally rich environment with the right tools: algorithms, programming & system support –Path to exploiting novel techniques and technology Explore design techniques for robust large-scale distributed systems –economic (or ecologic) approach Explore new ways of interacting with information –large paste-ups, two hands, sketching, 3D collaboration Investigate new technology –SMP nodes, gigabit Ethernet, SANs, VIA –NT, dCOM, Java beans, directory services –workbench displays, 3D icons, haptics, position sensors

16. Intel Visit 8/28/98Millennium16 Perspective Highly leveraged investment in a large scale infrastructure for studying problems of scale Deep commitment across the campus Sense of ownership and participation Rich research agenda

17. Intel Visit 8/28/98Millennium17 Nuts and Bolts

18. Intel Visit 8/28/98Millennium18 Current Environment All projects have servers and several desktops –last few in current shipment millennium.berkeley.edu domain established solaris/x86 shared servers with 60GB disks –www.millennium.berkeley.edu, MM.millennium.berkeley.edu –imap,... solaris/x86 sww served from CS servers NT domain server with 60 GB disks –MMNT.millennium.berkeley.edu –exchange, file,... First cluster ready for use NSF SimMillennium grant to cover network Two of three Millennium staff hired

19. Intel Visit 8/28/98Millennium19 Hardware Deployment (per Q2_98)

20. Intel Visit 8/28/98Millennium20 Basic Software Tools Standard Unix Tools /usr/sww/bin –484 packages GNU Tools –gcc, g++ (version 2.8.1), gdb, g... Sun ProWorks –workshop “development environment” –C, C++, F77 –Debugger Sun Performance Library/usr/sww/lib Sun Math Library NT dialtone

21. Intel Visit 8/28/98Millennium21 Cluster Developments 4 x 4 PentiumPro Cluster ready Full NOW environment –Glunix, MPI, Active Messages, Split-C –titanium protoype Transfering to Cory to debug deployment CS / Civil Eng. Consortium has been shaking it out –petc graph partitioner, finite element classes, … –ScaLapack (???) Have parts for three 8x2 Pentium II clusters –AstroPhysics –Soda Solaris x86 to be shipped out –Soda NT cluster

22. Intel Visit 8/28/98Millennium22 Grants NSF CISE Research Infrastructure grant –cluster networking –inter-cluster networking (campus mgmt) –computational economy staff support –visualization –devoted to CS research with disciplinary applns Bay Networks 70% discount up to 5 M$ –gigabit networks IBM SUR disk towers IBM CS extension to PDA access NERSC/LBL - DOE2000 Initiative => NSF Science and Tech. Centers in process

23. Intel Visit 8/28/98Millennium23 Networking Campus using Millennium to drive planning CNS is working closely with us. Testing gigabit ethernet Campbell, Evans, Davis moving forward CS will be able to cover the intracluster network (Myrinet) and gigabit switch in the group clusters –frees up some of the committed resources

24. Intel Visit 8/28/98Millennium24 Going forward Roll out group clusters over next few quarters Roll out gigabit interconnect Build up the cluster programming tools Intel Merced pushed out –campus cluster delayed –utilize NOW as an alternative –combine with NPACI infrastructure Exploring cluster technology –VIA –Synfinity, ServerNet, Gigabit Ethernet Exploring NT

25. Intel Visit 8/28/98Millennium25 A New Tier to Millennium Gigabit Ethernet PDAs Cell Phones Future Devices Wireless Infrastructure