Future Usage Environments & Systems Integration November 16 th 2004 HCMDSS planning workshop Douglas C. Schmidt (moderator) David Forslund, Cognition Group Jane W. S. Liu, Academia Sinica Raj Rajkumar, CMU Victoria L. Rich, U. Penn Douglas Rosendale, Veterans Affairs John R. Zaleski, Siemens

The Need: Better Tools Software developers depend on complex platforms, and increasingly work by extending or customizing with extra code –Web Services, J2EE, CORBA –Even Microsoft.NET The quality of these platforms and “tools” is a direct determinant of the quality of their applications and solutions TRUST September 13th 2004 NSF STC Review 2

Existing Platforms are Inadequate But existing platforms –Scale poorly and can “melt down” under stress –Are insecure; child’s play to disrupt or intrude –Are human intensive to deploy, configure –Are hard to repair when disruption occurs –Are costly to own and operate Example: publish-subscribe scalability issues within Yahoo, Amazon, other big data centers TRUST September 13th 2004 NSF STC Review 3

Why don’t platforms scale? Prevailing client-server communications model, combined with strong reliability goals, impose O(N) delays and O(N 2 ) performance degradation New technologies offer hope –Based on peer-to-peer interaction styles –Substitute probabilistic objectives for classic deterministic ones TRUST September 13th 2004 NSF STC Review 4

Kelips Affinity Groups: peer membership thru consistent hash 1N  N members per affinity group Map nodes to affinity groups

Kelips Affinity Groups: peer membership thru consistent hash 1N  Affinity group pointers N members per affinity group idhbeatrtt ms ms Affinity group view 110 knows about other members – 230, 30…

Kelips Affinity Groups: peer membership thru consistent hash 1N  Contact pointers N members per affinity group idhbeatrtt ms ms Affinity group view groupcontactNode …… 2202 Contacts 202 is a “contact” for 110 in group 2

Kelips Affinity Groups: peer membership thru consistent hash 1N  Gossip protocol replicates data cheaply N members per affinity group idhbeatrtt ms ms Affinity group view groupcontactNode …… 2202 Contacts resourceinfo …… cnn.com110 Resource Tuples “cnn.com” maps to group 2. So 110 tells group 2 to “route” inquiries about cnn.com to it.

Kelips Affinity Groups: peer membership thru consistent hash 1N  Gossip protocol replicates data cheaply N members per affinity group Lookup is routed by a contact node in group 2, then handled by node 110

Astrolabe: Scalable Monitoring Infrastructure. Captures system state hierarchically, using P2P protocol that “assembles a puzzle” without any servers NameLoadWeblogic?SMTP?Word Version … swift falcon cardinal NameLoadWeblogic?SMTP?Word Version … gazelle zebra gnu NameAvg Load WL contactSMTP contact SF ITH Paris San Francisco Ithaca SQL query “summarizes” data Dynamically changing query output is visible system-wide

Our team has many ideas like the two you’ve seen We’re challenging assumptions industry accepts as dogma (like client-server structure) Building real solutions that really work And finding ways to apply them in critical infrastructure settings –For example, we’ve worked with EPRI to explore applications of Astrolabe and Kelips to monitoring the electric power grid –Poor monitoring contributed to ’03 blackout TRUST September 13th 2004 NSF STC Review 11

Bio Dr. Douglas C. Schmidt is Full Professor in the Electrical Engineering and Computer Science Department, Associate Chair of the Computer Science and Engineering programs, and Senior Researcher, Institute for Software Integrated Systems (ISIS), at Vanderbilt University. His research has facilitated development of distributed real-time and embedded (DRE) middleware and applications on parallel platforms running over high-speed networks and embedded system interconnects. Dr. Schmidt is an internationally renowned and widely cited expert on distributed computing middleware patterns, middleware frameworks, and Real-time CORBA. He has published over 300 works in technical journals, conferences, and books, and has featured as guest editor and co-editor for various publications, editor of C++ Report magazine, and co-author for C/C++ Users Journal. Dr. Schmidt was Program Manager in the DARPA Information Exploitation Office (IXO) and Information Technology Office (ITO), where he led the national effort on QoS-enabled component middleware research in the Program Composition for Embedded Systems (PCES) program. He was Co-Chair for the Software Design and Productivity (SDP) Coordinating Group of the Federal government's multi-agency Information Technology Research and Development (IT R&D) Program. He served as Deputy Director of the DARPA Information Technology Office (ITO), where he helped set the national IT research and development agenda. Dr. Schmidt was formerly an Associate Professor in the Electrical and Computer Engineering Department at University of California, Irvine and an Associate Professor and Director of the Center for Distributed Object Computing, at Washington University in St. Louis. He has served in many capacities up to program chair, at leading industry conferences, and presented nearly 300 keynote addresses, invited talks, and tutorials. Dr. Schmidt has over fifteen years of experience developing DRE middleware and model-driven development tools. He led development of ADAPTIVE Communication Environment (ACE), and with colleagues at ISIS, Vanderbilt, used ACE to develop a high performance, real-time CORBA ORB endsystem called The ACE ORB (TAO). In turn, ACE and TAO form the basis for Component-Integrated ACE ORB (CIAO), a real-time CORBA Component Model implementation built by the DOC Group (which developed CoSMIC). ACE, TAO, CIAO, and CoSMIC have been used successfully by developers at hundreds of companies, and Dr. Schmidt and colleagues have applied these middleware platforms and tools on large-scale projects at Fortune 500 companies.