Peer-to-Peer Information Systems Week 14: Distributed Computing Old Dominion University Department of Computer Science CS 495/595 Fall 2003 Michael L. Nelson <mln@cs.odu.edu> 11/24/03

Grid Computing Compute Grids… Data Grids… Access Grids… etc. what do they mean? An early definition: “A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities” Foster & Kesselman, The Grid: Blueprint for a New Computing Infrastructure, 1998 http://books.elsevier.com/us//mk/us/subindex.asp?maintarget=&isbn=1558604758

An Updated Definition “…coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations…” “The sharing that we are concerned with is not primarily file exchange but rather direct access to computers, software, data, and other resources…” Foster, Kesselman & Tuecke, “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, International J. Supercomputer Applications, 15(3), 2001 http://www.globus.org/research/papers/anatomy.pdf (emphasis is mine - MLN)

The Grid for Virtual Organizations from: Foster, Kesselman & Tuecke, “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, International J. Supercomputer Applications, 15(3), 2001. http://www.globus.org/research/papers/anatomy.pdf

Grid Checklist Coordinates resources that are not subject to centralized control… (crosses domains of control/administration) …using standard, open, general-purpose protocols and interfaces… (general purpose, non-application specific APIs, services) …to deliver nontrivial qualities of service (complex and dynamic QoS; whole greater than sum of parts) from: Foster, “What is the Grid? A Three Point Checklist”, GRIDtoday, 1(6), 2002. http://www.gridtoday.com/02/0722/100136.html

Grid Misconceptions The Grid… is a next-generation Internet it builds on top of the Internet is a source of free cycles controlled sharing requires a distributed operating system TCP/IP model, not a virtual machine model requires new programming models proper abstraction and encapsulation should preserve current models makes high-performance computers superfluous low latency, high bandwidth problems not yet suitable for the Grid from: Foster, Kesselman & Tuecke, “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, International J. Supercomputer Applications, 15(3), 2001. http://www.globus.org/research/papers/anatomy.pdf

Grids and P2P In practice, we find that the technical focus of work in these domains has not overlapped significantly to date. One reason is that peer-to-peer and Internet computing developers have so far focused entirely on vertically integrated (“stovepipe”) solutions, rather than seeking to define common protocols that would allow for shared infrastructure and interoperability. … Another is that the forms of sharing targeted by various applications are quite limited, for example, file sharing with no access control, and computational sharing with a centralized server. As these applications become more sophisticated and the need for interoperability becomes clearer we will see a strong convergence of interests between peer-to-peer, Internet, and Grid computing. (emphasis is mine - MLN) from: Foster, Kesselman & Tuecke, “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, International J. Supercomputer Applications, 15(3), 2001. http://www.globus.org/research/papers/anatomy.pdf

P2P Computation Projects - Proto Grid? www.distributed.net cryptography, math puzzles www.chessbrain.net distributed chess playing www.intel.com/cure/ drug, protein research many others at: http://www.oreillynet.com/pub/t/73

SETI@Home http://setiathome.berkeley.edu

SETI Located in Arecibo, Puerto Rico SETI@Home SERENDIP antenna is a secondary antenna processed signals w/o storing; uses a dedicated super-computer focused on the hydrogen line (1.42 GHz) SERENDIP analyzes a 140 MHz band SETI@Home snail mail digital tapes back to Berkeley for processing analyzes a 2.5 MHz band, but does better analysis pictures from: http://www.naic.edu/

SETI@home Work Units Data sizes are chosen to be ~ 300Kb big enough to occupy your computer for a while, but small enough not to impose too much of a bandwidth load Data splitting is a computationally intensive task a cluster of workstations are dedicated to this task A Napster-style work dispatcher delegates work and receives results statistics pages are updated in batch mode

SETI@home Architecture from: http://www.computer.org/cise/articles/seti.htm from: http://setiathome.ssl.berkeley.edu/woody_paper.html

Aggregate TFLOPS (ca. 2001) ASCI White SETI@home 12.3 TFLOPS peak http://www.llnl.gov/asci/ http://www-1.ibm.com/servers/eserver/pseries/hardware/largescale/supercomputers/asciwhite/ 12.3 TFLOPS peak $110M 106 Tons SETI@home ~ $700k for 1 year of operation (not counting existing PCs) 1 day ~ 700k work units ~ 20 TFLOPS

Will Grids Have the Same Impact as P2P? ``Grid Computing is, according to the Grid Information Centre a way to "...enable the sharing, selection, and aggregation of a wide variety of geographically distributed computational resources." It is, in other words, an attempt to make Sun's famous pronouncement "The Network Is The Computer" an even more workable proposition. (It is also an instantiation of several of the patterns of decentralization that used to travel together under the name peer-to-peer.) IBM defines it more narrowly: Grid Computing is "... applying resources from many computers in a network-at the same time-to a single problem" , and the MIT Technology Review equated Grid technology with supercomputing on tap when it named Grids one of "Ten Technologies That Will Change the World.”’’ from: http://shirky.com/writings/grids.html (emphasis is mine - MLN)

Grids For The Masses? `` This view is wrong. Supercomputing on tap won't live up to to this change-the-world billing, because computation isn't a terribly important part of what people do with computers. This is a lesson we learned with PCs, and it looks like we will be relearning it with Grids.’’ from: http://shirky.com/writings/grids.html

coolness(“grid”) >> coolness(“distributed batch processing”) Shirky asserts that the “grid” is really just distributed batch processing (again) Email, web, games are primary reasons for selling PCs not calculation, spreadsheets, etc. General users are interested in peak performance to support their primary tasks (e.g., 3D graphics in games or movie editing) most users don’t have problems suitable for distributed batch processing “Of all the patterns supported by decentralization, from file-sharing to real-time collaboration to supercomputing, supercomputing is the least general.”

The Killer App for PCs? What if the killer applications for PCs are: file sharing game playing chatting video conferencing Its not that the grid won’t be useful, its just likely to be limited to a secondary impact for most users… however, distributed batch processing projects will be able to capitalize on the increasing number of powerful PCs purchased for their peak performance

Tapping Into the Sociology…