Grid: The Social Imperative B. Ramamurthy CSE Department University at Buffalo (SUNY) bina@cse.buffalo.edu http://www.cse.buffalo.edu/gridforce Partially Supported by NSF DUE CCLI A&I Grant 0311473 12/6/2018 Erir Community College

Erir Community College Topics for Discussion What does the title mean? Software trends What is Grid? Gridforce Project The Internet Revolution Evolution of Request/Response Model Grid Service Grid Concepts Case Studies: Applications we implemented Infrastructure Demos 12/6/2018 Erir Community College

Erir Community College Software Trends scale Multi-tier Server-side Grid Computing Virtualization Federation Provisioning Component programming Application complexity Client-server Classes Ex: J2EE Object-oriented programming monolithic Structured programming 1970 1980 1990 2000 Time (years) 12/6/2018 Erir Community College

Erir Community College Grid Technology Emerging enabling technology. Natural evolution of distributed systems and the Internet. Middleware supporting network of systems to facilitate sharing, standardization and openness. Infrastructure and application model dealing with sharing of compute cycles, data, storage and other resources. Promoted by NSF through its Network Middleware Initiative (NMI version 4). Publicized by prominent industries as on-demand computing, utility computing, etc. Move towards delivering “computing” to masses similar to other utilities (electricity and voice communication). 12/6/2018 Erir Community College

GridForce: Grid For Research, Collaboration & Education Hands-on Labs Courses/ Curriculum: CSE486/586: Distributed Systems CSE487/587: Information Structures Research Infrastructure LinuxGrid Globus infrastructure supporting secure service oriented architecture Sample labs: Grid Services Lab Design and implementation of Grid services with standard capabilities OS: Red Hat Linux 9.2 Grid: Globus3.0.2 Function: Deploy services Staging Server Production Server OS: FreeBSD Grid: Globus 3.0.2 Function: fileserver, firewall OS: Solaris 8.0 Function: Debug and test services Cerf Mills Vixen Postel Dissemination Package: Syllabus, Lecture Notes, Exams, Course Evaluations, Pedagogy, Applications, Lab descriptions, Solutions, Publications, and Infrastructure details. CSECCRGrid Collaboration with Center For Computational Research (CCR); Reusable old Sparcs offering Condor grid and NSF Middleware Initiative suite. Virtual Organization (VO) Lab Grid application design and implementation Symbolic representation of VO components Grid-based tax return filer. Registry IRSService HandleMap IRSServcie Factory Environment IRS Hosting EMPService EMPServcie Emp Hosting PerService PERServcie PER Hosting BNKService BNKServcie BNK Hosting TAX client Assessment Sample from Fall 2003 CSE486/586 Collaboration (SUNY Geneseo) Database MySQL ………………. Gatekeeper: Job submission scheduling tools Compute nodes: Data analysis and graph tools Internet : remote job submission Labs Illustrating Grid Use for Non-CS Majors 12/6/2018 Erir Community College

The Internet Revolution Began as an experimental network for collaboration among scientists. Enabling technology for the information age. Commercial success. Major impact on society and economy of the world. Fundamental concept: Request-response model. 12/6/2018 Erir Community College

Erir Community College Request-Response When you access a web page you request information at a location specifying its address using http (protocol). Response from the server to the request is typically in the form of a html document. Browser parses the html document and displays it for you. Now turn html tags into meaningful tags Extensible Markup Language (XML) Turn the information request into {operation, followed by operands} Simple Object Request Protocol (SOAP) Standard for SOAP over XML: Web Services (WS) WS is currently a very popular technology for application development. For example: Java 2 Enterprise Environment specification includes Web Services standard. Summarizing: html over http XML, SOAP over http WS  ? 12/6/2018 Erir Community College

Grid Service Framework Every grid service is a web service! with additional standard capabilities by Open Grid Services Architecture (OGSA) Open Grid Services Interface (OGSI) specifies capabilities defined in OGSA Globus Toolkit (GT3.0.2) is an implementation of OGSI. 12/6/2018 Erir Community College

Erir Community College Grid Service Logger object; Levels of logging: Info, .. Warn, Error, Fatal Filtering and redirecting to file, console For asynchronous messaging … Stores service properties and States; for discovery, monitoring, negotiations, etc. Services with ability to migrate Permanent services such as naming service that get activated and terminated with the container 12/6/2018 Erir Community College

Erir Community College Grid Concepts Virtualization Virtual Organization (VO) Federation Provisioning Service Oriented Architectures (SOA) 12/6/2018 Erir Community College

Erir Community College CSE486:Lab1: Web Services National Weather Service Web Site RMI Client 1 3 IP Socket API RMI Weather Server RMI http LAN 2 Relational Database Oracle 9i 4 Weather Web Service Container/Server SOAP XML Weather Web Service Web Client 6 5 http 12/6/2018 Erir Community College

CSE487:Lab 3: Tax Return Filer Registry IRSService HandleMap IRSServcie Factory IRS TAX Filer Hosting Environment EMPService EMPServcie EMPLOYEMENT PerService PERServcie PERSONAl BNKService BNKServcie BANK Hosting TAX client CSE487:Lab 3: Tax Return Filer Concepts illustrated: Virtual organization (VO) called IRS/Tax Filer that brings together virtualized capabilities of physical organizations of banking, personal profiles, and employment. Grid service handle (GSH) and Grid service reference (GSR), registry and handlemap, discovery of services, index services, application of notification, logging. . 12/6/2018 Erir Community College

Infrastructure: LinuxGrid Goal: To facilitate development of service-oriented applications for the grid. Two major components: Staging server and Production grid Server. Grid application are developed and tested on staging server and deployed on a production server. Production grid server: Three compute nodes with Red Hat Linux and Globus 3.0.2 instance. One utility gateway node with Free BSD and Globus 3.0.2. 12/6/2018 Erir Community College

Development Environment OS: Red Hat Linux 9.2 Grid: Globus3.0.2 Function: Deploy services Staging Server Production Server OS: FreeBSD Grid: Globus 3.0.2 Function: fileserver, firewall OS: Solaris 8.0 Function: Debug and test services Cerf Mills Vixen Postel 12/6/2018 Erir Community College

Infrastructure: SparcGrid Goal: To run jobs submitted in a distributed manner on a Condor-based computational cluster Condor. Composed of 50 Sun recyclable used Sparc4 machines, which form computational nodes, headed by a front-end Sun server. The installation scripts are custom-written facilitating running of jobs in a distributed manner. Partially supported by Center for Computational Research (CCR). 12/6/2018 Erir Community College

CSECCR Grid Internet : remote job submission MySQL Database ………………. Gatekeeper: Job submission scheduling tools Compute nodes: Data analysis and graph tools Internet : remote job submission Students work on computationally intensive application. Gene expression analysis; Markovitz model for financial portfolio picking; Students also learn About Condor, jon submission, scheduling, building grid infrastructure out of discarded computers. 12/6/2018 Erir Community College

SparcGrid Monitor (Ganglia) 12/6/2018 Erir Community College

Erir Community College Summary We will conclude the talk with some demos. In this talk, we covered salient features of the emerging grid technology. At University at Buffalo (UB) we have a established a comprehensive framework for grid education. A major social impact of grid will be computing accessible to the masses. Grid will enable a computational pipeline similar to how Internet enabled an information pipeline. 12/6/2018 Erir Community College