1. An Overview of Grid Computing and its Adaptation to CSE Curriculum
Bina Ramamurthy
CSE Department University at Buffalo (SUNY)
Partially Supported by NSF DUE CCLI A&I Grant
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

2. SIGCSE 2005 Panel, St. Louis, MO
Topics for Discussion
What is “The Grid”?
Motivation to explore the Grid
NSF Supported GridForce Project
Sample grid application
The Grid Conundrm
How to get involved?
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

3. How is the Internet evolving?
Technology Pipeline ……
......
Internet
Technology Pipeline ? ?
What is in the pipeline?
How is the Internet evolving?
What is next?
Answers: … later
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

4. SIGCSE 2005 Panel, St. Louis, MO
What is a grid?
Grid is a sophisticated framework that enables sharing of a variety of resources among distributed applications.
Open standard
Large scale operations
Automatic
Intelligent
Spontaneous
Interoperable
Service-oriented
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

5. What is a grid? (A formal definition)
Grid specifies a standard architecture, infrastructure, protocols and application program interface (API) for building an open enterprise system.
It can provide
Middleware supporting network of systems to facilitate sharing, standardization and openness.
Infrastructure and application model dealing with sharing of compute cycles, data, storage and other resources.
A framework for high reliability, availability and security.
Interoperation of batch-oriented and service-based architectures.
Standard service level feature definitions and higher level concepts for inter and intra-business collaboration.
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

6. SIGCSE 2005 Panel, St. Louis, MO
Beginnings of the Grid
Beginnings of the grid in Search for Extra Terrestrial Intelligence project)
The Wow signal
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

7. SIGCSE 2005 Panel, St. Louis, MO
Current Grid Users
A survey of 180 companies last summer by research firm Summit Strategies found that 4% of respondents had implemented a grid, and 12% were currently evaluating the technology.
Gartner predicted in 2002 that grid-based distributed systems will return by
Oracle Server 10g: g stands for grid. (Oracle 9i: i was for Internet)
Grid middleware from companies such as DataSynapse and Platform provides users the ability to manage workloads across the shared resources.
IBM used grid-base infrastructure for 2004 US Open: Enterprise Networks Aug 2004.
Burlington Coat Factory is investing its IT future on a grid-based, virtualized architecture: Enterprise Network June 2004.
HR (Human Resource) outsourcer Hewitt Associates put grid to work for crunching pension calculations. …
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

8. Financial: Build Portfolio Environment: Plan Forestation
(Grid takes you) Beyond Search Engines: Enabling Information Technology (IT) and Scientific Applications
Television Remote
Simple Search (stateless)
Financial: Build Portfolio
Medicine: Find Cure
Environment: Plan Forestation
Astronomy: Find and
Control Telescopes
On-demand
Complex multi-organization applications
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

9. Copyright: Mustafa Faramawi 2004 GridForce, University at Buffalo
Grid Growth
Copyright: Mustafa Faramawi 2004
GridForce, University at Buffalo
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

10. Copyright: Mustafa Faramawi 2004 GridForce, University at Buffalo
Global Grid Share
Copyright: Mustafa Faramawi 2004
GridForce, University at Buffalo
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

11. Technology Pipeline: Answer……
......
Internet
Technology Pipeline
What is in the pipeline?
Grid/GS ……
Web/WS
......
Internet
Technology Pipeline
WS: Web Service
GS: Grid Service
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

12. Adaptation of Grid Technology to CS-Curriculum
Introduce grid technology into the CS undergraduate curriculum.
Goal: Design and deploy grid services and applications. Study grid application models.
Focus on lab exercises to illustrate fundamental grid concepts, and development of grid services and applications.
Conduct seminars to industry.
Goal: Overview of grid technology landscape and its alignment to common technologies and application models.
Examine case-studies to expose potential uses of grid.
Introduce grid to potential users of grid
Goal: Publicize the usage models of grid.
Use grid infrastructure for entry level courses in Sciences and Engineering.
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

13. SIGCSE 2005 Panel, St. Louis, MO
GridForce
Our adaptation is evolving into a comprehensive framework we call GridForce (Grid For Collaboration and Education):
Course curriculum,
Laboratory exercises (labs),
Infrastructure to support labs,
Research projects,
Industrial outreach.
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

14. GridForce: Grid For Research, Collaboration & Education
Hands-on Labs
Courses/ Curriculum:
CSE486/586: Distributed Systems
CSE487/587: Information Structures
Research
Infrastructure
LinuxGrid Globus infrastructure supporting
secure service oriented architecture
Sample labs:
Grid Services Lab
Design and implementation of
Grid services with standard capabilities
OS: Red Hat Linux 9.2
Grid: Globus3.0.2
Function: Deploy services
Staging Server
Production Server
OS: FreeBSD
Grid: Globus 3.0.2
Function: fileserver,
firewall
OS: Solaris 8.0
Function:
Debug and test services
Cerf
Mills
Vixen
Postel
Dissemination Package:
Syllabus, Lecture Notes, Exams,
Course Evaluations, Pedagogy,
Applications, Lab descriptions,
Solutions, Publications,
and Infrastructure details.
CSECCRGrid Collaboration with Center
For Computational Research (CCR); Reusable old Sparcs
offering Condor grid and NSF Middleware Initiative suite.
Virtual Organization (VO) Lab
Grid application design and implementation
Symbolic representation of VO components
Grid-based tax return filer.
Registry
IRSService
HandleMap
IRSServcie
Factory
Environment
IRS Hosting
EMPService
EMPServcie
Emp Hosting
PerService
PERServcie
PER Hosting
BNKService
BNKServcie
BNK Hosting
TAX client
Assessment Sample from Fall 2003
CSE486/586
Collaboration
(SUNY Geneseo)
Database
MySQL
……………….
Gatekeeper: Job submission
scheduling tools
Compute nodes: Data analysis and graph tools
Internet :
remote job
submission
Labs Illustrating
Grid Use for
Non-CS Majors
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

15. Goals and Outcomes of GridForce: Grid For Research, Collaboration & Education
Bina Ramamurthy
CSE Department, University at Buffalo
NSF DUE CCLI A&I Grant
Dissemination Package:
Syllabus, Lecture Notes, Exams,
Course Evaluations, Pedagogy,
Applications, Lab descriptions,
Solutions, Publications,
and Infrastructure details.
Outcome 4: Multi-platform
Interoperable Research
& Educational Infrastructure & Testbeds
2. LinuxGrid Globus infrastructure supporting
secure service oriented architecture
OS: Red Hat Linux 9.2
Grid: Globus3.0.2
Function: Deploy services
Staging Server
Production Server
OS: FreeBSD
Grid: Globus 3.0.2
Function: fileserver,
firewall
OS: Solaris 8.0
Function:
Debug and test services
Cerf
Mills
Vixen
Postel
1.WSRF.NETGrid:
adapted from
University of Virginia
Outcome 1: Successful adaptation of Grid
CSE486/586: Distributed Systems
CSE487/587: Information Structures
Goal 1: Grid Courses/ Curriculum
Outcome 2: Involvement of smaller colleges and industries
collaborations
Goal 2: Research in Grid Services
Internet and Web Standards
Grid Standards
Applications
Goal 3: Collaboration for
Education & Dissemination
Industries, colleges and K-12
Outcome 3: Enrollment Strength:
Excellent
Outcome 5: Grid application
Models and grid accessibility
models
TAX client
Compute nodes: Data analysis and graph tools
Gatekeeper: Job submission
scheduling tools
Registry
IRSService
HandleMap
IRSServcie
Factory
Environment
IRS Hosting
……………….
3. CondorGrid
Collaboration with
Center For
Computational
Research (CCR);
Registry
EMPService
HandleMap
EMPServcie
Factory
Environment
Emp Hosting
Registry
PerService
HandleMap
PERServcie
Factory
Environment
PER Hosting
Registry
BNKService
HandleMap
BNKServcie
Factory
Environment
BNK Hosting
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO
Internet :
remote job
submission

16. Courses: CSE4/586 Distributed Systems
Learning outcome: fundamental concepts of distributed systems and grid.
Lab exercises to support concepts:
Three-tier client server system using Web Services.
A simple grid framework.
Design and implementation of a grid service.
Text: Distributed Systems: Concepts and Design (3rd Edition) by George Coulouris, Jean Dollimore, Tim Kindberg, Addison-Wesley Inc., 2000.
Prerequisites: Data structures and algorithms, object-oriented design and development, working knowledge of Java.
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

17. Courses: CSE 4/586: Lab Exercises
Topic
Learning Objective
Lab1
Webservices
To understand the alignment of the grid technology to Web Services, WS Definition Language (WSDL) and service description using WSDL.
Lab2
Grid
Infrastructure
A Webservices based grid.
Lab3
Programming
Design and implement a grid-base service using Globus 3.0.2
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

18. Courses: CSE4/587:Information Structures
Learning outcome: understand grid infrastructure and grid architecture, design and deploy grid services and grid applications.
Lab exercises support:
Enterprise application using Java 2 Enterprise Edition.
Grid application in high performance area.
Service-oriented grid application.
Text: “The Grid 2: Blueprint for a New Computing Infrastructure” by Ian Foster , Carl Kesselman, Morgan-Kauffmann, 2004.
Prerequisites: Data structures and algorithms, object-oriented design and development, working knowledge of Java, fundamentals of client/server architectures.
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

19. Courses: CSE4/587: Lab Exercises
Topic
Learning Objective
Lab1
Commercial
Application
Study requirements of a
commercial domain and
implement an application.
Lab2
High performance
Application.
Study requirements of scientific/business domain and implement compute intensive application.
Lab3
Defining a high-level grid service
Workflow service, a
business process, improvements to QoS
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

20. Services and Resources
Resources: data, storage
bandwidth
Services: scientific services,
business services and
personal services.
MyApplication
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

21. Grid Application using the services and resources
Resources: data, storage
bandwidth
Services: scientific services,
business services and
personal services.
MyApplication
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

22. StockExchangeService PortfolioAdvisoryService
Grid-based Stock Portfolio Building Application: Design Diagram By Faramawi, a Student in CSE4/586 Course
StockBrokerService
subscribe()
createPortfolio()
authenticate()
buy()
sell()
depositFunds()
withdrawFunds()
StockExchangeService
buy()
sell()
transactions
users
portfolios
Markowitz
Call & Listen
PortfolioAdvisoryService
advise()
StockClient
console/gui
history
Notify
Listen
Notify
random
HTML
StockMonitorService
authenticate()
monitor()
Data feed (HTTP)
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

23. StockExchangeService PortfolioAdvisoryService
Grid-based Stock Portfolio Building Application: Design Diagram By Faramawi, a Student in CSE4/586 Course
StockBrokerService
subscribe()
createPortfolio()
authenticate()
buy()
sell()
depositFunds()
withdrawFunds()
StockExchangeService
buy()
sell()
transactions
users
portfolios
Markowitz
Call & Listen
PortfolioAdvisoryService
advise()
StockClient
console/gui
history
Notify
Listen
Notify
random
HTML
StockMonitorService
authenticate()
monitor()
Data feed (HTTP)
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

24. Screen Shots from the Stock Monitoring Application:
Ross Goddard, CSE UG Student
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

25. Screen Shots from the Stock Monitoring Application:
Ross Goddard, CSE UG Student
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

26. OGSA Grid Internet (OSI)
Mappings
Applications
Written for users operating within VO environment
Applications
Written to run over the network: , FTP,
order-service, HR, SMTP, IIOP, CORBA
OGSA Architectured Services
Collective
Directory, Brokering, Monitoring,
Diagnostics, Data replication,
Policy, Membership
Java, C/C++, .Net
Presentation
Protocol and data conversion,
transport-independent policies: encryption, QoS routing, caching,
Security (SSL), CORBA, Data Rep.
Services with associated protocols
OGSI
Implemented by GT3 & OGSI.net
Web Services Technology
Session
Session Mgmt, checkpointing, reliable delivery: HTTP, SSL, SQL, NFS, RPC
SDKs with associated APIs
Workflow, Discovery & Registry
UDDI, ebXML Registries, IBM WSFL, MSXLang
Service Description Language
WSDL, WSCL (XML)
Resource
Transport
Flow control, error checking, guaranteed delivery:
TCP, UDP
Information
Management
Service Invocation & Messaging
SOAP (XML)
Network
Logical->physical address translation,
route and congestion management IP, ATM
Connectivity
Transport
HTTP, FTP
Communication
Authentication
Data Link
Packets-to-bits conversion, data frames:
Mac, LLC, ATM, PPP
Fabric (OGSA-enabled resources)
Logical
Security
Workflow
Database
File System
Directory
Messaging
Physical
Transmits raw bitstream over physical cable:
Ethernet band signalling, ISDN
Physical
Compute Nodes
Servers
Storage
Network
Sensors
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

27. The Grid Conundrum Technology Grid Internet State of the technology
Solid
Flux
System level
operation
Protocol level
operation
Where are the opportunities for CSE educators and students?
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO

28. SIGCSE 2005 Panel, St. Louis, MO
How to get involved?
Opportunities exist in:
Shaping the standards
Specifying accessibility models
Specifying application models
Redefining some older concepts such as trust and identity in the context of grid
Defining newer concepts such as resource use metering and service level agreement
Read online literature
Introduce it in your courses
Course curriculum, project descriptions, solutions and lecture material are available online at
Major challenge is lab infrastructure: GridForce has testbeds in various platforms available for use.
9/16/2018
SIGCSE 2005 Panel, St. Louis, MO