1. NMI Testbed GRID Utility for Virtual Organization
Art Vandenberg Director, Advanced Campus Services Georgia State University
19 November 2018

2. NMI Testbed GRID – “virtual organization” Participating sites
Overview
NMI Testbed GRID – “virtual organization”
Participating sites
Resources for VO
Catalog of grid applications
Example: genome alignment for VO
Plans for May-August 2004
11/19/2018

3. Vision – NMI Testbed GRID VO
NMI Integration Testbed Program
NSF #ANI
Explore grid capability – interoperability
Researchers & faculty
Across heterogeneous sites
Integrated with enterprise middleware
A utility grid using NMI components
Non-specialized, open, transparent
An inter-organizational model immediately calls for focus on interoperability - standards.
Consider certificate structures and policies: An application solution may work wonderfully well for that application - even when researchers are drawn from multiple institutions. And one could say, “See truly inter-organizational.” But campus enterprise infrastructures would say “It’s inter-organizational when the solution can be applied equally to other applications.” And we have a way to go to be able to claim that model (such as envisioned by OGSA web services model.)
11/19/2018

4. Collaborative environment VO
Beyond application specific grids
Leverage enterprise middleware
Identity management, authN, authZ...
Strive for transparent access
Portals
Ease of use: submit, monitor, retrieve data
Security policy & technology
Federation of cooperating sites
11/19/2018

5. Your Grid is here, now 19 November 2018
In the world of Middleware, this diagram has grown over the last 4+ years.
From a Network/Middleware/Application 3-level diagram to the 3-dimensional complexity shown.
GRIDs is a part of the picture…
19 November 2018

6. We want VO utility grid to be here... 19 November 2018
We need to think of GRIDs in the context of the whole picture. Not just a part.
19 November 2018

7. Participating sites - the VO
Testbed sites – push interoperation limits
Georgia State University
Texas Advanced Computing Center
University of Alabama at Birmingham
University of Alabama at Huntsville
University of Michigan
University of Southern California
University of Virginia
11/19/2018

8. Testbed sites – interoperation challenges
Site resources – VO
Testbed sites – interoperation challenges
GSU: Shibboleth, GridPort portal, REU & Grads, disk
TACC: REU student, portal, Enterprise CA, cluster
UAB: beowulf cluster, CA, Pubcookie, OGCE portal
UAH: application expertise, NASA IPG Certs
UMich: KX.509 & Kerberos, MGrid, ATLAS integration
USC: CA, Pubcookie, Shibboleth, Linux cluster, KX.509
UVa: Bridge CA model
Sites non-homogeneous – a VO challenge
11/19/2018

9. Catalog of grid applications
Knowledge base is important
REU students – Nicole Geiger, Anish Shindore
Graduate Research Asst – Manish Garg
NMI Testbed Sites initially
Researchers, schools, projects
Grid specific as well as grid potential
Started as spreadsheet, now online db
One activity that we think is useful is a catalog of GRID applications or potential applications. Note that Globus is collecting such info on GLOBUS TOOLKIT TM deployments. We’d like to do the same at campus level and maybe for an earlier stage of the deployment - what are applications that may be potentially grid-enabled?
11/19/2018

10. Catalog of grid applications
Catalog of GRID Applications (current version)
Expanding scope beyond testbed sites
18 schools/labs, 300 researchers & counting
Differentiated from Globus
Oriented to researcher, institutional level
Planning clustering, visualization modality
11/19/2018

11. Example: Genome Alignment for VO (GSU – UAB)
An opportunity for utility Grid VO
Nova Ahmed, CS grad with Dr. Yi Pan, GSU
dynamic programming algorithm for genome sequence alignment
Initial runs on GSU shared memory hydra
Limited access (grad student, shared cycles)
Algorithm improvement using multi-processor cluster across a grid?
11/19/2018

12. The Genome Alignment Problem
Alignment of DNA sequences
Sequence X: TGATGGAGGT
Sequence Y: GATAGG
Count the matching score as
1 => matching
0 => non-matching
Populate the Similarity matrix using:
Observation re Similarity Matrix:
Many zero values
Reduction of memory possible by reducing zero value elements
11/19/2018

13. Improved Parallel Algorithm for Genome Alignment
The parallel Method:
Similarity matrix is divided among processors
Processors calculate in parallel to match the partial sequence
Communication is done among the processors to match the whole sequence
The new Data Structure:
New algorithm calculates only non-zero values of the similarity matrix
Memory is dynamically allocated as needed
11/19/2018

14. Results on the Shared Memory Machine (Hydra)
Limitations
Can not allocate memory for long sequences
Ex: Largest sequence to align is 2000 x 2000
Number of processors are limited
Ex: For Hydra 12 processors
Not scalable
Performance
Computation time decreases with increased number of processors
11/19/2018

15. Results on the Beowulf Cluster of UAB
Using the beowulf cluster:
Longer genome sequence can be aligned
Highest sequence length can be 10,000 in the cluster
Limited scalability
Can increase the number of processors up to a certain limit
11/19/2018

16. Results via the GRID at UAB
Submitting genome alignment program using Globus and MPICH
Advantages:
Scalable – Can add new clusters to the grid
Easier job submission – Don’t need account on every node
Scheduling is easier – Can submit multiple jobs at one time
11/19/2018

17. Future Work Genome Alignment
Use MPICH-G2 (instead of MPICH) –
Use the power of Grid
Expand the computational resources –
Combine more clusters across the Grid
Develop program to align Multiple Genome Sequences (rather than two at a time) –
Requiring more computation resources
Use Georgia State certificate via Bridge CA
Via Shibboleth protected sector CA…?
11/19/2018

18. Portal for NMI Testbed GRID Integration of campus authN
Plans for May-August 2004
More resources
Contributed from current sites (others?)
Portal for NMI Testbed GRID
Cf. NPACI Hotpage
Integration of campus authN
UVa Bridge CA
More applications
Utility grid for grad research & education
11/19/2018

19. Catalog of Grid Applications
Plans for May-August 2004…
Documentation
Web site
Application docs and demos
Catalog of Grid Applications
Provide for self service contribution
Develop clustering (SOM), visualization options (“find researchers or projects like X”)
Auto-discovery of Grid researchers & apps based on reference sets (core sites)?
11/19/2018

20. Contact Information
Art Vandenberg
NMI Testbed GRID
11/19/2018