1. Collaborative eScience: Evolving Approaches
Charles Severance
NCeSS eCollaboration Workshop
June 28, 2006
This material is Copyright Creative Commons Attribution 2.5

2. Outline A look back at the past 15 years
Putting the “collab” in Collaborative eScience
The current tools of Collaborative eScience
Collaboration
Portals
Repository
Some Refections
A “future” eScience Case Study
Some sections are hidden to fit into 30 minutes…

3. The Founding Concepts Scientific Domain Groups of People
Common User Interface
Data Sharing
In the moment
Long-term
Experimental Equipment
Compute
Visualization

4. Over 15 Years of Collaborative eScience
2000
2001
2002
2003
2004
2005
2006
2007
SCIGate ?
OGCE Grid Portal
NEESGrid
NEESIT
Worktools
CHEF
Sakai
Upper Atmosphere Research Collaboratory
Space Physics Aeronomy Research Collaboratory
Globus Tool Kit
UARC/SPARC

5. What was SPARC?
Before
UARC..
SPARC - Space and Aronomy Research

6. What was SPARC?
UARC/
SPARC
SPARC - Space and Aronomy Research

7. SPARC 2/2001 600 users 800 data sources
An advanced users view of SPARC; lots of information in multiple SPARC pages!
The typical SPARC collaboration features down the left hand side, page information, resource list, active user list, and chat
Streaming video in the lower left.
Lots of data and model views throughout the rest of the screen shot.
2/ users 800 data sources

8. SPARC Software (1991-2001) Written from scratch
No Middleware
No Portal Technology
Three rewrites over 10 years
NextStep (Version 1)
Java Applets with server support (Version 2)
Browser based - kind of like a portal (Version 3)
At the end, in it was ready for another rewrite

9. Keys to SPARC Success Ten years of solid funding
Team consistency
Long enough to make and recover from “mistakes”
Long term relationship between IT folks and scientists - evolved over time - relationship was “grey”
Software rewritten several times over life of project based on evolving user needs and experience with each version of the program
Portion of effort was invested in evaluation of usability - feedback to developers
Funding ended in Project continued on inertia for a few years - but then faded away as its technology became dated wnd there was no further investment in “the little things” (I.e. no one to call and ask questions of).

10. After SPARC: Now What?
Getting people together is an important part of collaborative eScience
WorkTools - Based on Lotus Notes
CHEF - Collaborative framework - Based on Java and Jetspeed
Sakai - Collaboration and Learning Environment - Java
Critical point: Collaborative software is only one component of eScience
UM Focus: Building reusable user interface technologies for the people part of collaborative eScience
This is a bit of the UM perspective.

11. NEESGrid

12. Sakai CHEF and NEES - hidden
Cut to the “chase” - eliminate the intermediate steps…

13. Sakai General Collaborative Tools
Announcements
Assignments
Blog
Chat Room
Threaded Discussion
Drop Box
Archive
Message Of The Day
News/RSS
Preferences
Resources
Schedule
Web Content
Wiki
Worksite Setup
WebDAV

14. Sakai: Product Placement
Teaching
and
Learning
Sakai is a Collaborative Learning Environment. Sakai can be applied in both a teaching and learning environment as well as research collaboration.
Collaboration and eResearch

15. NMI / OGCE www.ogce.org NSF National Middleware Initiative
Indiana, UTexas, ANL, UM, NCSA

16. Chalk Talk:School of Portals (2004)
NEES 1.1
NEES 3.0
OGCE 1.2 ?
CHEF
OGCE 1.1
Jetspeed
OGCE 2
Sakai
GridPort 2
GridPort 3
GridPort
GridSphere
Alliance
uPortal
XCAT
Convergence
Competition
Collaboration

17. Chalk Talk:School of eScience Portals (2006)
NEES 1.1
GridSphere
CHEF
OGCE 1.1
OGCE 2
Jetspeed
SciGate ?
Sakai
GridPort 2
GridPort 3
SciDoc
uPortal
GridPort
Alliance
XCAT
Convergence
Competition
Collaboration

18. … … Integration and Administration Applications and Users Atlas ITER
CMS
Configure: Atlas Portal
Experiment
Process
Control
Knowledge Store
Sakai
SRB
Opal
Clarens
Metadata
Gateway
Technologies
Portal Gateway
Desktop Gateway …
Management
Components
Control
Experiment
Simulation
Process
KnowledgeStore … …
Configure: ITER Portal
Experiment
Process
Control
Knowledge Store
Sakai
SRB
Opal
Clarens
Metadata
Services and
Components
Identity
Security
MetaData
SRB
Sakai
Opal
Clarens
Globus
BlueGene
ORNL …
SciGate
Production
Petascale
Compute
Petascale
Data
Resources

19. Some Reflections

20. Some Reflections Computer Scientists like to “stay in their box”
Many of the technology solutions work well in their “initial domain”
Once an eScience team “adopts” a technology (often step 1) their further progress (and focus) is limited by the technology.
Agility is very important in the early phases of eScience
Builders of components *must* make their component as interoperable as possible

21. Sakai and Web 2.0
Web 2.0 is about making sure data is available in some form beyond just displayed in the Sakai Tool Set.
Formats
RSS
Resource Description Framework (RDF)
HTML
Protocols
RSS / getData / SOAP / REST
Consuming Applications
Portals
Google
delic.io.us

22. Sakai Data Interoperability
Portal
Environment
Personal
Environment
LMS Systems
Collaboaration
Environment
Authoring
Environment
Content
Management
One word people use for this is an “ecology”. An rich, nurturing supportive environment where stuff “grows”.
Mention SakaiBrary.
Identity
ACL
Data
Repository
... interoperability and data portability are key elements...

23. Sakai “Swiss Army Knife”
HTML
HTML
CalDav
RSS
Collaboration
and
Learning
Collaboration
and
Learning
WebDav
SOAP
WebDav
SOAP
To allow Sakai to be used as a component as much as possible, we need to implement any and every protocol that deployers will need to integrate Sakai into any environment.
SPARQL
iCal
Current Sakai
REST
Future Sakai

24. Interoperability at the UI
RSS, ATOM, RDF,
SOAP, REST, HTML
Interoperability at the UI
The SOAUI
Higher Education Academy (HEAcademy) Tomorrow at Cambridge - Keynote.

25. Reflecting on Middleware and Virtual Organizations
(slides not hidden)

26. Where is the Middleware?
“..composing and orchestrating many technologies…”
“..interoperability is key…”
Portal
Technology
Data
Sources
Collaborative
Tools
Middleware is a great thing…
Identity
ACL
Data
Repository
Shared
Compute
Knowledge
Tools

27. Is Middleware The Universal Connector?
Portal
Technology
Collaborative
Tools
Data
Sources
Middleware
Identity
ACL
Knowledge
Tools
Shared
Compute
I call this model: “middleware in the middle”.
Effectively nothing happens unless the middleware is involved.
The middleware’s performance, complexity, reliability, and ease of use become the rate-limiting factor…
To meet this need, middleware must be very fast, very simple, very easy to understand, very stable (I.e. not change so much) work with many different languages.
Middleware must stand the test of time in production environments…
Data
Repository

28. The Universal Connectors
Portal
Technology
Collaborative
Tools
Data
Sources
tcp/ip
http/https
Identity
ACL
web services
Knowledge
Tools
Shared
Compute
If we want cross-language, cross environment, there are actually very few technologies which reach the “universal connector” status.
Data
Repository

29. Is Middleware “inside” each application?
Portal
Technology
Data
Sources
Collaborative
Tools
There is no middleware that is sufficient to be a part of every single application ever written across all languages.
So where is it?
Identity
ACL
Data
Repository
Shared
Compute
Knowledge
Tools

30. Middleware is simply another component - used as needed
Portal
Technology
Data
Sources
Collaborative
Tools
Middleware
Identity
ACL
Middleware is no better, worse, or even that much different than any of the other components. It is used by the other components *if and only if* those components find the services provided by the middleware useful.
Middleware gets no special “must be used” status - if it adds value to an eScience application it will be used - if not - developers/integrators will find another way.
Data
Repository
Shared
Compute
Knowledge
Tools

31. Identity and Access Control: A very important function of Middleware
Portal
Technology
Data
Sources
Collaborative
Tools
Middleware
Lets Talk about This
Identity
ACL
Middleware is no better, worse, or even that much different than any of the other components. It is used by the other components *if and only if* those components find the services provided by the middleware useful.
Middleware gets no special “must be used” status - if it adds value to an eScience application it will be used - if not - developers/integrators will find another way.
Data
Repository
Shared
Compute
Knowledge
Tools

32. Chalk Talk:Identity and Access Control
CAS ??
Shibboleth
MyProxy
GridShib
Globus
Identity
ACL
K.X509
Kerberos
???
LDAP
What to do and who will fund it?
For portals this took 5 years - once we set out to do it.
Cosign
PubCookie
Convergence
Competition
Collaboration

33. Identity and ACL: Goal State
One server - one software distribution
Virtual Organization Software
Supports all protocols
Globus Certificate Authority
Shibboleth
LDAP
MyProxy
Kerberos
Who will do this? Who will fund this? Who can get these competitors to cooperate?

34. AUTHN/AUTHZ Meetings

35. My eScience Fantasy

36. The pre-requisites My net worth is $5B (I give myself grants)
I encounter some tech-savvy scientists in a field who are using technology to do world-class research…
They have never been visited by any other computer scientist…
They are working in groups of 1-30 geographically distributed around the world
They all work on a beach with Internet2 connections and wide-open wireless and favourable exchange rates

37. D A E B F C Experiments Compute Remote Observation Data Models
Vol 4 F
Vol 3
Vol 2
Vol 1 C
eDocuments
Tutorials

38. Step 1: Visit The Scientists
Understand what they are doing and how they are doing it?
Ask them how they would like to improve it.
Show each application to other scientists. Ask the other scientists how they would improve it.
Help each group improve their work - help them using whatever technology they are currently using

39. Step 2: Add some technology
Install the super-multi-protocol Virtual Organization software and provide a NOC for the VO software - identity and simple attributes - make sure the VO is easy to use!
Install Sakai - point it at the VO software for identity add icon at the top of Sakai
Give each scientist an account in the VO
Give each effort in the field a site within Sakai
The VO is the “first impression” that most users get of collaborative eScience. Those Computer Scienctists who are “security experts” are the least user-friendly of all computer scientists.
Most folks deploying VO’s want to push the envelope on the very definition of “what is a VO” with no concern as to what the users really want or will tolerate.

40. Heart Study Collaboratory
Login
My Workspace A B C D E
Open Forum
Home
Chat
Resources
Tutorials
Site B
Mail List
Live Meetings
All this step does is provide simple access to the work that is being done and give each one a tab. Let folks “learn about one another”.
Keep it simple - First, do no harm.

41. Step 2: Use the VO
For those who want to protect their information, help them add SSO to their sites, backed by the VO service
Since it is multi-protocol - likely there will be no modification of the underlying science code - only a server configuration change
Identity
ACL

42. D A E B F C Identity ACL Experiments Compute Remote Observation
Data Models
Heart Study Collaboratory
Login
My Workspace A B C D E
Open Forum
Home
Chat
Resources
Tutorials
Site B
Mail List
Live Meetings
Vol 4 F
Vol 3
Vol 2
Vol 1 C
Identity
ACL
eDocuments
Tutorials

43. Step 4: Unique Identifier Service
Come up with a way for any member of the VO to “get” a unique identifier
Demand some information (build a little data model)
Person’s name and organization (implicit from request)
What kind of thing this will represent (experiment, document, image series)
Simple description
Keyword/value extensions
Build an simple way request and retrieve these through a simple web service - capture implicit metadata from request (when, IP address, etc). Make sure it works from perl!
Encourage community to start marking “stuff” with these identifiers in their stovepipes

44. Step 5: Data Models
Begin to work with subsets of the field to try to find common data models across stovepipes
Start simple - use very simple RDF - human readable
Broaden / deepen model slowly - explore variations
Define simple file-system pattern for storing metadata associated with a file and/or a directory

45. Step 6: A Backup-Style Repo
Build a data repository which will function as a backup
Basic idea - each time you get identifier - this enables backup space - any data and/or metadata can be uploaded under that particular identifier and left in the repository
Make the repo multi-protocol, FTP, DAV, Web-Service with attachments, GridFTP, etc.
Make it so there can be a network of cooperating repositories

46. D A E B F C Local Repo Central Repo GUID Service Identity ACL Local
Experiments
Compute E B
Remote Observation
GUID
Service
Data Models
Vol 4 F
Vol 3
Vol 2
The key of this sequence is how little each of the science activities has been affected so far.
I understand this is a fantasy. But the underlying principle is the Hippocratic oath - “first, do no harm”.
Vol 1 C
Identity
ACL
Heart Study Collaboratory
Login
My Workspace A B C D E
Open Forum
Home
Chat
Resources
Tutorials
Site B
Mail List
Live Meetings
eDocuments
Local
Repo
Tutorials

47. Year 4 and on…
Once the basic stovepipes have been “brought in from the cold” and made part of a community with no harm, the next steps are to begin to work “cross-stovepipe”
Evolve data models to be far richer with many variants
Build value added tools that are aware of the data models and are usable across stovepipes
Teach the community to build and share tools - gently encourage development standards - Java / JSR-168 perhaps
Most important: Always listen to the users

48. Science at the center of eScience
… start at the center and work outwards…
New Tools
Data Models
Connect
Science
Scientists
Priority
New Technologies
Repositories
Communicate
Enhance
Data Storage
… apply technology when the users will see it as a “win” …
New Approaches

49. Conclusion Many years ago, eScience had science as its main focus
Custom approaches resulted in too many unique solutions
Computer scientists began a search for the “magic bullet” - each group found a different magic bullet
Each group now competes for mind share (and funding) to be the “one true” magic bullet

50. Conclusion (cont)
One way to solve the “many competing technologies” solution is to form “super groups” which unify the technologies
No single technology gets to claim “they are the one” (Middleware cannot be “in the middle” because then it gets “in the way”)
Each technology needs to become a drop-in service/component which is available for use only when appropriate
Once we can get past looking at the technologies as the main focus, we get back to science as the main focus
Infrastructure is what is in the middle. There is a long way between middleware and infrastructure.

51. Lets remember why we started this whole field in the first place…
Scientific Domain
Groups of People
Common User Interface
Data Sharing
In the moment
Long-term
Experimental Equipment
Compute
Visualization
To download
“Chuck’s Talks”