1. TGSAF - Grid Storage Access Framework
(A simple paradigm to design Grid “Data Based Application”)
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid
Catania,
Salvatore Scifo
Cometa Consortium (INFN), Catania – Italy
Salvatore Parisi
IR&T engineering s.r.l.

2. “Data Grid” Based Application
“Data Grid Application” context
application that uses gLite Data Management System as Digital Repository
This set of application collects a large type of application:
e-learning platforms
health care applications
digital archives for cultural heritage
earth e-science repositories
other…
TGSAF is SDK for Digital Libraries
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid - Catania 04/07/2008

3. Digital Libraries Architecture
Every application is customized according to its own purposes
vertical dimension = specialization / customization
Looking at the layered architecture we see how DATA ACCESS LAYERs share functions each other
horizontal dimension = shared functions / common problems
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid - Catania 04/07/2008

4. Grid development troubles
Grid Data Services are independent from each others (SOA)
They work in a “standalone” mode (API fragmentation)
No coherence is ensured (uncontrollable resources)
Fragmentation effects
software engineers must consider a vertical architecture
applications must take care themselves (at Business Logic Level), about the atomicity, coherence and synchronization of data manipulation (big development effort!)
knowledge gap (traditional development vs. Grid development)
code replication in different use cases for the same capabilities
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid - Catania 04/07/2008

5. GSAF collects shared features
( common features = common problems )  Design Pattern
Built on top of the Grid Metadata Service and Grid Data Service
collects and implements functionalities shared among applications according to “write once use anywhere” principle
reduces the knowledge gap
hiding the complexity and the fragmentation of the several underlying APIs
exposing a unified interface more near to the developer mind (design patterns) rather than the Grid stuff details (API syntaxes)
works as a black box providing
classes and related methods for applications located above
interfaces to extend the implemented capabilities
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid - Catania 04/07/2008

6. GSAF Logical Architecture
The Framework is designed to be a plug-in
It logically includes all vertical specialized modules
It provides a unique interface for programmers
Developers just need to learn only one API
The model is extensible to include other middleware or classic File Systems (also distributed)
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid - Catania 04/07/2008

7. gLite C/C++ API weakness
LGC Utils are not a framework
LGC Utils are not GOOD for Data Grid Applications
LGC Utils provide the only super interface
Doesn’t include GSIFTP
Doesn’t include GFAL
Misses Metadata integration (AMGA)
Messy overlap among different semantic APIs
GFAL is only a POSIX-like file manager interface, it should not wrap LFC API because just RFIO would be enough
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid - Catania 04/07/2008

8. gLite 100% Java Solution New AMGA is supported
LFC API
CNS API
AMGA is supported
only one super interface for Data Grid Application
GSAF is an Object Oriented Framework
Java 100% - Design Pattern based
Clear design : object modeling / functional modeling
Solves the fragmentation of DMS APIs
Solves the natural (due to OGSA) inconsistence of services
Provides ACID Transactional Model (coming soon)
Road Map
RFIO java native implementation
SRM encapsulation
GRelC integration
Secure Storage integration
modular and extensible
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid - Catania 04/07/2008

9. Transactional GSAF Transaction Manager means Challenge
ACID: Atomicity, consistence, Isolation, Duration
Provide developers with a Transaction Pattern
autocommit(), execute(), commit(), rollback()
performs several data manipulation in Atomic Mode
Hold Runtime Exceptions to limit Data Inconsistence
System Crash Recovery to avoid Data Loss
Challenge
SOA of Grid middleware makes transactions impossible at server side
GSAF works at client side
No full control to resources and services (resource locking)
No cache mechanism to save temporary data (action tracing)
No backup system to store previous data status (backward restore)
No full awareness of distributed resources usage (centralized access)
…..
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid - Catania 04/07/2008

10. Weakness/Solutions
Only datasets under the control of a given application are protected by T-GSAF
Metadata deletion/update can be very hard to compensate
Consider the command rm /dir/dir1/dir1.2/*; how can we restore it without preventively save all related information?
Sub directories, files, permission, stats, owner, ACLs, …
However, if a single DMS subsystem is itself transactional we can suppose to solve this problem enabling local transaction according to the nested transaction model
T={T1, …, Tn}
T.commint()  T1.commit() ∩ … ∩ Tn.commit()
T.rollback() if Tj T : Tj.rollback()
File deletion is impossible to compensate without a Cache/Backup mechanism
But it could be always the last operation, than if it fails we don’t need to compensate 
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid - Catania 04/07/2008

11. AMGA Web Interface
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid - Catania 04/07/2008

12. Grid Authoring & Provisioning
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid - Catania 04/07/2008

13. ADAT – Archivio Digitale Antichi Testi
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid - Catania 04/07/2008

14. References Downlod JARs file here Developer Guide Publications
Developer Guide
Publications
GSAF Grid Storage Access Framework. Enabling Technologies: Infrastructure for Collaborative Enterprises, WETICE Evry, France, June 2007 .
AMGA Web Interface - Proceedings of the 30st International Convention MIPRO, Conference on Grid and Visualization Systems (GVS). Opatija, Croatia May 2007.
A new paradigm to design, implement and deploy Grid oriented application: a biomedical use case. Proceedings of the Symposium Grid open days at the University of Palermo, pp Palermo, Italy, December 2007.
Incontro dei Progetti PON Avviso 1575 con il ROC di INFN Grid - Catania 04/07/2008