1. EGrid Software Packages Overview

2. EGrid Introduction Egrid Introduction : A description of the main software packages EGrid Inside : A detailed description of the software packages EGrid Inside

3. Data Management Primitive data management operations (upload, download, withdrawal, etc.). Operations common to all end users. User requirements Data managementTransformationsFilters

4. Transformations Advanced data management operations. Operations common to restricted group of persons or to individual scientists. The same computational infrastructure is used by all project participants. User requirements Data managementTransformationsFilters

5. Advanced data management operations. Operations particular to each researcher. User requirements Data managementTransformationsFilters

6. EGrid Inside Choose your directions:  Data Management Data Management  Transformations Transformations  Filters Filters

7. Data Management Operational Classification Data upload Data download Data removal Data movements Data browsing Functional Classification Cache management. File management.

8. Data Upload Uploads data in the local cache :  egrid-cache-upload Example: egrid-cache-upload./nyse-2002-01.tar.gz lfn:/00010/cd/nyse-2002-01.tar.gz Uploads data in the main storage :  egrid-upload Example: egrid-upload./nyse-2002-01.tar.gz lfn:/00010/cd/nyse-2002-01.tar.gz cache Main storage

9. Data Download Downloads a file from the Grid to the local machine :  egrid-download … Example: egrid-download lfn:/00010/cd/nyse-2002-01.tar.gz cache Main storage storage

10. Data Withdrawal Removes a file from the local cache :  egrid-cache-remove Example: egrid-cache-remove lfn:/00010/cd/nyse-2002-01.tar.gz Removes a file from the Grid :  egrid-remove-file Example: egrid-remove-file lfn:/00010/cd/nyse-2002-01.tar.gz cache Main storage storage

11. Data Movement - Replication Replicates a file from the local cache to the main storage :  egrid-replicate-to-main | … Example: egrid-replicate-to-main lfn:/00010/cd/nyse* Replicates a file from the main storage to the local cache :  egrid-replicate-to-cache | … Example: egrid-replicate-to-cache lfn:/00010/cd/nyse* cache Main storage

12. Data Movement - Copying Copy a file from the local cache to the main storage with a new logical filename :  egrid-copy-to-main Example: egrid-copy-to-main lfn:/00010/cd/nyse-2001.tar.gz lfn:/brownlees/nyse-2001.tar.gz Copy a file from the main storage to the local cache with a new logical filename :  egrid-copy-to-cache Example: egrid-copy-to-main lfn:/brownlees/nyse-2001.tar.gz lfn:/brownlees/nyse-test.tar.gz cache Main storage

13. Data Browsing Shows files in the local cache:  egrid-cache-list-files Example: egrid-cache-list-files lfn:/*brownlees* Shows files in the Grid:  egrid-list-files Example: egrid-list-files lfn:/*brownlees* Shows where a file is located:  egrid-where Example: egrid-where lfn:/*brownlees* cache Main storage storage

14. Transformations Operations regarding two classes of users  End users – users performing transformations.  Software developers – users developing transformation programs.

15. Transformations – End Users 1 Are mainly used to transform data CDs from a proprietary format to ASCII. Are performed using the job submission mechanisms. Are independent on the data format offering the possibility to reuse the same infrastructure in other contexts. The module can be used as a starting point for custom transformations.

16. Transformations – End Users 2 Transforms a file applying a certain transformation :  egrid-transform Example: egrid-transform taq-trades lfn:/brownlees/nyse-test.tar.gz lfn:/brownlees/test-data/  egrid-transform [ ] Example: egrid-transform taq-trades lfn:/brownlees/nyse-test.tar.gz lfn:/brownlees/test-data/ tickers.conf Generates to standard output a.JDL file which can be further customized :  egrid-transform –n Example: egrid-transform -n taq-trades lfn:/brownlees/nyse-test.tar.gz lfn:/brownlees/test-data/  egrid-transform –n [ ] Example: egrid-transform -n taq-trades lfn:/brownlees/nyse-test.tar.gz lfn:/brownlees/test-data/ tickers.conf

17. Transformations – Software developers Guidelines are provided to software developers on how to develop transformation programs which can be interconnected with the Grid technologies.  What interfaces the programs must implement.  What the runtime environment is.  Where the programs are installed.

18. Filters Are custom operations which are dependent on the researcher needs. Software modules will be implemented on demand.  Example : Requirement : Get all the files containing data in a time interval. Solution : A program which analyses the LFNs in RLS and decides if the file contains data in the requested interval.