1. S.Cozzini www.egrid.it Den Haag, 25th November 2004
The EGRID project
S.Cozzini
Den Haag, 25th November 2004

2. Agenda The EGRID project (short overview) The applications for EGEE
The added value for EGRID
The added value for EGEE
Our commitments

3. The EGRID project A FISR project funded by MIUR (2003) Original Aim:
Use of grid paradigm for research on complex system in economics and finance
MIUR asked to similar FISR projects to coordinate together
ICTP new task:
Provide a National Italian Facility for economical and financial data using grid technology

4. EGRID and its friends Area project (M.Fanni)
“Applicazioni di tecniche di soft
computing (neural network/genetic algorithm)
per analisi di sistemi complessi
applicati alla finanza”
Financial Databases
INFM project (R. Mantegna)
“Dinamica di altissima frequenza
nei mercati finanziari”
High Frequency Financial Data
User Requirements
ICTP EGRID project (S.Baroni)
“Utilizzo di tecnologie Grid in modellazione sistemi
complessi in ambito economico e finanziario”
Provide the infrastructure using
Grid computing

5. EGRID project: three lines ..
Research in Grid Computing (G.Pau UCLA in collaboration with C.Kesselman)
Research in finance/economics on the Grid (M. Gerla UCLA, S.D’addona Columbia)
Implementation of the EGRID infrastructure (ICTP Trieste)

6. EGRID national infrastructure
Version 1.0 released: 8-9 october 2004 (meeting in Trieste)
Guide lines:
Use available technology
Develop and integrate only missing tools
Grid provider: Grid.it
Built after clearly identifying user requirements

7. User requirements in a nutshell
High Frequency Financial Data: raw data
Different Stocks Exchanges (NYSE/LSE/Paris/BorsaMilano) .
Restricted access to raw data acquired under legally binding disclosure policies ( privacy and security)
Raw Data should be:
Safe kept/backed-up/shared on the GRID
Actions on the raw data: transformations
Filtering procedures (size reduced)
Building procedures (size increased)
Authorized and authenticated access to financial databases

8. The EGRID community
10 Different research groups in Italy (distributed community)
2 different groups in USA
~ 40 researchers
European groups very much interested in joining our infrastructure..

9. EGRID 1.0: basic architecture
Version 1.0 released: 8-9 october 2004 (meeting in Trieste)
Globus based EDG middleware.
Specific VO within Grid.it. ( our Grid provider..)
A star shaped topology for data management:
Central SE (large and backed up storage ) hosted in grid.it
Local SE hosted in research centres acting as a temporary storage space.
Data management based on EDG Replica manager

10. A graphical view AN Roma PADOVA CE WN WN Main Storage TS WN WN
SuperNode:
CE+SE+WN FI PA AL

11. EDG & Egrid EDGLimits Other problems EGRID solutions
Data privacy not guaranteed
EDG software installations: complex/invasive…
Other problems
Limited Bandwidth among different sites.
EGRID solutions
Temporary Work-around on privacy/security
Temporary cache system based on star-topology
Ad hoc procedures for installing (Live CD for UI and SuperNode)

12. EGRID solution: UI (2) Egrid UI now also on Live CD:
Based on Knoppix 3.6

13. EGRID applications in EGEE
Filtering/Building procedures
Now inserted in a software layer built on the top of EDG commands and based on the EGRID architecture
Insert them in web portal (GENIUS portal ?)
Pricing financial derivatives by means of MCS
Already parallel (almost embarrassingly)
Under porting on GTK3 using mpich-g2 (USA partners)
Under porting on the EGRID infrastructure

14. EGRID added values in joining EGEE
EGEE middleware is addressing some of our requirements
Egrid needs to de aligned with major european grid research avoiding duplications
Egrid partners want to include European research group: infrastructure needs to be at European scale
EGRID is now addressing Banks and Financial Institutions interested in using a continental grid instead of a national one.

15. EGEE added values in accepting EGRID
very important security/privacy issues addressed
A well trained and motivated group
Scientific & academic community with a large technological transfer
Financial & economical entities can play a determinant role in spreading grid technologies outside academic/scientific world

16. EGRID working plan within EGEE
Applications domain
Development of a portal for Filtering procedures
Porting of parallel applications:
pricing financial derivatives (USA partners)
Other financial applications
Data management domain:
SRM ad hoc implementation to be used as a secure and general purpose caching system
EGRID is now participating actively in the STORM server deployment together with INFN/CNAF

17. Other partnership/collaborations
ITC-IRST Trento
Collaboration on porting similar applications on the GRID
INAF-Trieste
Some contacts to work together on similar needs
A local grid Testbed
INFN/CNAF
Storm server

19. EGRID solution: gridmap file
pool account only on CE e WN
security holes persist but less pronounced
Personal accounts on SE:
ldap server maintained by Egrid team
Scripts/automatic procedures for creating/managing users

20. EGRID solution: UI (1)
UI is independent from Linux distribution (mandrake/debian/fedora/Suse etc...)
No need to use LCFG server
UI includes only needed software and EGRID add-ons
No root privileges needed: user can install it in home dir
Freely Available
~ 500MB

21. EGRID solution: the SN Egrid SuperNode Live CD Based on Knoppix 3.6
Standard tool for installing local nodes SN SE CE WN

22. Examples of EGRID procedures
Loading files from UI and storing them on local and central SEs:
EGRID command:
egrid-upload <file> <lfn>
Example:
egrid-upload ./nyse tar.gz lfn:/00010/cd/nyse tar.gz
Transforming data on the grid:
egrid-transform <transformation> <lfn> <lfnprefix>
egrid-transform taq-trades lfn:/brownlees/nyse-test.tar.gz lfn:/brownlees/test-data/
Transformation: application ported to the grid
lfn: input logical filename
lfnprefix: prefix for the output filename

23. EGRID caching system Avoid multiple download of the same file
Coherence with access control procedure on data
Minimize latencies for data trasfer
overcome star topology adopted so far

24. What next: Full Caching system for EGRID
Collaboration with Grid.it on common requirements:
SRM on parallel filesystems
Parallel Runs
I/O performances on filtering procedures..
Accounting system:
Important requirement in near future (it must present in final release)