10 April 2003Deploy Grid in Israel Universities1 Deploy Grid testbed in Israel universities Lorne Levinson David Front Weizmann Institute

10 April 2003Deploy Grid in Israel Universities2 The Grid Problem Seamless, flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources Enable virtual organizations (VO)s to share geographically distributed resources as they pursue common goals assuming the absence of: central location, central control, trust relationships.

10 April 2003Deploy Grid in Israel Universities3

10 April 2003Deploy Grid in Israel Universities4 DataGRID Job submission UI JDL ResourceBroker (Match requests to resources) Job Submission Service (Condor G) Storage Element (SE) Compute Element (CE) InformationService Job Status ReplicaCatalogue DataSets info Author. &Authen. Job Submit Event Job Query Job Status Input “sandbox” Input “sandbox” + Broker Info Globus RSL Output “sandbox” Job Status Publish grid-proxy-init Expanded JDL SE & CE info Logging & Book-keeping

10 April 2003Deploy Grid in Israel Universities5 File replication Read StorageElement InformationService Replica Catalogue (RC) Translates logical FN to physical FN Ask for a LFN Ask for LFN Return name & protocol Return PFN Publish SE & CE info Write Writes a file RC only tells where files are In order to get them, one may use Gridftp

10 April 2003Deploy Grid in Israel Universities6

10 April 2003Deploy Grid in Israel Universities7 Data Management future : Reptor: The Next Generation Replica Manager Replica Manager Replica Metadata Replica Location File Transfer Optimization Transaction Consistency Preprocessing Postprocessing Subscription Client Reptor Giggle RepMeC Optor GDMP

10 April 2003Deploy Grid in Israel Universities8 The Grid is NOT (yet) Not: Super computing –The trend: Many PCs rather than a big computer Not: Can not guarantee fast response, mainly because of network limitations Not (yet) : A total solution for distributed computing –Batch rather than interactive or stream oriented: Examples: the grid is not geared to Support games Show a lecture on line Not (yet) : Ultimate parallel computing solution –Supported class: Embarrassingly Distributed (ED)

10 April 2003Deploy Grid in Israel Universities9 Classes of parallel computing Supported by Grid

10 April 2003Deploy Grid in Israel Universities10 Assumptions Grid should be functional Users are scientists Prefer open source Run on Linux RedHat 7.3

10 April 2003Deploy Grid in Israel Universities11 How? Define priorities for required functionality and suppliers of relevant SW packages Support wide functionality scope. Each institution will select its subset Find best, up-to-date, reasonably tested, set of SW packages Honor Glue - USA-Europe collaboration The actual content and versions will change because SW is evolving Stick to one SW provider that does exactly that

10 April 2003Deploy Grid in Israel Universities12 LHC Computing Grid - LCG LCG is the project that selects at tests computing grid SW for LHC experiments in CERN LCG will supply highly integrated Grid SW Utilizes American and European Grid SW Honors GLUE: America-Europe collaboration

10 April 2003Deploy Grid in Israel Universities13 LCG main SW suppliers Globus: A bag of Grid services –GSI: Grid Security Infrastructure –GRAM: Grid Resource Allocation & Management –MDS: Monitoring and Discovery Service –GridFTP: file transfer over the net VDT: –Condor-G: Submit jobs and monitor job consistency EDG: –Resource Broker: Select on what Computing Element (CE) to submit job –Data replication –VOMS: Virtual Organization Management System –TBD : LCFG: Installation and configuration GLUE: –Make sure that USA and Europe grid SW work together

10 April 2003Deploy Grid in Israel Universities14 LHC Computing Grid - LCG USAEurope

10 April 2003Deploy Grid in Israel Universities15 LCG-1 Components 1/2003 VDT Globus patches ClassAds: Condor: Condor-G: EDG Certificates EDG CRL Update: MDS patches + static GLUE schema EDG mkGridmap EDG GDMP ( to be migrated to 4.0) Replica Catalogue Server ( to be migrated to RLS) edg-rc-server Replica Manager edg-replica-manager Replica Catalogue API /CLI ReplicaCatalogue [4] Resource broker: Workload Management (informationindex , jobsubmission- [glue-aware], lbserver , locallogger , proxy , userinterface ) LCG lcg-version Configuration solutions DataTAG 1.0 GLUE Information Providers Edt-monitor

10 April 2003Deploy Grid in Israel Universities16 LCG-1 Components 3-5/2003 VDT 1.x.x 3/2003 Globus 2.2.x RLS pre-release Condor: Info Providers (partial Glue set) EDG /2003 R-GMA x.x.x (To be evaluated) Reptor x.x RepMecC x.x Resource Broker x.x (Glue) VOMS x.x.x (To be evaluated ) LCG 3/2003 lcg-version Test-suite DataTAG 1.0 3/2003 Edt-monitor Dilemma: Should the more advanced, but less tested version be used?

10 April 2003Deploy Grid in Israel Universities17 Grid functionality: Security FunctionalitySupplied byPriority Authentication: Certificate Authority (CA): Allocate user and host certificates Europe, USA: Many certification authorities Israel: Machba to be a certification authority Essential single sign-onGlobus: Grid Security Infrastructure (GSI) Essential Certificate delegationGlobus: GSIEssential Authorization: Virtual Organization (VO) management Globus: Mapfile - Map local users to grid users EDG mkGridmap EDG 2.0: VOMS High Firewall: Submit grid jobs securely through firewalls Not supplied yetHigh

10 April 2003Deploy Grid in Israel Universities18 Grid functionality: Job submission FunctionalitySupplied byPriority Job submissionGlobus: Grid Resource Allocation & Management (GRAM) Essential Job consistencyCondor-GVery high Resource Broker (RB): Select a Computing Element (CE) for job submission EDG (Glue)The bigger the number of CE’s, The more a RB is needed Scheduling policy: Control local resource allocation according to policy Not supplied yetHigh Job dependenciesEDG 2.0 based on Condor DAGMan Support interactive jobsEDG 2.0

10 April 2003Deploy Grid in Israel Universities19 Grid functionality: Data management FunctionalitySupplied byPriority Copy files over the netGlobus: GridFTPEssential Data mirroringEDG: GDMP Very high Replica Catalog+ management EDG: Replica catalog EDG 2.0: reptor The bigger the number of (big) files to be read at many sites, The more replica management is needed Virtual DataNot available

10 April 2003Deploy Grid in Israel Universities20 Grid functionality: Information services FunctionalitySupplied byPriority Information services LDAP based Globus: Monitoring and Discovery Service (MDS). MDS resource information models physical and logical components of a compute resource as a hierarchy of elements, using LDAP Essential RDB basedEDG 2.0: R-GMA Essential Job monitoringNot available?High

10 April 2003Deploy Grid in Israel Universities21 Grid functionality: Graphical User Interface (GUI) FunctionalitySupplied byPriority Graphical (web based) User Interface (GUI) Genius: EDG simple portal for job submission EDG 2.0 will have more GUI High

10 April 2003Deploy Grid in Israel Universities22 Links l Globus Project™ – l Global Grid Forum – l European Data Grid EDG – datagrid / datagrid /