E-science grid facility for Europe and Latin America OurGrid E2GRIS1 Rafael Silva Universidade Federal de Campina Grande (Brazil) Itacuruça (Brazil), 2-15 November 2008

Overview Itacuruça (Brazil), E2GRIS1, OurGrid Architecture Infrastructure Jobs Installing an OurGrid Site –XMPP Server –Peer –Worker –Broker Running Jobs References

OurGrid OurGrid is an open, free-to-join, cooperative grid in which labs donate their idle computational resources in exchange for accessing other labs’ idle resources when nedeed Applications Bag-of-Tasks (BoT) –However, a single OurGrid task may itself be a parallel tightly- coupled application (written in MPI, for example) Itacuruça (Brazil), E2GRIS1,

Architecture Itacuruça (Brazil), E2GRIS1,

Infrastructure Broker –Client interface for job submission Worker –Executes the jobs submitted Peer –Controls the donating and consuming of the machines Discovery Service –Discover resources on the Grid Network of Favors (NoF) –Resource share incentive –Prevent free riding  Peers that only consumes resources Itacuruça (Brazil), E2GRIS1,

Communication Infrastructure Commune –Built on top of XMPP (Extensible Messaging and Presence Protocol). Thus, every communication between OurGrid components are intermediated by a XMPP server. –OpenFire Server (WildFire)  Itacuruça (Brazil), E2GRIS1,

Jobs Job Description File (JDF) Each job is a set of tasks, where each task can have three parameters: –Init  File list to be transferred to the worker –Remote  Command to be executed in the worker –Final  Retrieve the result of the task Itacuruça (Brazil), E2GRIS1,

Jobs (2) Requirements –Actually, the OurGrid only has two types of requirements specification available:  Operating System  Memory Size Itacuruça (Brazil), E2GRIS1, requirements: (os == windows) requirements: (mem == 1024) requirements: (os == windows) requirements: (mem == 1024)

Jobs (3) Types of Storage –$PLAYPEN (temporary storage) –$STORAGE (permanent storage) Other Variables –$PROC  Contain the grid machine chosen to run the task –$JOB  Job unique number –$TASK  Task unique number (within a job) Itacuruça (Brazil), E2GRIS1, put store

An example of a JDF file Itacuruça (Brazil), E2GRIS1, job : label : SimpleJob task: init : put Fat.class Fat.class remote : java -cp. Fat output-$JOB.$TASK final : get output-$JOB.$TASK output-$JOB.$TASK task : remote : hostname -f job : label : SimpleJob task: init : put Fat.class Fat.class remote : java -cp. Fat output-$JOB.$TASK final : get output-$JOB.$TASK output-$JOB.$TASK task : remote : hostname -f

Installing an OurGrid Site To install an OurGrid Site, it is only necessary to install the following services: –XMPP Server –Peer –Workers –Broker In addition to these services, the only requirement is Java 1.5 or higher Itacuruça (Brazil), E2GRIS1, Two ways to configure: - text mode - gui mode

Installing the XMPP Server Download the openfire package from: – Extract the tarball and start it – /bin/openfire start Open your browser and type: – :9090 And follow the instructions of the following link: – mid=2&lang=en#Anc_2http:// mid=2&lang=en#Anc_2 Itacuruça (Brazil), E2GRIS1,

XMPP Server Itacuruça (Brazil), E2GRIS1,

Peer Installation Download the peer package from the OurGrid’s website, uncompress it and start the graphical interface Configuring the peer Itacuruça (Brazil), E2GRIS1, $ unzip peer-.zip $ cd peer- $ sh peer gui $ unzip peer-.zip $ cd peer- $ sh peer gui

Peer Installation (2) Adding Workers Itacuruça (Brazil), E2GRIS1,

Peer Installation (3) Site Description File (SDF) Itacuruça (Brazil), E2GRIS1, workerdefaults : copyTo : scp $localfile $machine:$remotefile remExec : ssh -x $machine $command copyFrom : scp $machine:$remotefile $localfile worker : OS : linux servername : xmpp.ourgrid.org username : worker-001 worker : OS : windows servername : xmpp.ourgrid.org username : worker-002 workerdefaults : copyTo : scp $localfile $machine:$remotefile remExec : ssh -x $machine $command copyFrom : scp $machine:$remotefile $localfile worker : OS : linux servername : xmpp.ourgrid.org username : worker-001 worker : OS : windows servername : xmpp.ourgrid.org username : worker-002

Peer Installation (4) Adding Brokers Itacuruça (Brazil), E2GRIS1,

Worker Installation Download the worker package from the OurGrid’s website, uncompress it and start the graphical interface Then, configure the same variables again: –XMPP User Name –XMPP User Password –XMPP Server Name IMPORTANT! –Put the public key of the peer in the “Worker Configuration” tab. Itacuruça (Brazil), E2GRIS1, $ unzip worker-.zip $ cd worker- $ sh worker gui $ unzip worker-.zip $ cd worker- $ sh worker gui

Broker Installation Download the broker package from the OurGrid’s website, uncompress it and start the graphical interface Then, configure the same variables again: –XMPP User Name –XMPP User Password –XMPP Server Name Itacuruça (Brazil), E2GRIS1, $ unzip broker-.zip $ cd broker- $ sh broker gui $ unzip broker-.zip $ cd broker- $ sh broker gui

Broker Installation (2) Now, it is necessary specify the peers the broker will be able to submit jobs Grid Description File (GDF) Itacuruça (Brazil), E2GRIS1, peer: label: username : peer-rafael servername : rs peer: label: username : peer-rafael servername : rs

Running Jobs Demonstration Itacuruça (Brazil), E2GRIS1,

References OurGrid web site – XMPP Community – Openfire – Itacuruça (Brazil), E2GRIS1,

Questions… Itacuruça (Brazil), E2GRIS1,