Current ALICE Grid activities in Mexico Grid of the Americas workshop Luciano Diaz ICN-UNAM February /2/2011Grid of the Americas Workshop1

Outline Introduction History 2006 – 2011 Current status site ICN-UNAM Conclusions 11/2/2011Grid of the Americas Workshop2

Introduction The ALICE experiment has based their data and computing model on a GRID architecture based on several sites distributed around the world. One Tier 0 located at CERN 88 sites, most of them Tier 1 and 2 11/2/2011Grid of the Americas Workshop3

Mexico is part of ALICE Grid since 2007 when it joined the ALICE production infrastructure through the site ICN-UNAM. 11/2/2011Grid of the Americas Workshop4

History On 2006 the adventure began with tochtli, a cluster with 16 nodes: 32 Xeon 2.4GHz cores and 500MB of memory per core. A 30 Mbps CUDI connection used mainly for videoconferences. The opportunity to join the EELA testbed and ALICE production infrastructure. 11/2/2011Grid of the Americas Workshop5

Site ICN-UNAM on 2006 tochtli 11/2/2011Grid of the Americas Workshop6

2006 June 2006 First hardware upgrade of tochtli to run ALICE jobs: Upgrade of memory: 500 MB to 1.5 GB per core. 1TB of disk for a Storage Element required Site manager registered in the EELA - VO 11/2/2011Grid of the Americas Workshop7

2006 July 2006 Decision to install a full gLite/LCG site and not a pure AliEn site. First steps to create a Resource Center with EELA: site EELA-UNAM August 2006 Site manager registered in ALICE - VO 11/2/2011Grid of the Americas Workshop8

2006 September 2006 Problems to setup ALICE VoBOX SE required First test job arrived on 25 September, but RB problems due to network connectivity with CERN and UFRJ, several minutes to receive a job. Also middleware problems to use the same CN with more than one VO, voms extension will arrive later. 11/2/2011Grid of the Americas Workshop9

2006 At the end of 2006: Many network problems: Mixed routing between CUDI and commodity links. Low bandwidth, better from home Not all was bad: EELA was working on the creation of several Certification Authorities in LA, including Mx-UNAM 11/2/2011Grid of the Americas Workshop10

2007 On January 30, 2007 Latchezar wrote: Dear Luciano, Good job - the site is running fine and we are getting first batch of jobs done at UNAM! Cheers, Latchezar 11/2/2011Grid of the Americas Workshop11

2007 The network latency continue to high. March 2007 More Grid resources added to UNAM (IBT,CCG,FESC and Super), is necessary to split the resources on "production" and "certification" status on the EELA testbed: EELA-UNAM (production) EELA-UNAM-CERT (certification) EELA-UNAM-ALL (both) 11/2/2011Grid of the Americas Workshop12

2007 April 2007 Network characterization, improvement of the network configuration: UNAM-CUDI-CLARA TCP tuning on nodes Symmetric routing Avoid traffic to Europe from Sao Pablo May 2007 Air condition problems, improvement of the room Also UPS overload, firsts time the cluster runs 100% load. 11/2/2011Grid of the Americas Workshop13

Band width test from CERN to ICN using 1,4,8 and 16 conecctions and 264 KB of TCP windows size 11/2/2011Grid of the Americas Workshop14

2007 September 2007 Exploring T-2 relationship UNAM-Hou. Start network testing Request of jumbo frames support News: UNAMGRID-CA waiting for green light to start operations 11/2/2011Grid of the Americas Workshop15

2008 January 2008 VOBOX migration to gLite 3.1 February 2008 UNAMGRID-CA: accredited by IGTF, start operations More network tests UNAM CERN March 2008 Maintenance of the multimode optical fiber from ICN to campus network,link at 100Mbps. 11/2/2011Grid of the Americas Workshop16

2008 April 2008 Middleware upgrade from gLite 3.0 to gLite 3.1 Testing UNAM-Hou, Jumbo frames support request, next problem: the end point (ICN) is connected at 100Mbps, problematic with existing routing. May 2008 Call to collect information about projects with high BW requirements. 8 projects signed a document to request BW improvement. 11/2/2011Grid of the Americas Workshop17

2008 June 2008 Arrives the first 64bits processors: 16 cores in 2 nodes 30 TB of storage for ALICE and AUGER but no shared with Grid due to BW problem. December 2008 Second E3 to CUDI, duplicated the BW to 60Mbps 11/2/2011Grid of the Americas Workshop18

2009 February 2009 ICN resources create the site: ICN-UNAM Creation of grid.unam.mx domain for Grid resources distributed within the UNAM. Problems renewing certificates from GRIDUNAM-CA March 2009 New worry: ROC-CERN will disappear March 3: The idea is born to create the ROC-LA to support HEP sites in the LCG in Latin America. March 20: UNAM is invited to be certified by ROC-CERN to join the EGEE infrastructure. 11/2/2011Grid of the Americas Workshop19

2009 September 2009 First advice: The 32bits arch will be discontinued from ALICE production sites September 17: New monomode optical fiber, 1Gbps at ICN to DGSCA, including a subnet with 50 IPs Direct link from ICN to I2 with jumbo frames support. September 30:ROC-LA start operations December new 64 bits nodes: Creation of tochtli64 with 40 cores 11/2/2011Grid of the Americas Workshop20

2010 January 2010 UNAM site blocked from ALICE production due to obsolete 32 bits VOBOX. March 2010 UNAM back to ALICE production after install tochtli64 with gLite 3.2. Now all 64 bits, including a CREAM- CE and VOBOX Problems renewing certificates automatically in the GRIDUNAM-CA. 11/2/2011Grid of the Americas Workshop21

2010 June 2010 New cluster, financed by CTIC arrives all 64 bits, 104 cores added to tochtli64 for local and grid use. 11/2/2011Grid of the Americas Workshop22

2010 October 2010 The consolidation of efforts of last years: 1 Gbps link from UNAM to academic networks in USA. 11/2/2011Grid of the Americas Workshop23

2011 Current status Site ICN-UNAM on ALICE and AUGER production infrastructure VO’s supported: AUGER, EELA, BIOMED Certified site on EGI infrastructure through ROC-LA Support of the GISELA applications through EELA VO Up to 146 cores with 64 bits Still operating 32 bits nodes: 18 cores Direct connection from ICN to I2 at 1Gbps with Jumbo frames support 11/2/2011Grid of the Americas Workshop24

Site ICN-UNAM on 2011 tochtli64 11/2/2011Grid of the Americas Workshop25

Conclusions Installing and maintaining a grid site in production is definitely not a one-click process. Double click to install a site 11/2/2011Grid of the Americas Workshop26

The site ICN-UNAM has survived thanks to the really big effort from the UNAM to improve and provide all the necessary infrastructure for the ALICE experiment and the invaluable support from the ALICE Grid experts. 11/2/2011Grid of the Americas Workshop27

Thanks for all the support provided: ICN Director: Alejandro Frank HEP visionary group: Guy Paic, Lukas Nellen, Alejandro Ayala Computer department: Antonio Ramirez CERN Federico Carminati, Latchezar Betev, Patricia Mendez CTIC and CONACyT EELA, CUDI and ROC-LA people We would like to thank the DGTIC (before DGSCA) for their interest and support of the GRID activities in the ICN since the very beginning Director: Ignacio Ania Security GRIDUNAM-CA: Ruben Aquino Telecommunications: Oscar Fermandez, Salvador Cardenas, Hans Reyes (CUDI / UNAM) Supercomputing: Jose Luis Gordillo, Eduardo Murrieta 11/2/2011Grid of the Americas Workshop28

Through 2006 to 2010 several things has been improved and deployed, now we have the conditions to grow up and provide resources to the ALICE Grid. Projects / Applications ALICE, AUGER, HAWC, non HEP Knowledge of the Middleware gLite Grid Infrastructure ROC-LA GISELA Better Network 1Gb/s to I2, Jumbo frames Security GRIDUNAM -CA Tier -1 Supercomputing expertice KanBalam, Bakliz, Mixbaal 11/2/2011Grid of the Americas Workshop29 Our next challenge is large-scale data management at PB scale.

The ICN-UNAM was the first ALICE node in Latin America until 2010 and is interested to be the first Tier-1 for ALICE in the region. 11/2/2011Grid of the Americas Workshop30