1. October,10-14 2005 HEPiX Fall 2005 at SLACSLAC Site Report Roberto Gomezel INFN

2. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 2 Outline of Presentation New Computing Committee New Computing Committee Computing Environment Computing Environment Security Security Services Services Network Network AFS AFS INFN Farms INFN Farms Tier1@CNAF Tier1@CNAF

3. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 3 New Computing and Networking Committee Last June the previous Computing and Networking committee expired so a new one was formed Last June the previous Computing and Networking committee expired so a new one was formed Mauro Morandin (INFN-Padova) is the new chairman of the committee and some members have been replaced Mauro Morandin (INFN-Padova) is the new chairman of the committee and some members have been replaced This committee has been charged with the following explicit mission: This committee has been charged with the following explicit mission: –To coordinate implementation of computing farm with particular regard to LHC Tier-1, Tier-2 and Tier-3 –To participate to national and international coordination committees focused on topics related to CNC interests –To promote innovation and technological coordination of computing and networking of INFN sites –To coordinate and to finance technological development and maintenance of computing resources

4. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 4 Computing Environment and security Most of boxes are PCs running Linux or Windows Most of boxes are PCs running Linux or Windows Mac OS boxes keep on living Mac OS boxes keep on living VPNs available in many sites VPNs available in many sites –Cisco and Netscreen boxes using IPsec –SSL VPNs are currently used by some sites Interested results at LNF using Cisco VPN Concentrator Interested results at LNF using Cisco VPN Concentrator Network Security Network Security –Dedicated Firewall machines just in a few sites –Implemented with access lists on router connected to WAN INFN Site Report – R.Gomezel

5. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 5 Desktop PCs running Linux, Windows and Mac OS PCs running Linux, Windows and Mac OS SL and SLC are equally used SL and SLC are equally used A few sites use Caspur BigBox release A few sites use Caspur BigBox release Some units are taking advantage of outsource support for windows desktop environment because of lack of personnel Some units are taking advantage of outsource support for windows desktop environment because of lack of personnel INFN Site Report – R.Gomezel

6. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 6Backup Tape Libraries used: Tape Libraries used: –IBM Magstar – just used at LNF –DLT, LTO2 – wide spread –LTO3 will naturally replace LTO2 drives in the next future Backup tools: Backup tools: –IBM Tivoli – quite used –HP Omniback – quite used –Atempo Time Navigator – just a few sites –Domestic tools - widespread INFN Site Report – R.Gomezel

7. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 7 Wireless LAN Access point running standard 802.11a,b,g Access point running standard 802.11a,b,g All sites are using wireless connection during meeting or conferences All sites are using wireless connection during meeting or conferences Most of them use it to give connection to laptop computers Most of them use it to give connection to laptop computers A specific working group keeps on investigating in order to provide a common solution to solve security issues A specific working group keeps on investigating in order to provide a common solution to solve security issues –To go beyond the permission based on Secure Port filtering (MAC Address) – (very poor) 802.1X is a d solution but it is not implemented and working well on all platforms in use 802.1X is a good solution but it is not implemented and working well on all platforms in use –To investigate the standard 802.16 (WiMAX) INFN Site Report – R.Gomezel

8. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 8E-mail Mail Transfer Agent Mail Transfer Agent –Sendmail – widespread and more used (70%) –Postfix – a few sites (30%) (increased if compared with last report to confirm the trend reported last year) Hardware and OS Hardware and OS INFN Site Report – R.Gomezel

9. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 9 E-mail user agent All INFN sites provide an HTTP mail user agent All INFN sites provide an HTTP mail user agent –IMP –SQUIRREL (increased use is due to its light impact and good response time) –Others: IMHO, Open WebMail, Cyrus+Roxen… IMHO, Open WebMail, Cyrus+Roxen… Other mail user agents commonly used: Other mail user agents commonly used: –Pine, Internet Explorer, Mozilla, Thunderbird… INFN Site Report – R.Gomezel

10. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 10 E-mail antispam The last Computing and Networking Committee decided to subscribe a nation wide license for using Sophos as common tool to reduce junk e-mail and to provide antivirus control The last Computing and Networking Committee decided to subscribe a nation wide license for using Sophos as common tool to reduce junk e-mail and to provide antivirus control Some sites used RAV or SPAM Assassin Some sites used RAV or SPAM Assassin By the end of this year every site is supposed to move to Sophos not only for the pure message functionality but also as antivirus tool for PCs By the end of this year every site is supposed to move to Sophos not only for the pure message functionality but also as antivirus tool for PCs Only authorized mail relays are allowed to send and receive mail for a specific site Only authorized mail relays are allowed to send and receive mail for a specific site An increasing number of sites are filtering outbound connections on port 25 to prevent users from sending viruses An increasing number of sites are filtering outbound connections on port 25 to prevent users from sending viruses unconsciously INFN Site Report – R.Gomezel

11. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 11 INFN network LAN backbone network mainly based on Gigabit Ethernet LAN backbone network mainly based on Gigabit Ethernet –10 Gbit Ethernet switches used in computing farm The INFN WAN network is completely integrated into the GARR, providing a backbone connectivity at 54 Gbps The INFN WAN network is completely integrated into the GARR, providing a backbone connectivity at 54 Gbps –POP typical access bandwidth for INFN sites: 34Mbps, 155 Mbps, 622 Mbps and Gigabit Ethernet –CNAF Tier-1 will be connected at 10Gbps soon –There are still just a few small research groups connected via multiple 2Mbps links because of lack of efficient telecommunication infrastructure –Access to GEANT2: N * 10Gbps links soon INFN Site Report – R.Gomezel

12. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 12AFS INFN sites keep on using AFS services to share data and software throughout sites INFN sites keep on using AFS services to share data and software throughout sites Local cells have completely moved or are moving to Linux boxes running OpenAFS software Local cells have completely moved or are moving to Linux boxes running OpenAFS software The migration of INFN.IT authentication servers from Kerberos IV to Kerberos V was accomplished last June The migration of INFN.IT authentication servers from Kerberos IV to Kerberos V was accomplished last June –A Kerberos V master server has been installed on a Linux machine: k5.infn.it –The former 3 AFS authentication servers (CNAF,Naples and Rome) have been reconfigured as Kerberos V slave servers K5 WG is now working in order to test the usage of trust relationship authentication between different INFN cells K5 WG is now working in order to test the usage of trust relationship authentication between different INFN cells INFN Site Report – R.Gomezel

13. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 13 INFN Site Farm: update A lot of sites are configuring and integrating computing facilities and local experiment-specific farm into a unique computing farm A lot of sites are configuring and integrating computing facilities and local experiment-specific farm into a unique computing farm Widespread deployment of SAN infrastructure to connect storage systems and computing units Widespread deployment of SAN infrastructure to connect storage systems and computing units –GPFS file system is becoming the most adopted as an efficient way of providing a cluster file system and volume manager –The increasing usage allows people to have support from other sites when problems arise –Even though Tier-1 is evaluating to move to Lustre because of lack of support from IBM on GPFS within a heterogeneous environment There is an increasing use of LSF as tool for submitting jobs to computing farm using different queues There is an increasing use of LSF as tool for submitting jobs to computing farm using different queues –Server license hosted at CNAF – Tier1 –Incoming sites can take advantage of the increasing experience coming from Tier1 and other units like Padua, Pisa and Catania INFN Site Report – R.Gomezel

14. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 14 Storage WG The last CNC promoted the creation of a storage working group The last CNC promoted the creation of a storage working group This group has been working since march 2005 This group has been working since march 2005 Main tasks Main tasks –To evaluate the opportunity of using Fibre Channel technology as common infrastructure for Computing Facility at each site –To investigate on the most common distributed file systems available evaluating performance and reliability With particular regard to the startup of next Tier-2 farms With particular regard to the startup of next Tier-2 farms –To keep in touch with the HEPiX Storage task force activity –To take into account the impact of GRID requirements on storage file system First status report at CNC meeting next week First status report at CNC meeting next week INFN Site Report – R.Gomezel

15. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC TIER-1@CNAF Status Report: Introduction TIER-1@CNAF Status Report: Introduction Location: INFN-CNAF, Bologna (Italy) Location: INFN-CNAF, Bologna (Italy) –one of the main nodes of GARR network Computing facility for INFN HNEP community Computing facility for INFN HNEP community –Partecipating to LCG, EGEE, INFNGRID projects Multi-Experiment TIER1 Multi-Experiment TIER1 –LHC experiments –VIRGO –CDF –BABAR –AMS, MAGIC, ARGO, PAMELA,… Resources assigned to experiments on a yearly basis. Resources assigned to experiments on a yearly basis. CNAF Tier-1 Report – L.Dell’Agnello

16. October,10-14 2005 HEPiX Fall 2005 at SLAC SLACInfrastructure Hall in the basement (-2 nd floor): ~ 1000 m 2 of total space Hall in the basement (-2 nd floor): ~ 1000 m 2 of total space –Easily accessible with lorries from the road –Not suitable for office use (remote control) Electric power Electric power –220 V mono-phase (computers) 4 x 16A PDU needed for 3.0 GHz Xeon racks 4 x 16A PDU needed for 3.0 GHz Xeon racks –380 V three-phase for other devices (tape libraries, air conditioning etc…) –UPS: 800 KVA (~ 640 KW) needs a separate room (conditioned and ventilated). needs a separate room (conditioned and ventilated). –Electric Generator: 1250 KVA (~ 1000 KW)  up to 160 racks (~100 with 3.0 GHz Xeon)  Expansion under evaluation CNAF Tier-1 Report – L.Dell’Agnello

17. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC HW Resources (1/2) CPU: CPU: –700 biprocessor boxes 2.4 – 3 GHz (+70 servers) –150 new Opteron biprocessor boxes 2.6 GHz 1300 KSi2k Total 1300 KSi2k Total Decommissioning ~ 100 WNs (~ 150 KSi2K) moved to test farm Decommissioning ~ 100 WNs (~ 150 KSi2K) moved to test farm –Each CPU equipped with FE switch with 2xGb uplinks to core switch Disk: Disk: –FC, IDE, SCSI, NAS technologies –470 TB raw (~ 430 FC-SATA) –Disk servers connected via GE to core switch CNAF Tier-1 Report – L.Dell’Agnello

18. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC 18 HW Resources (2/2) Tapes: Tapes: –Stk L180 18 TB –Stk 5500 6 LTO-2 with 2000 tapes  400 TB 6 LTO-2 with 2000 tapes  400 TB 2 9940B with 800 tapes  200 TB 2 9940B with 800 tapes  200 TB Networking Networking –30 rack switches  46 FE UTP + 2 GE FO –2 core switches  96 GE FO + 120 GE FO + 4x10 GE –Foreseen backbone upgrade to 10 Gbps –3x1Gbps links to WAN (on going upgrade to 10 Gbps) 1 Gbps production link 1 Gbps production link 10 Gbps Service Challenge (LHCOPN) link 10 Gbps Service Challenge (LHCOPN) link CNAF Tier-1 Report – L.Dell’Agnello

19. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC Farm status Farm status SLC 3.0.5/LCG 2.6 installed on farm SLC 3.0.5/LCG 2.6 installed on farm –Installation via quattor (project-lcg-gdb-quattor-wg@cern.ch) project-lcg-gdb-quattor-wg@cern.ch Deployed upgrade to 500 nodes in one day Deployed upgrade to 500 nodes in one day –Standard configuration of WNs for all experiments Migration from torque+maui to LSF (v6.1) last Spring Migration from torque+maui to LSF (v6.1) last Spring –LSF farm running successfully –Fair sharing model for resource access 1 queue/experiment (at least) 1 queue/experiment (at least) –Special MPI queue on dedicated resources (InfiniBand) –Progressive inclusion of CDF farm into general one Access to resources centrally managed with Kerberos (authc) and LDAP (authz) Access to resources centrally managed with Kerberos (authc) and LDAP (authz) –Group based authorization CNAF Tier-1 Report – L.Dell’Agnello

20. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC Access to Batch system “Legacy” non Grid Access CELSF Wn1WNn SE Grid Access UI Grid CNAF Tier-1 Report – L.Dell’Agnello

21. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC c=it o=infn U AFS: infn.it G o=cnaf UG Generic CNAF users infn user public view ou=afs Authorization with LDAP private view ou=cnaf G U A R R U G ou=people ou=group A ou=role ou=automount N ou=people-nologin N CNAF Tier-1 Report – L.Dell’Agnello

22. October,10-14 2005 HEPiX Fall 2005 at SLAC SLAC Storage status Physical access to main storage (Fast-T900) via SAN Physical access to main storage (Fast-T900) via SAN –Level1 disk servers connected via FC Usually also in GPFS cluster Usually also in GPFS cluster –Easiness of administration –Load balancing and redundancy –Lustre under evaluation –Can be level2 disk servers connected to storage only via GPFS LCG and FC dependencies on OS decoupled LCG and FC dependencies on OS decoupled WNs are not members of GPFS cluster (no scalability on large number of WNs) WNs are not members of GPFS cluster (no scalability on large number of WNs) –Storage available to WNs via rfio, xrootd (BABAR only), gridftp/SRM or NFS (sw distribution only) CASTOR HSM system (SRM interface) CASTOR HSM system (SRM interface) –STK library with 6 LTO2 and 2 9940B drives (+4 to install) 1200 LTO2 (200 GB) tapes 1200 LTO2 (200 GB) tapes 680 9940B (200 GB) tapes 680 9940B (200 GB) tapes CNAF Tier-1 Report – L.Dell’Agnello