Presentation is loading. Please wait.

Presentation is loading. Please wait.

STEINBUCH CENTRE FOR COMPUTING - SCC www.kit.edu KIT – University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association.

Published byJennifer Rhodes Modified over 10 years ago

Similar presentations

Presentation on theme: "STEINBUCH CENTRE FOR COMPUTING - SCC www.kit.edu KIT – University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association."— Presentation transcript:

1 STEINBUCH CENTRE FOR COMPUTING - SCC www.kit.edu KIT – University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association GridKa Site Report Andreas Petzold

2 INSTITUTS-, FAKULTÄTS-, ABTEILUNGSNAME (in der Masteransicht ändern) Steinbuch Centre for Computing Andreas Petzold – GridKa Site Report – HEPiX Ann Arbor 2013 GridKa Batch Farm Univa Grid Engine is running fine ~150kHS06 ~10k job slots 98 replacement machines this summer SysGen 2U 4 node chassis 2x Intel Xeon E5-2670 (8-core, 2,6 GHz, 312 HS06) 3GB/core 3x 500GB HDD

3 INSTITUTS-, FAKULTÄTS-, ABTEILUNGSNAME (in der Masteransicht ändern) Steinbuch Centre for Computing Andreas Petzold – GridKa Site Report – HEPiX Ann Arbor 2013 WN Migration to SL6 Migration of GridKa compute fabric to SL6 finished Performance: +5.4% Intel Xeon E5-2670 (8 cores, 2.6 GHz) HT off / HT on: SL5 + default compiler:267 HS06/335 HS06 SL6 + default compiler:283 HS06(+5.8%)/348 HS06(+3.9 %) SL5 + gcc-4.8.1:289 HS06/353 HS06 AMD Opteron 6168 (12 cores, 1.9 GHz): SL5 + default compiler:183 HS06 SL6 + default compiler:193 HS06 (+ 5.6 %) SL5 + gcc-4.8.1:187 HS06

4 INSTITUTS-, FAKULTÄTS-, ABTEILUNGSNAME (in der Masteransicht ändern) Steinbuch Centre for Computing Andreas Petzold – GridKa Site Report – HEPiX Ann Arbor 2013 Ivy Bridge Benchmarks New Intel Ivy Bridge processors on the market (E5-26## v2) Manufacturing process: 0.022 micron Sandy Bridge: 0.032 micron Up to 12 cores Sandy Bridge: up to 8 cores Increasing HS06 score according to number of cores: E5-2670 (8 cores, 2.6 GHz, HT on, SL6, default compiler) 348 HS06 E5-2670 v2 (10 cores, 2.5 GHz, HT on, SL6, default compiler) 411 HS06 Power saving of around 25...30 % Thanks to DELL for providing test machine

5 INSTITUTS-, FAKULTÄTS-, ABTEILUNGSNAME (in der Masteransicht ändern) Steinbuch Centre for Computing Andreas Petzold – GridKa Site Report – HEPiX Ann Arbor 2013 Power Efficiency Power Usage (W) per Performance Score (HS06) Worker node class machines at GridKa / E5-2670v2 is a test system provided by DELL 2004200520062007200820092010201120122013 AMD 6168 Intel E5430 Intel L5420 AMD 246 AMD 270 Intel 5160 Intel E5345 Intel E5520 (HT on) Intel E5-2670 (HT on) Watts per HS06 Intel E5-2670v2 (HT on)

6 INSTITUTS-, FAKULTÄTS-, ABTEILUNGSNAME (in der Masteransicht ändern) Steinbuch Centre for Computing Andreas Petzold – GridKa Site Report – HEPiX Ann Arbor 2013 GridKa dCache & xrootd 6 production dCache instances + pre-production setup 5 instances running 2.6, 1 running 2.2 9 PB, 287 pools on 58 servers Upgrade to 2.6 instead of 2.2 recommended by dCache.org last minute decision one week before planned downtime full support for SHA-2 and xrootd monitoring great support from dCache devs CMS disk-tape separation most CMS tape pools converted to disk-only pools last CMS config changes today GridKa 1 st CMS T1 successfully migrated two xrootd instances for ALICE 2.7PB 15 servers

7 INSTITUTS-, FAKULTÄTS-, ABTEILUNGSNAME (in der Masteransicht ändern) Steinbuch Centre for Computing Andreas Petzold – GridKa Site Report – HEPiX Ann Arbor 2013 GridKa Disk Storage 9x DDN S2AA9900 150 enclosures 9000 disks 796 LUNs SAN Brocade DCX 1x DDN SFA10K 10 enclosures 600 disks 1x DDN SFA12K 5 enclosures 360 disks 14PB usable storage

8 INSTITUTS-, FAKULTÄTS-, ABTEILUNGSNAME (in der Masteransicht ändern) Steinbuch Centre for Computing Andreas Petzold – GridKa Site Report – HEPiX Ann Arbor 2013 Evaluating new Storage Solutions DDN SFA12K-E allows to run server VMs directly in storage controller DDN are testing complete dCache instance inside controller expected benefits shortening long IO paths: no SAN + FC HBAs, reduced latency less hardware: less power consumption, improved MTBF possible drawbacks limited resources in storage controllers for VMs loss of redundancy

9 INSTITUTS-, FAKULTÄTS-, ABTEILUNGSNAME (in der Masteransicht ändern) Steinbuch Centre for Computing Andreas Petzold – GridKa Site Report – HEPiX Ann Arbor 2013 DDN SFA12K-E

10 INSTITUTS-, FAKULTÄTS-, ABTEILUNGSNAME (in der Masteransicht ändern) Steinbuch Centre for Computing Andreas Petzold – GridKa Site Report – HEPiX Ann Arbor 2013 Glimpse at Performance Preliminary performance evaluation IOZONE testing 30-100 parallel threads on XFS file system still a lot of work ahead no tuning file system + controller setup tuning

11 INSTITUTS-, FAKULTÄTS-, ABTEILUNGSNAME (in der Masteransicht ändern) Steinbuch Centre for Computing Andreas Petzold – GridKa Site Report – HEPiX Ann Arbor 2013 GridKa Tape Storage 2x Oracle/Sun/STK SL8500 2x 10088 slots 22 LTO5, 16 LTO4 drives 1x IBM TS3500 5800 slots 24 LTO4 drives 1x GRAU XL 5376 slots 16 LTO3, 8 LTO4 >20k cartridges 17PB Migration to HPSS planned for 2014

12 INSTITUTS-, FAKULTÄTS-, ABTEILUNGSNAME (in der Masteransicht ändern) Steinbuch Centre for Computing Andreas Petzold – GridKa Site Report – HEPiX Ann Arbor 2013 100G WAN at GridKa Current WAN setup 7x10Gb/s links to LHCOPN, LHCONE, German research network, FZU Prague + 1x1Gb/s link to Poznan participation in 100G tests at SC2013 100G equipment provided by CISCO 100G connection provided by DFN, time-shared by Aachen, Dresden, KIT plan to move LHCOPN, LHCONE to 100G link in 2014 replace old Catalyst border routers procurement of new Nexus 7k with 100G line cards already underway requires new arrangement of LHCOPN operation between KIT and DFN

13 INSTITUTS-, FAKULTÄTS-, ABTEILUNGSNAME (in der Masteransicht ändern) Steinbuch Centre for Computing Andreas Petzold – GridKa Site Report – HEPiX Ann Arbor 2013 Configuration Management Still mostly using Cfengine 2 Middleware services used as testbed for puppet Started in early 2012 Still based on old homegrown deployment infrastructure CluClo Very smooth operation Now starting to draw up plans for puppet migration wed like to try many new things: git integration, deployment management with Foreman, MCollective, … Will be step by step process

14 bwLSDF News from SCC/KIT outside GridKa new services for state of Baden-Württemberg run by SCC/KIT bwSync&Share dropbox for scientists winner of software evaluation: PowerFolder start of production Jan 1 st 2014 expect active 55k users from all universities, 10GB quota bwFileStorage simple/overflow storage for scientific data access via SCP, SFTP, HTTPS (r/o) provided by IBM SONAS start of production Dec 1 st 2013 bwBlockStorage iSCSI storage over WAN for universities all services based on storage hosted at Large Scale Data Facility Andreas Petzold – GridKa Site Report – HEPiX Ann Arbor 2013

15 bwIDM The bwIDM Project: Vision Federated access to services of the State of Baden-Württemberg Access control based on local accounts of the home organizations bwIDM is not about establishing IDM systems, its about federating existing IDM systems and services. bwServices 20.6.2013M.Nussbaumer@ISC2013 | Federating HPC access via SAML 15

16 bwIDM bwIDM Overview bwIDM –…federation of 9 universities of the state of Baden-Württemberg (non) web-based services –…federates the access to non web-based services such as grid, cloud, and HPC resources. LDAP Facade –Deployable, operable, and maintainable approach to federate non web-based services: LDAP facade makes active use of the SAML-ECP and AssertionQuery profile LDAP facade offers users a high usability in trustworthy federations LDAP facade facilitates temporary trust for scientific portals Easy-to-deploy solution for service collaborations of universities, research centres or companies Single registration process per service service access Successfully deployed in testing environments Deployed Services –Federated HPC Service bwUniCluster (8640 cores, 40.8 TIB Ram, IB FDR) going live in Q4/2013 –Federated Sync&Share Service going live in Q1/2014 Any Questions? Feel free to contact me: martin.nussbaumer@kit.edu.martin.nussbaumer@kit.edu 20.6.2013 16 M.Nussbaumer@ISC2013 | Federating HPC access via SAML If you have to bring non web-based services together with SAML, make use of the LDAP facade!

Download ppt "STEINBUCH CENTRE FOR COMPUTING - SCC www.kit.edu KIT – University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association."

Similar presentations

1 ALICE Grid Status David Evans The University of Birmingham GridPP 14 th Collaboration Meeting Birmingham 6-7 Sept 2005.

1 ALICE Grid Status David Evans The University of Birmingham GridPP 14 th Collaboration Meeting Birmingham 6-7 Sept 2005.
Tony Doyle - University of Glasgow GridPP EDG - UK Contributions Architecture Testbed-1 Network Monitoring Certificates & Security Storage Element R-GMA.

Tony Doyle - University of Glasgow GridPP EDG - UK Contributions Architecture Testbed-1 Network Monitoring Certificates & Security Storage Element R-GMA.
The RHIC-ATLAS Computing Facility at BNL HEPIX – Edinburgh May 24-28, 2004 Tony Chan RHIC Computing Facility Brookhaven National Laboratory.

The RHIC-ATLAS Computing Facility at BNL HEPIX – Edinburgh May 24-28, 2004 Tony Chan RHIC Computing Facility Brookhaven National Laboratory.
12th September 2002Tim Adye1 RAL Tier A Tim Adye Rutherford Appleton Laboratory BaBar Collaboration Meeting Imperial College, London 12 th September 2002.

12th September 2002Tim Adye1 RAL Tier A Tim Adye Rutherford Appleton Laboratory BaBar Collaboration Meeting Imperial College, London 12 th September 2002.
STEINBUCH CENTRE FOR COMPUTING - SCC www.kit.edu KIT – University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association.

STEINBUCH CENTRE FOR COMPUTING - SCC KIT – University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association.
Service Data Challenge Meeting, Karlsruhe, Dec 2, 2004 Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft Plans and outlook at GridKa Forschungszentrum.

Service Data Challenge Meeting, Karlsruhe, Dec 2, 2004 Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft Plans and outlook at GridKa Forschungszentrum.
Tier-1 Evolution and Futures GridPP 29, Oxford Ian Collier September 27 th 2012.

Tier-1 Evolution and Futures GridPP 29, Oxford Ian Collier September 27 th 2012.
Computing Infrastructure

Computing Infrastructure
UWF Computing Hardware Standards ITS Annual Recommendations for 2003 - 2004.

UWF Computing Hardware Standards ITS Annual Recommendations for
Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft Jos van Wezel Doris Ressmann GridKa, Karlsruhe TSM as tape storage backend for disk pool managers.

Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft Jos van Wezel Doris Ressmann GridKa, Karlsruhe TSM as tape storage backend for disk pool managers.
Leaders Have Vision™ visionsolutions.com 1 Easy migration into the cloud Simple “on demand” disaster recovery With Double Take and HyperV Gabriel Chadeau.

Leaders Have Vision™ visionsolutions.com 1 Easy migration into the cloud Simple “on demand” disaster recovery With Double Take and HyperV Gabriel Chadeau.
Cloud Storage in Czech Republic Czech national Cloud Storage and Data Repository project.

Cloud Storage in Czech Republic Czech national Cloud Storage and Data Repository project.
1 Chapter 11: Data Centre Administration Objectives Data Centre Structure Data Centre Structure Data Centre Administration Data Centre Administration Data.

1 Chapter 11: Data Centre Administration Objectives Data Centre Structure Data Centre Structure Data Centre Administration Data Centre Administration Data.
KIT – University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association Steinbuch Centre for Computing (SCC) www.kit.edu.

KIT – University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association Steinbuch Centre for Computing (SCC)
Introduction to Storage Area Network (SAN) Jie Feng Winter 2001.

Introduction to Storage Area Network (SAN) Jie Feng Winter 2001.
Alastair Dewhurst, Dimitrios Zilaskos RAL Tier1 Acknowledgements: RAL Tier1 team, especially John Kelly and James Adams Maximising job throughput using.

Alastair Dewhurst, Dimitrios Zilaskos RAL Tier1 Acknowledgements: RAL Tier1 team, especially John Kelly and James Adams Maximising job throughput using.
STEINBUCH CENTRE FOR COMPUTING - SCC www.kit.edu KIT – University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association.

STEINBUCH CENTRE FOR COMPUTING - SCC KIT – University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association.
Site Report US CMS T2 Workshop Samir Cury on behalf of T2_BR_UERJ Team.

Site Report US CMS T2 Workshop Samir Cury on behalf of T2_BR_UERJ Team.
Introduction to DoC Private Cloud

Introduction to DoC Private Cloud
CPP Staff - 30 CPP Staff - 30 FCIPT Staff - 35 IPR Staff - 500 IPR Staff - 500 ITER-India Staff - 150 ITER-India Staff - 150 Research Areas: 1.Studies.

CPP Staff - 30 CPP Staff - 30 FCIPT Staff - 35 IPR Staff IPR Staff ITER-India Staff ITER-India Staff Research Areas: 1.Studies.

Similar presentations

Ads by Google