1. Conditions Data access using FroNTier Squid cache Server
dimanche 9 septembre 2018dimanche 9 septembre 2018
Conditions Data access using FroNTier Squid cache Server
Farida Fassi
LCG-T1 & AF

2. Outline Motivation Implementation Overview Deployment
Squid Hardware details

3. Motivation What Condition data are?
All parameters describing the detector run condition such as calibration, alignment and configuration info
Data needed for Offline detector event data processing
and reconstruction
Vary with time and may exist in several versions
A given object may be used by thousands of jobs
Caching such info close to the processing activity (CMS Software application) provides significant performance gains

4. CMS Software Stack: CMSSW overview
User Application
EDM EventSetup assures the correct Condition Data is accessed and available for the user application
POOL-ORA (Object Relational Access) is used to map C++ objects to Relational schema
A CORAL-Frontier plugin provides read-only access to the POOL DB objects via Frontier.
EDM Framework
EDM EventSetup
POOL-ORA
CORAL
Oracle
Frontier
Frontier/Cache
Database

5. Implementation: Overview
Pool and CORAL generate SQL queries from the CMSSW C++ objects
The FroNTier client converts the SQL into an HTTP GET and sends it over the network to the FroNTier server
The FroNTier server (a servlet under Tomcat) unpacks the SQL request, sends it to the DB server, and retrieves the needed data and then packed into an HTTP formatted stream sent back to the client.
Having a Squid caching server between the FroNTier server and client caches requested objects, improving significantly performance and reducing load on the DB

6. Deployment Clients: WNs
Redundant Tomcat + Squid servers are deployed at Tier 0 (launchpad).
Squids are deployed at Tier1, and TierN sites
Applications needing access to conditions data are configured with a list of servers, and proxies, for example:
FRONTIER_SERVER1= export
FRONTIER_SERVER2= export
FRONTIER_PROXY1= domain.net:10000
Tier N
Squid
Squid
Squid
Tier 1
Squid
Squid
Squid
Squid(s)
Tomcat(s)
Tier 0
Launchpad DB

7. Marc Paterno, CHEP 2004
T1/T2: Squid
T0: Squid
T0: Tomcat
T0: Oracle

8. Deployment
Conditions data access via Frontier already successfully tested in CSA 2006.
Many performance improvements since that time (improve cache reuse, fix many problem)

9. Cache Coherency issue
CMS adapted the following policy to deal with CC:
All objects that are cached have expiration times
Short expiration time: metadata (Tag and IOV info)
Long expiration time: data payloads (condition data)
The values of the short and long times are adjusted according to where the data is being used. For example:
Tier 0: calibrations change on the order of a few hours as new
runs appear for reconstruction.
Tier 1 +: Conditions data may be stable for weeks.
The value of the short and long expirations is modified at the central FroNTier server, so they can easily be tuned as needed. For example short ~day, long ~month.

10. Service Availability Monitoring (SAM)
Available from Frontier twiki page:
Heartbeat monitoring tests are run every 2 hours, 40 sites included
Check integrity of local squid, and
Overall Frontier system and connection to CERN

11. Local site Squid Hardware details
Minimum specs: 1GHz CPU, 1GByte mem, GBit network, 100 GB disk
Multi-core machines: CPU resources under utilized because of single
Squid process utilizes only one CPU
Multiple network interfaces can resolve this; two solution tested
Multi-homed (2 ip addresses): machine looks like multiple nodes
Bonded interfaces (1 ip address): multiple NICs used together to increase throughput.
Bonded approach implemented at Fermilab (FNAL)
Works best with specific load balancing approach
Requires 2 squids on same server machine, Two Gbit network interface
~200 MBps throughout can be achieved
Many sites prefer just adding more machines to make network
management simpler and reduce configuration issues