FroNtier: High Performance Database Access Using Standard Web Components in a Scalable Multi-tier Architecture Marc Paterno Fermilab CHEP 2004 Sept. 27-Oct. 1, 2004 Interlaken Switzerland

2 Credits Fermilab, Batavia, Illinois Jim Kowalkowski, Sergey Kosyakov, Dmitri Litvintsev, Lee Lueking, Marc Paterno, Stephen White Johns Hopkins University, Baltimore, Maryland Barry Blumenfeld, Petar Maksimovic, Mark Mathis Rovaniemi Polytechnic, Rovaniemi, Finland Lauri Autio (through collaboration with Fermilab and The University of Helsinki)

3 Outline Motivation Motivation Overview Overview Testing Testing Deployment Deployment Status and Plans Status and Plans

4 Motivation CDF was CDF was –concerned about the scalability of their current database access layer –looking for a straightforward path to move to a multi- tier data delivery architecture using commodity tools Why a multi-tier approach with caching? Why a multi-tier approach with caching? –Ideal for write once, read many data –Reliability and high performance w/o expensive DB servers –Readily scalable to serve thousands of clients –Decouples client and database development and maintenance

5 Transport Protocol Several choices for transport protocol were possible Several choices for transport protocol were possible –many projects use home-grown protocols, based on bare sockets The use of HTTP for the server-client protocol provides a starting point The use of HTTP for the server-client protocol provides a starting point –Opens the door to many well understood commodity tools and techniques used routinely on the internet. –These tools are easily deployed and have low administrative overheads.

6 A Multi-tier Architecture Database (Oracle, MySQL, etc.) Database (Oracle, MySQL, etc.) Database Access Layer Database Access Layer –Tomcat: servlet management engine –JDBC: Database connection –Database Connection pool management Caching and proxy layer: Squid Caching and proxy layer: Squid –widely used, highly configurable, caching proxy server Client Client –Client API provides simple interface packaged in a compact client library

7 FroNtier Overview CDF Persistent Object Templates (Java) FroNtier components in yellow Client Caching FroNtier Server Database FroNtier Client API Library Squid Proxy/Caching Server FroNtier Servlet running under Tomcat Database (or other persistency service) XML Server Descriptors DDL for Table Descriptions C++ Header and Stubs JDBC HTTP

8 FroNtier Servlet Design 1.Client sends request (URI) 2.Command Parser translates URI into commands + values 3.Servicer Factory gets XSD (XML Server Description) from database and 4.Instantiate a Servicer 5.Servicer queries database and 6.Results sent for encoding 7.Encoder marshals (serializes) the data to requestor client Database CommandParser ServicerFactoryServicer Encoder Client

9 FroNtier XML Server Descriptor Object name and version information Object name and version information Response description Response description The SQL mapping to the database The SQL mapping to the database –Select statement –From statement –Where clause –Special modifiers (order by, etc) calib_run, calib_version, data_status CalibRunLists cid

10 FroNtier use of Squid Cache Squid (v 2.5STABLEn) – Open source Squid (v 2.5STABLEn) – Open source –Well documented, widespread operational experience –Easily installed and maintained –Highly configurable for access control, disk cache tuning, distributed cache peer relationships,… –Monitoring built in through SNMP-2 interface Refreshing the cache (3 options) Refreshing the cache (3 options) –Driven by servlet: expiration time sent in HTTP header –Driven by client: forced object refresh through request –Delete each Squid’s cache files and rebuild the cache However, the objects being delivered are generally not changing, so a static cache matches the requirements. However, the objects being delivered are generally not changing, so a static cache matches the requirements.

11 FroNtier client API features C and C++ interfaces C and C++ interfaces Portable Portable –32 and 64 bit systems tested Transparent object access Transparent object access –Type conversion detection –Preserves data integrity Multi-object requests Multi-object requests Easy runtime configuration Easy runtime configuration Extensive error reporting Extensive error reporting –Adjustable logs FroNtier Service User application FroNtier API

General Deployment Overview General Deployment Overview Dedicated Squid Node Tomcat Node 1 Tomcat Node N Tomcat Node 2 CAF 1 Processor Farm Database Dedicated Squid Node CAF N Processor Farm Dedicated Squid Node CAF 2 Processor Farm Squid Node N Squid Node 2 Cache Layer local to each CAF Gbit ethernit Intermediate Cache Layer DB Access Layer Squid Node 1 Load Balancing and Failover Launchpad WAN

13 FroNtier Testing at FNAL/SDSC (San Diego Super Computing Center) 0.1 MB/s 100 MB/s 5 MB/s FNAL Launchpad SDSC Squid SDCS CAF CDF Oracle No Caching Caching at FNAL Launchpad Caching local to SDSC

14 Client Side: FNAL/SDSC No Caching Caching at SDSC Squid 1000 Sec 1 Sec

15 Support Plan The launchpad hardware configuration has been designed to be stable, redundant, and reliable. It should not be a major support load. The launchpad hardware configuration has been designed to be stable, redundant, and reliable. It should not be a major support load. The software on the launchpad (Tomcat + Squid) has been reliable in our testing. Maintained by CDF database support personnel. The software on the launchpad (Tomcat + Squid) has been reliable in our testing. Maintained by CDF database support personnel. Squid at remote sites is the responsibility of the remote admin people. FroNtier team will help set up and configure as needed. Squid at remote sites is the responsibility of the remote admin people. FroNtier team will help set up and configure as needed. A monitoring system is in place using SNMP (for squid) and Log file mining (Tomcat and Squid) to assure the system is performing as expected. A monitoring system is in place using SNMP (for squid) and Log file mining (Tomcat and Squid) to assure the system is performing as expected.

16 FroNtier Status and Plans CDF Production ready CDF Production ready –Client library is included in CDF production code –Production executables built and tested with FroNtier. –Widespread use will begin as new releases are built Plan to extend the use in CDF beyond calibration data to trigger and other static data needed offline. Plan to extend the use in CDF beyond calibration data to trigger and other static data needed offline. Exploring a prototype of a FroNtier “plug-in” for POOL (LCG). Some interest in CMS to use a web-based approach for DB access. Exploring a prototype of a FroNtier “plug-in” for POOL (LCG). Some interest in CMS to use a web-based approach for DB access.

17 Summary FroNtier is a multi-tier architecture providing clients high throughput, low latency, scalable access to a persistent store, such as a database. FroNtier is a multi-tier architecture providing clients high throughput, low latency, scalable access to a persistent store, such as a database. The CDF DB access framework has been adapted to use the FroNtier approach. Extensive testing was performed to confirm the integrity, performance, and reliability of the system. The CDF DB access framework has been adapted to use the FroNtier approach. Extensive testing was performed to confirm the integrity, performance, and reliability of the system. We are in the process of executing a deployment plan and will soon have real-world production experience with the system. We are in the process of executing a deployment plan and will soon have real-world production experience with the system.

18 References FroNtier Talks: FroNtier Talks: – FroNtier working page: FroNtier working page: –