1. LHCbPR V2 Sasha Mazurov, Amine Ben Hammou, Ben Couturier 5th LHCb Computing Workshop 2015-05-19

2. Content Objectives – LHCbPR V1 Experience – Why V2? New architecture and tools RESTful service Web frontend 5th LHCb Computing Workshop22015-05-19

3. LHCbPR “LHCb Performance and Regression(PR) is a service designed to record important measures about releases of the version of the LHCb software” Current version (V1): https://lhcb- pr.web.cern.ch/lhcb-pr/ 2015-05-195th LHCb Computing Workshop3

4. LHCbPR V1 experience (1) Web frontend Main form for jobs searching: lack of dynamic filtering for options, platforms,... 5th LHCb Computing Workshop42015-05-19

5. LHCbPR V1 experience (2) 5th LHCb Computing Workshop5 Admin backend Automatic admin site scaffolding is not usable for creating a new job description – too many unobvious steps. A wizard for such task is neede to be written 2015-05-19

6. LHCbPR V1 experience (3) 5th LHCb Computing Workshop6 New handlers and analysis development Lack of python and javascript libraries for common LHCbPR programming tasks. A lot of copy-paste code. Unreadable raw sql in the source code. To get a readable code we need to use standard Django database model classes 2015-05-19

7. Why V2? First version of LHCbPR is a good program for understanding the real needs for regression analysis. We have collected a list of features that hard to implement in V1 (thresholds, attributes management, separate versions and applications,…). We need to have a common javascript libraries and components specific for LHCbPR that can be shared among developers (now we have a spaghetti code) Decouple data access application and presentation sites. Avoid using raw SQL. 5th LHCb Computing Workshop72015-05-19

8. New database entities and changes Versions table – separate application and its version entities. Has timestamp field that is inserted from its first job data, so we can arrange releases’ and nightly builds’ versions. Nightly slots table – store name of nightly slot (“lhcb-head”, “lhcb- gaudi-head”,…). Versions table can have a link to this table. Thresholds table for attributes – we can check attributes ranges Attributes’ group table – attribute can belong to several groups We have a tool that import data from V1 database. 5th LHCb Computing Workshop82015-05-19

9. New architecture LHCbPR as a RESTful data service: – Separate data access and data representation. Interoperate using JSON format – Web site is only one of the clients of the service – Easy to write command line scripts (for nightlies, reports,…) 92015-05-195th LHCb Computing Workshop

10. Tools RESTful service – Django 1.8 with rest framework application. Web site – AngularJS JavaScript MVC framework – Bootstrap based CSS framework – No backend 102015-05-195th LHCb Computing Workshop

11. REST API 11 http://amazurov.web.cern.ch/amazurov/lhcbpr-apihttp://amazurov.web.cern.ch/amazurov/lhcbpr-api - outputs all possible service endpoints 2015-05-195th LHCb Computing Workshop

12. REST API features We can use it for all CRUD operations (create, read, update, delete) Can output arbitrary JSON (examples later) Authentication plugins: e.g. shibboleth for web and tokens for console applications Output pagination, throttling and requests’ cache allow to avoid service overloading. 5th LHCb Computing Workshop122015-05-19

13. Examples Jobs list - /jobs Jobs list Job details - /jobs/178 Job details – We can expand any object’s relationships. Nested requests - /jobs/178/results/ Nested requests – Output job attributes’ values Output filtering - /compare/?ids=32,33&contains=Event Output filtering – Show and group attributes of jobs with id 32 and 32. Attribute name should contain “Event” string 5th LHCb Computing Workshop132015-05-19

14. Web Frontend 5th LHCb Computing Workshop14 Only http requests from browser to REST server, no backend behind http://amazurov.web.cern.ch/amazurov/lhcbpr-devhttp://amazurov.web.cern.ch/amazurov/lhcbpr-dev - DEMO 2015-05-19

15. Reusable Web Components 15 = <div search-jobs="search-jobs" on-found="onJobsFound(searchParams)”> onJobsFound – client’s custom callback function that get search form results: ids of application, options, Versions More components in development 2015-05-195th LHCb Computing Workshop

16. Command line / Scripts 16 Can be easily parsed with standard python libraries $> curl http://amazurov.web.cern.ch/amazurov/lhcbpr-api/jobs/ 2015-05-195th LHCb Computing Workshop

17. For Developers We have prepared a set of reusable JavaScript components that simplifies work with REST service and introduce common UI elements for LHCbPR Well organized modules’ directory structure 5th LHCb Computing Workshop172015-05-19

18. Modules 5th LHCb Computing Workshop18 Controllers are written in JavaScript Views use Jedi template engine Modules support lazy JavaScript loading, so only required libraries are used in modules. 2015-05-19

19. Summary API – Updated database schema for proper support of application versions, build slots, thresholds, attribute groups. Import script from V1 database – Ready for production RESTful service that can be used from any application (currently in readonly mode) – https://github.com/LHCbDev/lhcbpr-api https://github.com/LHCbDev/lhcbpr-api Frontend – Fixed a structure of modules in web frontend – Prepared a set of web components and services. – Compare jobs’ results module. Framework is ready for developing (migration) of the rest of analysis modules. – https://github.com/LHCbDev/lhcbpr-www https://github.com/LHCbDev/lhcbpr-www 5th LHCb Computing Workshop192015-05-19