1. 1 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland Experience with Field Quality Analysis Software and Future Projects Laurent Deniau Laurent.Deniau@cern.ch

2. 2 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland Content Part I: Past Projects  Series Tests Requirements  First System  Second System  Learning from experience Part II: Future Projects  Future Requirements  Proposal I  Proposal II  Proposal III

3. 3 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland PART I Past Projects

4. 4 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland Series Test Requirements Test of ~1300 Main Dipoles  Field Quality: 100% [  20%] Test of ~400 Main Quadrupoles  Field Quality: 100% [  15%] Test of ~100 Special Quadrupoles  Field Quality: 100% [  10%] Test of ~500 Correctors (over ~8000)  Field Quality: 100% [  20%] System usage (2000 – 2007+)  ~25 users [  ~10 users]  Inside and outside CERN [  inside CERN]  Cold and warm measurements [  cold measurements]  15000 measurements/magnets ~150MB [  5000 ~50MB]

5. 5 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 1 st System Development1999 Deployment1999 – 2000 Improvement2000 – 2003 Resources~1 m.year over 3 years Usage2000 – 2007 Effective lines of code:  C:58300 (59.55%)33079 (57%) generated  Perl:38162 (38.98%)  Sh:295 (0.30%)  Other:1142 (1.17%)

6. 6 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 1 st System – Aims [all achieved] Deal only with rotating coil devices  long shaft, scanner, mole Parametrizable transformations from measures to magnet integral field and harmonics  Measure + Calibration  Field Automatic calibration (never used)  Measure + Field  Calibration Generic processing of devices and magnets Data extraction (e.g. b3 vs I, b3 vs b1,...) and plots Command line tools (Server) Web interface (Client)

7. 7 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 1 st System – Architecture Each block is a command line program  Communication is through pipes (files transfer)

8. 8 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 1 st System – Web Interface

9. 9 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 1 st System – Technologies Oracle for DB  Imposed by CERN – IT C for the tools and libraries  Availability, portability, simplicity, efficiency Perl for the web scripts  Imposed by developers knowledge Linux and Apache on 5 web servers  Imposed by CERN – IT Central repository  CVS for sources  AFS (Andrew File System on CERN servers) Only free/open source software (except DB client)

10. 10 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 1 st System – Experience System not heavily used  ~10 users  Less magnets and measurements than foreseen (15x) System is oversized and not enough flexible (standard) Users prefer graphical interface for simple non- repetitive tasks Users prefer scripting for flexible, configurable, parameterizable and/or repetitive tasks Some Users are ready to help for supervised programming (assume a framework) Error handling/messages are of first importance! Reliability does matter (confidence in the system) Speed does not matter so much (DBs are slow)

11. 11 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 2 nd System Development2002 Deployment2002 – 2003 Improvement2003 – today Resources~1 m.year over 3 years Usage2002 – 2007+ Effective lines of code:  C89:15956  PHP:28563

12. 12 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 2 nd System – Aims [none achieved*] Process all measurements from all devices  Generic processing of devices and magnets long shaft, scanner, mole, hall probe, SSW, NMR, …  More parametrizable transformation and calibration More generic data processing (large data set)  Process cycles, history and integral over many magnets  More data manipulation and DBs access  More data extraction facilities Flexible and extensible (not data oriented) Unification  The data and DBs access (parser, db engine)  The jungle of configs, tools, tables and scripts (many, …) 4 layers: www  script  tool  libraries

13. 13 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 2 nd System – Web Interface

14. 14 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 2 nd System – Technologies As 1 st System  Oracle, C, Linux, Apache, CVS, AFS PHP for the web  Imposed by developer knowledge Improvement of C libraries  Better code insulation  Better filters

15. 15 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 2 nd System – Experience Much better web interactivity  Windows-like application Multiple windows, interactivity, error handler, contextual help  Better configurable DB interface  Very extensible and flexible Magnet Follow-Up  A lot of work for a little use (many plots and parameters) Id Cards Warm-Cold extrapolation Many data representation and plots Better filters and alarms Very little advancement for analysis and unification  No aim reached for the moment (reorientation of aims)

16. 16 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland Learning from Experience 1/3 Analysis of measurements always looks the same  Dataflow oriented (left to right) Transformations to physical quantities Device calibrations Reduction Transformation Extraction Combination Visualization Measurements Files or DB Query Configuration/Settings Results Files External DBMeasure DB

17. 17 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland Learning from Experience 2/3 Final specifications never exist  R&D needs evolutive prototyping  Prototyping requires iterations and never ends Outsourcing  Not efficient during prototyping and much less evolutive 5x – 10x slower than local development Low control on code quality (result oriented)  Efficient for converting prototype to production software  Efficient for maintenance and support Commercial software  Problem for long term projects (cycle > 5 years)  Problem for open problems (must fit the needs)

18. 18 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland Learning from Experience 3/3 Users wish-list  Simplicity for simple tasks (graphical interface)  Flexibility for complex tasks (script)  Automation for repetitive tasks (script)  Clear multi-level checks/alarms (error handler)  Access from the web (even for script)  Low level code control (advanced users) Developers wish-list  Development framework (very important)  Language (very important)  Code test (test-driven development)  Less (no) graphics, plots and interface (take time)

19. 19 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland PART II Future Projects

20. 20 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland Future Requirements – 2007+ Magnet Rescue [MAR]  Test for 1-2 magnets/month  Same as series tests (simplified version) Fast Measurement Equipment [FAME]  Continuously rotating coils  Large amount of data (> series tests)  New equipments Field Description for the LHC [FIDEL]  Relies on FAME  Network and DB activities  Very large amount of data (>> series tests, (cycle, history))  May require to be online one day (fast response time) Easy prototyping and configuration

21. 21 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland Resources Availability Technical Students ~8 – 12 months [Easy]  Short term development, require framework and tutorial  Simple programming (except for computer scientists) Doctoral Students ~2 – 3 years (part time) [Medium]  More time for training (can learn new languages)  Medium – long term development, require framework  Scientific and/or computer skills Fellows & P.A ~2 – 5 years [Medium  Hard]  Long term development, develop framework, open training  Scientific and/or computer skills Skills availability (in CERN – AT)  Physicist [Easy]  Computer scientists [Hard]

22. 22 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland Technologies Already selected or imposed  Labview for acquisition and control  Oracle for DB  PHP & Apache for web  Linux PCs for servers Analysis interface  Integrated and/or communicate with acquisition Accessible and/or compatible with Labview Semi-open questions  Scripting language for users  Programming language for developers/advanced users  Web servers (central) or local program (distributed)  Support and maintenance

23. 23 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland Language Requirements Widely known (or very easy to learn) Highly Portable (including 64 bits) Normalized (ISO) or formally specified (and stable) Object Oriented  Very good code insulation  Simple memory management  Non-local error management  Reflection Simple and Efficient Large set of available libraries Good operating system API (POSIX) Good database API (OCI)

24. 24 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland Open source Debian GNU/Linux system includes 230 millions of effective line of code (SLOC) in all fields from all horizons :  C%57*  C++%16.8*  Shell%9  Lisp%3  Perl%2.8  Python%1.8  Java%1.6  Fortran%1.2*  PHP%0.93  Pascal%0.62*  Ada%0.61* Languages Widely Used Blue languages are normalized (ISO) Green languages are formally specified Red stars indicate languages compiled to native executables, others are interpreted

25. 25 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 1 st Proposal – Resources Oriented Provide only integrated ‘black boxes’ for analysis Build prototypes internally (next point  Labview)  Rely on sparse resources  Incomplete specification Enough for fast interactive prototyping  Build prototypes Validation of analysis ‘black boxes’ Possibility to extend/improve ‘black boxes’ in C Hard to use for large projects (unification, refactoring) Delegate production development Only AB – CO can take over Labview everywhere, including prototyping Design is fixed Delegate support & maintenance

26. 26 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 2 nd Proposal – Users Oriented Provide a complete framework (extend 2 nd system)  3 layers: web  script  libraries of scripts Users can use and/or write advanced scripts  Powerful and flexible  Require users training and tutorial  Hard to control modules unification/integration Scripting Language  Python (interpreted) – Lisp (interpreted + compiled) Easy, available, extensible in C Python not standardized, Lisp is functional  Java – (virtual machine) Easy, available (?), hard to extend, needs virtual machine Not standardized but well specified and supported Maybe blocked by efficiency and/or memory

27. 27 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland 3 rd Proposal – All-in-One Provide a complete framework (extend 2 nd system)  4 layers: web  script  tool  libraries Users can use and write simple scripts  Powerful but less flexible (focus on simplicity)  No training, small tutorial and examples  Easy to control unification/integration/extension Advanced users can write plug-in (classes)  Some training, small manual (require knowledge of C)  Powerful and flexible (no limit), still allow Python (or Perl) Language: Object Oriented C (ISO C89)  Allow Java-like programming in C with all OO features and more!  Easy, available, portable, standard, natively extendable  No problem with efficiency, memory, libraries or APIs

28. 28 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland Proposals – Architectures DB Data Analysis and Processing User WWW Data Analysis and Processing Acquisition and Control Online Acquisition (many sites) Remote Analysis (central) Sun local copy of tool 1 st proposal  Labview 2 nd proposal  Python 3 rd proposal  OOC Analysis Servers Linux DB Servers Oracle Acquisition Systems Labview on Sun 2 nd proposal  Python 3 rd proposal  OOC Script 2 nd and 3 rd proposals only! Script 2 nd and 3 rd proposals only http://sma.cern.ch

29. 29 / 29 Laurent Deniau CERN – AT – MTM Laurent.Deniau@cern.ch 14th International Magnetic Measurement Workshop 26-29 September 2005, Geneva, Switzerland Proposals – Summary UsersDevelopersResourcesTechnologies 1 st Proposal Resources Settings only Labview Constraint Sparse + Outsourced Labview 2 nd Proposal Users Full flexibility Flexible 1-2 m.years + ½ m.years www + Python [+ C] 3 rd Proposal All-in-One Simplicity and full flexibility No limit Flexible Standard 2-3 m.years + ½ m.years www + OOC