1. Ideas on the LCG Application Architecture Application Architecture Blueprint RTAG 12 th June 2002 P. Mato / CERN

2. 24/10/2001LCG Application Architecture2 Application Domains  Event Processing Applications –Trigger, Simulation, Reconstruction, Selection programs  Data Analysis –Event/Detector display, data presentation programs  Detector calibration –Calibration and Alignment programs  Job configuration, submission, monitoring and control –Grid awareness Mainly batch oriented. Interactive for development & debugging. Real-time. Mainly interactive Batch and interactive

3. 24/10/2001LCG Application Architecture3 No Disjoint Domains  For example, the LHCb requirements for interactive analysis: –Better [than PAW] integration with experiment framework »consistent with the analysis batch environment, use the same toolkits and experiment algorithms/tools »access the experiment data objects and allow browsing »integrated with event display »allow interactive reconstruction and simulation

4. 24/10/2001LCG Application Architecture4 Architectural styles  General categorization of systems [1] user-centricfocus on the direct visualization and manipulation of the objects that define a certain domain data-centricfocus upon preserving the integrity of the persistent objects in a system computation-centricfocus is on the transformation of objects that are interesting to the system [1] G. Booch, “Object Solutions”, Addison-Wesley 1996

5. 24/10/2001LCG Application Architecture5 Different style in different domains User Interface Interactivity Scripting Object store Database Data Integrity Algorithms Computation computation-centric data- centric user- centric The applications in the different domains may have different emphasis in: Interactivity Database Computation Elements of all three are present in all applications

6. 24/10/2001LCG Application Architecture6 Computation-centric: GAUDI  Framework adequate for “all” event processing applications  Algorithms process “event data” with the help of “services” and using “detector data”.

7. 24/10/2001LCG Application Architecture7 User-centric: PAW  Provides interactive graphical presentation and statistical or mathematical analysis, working on objects familiar to physicists like histograms, event files (Ntuples), vectors, etc.  Is based on several components of the CERN Program Library. Easy to use and freely available  No integration within experiments framework  Offline analysis done in 2 separate steps “batch” and “interactive”, the user decides how much to do in each step

8. 24/10/2001LCG Application Architecture8 The Challenge  Many components are shared among the different applications in the different domains  We would like to profit from the best of the existing solutions (implementations) for each component but not duplicate everything. >>> INTEGRATION <<<

9. 24/10/2001LCG Application Architecture9 Component interface specifications  Integration technologies –Mainly for distributed computing –CORBA, DCOM (.NET), Web Services, JavaBeans  Abstract interfaces –C++ “in process” integration –Robust and efficient. A bit awkward for end-users.  Scripting extension modules –Ideal for prototyping. Rapid application development (RAD). –Dependent of the scripting tool Probably the best solution but unrealistic in our environment Works well within a given Framework Minimal coupling, maximum flexibility. Lacks performance.

10. For user-centric applications Interactive analysis, configuration, monitoring applications, etc.

11. 24/10/2001LCG Application Architecture11 Software Bus  Define a “bus” specifications equivalent to a hardware bus (e.g. VME) where modules implementing different functionality can be plugged easily. Bus math shellGUI Experiment Framework (Gaudi) Analysis Tools (Lizard) DatabaseGrid

12. 24/10/2001LCG Application Architecture12 Python as a Software-Bus  Python is an object-oriented scripting language commonly used in a wide variety of domains  But, it could also be seen as framework where you can plug easily “extension modules” in binary form implemented using other languages. –Very easy and natural to interface to C++ classes (C++ API) –Python should only be the “glue” between modules developed in C++ or other languages –The interface (API) for Python extension modules is quite simple and at the same time flexible

13. 24/10/2001LCG Application Architecture13 Module Communication  Communication between two extension modules is always possible using Python primitive types (a) –Example: I would like to display an histogram of Gaudi Gaudi (C++)Lizard (C++) Python IH1D IH1D* Plotter Python Primitives  It is more efficient to communicate using a reference to an interface (AIDA?) (b) –The objects in Python are only references (a) (b) Both ways are possible!!

14. 24/10/2001LCG Application Architecture14 Why Python?  Interpreted –However it is quite fast (byte code idea from Java)  Dynamically typed –Not need to declare any variable  Simple syntax –Emphasis by the authors to minimize typing  Shells  Powerful built-in types and modules  Ideal for Scripting and Prototyping

15. 24/10/2001LCG Application Architecture15 GUI What is Feasible? Python math shell gaudipython DatabaseEDG API GUI Very rich set of Python standard modules Several GUI toolkits XML Very rich set specialized generic modules Gaudi Framework rootpython Root Classes PVSS JPE Java Classes LHC modules Gateways to other frameworks

16. 24/10/2001LCG Application Architecture16 Example Filling an Excel spreadsheet from a ROOT ntuple # Get the ntuple from the ROOT file import rootmodule hfile = TFile('hsimple.root') ntuple = gROOT.FindObject('ntuple') entries = ntuple.GetEntries() nvar = ntuple.GetNvar() tuple = ntuple.GetArgs() # Initialize Excel import win32com.client excel = win32com.client.Dispatch('Excel.Application') wbook = excel.Workbooks.Add() wsheet = wbook.WorkSheets.Add() wsheet.Name = ntuple.GetTitle() # Fill Excel sheet for i in xrange(500) : ntuple.GetEntry(i) for j in range(nvar) : wsheet.Cells(i+1,j+1).value = tuple[j] # Make Excel sheet visible excel.Visible = 1

17. 24/10/2001LCG Application Architecture17 Ideal Interactive Application >>> execfile(‘mymacro’) >>> for c in cands : plot(c) >>> Scripting Window Canvas (histograms, 3D graphics, etc.) Menu GUI Data Browser

18. 24/10/2001LCG Application Architecture18 Where are the problems?  The Software-Bus architecture is not applicable everywhere –I would not like to implement the LHCb 2 level trigger with that ! –Software-Bus + Abstract interfaces is a good compromise for user-centric applications (fast prototype)  In addition to “standardize” to the Python extension API we need more –AIDA ?, interoperability between Boost/Swig? …  Mixing several GUI toolkits, thread libraries, etc. may pose serious technical problems.