SySal Analysis tools: Status and outlook Cristiano Bozza Salerno Emulsion Group Bern, March 2004

Software architecture basics.NET (Windows), Mono (Linux) Most class libraries and executables do not need OS-specific binaries Platform Most code is written in C# Libraries can be used by programs in any.NET-supported language: C++, C#, BASIC, FORTRAN,... Languages No recompilation or adaptation required In the real OPERA data-taking, quasi-on-line analysis will be needed to drive the microscopes Dedicated servers will run automated, distributed analysis tasks A natural framework for distributed computing is needed .NET / Mono Why this choice?

Software architecture overview SySalCore, Imaging, Tracking2, Scanning2, VolumeTracks NumericalTools, MathExpression Basic libraries QuickMapper, StripesFragLink2, AlphaOmegaReconstruction VolumeGeneration Most commonly used computation libraries Additional libraries Basic classes libraries Common numerical operations and a powerful math parser Pattern matching, fragment linking, volume alignment and vertex-track topological tree reconstruction Customizable generation of vertex-track trees in a background of passing-throughs and uncorrelated tracks (Compton electrons, delta rays, etc...)

Software architecture overview GraphicalAnalysis Data analysis libraries Produces several statistical plots that can be displayed or saved to a file in several popular formats (GIF, JPEG, EPS) StatisticalGraphics Passive GUI control that can be placed into user applications StatisticalAnalysisManager Active GUI control that can be placed into user applications Manages several datasets, applies cuts, adds variables, etc...

Software architecture overview Scanning2 and VolumeTracks support file-based persistence for: I/O libraries OperaDB supports Opera DB - based persistence for SySal.Scanning.Plate.IO.Opera.LinkedZone SySal.Scanning.Plate.IO.Opera.RawData.Catalog SySal.Scanning.Plate.IO.Opera.RawData.Fragment SySal.TotalScan.Volume SySal.OperaDb.Scanning.LinkedZone SySal.OperaDb.TotalScan.Volume + SySal.OperaDb.Scanning.Batch SySal.OperaDb.ComputingInfrastructure.User SySal.OperaDb.ComputingInfrastructure.UserPermission SySal.OperaDb.ComputingInfrastructure.Machine SySal.OperaDb.ComputingInfrastructure.ProgramSettings

Software architecture overview Level 1 Support for Opera Emulsion Scanning DB OperaDbConnection, OperaDbCommand, OperaDbDataAdapter to issue SQL commands Level 2 LinkedZone, Volume, ProgramSettings used for storage/retrieval Level 3 Integrated Oracle-OPERA-OS authentication (OperaDbLoginTools) Level 4 File paradigm for DB operations (OperaPersistence)

Software architecture overview Level 1 Support for Opera Emulsion Scanning DB C# OperaDbConnection conn = new OperaDbConnection(“operadbsrv.mydomain.eu”, “user”, “pwd”); conn.Open(); DataSet ds = new DataSet(); OperaDbDataAdapter da = new OperaDbDataAdapter(“SELECT DOWNSTREAMSLOPEX, DOWNSTREAMSLOPEY FROM TB_VOLUMETRACKS”, conn, null); da.Fill(ds); int i; for (i = 0; i < ds.Tables[0].Rows.Count; i++) { Console.WriteLine(“{0} {1}”, ds.Tables[0].Rows[i][0].ToString(), ds.Tables[0].Rows[i][1].ToString()); } conn.Close(); C++ OperaDbConnection *conn = new OperaDbConnection(“operadbsrv.mydomain.eu”, “user”, “pwd”); conn->Open(); DataSet *ds = new DataSet(); OperaDbDataAdapter *da = new OperaDbDataAdapter(“SELECT DOWNSTREAMSLOPEX, DOWNSTREAMSLOPEY FROM TB_VOLUMETRACKS”, conn, 0); da->Fill(ds); int i; for (i = 0; i Tables[0]->Rows->Count; i++) { Console::WriteLine(“{0} {1}”, ds->Tables[0]->Rows[i][0]->ToString(), ds->Tables[0]->Rows[i][1]->ToString()); } conn->Close();

Software architecture overview Level 2 Support for Opera Emulsion Scanning DB C# OperaDbConnection conn = new OperaDbConnection(“operadbsrv.mydomain.eu”, “user”, “pwd”); conn.Open(); LinkedZone lz = new LinkedZone(2767, conn, null); conn.Close(); C++ OperaDbConnection *conn = new OperaDbConnection(“operadbsrv.mydomain.eu”, “user”, “pwd”); conn->Open(); LinkedZone *lz = new LinkedZone(2767, conn, 0); conn->Close(); The library takes care of issuing all the proper SQL commands to retrieve a specific complex object from the DB

Software architecture overview Level 3 Support for Opera Emulsion Scanning DB C# OperaDbConnection conn = OperaDbCredentials.CreateFromRecord().Connect(); conn.Open(); LinkedZone lz = new LinkedZone(2767, conn, null); conn.Close(); C++ OperaDbConnection *conn = OperaDbCredentials::CreateFromRecord()->Connect(); conn->Open(); LinkedZone *lz = new LinkedZone(2767, conn, 0); conn->Close(); The user doesn’t need to supply the username/password pair for Oracle login: they are recorded in an encrypted file

Software architecture overview Level 4 – in a program Support for Opera Emulsion Scanning DB LinkedZone lz = (LinkedZone)OperaPersistence.Restore(“db:\2354.tlg”, typeof(LinkedZone)); The OperaPersistence library takes care of discriminating between DB references and files. DB objects have a virtual file path. OperaPersistence automatically logs onto the DB when needed. Level 4 – OS shell C:> OperaCopy db:\2354.tlg c:\myfile.tlg It is possible to treat DB objects as if they were files in a generalized file system. (OperaPersistence implements a GUFS: Grand-Unified File System...) OperaPersistence automatically logs onto the DB when needed. If a program is able to work with data files, it is immediately able to work with an OPERA DB This cannot replace smart use of DB, i.e. ability to extract only the interesting information units (e.g. a single base track for scanback)!!!

Software applications Ready-made programs EasyLink.NET – GUI interactive utility for fragment linking EasyReconstruct – GUI interactive utility for volume alignment, vertex detection, topology reconstruction, interactive display and alignment data analysis QuickDataCheck – GUI interactive data analysis program that natively supports TLGs, RWDs, TSRs as well as ASCII n-tuples TLGSel* – Command-line program that extracts subsets of tracks from a TLG file BatchLink* – Command-line program that links fragments, and optionally performs automatic shrinkage correction and camera spot elimination BatchReconstruct* – Command-line program that performs alignment, vertex detection and topology reconstruction ASCII2TLG – Command-line utility that builds a TLG from n-tuples Qmap* – Command-line track pattern-matching utility OperaCopy* – shell executable that copies objects between the file system and the DB OperaDbZoneLoader – GUI interactive utility that loads linked zones into the DB * = supports OperaPersistence  “Grand-Unified File System”

Software applications QuickDataCheck Produces several plots and fits Easy to use, high interactivity

Software applications EasyLink Interactive fragment linking tool

Software applications EasyReconstruct Interactive alignment + analysis + vertex / topology reconstruction

Software applications BatchLink Automatic shrinkage correction (“beam-like” precondition is not required)

Software applications Coming soon BatchIntercalibrate – Command-line program that computes the intercalibration parameters from tracks and generates a new mark map in the intercalibrated reference frame BatchPlot – Command-line program that generates analysis plots for automatic data monitoring “Driver programs” are also under development: Free scanning and automatic fragment linking Vertex location (includes scanning, linking, intercalibration, alignment and vertex detection) Vertex selection Brick management in a scanning cluster Scanning cluster management

Software development Recent improvements of SySal.NET algorithms Vertex reconstruction in AlphaOmegaReconstruction is being extensively tested and validated. Memory and network management in StripesFragLink2 has been improved to work in extreme conditions (high density and areas > 30 cm 2, multi-GB datasets easily handled) Work on these tools has been triggered by the vertex location test An ideal playground to set up and test methods, precisions and strategies for OPERA!

Next future The software is improving very steadily towards its final shape In OPERA we will have: Lower background Smaller data sets  easier than now! “Stay tuned!”... new developments are coming soon...