1. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 1 Using GraphViz in Dependancy Analysis of BaBar Software Igor A. Gaponenko LBL / NERSC / HENPC

2. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 2 Contents What is GraphViz and why here? –A problem of migrating CDB (Condition/DB of BaBar) from Objectivity/DB to ROOT I/O and MySQL –Dependency analysis of BaBar software BaBar SRT (Software Release Tools) –A short introduction into relevant (in a context of this talk) aspects of SRT –Generating graphs More on GraphViz and its applications

3. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 3 What is GraphViz? And why here?

4. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 4 So, What is GraphViz? WWW.GRAPHVIZ.ORG Free Graph Visualization Software –Developed at AT&T (to visualize telephone networks?) (Relatively) simple graph description language: DOT Graph presentation is (sort of) separated from its (graph) description More details to come in a dedicated section… But first, why we got so interested in it.

5. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 5 Migrating CDB of BaBar from Objectivity A complex problem involving –>200 persistent classes –A few more hundreds of relevant transient classes –The Objectivity API specific code: ~100 software packages –Direct or indirect package dependencies onto Objectivity API from physics code –1400 binary applications built in latest releases (many of them need CDB) “Certain restrictions apply” –Working on a frozen snapshot of code is not an option –The migration must be incremental Facilitate testing Do things in parallel with other developments –The BaBar software must keep functioning –Both technologies should co-exist w/ no code duplication at a level of physics code Hence –A need in major code re-factoring, reorganizing packages to take out technology- specific code and confine it in specialized packages –A need in instruments helping to analyze code dependencies, monitor an evolution of dependencies in a course of the migration tough transition period

6. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 6 CONCRETE FACTORY What’s wrong with these packages? WHAT WE HAVE WHAT WE NEED CLIENTCONCRETE FACTORYOBJECTIVITYmain()CLIENTCONCRETE FACTORYOBJECTIVITYmain()ABSTRACT FACTORYROOT instantiateuse instantiateuse

7. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 7 PICTURE GENERATED BY GraphViz

8. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 8 More Examples from BaBar A bit more complex package dependency diagram: –http://www.slac.stanford.edu/~gapon/TALKS/2006_May_LBNL_Grou pMeeting/MooseApp_BdbCond.pnghttp://www.slac.stanford.edu/~gapon/TALKS/2006_May_LBNL_Grou pMeeting/MooseApp_BdbCond.png A dependency diagram for configuration files of BaBar Framework based applications: –http://www.slac.stanford.edu/~gapon/TALKS/2006_May_LBNL_Grou pMeeting/mod2tcl_CdbBdbInit.pnghttp://www.slac.stanford.edu/~gapon/TALKS/2006_May_LBNL_Grou pMeeting/mod2tcl_CdbBdbInit.png –Comments: BaBar Framework uses the TCL scripting language for that The above shown diagram presents how TCL files depend on some Framework module. Modules are specially designed configuration objects Framework application are linked with. TCL provides an interactive interface to these objects. A module shown on the diagram is used to control general parameters of CDB.

9. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 9 BaBar SRT

10. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 10 Generating Graphs from BaBar code A For packages, we’re not using source (C++) code! A structure of BaBar SRT is used instead. SRT is based on GNU Make. Plus shell and Perl. SRT has a flat and static dependency model for packages. ALLOWED LIBRARY DEPENDENCIES B C. Y Z ACTUAL LIBRARY DEPENDENCIES A C Y Z D Cyclic dependencies for libraries are not allowed!

11. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 11 SRT release and packages A release is a consistent collection of packages A package defines: one library (optionally) binary(-s) (optionally) exe.scripts Source of package dependencies (defined through special files included into the package makefile): link_A.mk [ bin_A.mk ] [ bin_BinApp.mk ] A C Y Z D A / BinApp.cc E - For binary applications dependency restrictions are relaxed. They’re allowed to have upstream dependencies! - Though, binary always depends on its package’s library.

12. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 12 Generating Graphs for GraphViz Developed the scanners - Perl scripts to scan special (.mk) files in packages –A compact database of dependencies is build – one per SRT release Developed analyzers – Perl scripts taking queries, using the databases and generating answers as: –Human readable dependency printouts –Graph descriptions (in the DOT language) for GraphViz Types of queries supported by analyzers: “..I have an application MyApp. Tell me how my application depends on Objectivity/DB?..” “..I have three applications: App1, App2 and App3. What’s a combined dependency diagram for all of them on these packages: A, B, C and Z?...” “..I have a package called Pravda. Which packages depend directly or indirectly on my package? Asking this because I’m going to do som eserious changes to my packages and I wish to know which applications I may potentially break..”

13. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 13 Automate Everything! The following ideas exist : –Build dependency database at a time when an SRT release is built. –Set up a Web service (CGI, whatever) to provide an interface (HTML forms?) for making queries like those one mentioned on the previous slide. –Generate full dependency diagrams for each application and for each package at a release built time. Publish this information (in a form of DOT programs) in a documentation section of the release. –Generate “interactive” graphs with a URL behind each package box. That URL will bring me into a Web page with… No time to implement them 

14. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 14 An Overview of GraphViz

15. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 15 The DOT Graph Description Language A trivial example of the DOT program and its effect : graph G { run -- intr; intr -- runbl; runbl -- run; run -- kernel; kernel -- zombie; kernel -- sleep; kernel -- runmem; sleep -- swap; swap -- runswap; runswap -- new; runswap -- runmem; new -- runmem; sleep -- runmem; }

16. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 16 Sub-graphs Nested graphs (sub-graphs) can easily be defined as well : digraph G { subgraph cluster_0 { label = "hello world"; a -> b; a -> c; color = hot_pink; } subgraph cluster_1 { label = "MSDOT"; style = "dashed"; color = purple; x -> y; x -> z; y -> z; y -> q; } top -> a; top -> y; y -> b; }

17. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 17 Example: ER Graph

18. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 18 Example: Memory References Graph

19. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 19 Graph Layouts The same graph can be presented by 5 algorithms : –dot makes ``hierarchical'' or layered drawings of directed graphs. The layout algorithm aims edges in the same direction (top to bottom, or left to right) and then attempts to avoid edge crossings and reduce edge length. –neato, fdp make ``spring model'' layouts. neato uses the Kamada-Kawai algorithm, which is equivalent to statistical multi-dimensional scaling. fdp implements the Fruchterman- Reingold heuristic including a multigrid solver that handles larger graphs and clustered undirected graphs. –twopi radial layout, after Graham Wills 97. –circo circular layout, after Six and Tollis 99, Kauffman and Wiese 02. Suitable for certain diagrams of multiple cyclic structures. Lots of examples in here: –http://www.graphviz.org/webdot/demo.htmlhttp://www.graphviz.org/webdot/demo.html

20. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 20 Algorithm: DOT

21. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 21 Algorithm: NEATO

22. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 22 Algorithm: TWOPI

23. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 23 Presentation Attributes In the DOT language a graph definition is combined with its presentation parameters. All attributes: –http://www.graphviz.org/doc/info/attrs.htmlhttp://www.graphviz.org/doc/info/attrs.html Nodes attributes (shapes) : –http://www.graphviz.org/doc/info/shapes.htmlhttp://www.graphviz.org/doc/info/shapes.html Edges attributes : –http://www.graphviz.org/doc/info/arrows.htmlhttp://www.graphviz.org/doc/info/arrows.html Colors : –http://www.graphviz.org/doc/info/colors.htmlhttp://www.graphviz.org/doc/info/colors.html

24. 19-MAY-2006GraphViz in Dependancy Analysis of BaBar SoftwarePage: 24 What else can be tuned? Output formats –http://www.graphviz.org/doc/info/output.htmlhttp://www.graphviz.org/doc/info/output.html In fact, all known graphic formats supported I’ve also heard that it’s also possible to split an output picture of a complex graph onto smaller ones for printing purposes. Never found how to do it though. Had to use a separate tool.