2. BTeV Software Workshop - FERMILAB October 2001 1 Dario Menasce I.N.F.N. Milano Code Browser Philosophy

3. BTeV Software Workshop - FERMILAB October 2001 2 Let’s take for granted that we want a code browser in BTeV: what’s out there in the market that suits our needs? Usually commercially available IDE (like Microsoft Visual C++) provide excellent code browsing capabilities. Why not use an IDE? an IDE usually works on a SINGLE platform: no tools are provided to allow concurrent developers to manage a single project through the network, or to synchronize access (files locking etc…) or provide smooth access to a CVS repository (at least as far as i know…:) good IDE are usually commercial products available on a Windows platform cost portability and/or compatibility with Linux some exists also on Linux, but code generated with an IDE makes it cumbersome porting to other platforms (make files are generated on the fly…)

4. BTeV Software Workshop - FERMILAB October 2001 3 Example of an IDE: Microsoft’s Visual C++

5. BTeV Software Workshop - FERMILAB October 2001 4

6. 5 So an IDE is good for individual development but impractical for a large scale collaboration effort with centralized CVS repository and remote WEB access. So what else is available to help developers? WEB-based source code browsers are beginning to be available: FOCUS had one of those for FORTRAN, custom developed for the experiment. The browsing was made available to the user in a two stage process: Creation of a database of all tokens (variables, functions etc…) was done each night parsing (compiling) all files pertaining to the code: f77 (only on alpha platforms) can be invoked with a keyword to produce a formatted listing of all tokens in the code being compiled. What I did was just writing a perl parser of that particular output format. Nice but only works for FORTRAN on a particular platform Users could browse the code using forms available on the Web. Perl scripts got input from the user, checked against the database and produced results.

7. BTeV Software Workshop - FERMILAB October 2001 6

8. 7 Regexp!

9. BTeV Software Workshop - FERMILAB October 2001 8

10. 9

11. 10 This tool provided several kinds of helps to the users search based BOTH on language tokens (variables, functions, declarations, etc…) AND on simple strings. In both cases use of regular expressions allowed to implement one-line powerful selection criteria to pin-down code instances. reference to function calls or include files were hyperlinked (navigation tool) (limited) access to the CVS repository was provided (history, differences etc…) graphical display of hierarchical dependencies was provided as well (help to locate were you are in the code once an instance of what you are looking for has been pinpointed tables of errors/warnings found by the compiler (broken code that made it into the distribution) as well as list of potential declaration inconsistencies (common blocks both declared in include files and handwired in the code) a topographical view of the entire code (files grouped by library, dependencies,…)

12. BTeV Software Workshop - FERMILAB October 2001 11 So why not use this marvel? works only for FORTRAN and only on a specific platform (at least the parser) I wrote it when I was in my infancy: difficult to extend/modify/maintain Me, Francesco Prelz and Stefano Magni tried a novel approach Start from scratch, make ourselves available a FORTRAN parser, a C and a C++ parser, customize it, write an ad-hoc database and create an efficient, platform independent token database generator. Results were encouraging, at least in the beginning, BUT: writing a parser is extremely difficult: we just modified egcs and f2c in order to produce a formatted output suitable to be entered in a database. The idea of using those parsers was that, should the project be successful we could ask for the egcs developers to include this feature in the standard distribution. even this “simple” task proved to be difficult

13. BTeV Software Workshop - FERMILAB October 2001 12 On the other hand navigating a source code means much more than just locating were an instance of something is: you would like to know if and when that statement has changed (who did it, why), how’s that piece of code looking like in the development or release version and so on (“navigability” can be broadly defined). whatever source code browser we will end-up using, we will for certain be able to add features, but in order to do that an efficient and up-to-date parser is needed C++ is still undergoing development (STL in particular) so a parser tied to the latest release of the compiler might be extremely valuable: we could devise any kind of new options we like... No source code browser I know of has a real parser at it’s foundation, only very “smart” perl based regular expression parses.

14. BTeV Software Workshop - FERMILAB October 2001 13 What’s out there in the OpenSource community? Lxr: an excellent product (in use by several large experiments at this time) Source is available (even a customized version by CDF) (good) The foundation is pure perl, difficult to make this product up-to-date with coming improvements in the ANSI C++ (at least as far is know) (bad) Not everything is hyperlinked; not necessarily a bad thing, but then no regular-expression engine is provided to restrict searches. Regexp are available ONLY for plain text searches, but that way we loose much of the searching power... Back-end database are Postgres and MySQL WhereTheHeck: our own effort SourcePublisher C++: a commercial product (reminds me a lot of our own stuff…)

15. BTeV Software Workshop - FERMILAB October 2001 14 Our new approach to WhereTheHeck

16. BTeV Software Workshop - FERMILAB October 2001 15 Multiframe WEB browser page Multiframe WEB browser page

17. BTeV Software Workshop - FERMILAB October 2001 16

18. BTeV Software Workshop - FERMILAB October 2001 17

19. BTeV Software Workshop - FERMILAB October 2001 18

20. BTeV Software Workshop - FERMILAB October 2001 19

21. BTeV Software Workshop - FERMILAB October 2001 20

22. BTeV Software Workshop - FERMILAB October 2001 21 Find where a token containing the or string zmin or the string zmax is located in the whole source code of the project mcfast, but only in places where its value gets modified Find where a token containing the or string zmin or the string zmax is located in the whole source code of the project mcfast, but only in places where its value gets modified

23. BTeV Software Workshop - FERMILAB October 2001 22

24. BTeV Software Workshop - FERMILAB October 2001 23 Even simple strings can be searched for, either as plain strings or as regular expressions Even simple strings can be searched for, either as plain strings or as regular expressions

25. BTeV Software Workshop - FERMILAB October 2001 24

26. BTeV Software Workshop - FERMILAB October 2001 25 22) /vtx28/winner/btev/mcfast/v2_6_2/mcfast/src/geom/load_beampipe.f

27. BTeV Software Workshop - FERMILAB October 2001 26

28. BTeV Software Workshop - FERMILAB October 2001 27

29. BTeV Software Workshop - FERMILAB October 2001 28

30. BTeV Software Workshop - FERMILAB October 2001 29

31. BTeV Software Workshop - FERMILAB October 2001 30

32. BTeV Software Workshop - FERMILAB October 2001 31

33. BTeV Software Workshop - FERMILAB October 2001 32

34. BTeV Software Workshop - FERMILAB October 2001 33 Floating Integer….

35. BTeV Software Workshop - FERMILAB October 2001 34

36. BTeV Software Workshop - FERMILAB October 2001 35 LXR as used by the CDF collaboration

37. BTeV Software Workshop - FERMILAB October 2001 36

38. BTeV Software Workshop - FERMILAB October 2001 37

39. BTeV Software Workshop - FERMILAB October 2001 38

40. BTeV Software Workshop - FERMILAB October 2001 39

41. BTeV Software Workshop - FERMILAB October 2001 40

42. BTeV Software Workshop - FERMILAB October 2001 41

43. BTeV Software Workshop - FERMILAB October 2001 42

44. BTeV Software Workshop - FERMILAB October 2001 43 http://scitools.com/sp_examples/apache/ SourcePublisher C++

45. BTeV Software Workshop - FERMILAB October 2001 44

46. BTeV Software Workshop - FERMILAB October 2001 45

47. BTeV Software Workshop - FERMILAB October 2001 46

48. BTeV Software Workshop - FERMILAB October 2001 47 Conclusion Not many tools exist around. In principle we have part of the technology needed to build our own and have full control. I would prefer this solution since I wrote my first code browser six years ago, but the manpower to finish the project?... The existing tools are already in use but suffer of still limited capabilities: situation can change in the future A practical possibility is to get LXR working in our context and eventually collaborate with the developers (I already proposed them to collaborate three years ago but they declined…)