1. Experiences Reviewing Scientific C++ Code Marc Paterno f CD/Special Assignments

2. Outline Background Background Abstraction Abstraction Need, use and misuse Need, use and misuse Libraries Libraries Standard library Standard library HEP libraries HEP libraries

3. Review experience J. Kowalkowski & I are part of the CD/SA department. J. Kowalkowski & I are part of the CD/SA department. One of our major tasks is reviewing C++ designs and implementations for experiments. One of our major tasks is reviewing C++ designs and implementations for experiments. 7 major reviews in last 18 months 7 major reviews in last 18 months Analysis objectsTrack reconstruction (x2) Analysis objectsTrack reconstruction (x2) Calorimeter reconstructionMuon reconstruction Calorimeter reconstructionMuon reconstruction Detector simulation(Un)packing of raw data Detector simulation(Un)packing of raw data Most reviews are of significant subsystems Most reviews are of significant subsystems 3-10 packages (libraries)50+ classes 3-10 packages (libraries)50+ classes

4. Most useful feature of C++ Clearly, the most useful: support for abstraction, both in OO design and generic programming Clearly, the most useful: support for abstraction, both in OO design and generic programming To appreciate this, let’s have a look at where we’re coming from… minimally structured FORTRAN.

5. SUBROUTINE DECOMP(NDIM,N,A,COND,IPVT,WORK) IMPLICIT DOUBLE PRECISION (A- H,O-Z) DIMENSION A(NDIM,N),WORK(N),IPVT(N) IPVT(N)=1 IF(N.EQ.1)GOTO 80 NM1=N-1 ANORM=0.0 DO 10 J=1,N T=0.0 DO 5 I=1,N T=T+DABS(A(I,J)) 5 CONTINUE IF(T.GT.ANORM)ANORM=T 10 CONTINUE DO 35 K=1,NM1 KP1=K+1 M=K DO 15 I=KP1,N IF(DABS(A(I,K)).GT.DABS(A(M,K)))M=I 15 CONTINUE IPVT(K)=M IF(M.NE.K)IPVT(N)=-IPVT(N) T=A(M,K) A(M,K)=A(K,K) A(K,K)=T C IF(T.EQ.0.0)GOTO 35 C DO 20 I=KP1,N A(I,K)=-A(I,K)/T 20 CONTINUE C DO 30 J=KP1,N T=A(M,J) A(M,J)=A(K,J) A(K,J)=T IF(T.EQ.0.0)GOTO 30 DO 25 I=KP1,N A(I,J)=A(I,J)+A(I,K)*T 25 CONTINUE 30 CONTINUE 35 CONTINUE

6. DO 50 K=1,N T=0.0 IF(K.EQ.1)GOTO 45 KM1=K-1 DO 40 I=1,KM1 T=T+A(I,K)*WORK(I) 40 CONTINUE 45 EK=1.0 IF(T.LT.0.0)EK=-1.0 IF(A(K,K).EQ.0.0)GOTO 90 WORK(K)=-(EK+T)/A(K,K) 50 CONTINUE DO 60 KB=1,NM1 K=N-KB T=0.0 KP1=K+1 DO 55 I=KP1,N T=T+A(I,K)*WORK(K) 55 CONTINUE WORK(K)=T M=IPVT(K) IF(M.EQ.K)GOTO 60 T=WORK(M) WORK(M)=WORK(K) WORK(K)=T 60 CONTINUE YNORM=0.0 DO 65 I=1,N YNORM=YNORM+DABS(WORK(I)) 65 CONTINUE CALL SOLVER(NDIM,N,A, WORK,IPVT) ZNORM=0.0 DO 70 I=1,N ZNORM=ZNORM+DABS(WORK(I)) 70 CONTINUE COND=ANORM*ZNORM/YNORM IF(COND.LT.1.0)COND=1.0 RETURN 80 COND=1.0 IF(A(1,1).NE.0.0)RETURN 90 COND=1.0E32 RETURN END

7. The most problematical feature… Unfortunately the most problematical feature of C++ has also been abstraction, generally in the context of OO design. Unfortunately the most problematical feature of C++ has also been abstraction, generally in the context of OO design. It is not always easy to determine the right level of abstraction for a given use. For many physicists, previous programming experience doesn’t apply -- new techniques have to be learned.

8. Common mistakes with abstraction Missing the useful abstraction Missing the useful abstraction Classes used as structs, if used at all Classes used as structs, if used at all Procedural code, where OO would be better Procedural code, where OO would be better Too many abstractions Too many abstractions Many levels of inheritance Many levels of inheritance Base classes never used Base classes never used Often, casting required Often, casting required Classes that are too large Classes that are too large Too many tasks performed by a single class Too many tasks performed by a single class Resulting classes are difficult to understand and use Resulting classes are difficult to understand and use

9. Missing the Abstraction Common Mistake #1: missing class Common Mistake #1: missing class Set of functions pass around a common set of arguments, or perhaps an array Set of functions pass around a common set of arguments, or perhaps an array Users have to keep track of the arguments themselves Users have to keep track of the arguments themselves Common Mistake #2: do-nothing class Common Mistake #2: do-nothing class Contains set & get methods Contains set & get methods Users extract values, perform calculations themselves Users extract values, perform calculations themselves Often not hard to solve: refactoring Often not hard to solve: refactoring

10. XList* Alg::createXList(const Hit& hit1, const Hit& hit2) { XList* list = new XList; double x1 = hit1.pos().pos().x(); double y1 = hit1.pos().pos().y(); double z1 = hit1.pos().pos().z(); double d2 = hit1.hit().drift(); double x2 = hit2.pos().pos().x(); double y2 = hit2.pos().pos().y(); double z2 = hit2.pos().pos().z(); double d2 = hit2.hit().drift(); Point p11( x1, y1-d1, z1 ); Point p12( x1, y1+d1, z1 ); Point p21( x2, y2-d2, z2 ); Point p22( x2, y2+d2, z2 ); list->push_back(new X(p11, p21)); list->push_back(new X(p11, p22)); list->push_back(new X(p12, p21)); list->push_back(new X(p12, p22)); return list; } Missing class functionality

11. XList* Alg::createXList(const Hit& hit1, const Hit& hit2) { List* li = new XList; li->push_back(new X(hit1.negPoint(), hit2.negPoint())); li->push_back(new X(hit1.negPoint(), hit2.posPoint())); li->push_back(new X(hit1.posPoint(), hit2.negPoint())); li->push_back(new X(hit1.posPoint(), hit2.posPoint())); return li; } Refactoring Exception safety is also a common problem

12. Typical review comment …we had a difficult time telling what exactly the algorithm was and what it was doing. One of our concerns is that we could not tell what manipulations are completely defined by the detector design (fixed angle calculation, fixed relationships of components) and which were actually algorithmic… The source of this problem is that the algorithm has not been decomposed properly into manageable units. Many of the tasks are performed directly by the algorithm code, as opposed to utilities and tools. The tasks perform operations on dumb data structures rather then real objects with behavior.

13. Too many abstractions Common Mistake #3: unused base class Common Mistake #3: unused base class Abstract base class introduced, but only one subclass is ever produced Abstract base class introduced, but only one subclass is ever produced Cost both in program efficiency and maintenance Cost both in program efficiency and maintenance Common Mistake #4: very deep hierarchies Common Mistake #4: very deep hierarchies Add a new layer of inheritance for one or two new functions; no users are interested in the middle layers of the hierarchy Add a new layer of inheritance for one or two new functions; no users are interested in the middle layers of the hierarchy Much more difficult for users to understand the code Much more difficult for users to understand the code

14. Needless Abstraction Only one subclass of each abstract class exists Only one subclass of each abstract class exists Users who get an AbsComponent* from the AbsThing interface have to (dynamic) cast before they can use it to get at function h(). Users who get an AbsComponent* from the AbsThing interface have to (dynamic) cast before they can use it to get at function h(). +f() : AbsComponent AbsThing +g() : void AbsComponent +h() : void ComponentThing Needless complexity makes code harder to understand.

15. Classes that do too much Fat interface Fat interface Scores of unrelated functions Scores of unrelated functions Often, functions form several related groupings Often, functions form several related groupings Multiple purposes Multiple purposes Member datum that says if we are to perform task A or task B Member datum that says if we are to perform task A or task B … even worse if we have to query this state from outside the class … even worse if we have to query this state from outside the class

16. A fat interface example “The [BaseObject] class provides default behavior and protocol for all objects … It provides protocol for object I/O, error handling, sorting, inspection, printing, drawing, etc. Every object which inherits from [BaseObject] can be stored in the … collection classes.” What happens when some subclass can’t sensibly implement the entire interface? We get functions that don’t make sense. What happens when some subclass can’t sensibly implement the entire interface? We get functions that don’t make sense.

17. Abstraction: summary It seems to be quite difficult to find just the right level of abstraction. Cases of too little abstraction seem easier to repair than cases of too much abstraction. This may be partially sociological. Wrong abstraction makes code that is difficult to understand -- and difficult to (re)use.

18. We need… a few good libraries Dearth of high-quality scientific C++ libraries. Dearth of high-quality scientific C++ libraries. Many projects were started with early versions of C++. Many projects were started with early versions of C++. It has been difficult for these to catch up with modern C++: It has been difficult for these to catch up with modern C++: Home-grown string, collection classes Home-grown string, collection classes No use of exceptions -- query isGood() to check objects No use of exceptions -- query isGood() to check objects Lack of exception safety Lack of exception safety No use of templates No use of templates The result is often not robust, less efficient than possible, and more difficult to maintain than need be. The result is often not robust, less efficient than possible, and more difficult to maintain than need be.

19. How is the Standard Library Used? Containers are used frequently Containers are used frequently Except where use of some other library’s collection(s) have taken over! Except where use of some other library’s collection(s) have taken over! New users have some trouble choosing the right container from the Standard Library New users have some trouble choosing the right container from the Standard Library Iterators are used simplistically Iterators are used simplistically Commonly used in loops Commonly used in loops Rarely see a pair of iterators used as a range Rarely see a pair of iterators used as a range Algorithms are chronically under-used Algorithms are chronically under-used Rarer still are user-invented generic algorithms Rarer still are user-invented generic algorithms

20. Summary We have made vast improvement over barely-structured Fortran. We have made vast improvement over barely-structured Fortran. Making good use of abstraction is difficult. Making good use of abstraction is difficult. We don’t take sufficient advantage of the Standard Library. We don’t take sufficient advantage of the Standard Library. In short, we see too little use of modern C++.