1. Kalman Filter based Track Fit running on Cell S. Gorbunov 1,2, U. Kebschull 2, I. Kisel 2,3, V. Lindenstruth 2 and W.F.J. Müller 1 1 Gesellschaft für Schwerionenforschung mbH, Darmstadt, Germany 2 Kirchhoff Institute for Physics, University of Heidelberg, Germany 3 Laboratory of Information Technologies, JINR, Dubna, Russia IBM, Böblingen February 13, 2007

2. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg2/20Terminology Kalman Filter based Track Fit running on Cell

3. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg3/20 The Kalman Filter 1/3 The Kalman filter is a recursive algorithm which estimates the state of a dynamic system from a series of incomplete and noisy measurements. The Kalman filter is a recursive algorithm which estimates the state of a dynamic system from a series of incomplete and noisy measurements. The filter was developed in papers by Swerling (1958), Kalman (1960), and Kalman and Bucy (1961). The filter is named after Rudolf E. Kalman. An example of an application would be to provide accurate continuously-updated information about the position and velocity of an object given only a sequence of observations about its position, each of which includes some error. It is used in a wide range of engineering applications from radar to computer vision. simpleextendedinformationsquare-root A wide variety of Kalman filters have now been developed, from Kalman's original formulation, now called the simple Kalman filter, to extended filter, the information filter and a variety of square-root filters.

4. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg4/20 Example: Radar Applications 2/3 In a radar application, where one is interested in following a target, information about the location, speed, and acceleration of the target is measured at different moments in time with corruption by noise. r = { x, y, z, v x, v y, v z }  2 x  2 y …  2 y …  2 z  2 z  2 vx  2 vx …  2 vy …  2 vy  2 vz  2 vz C = December 21, 1968. The Apollo 8 spacecraft has just been sent on its way to the Moon. 003:46:31 Collins: Roger. At your convenience, would you please go P00 and Accept? We're going to update to your W-matrix. State vector Covariance matrix position velocity error of x

5. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg5/20 The Kalman Filter Algorithm 3/3 The Kalman filter exploits the dynamics of the target, which govern its time evolution, to remove the effects of the noise and get a good estimate of the location of the target filtering at the present time (filtering), prediction at a future time (prediction), or interpolationsmoothing at a time in the past (interpolation or smoothing). recursive The Kalman filter is a recursive estimator – only the estimated state from the previous time step and the current measurement are needed to compute the estimate for the current state.n n+1 n mean value over n measurements n+1 mean value over n+1 measurements previous estimation new measurement correction weight PredictionorExtrapolation UpdateorFilter

6. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg6/20Terminology Kalman Filter based Track Fit running on Cell

7. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg7/20 The Compressed Baryonic Matter Experiment (GSI, Darmstadt) 1/3 CBM is a typical modern high energy physics experiment CBM is a typical modern high energy physics experiment Challenge: ~ 1000 charged particles/collision 10 7 Au+Au collisions/sec high speed data acquisition and trigger system Track == trajectory

8. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg8/20 Data Acquisition System 2/3 EventBuilderNetwork 100 ev/slice Detector PC Farm 10 7 ev/s 10 5 sl/s 50 kB/ev 5 MB/slice N x M SchedulerScheduler Sub-Farm RU RU RU RU RU RU RU RU RU RU RU RU RU RU RU RU Farm Control System Sub-Farm SF n available SF n tt MAPSSTSRICHTRDECAL SF n tt MAPSSTSRICHTRDECAL SF n tt tt tt tt … Cell, Cell, Cell, Cell … ~1000 PCs

9. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg9/20 Stages of Data Reconstruction 3/3 Track finding Track fitting Vertex finding/fitting Ring finding (PID) Time consuming!! ! Kalman Filter Combinatorics Conformal Mapping Conformal Mapping Hough Transformation Hough Transformation Track Following + Kalman Filter Track Following + Kalman Filter Cellular Automaton + Kalman Filter Cellular Automaton + Kalman Filter

10. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg10/20Terminology Kalman Filter based Track Fit running on Cell

11. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg11/20 Kalman Filter for Track Fit 1/3 detectors measurements e-e-e-e- (r, C) track parameters and errors

12. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg12/20 The Kalman Filter for Track Fit 2/3 arbitrary large errors non-homogeneous magnetic field as large map multiple scattering in material small errors weight for update >>> 256 KB of Local Store not enough accuracy in single precision

13. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg13/20 Modifications of the Fitting Algorithm 3/3 The initial track parameters are directly estimated from the input data. The propagation step is performed directly from measurement to measurement without intermediate steps. Matrix multiplications have been replaced by direct operations on only non-trivial matrix elements. Most loops have been unrolled in order to provide additional instructions for interleaving. All branches have been eliminated from the algorithm to avoid branch misprediction penalty. Calculations have been reordered for better use of the processors pipeline.

14. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg14/20Terminology Kalman Filter based Track Fit running on Cell

15. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg15/20 Cell Processor: Supercomputer-on-a-Chip 1/5 Approach: Run SPEs independently (one collision per SPE)Run SPEs independently (one collision per SPE) Vectorization (SIMDization)Vectorization (SIMDization) Universality (any CPU architecture)Universality (any CPU architecture)

16. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg16/20 Porting the Kalman Filter on Cell 2/5 1.GSI-Linux 2.Virtual machine: Red Hat (Fedora Core 4) Red Hat (Fedora Core 4) Cell Simulator: Cell Simulator:  PPE  SPE 3.Cell Blade SSE2 SSE2 AltiVec Specialized SIMD Data Types: Platform: Use headers to overload +, -, *, / operators --> the source code is unchanged ! c = a + b Scalar double Scalar double Scalar float Scalar float Pseudo-vector (array) Pseudo-vector (array) Vector (4 float) Vector (4 float)

17. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg17/20 SPE Statistics 3/5

18. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg18/20 Modifications of the Fitting Algorithm 4/5 Intel P4 Cell

19. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg19/20 Kalman Filter on Intel Xeon, AMD Opteron and Cell 5/5 Fit of a single track: Fit of thousands of tracks: lxg1411 eh102 blade11bc4 2.1 1.6 1.7 1.9 Cell SPE is: 1.5 times faster than Intel Xeon and 2 times faster than AMD Opteron and 2 times faster than AMD Opteron Cell SPE is: 1.5 times faster than Intel Xeon and 2 times faster than AMD Opteron and 2 times faster than AMD Opteron

20. 13 February 2007, IBM BöblingenIvan Kisel, KIP, Uni-Heidelberg20/20 Summary and Plans Sub-Farm … Cell, Cell, Cell, Cell … On-line Selection (Trigger) DevelopmentDevelopment 1. Distribution of Data 2. Track Finding 3. Track Fit Kalman Filter Intel, AMD and Cell ? Cellular Automaton ? Sub-Farm Demonstrator