1. L2-Cache Miss Profiling on the p690 for a Large-scale Database Application Trevor Morgan, Diana Villa, Patricia J. Teller, and Jaime Acosta The University of Texas at El Paso Department of Computer Science Bret Olszewski IBM Corporation-Austin 4 th Annual Austin CAS Conference – 21 February 2003

2. 4th Annual Austin CAS Conference - 21 February 2003 Outline Motivation Methodology  Events profiled  Information collected Results Conclusions and Future Work

3. 4th Annual Austin CAS Conference - 21 February 2003 Motivation Overall research goal General workload characterization model Project goal  Study load access patterns of TPC-C  Identify and remedy performance impediments

4. 4th Annual Austin CAS Conference - 21 February 2003 Platform & Workload IBM eserver p-Series 690 architecture 8-way and 32-way configurations TPC-C benchmark Data collected via event trace sampling: Timestamp Effective instruction and data addresses CPU id Process id Thread id

5. 4th Annual Austin CAS Conference - 21 February 2003 Servicing of L2-cache miss events L2.5  L2.5 shared  L2.5 modified L2.75  L2.75 shared  L2.75 modified L3 L3.5

6. 4th Annual Austin CAS Conference - 21 February 2003 GP X X X X L2 X X X X GP L2 L3 L2-cache miss events L2.5 Modified - 73 cycles MCM 0 MCM 1

7. 4th Annual Austin CAS Conference - 21 February 2003 GP X X X X X X X X L2 L3 L2.5 Shared - 73 cycles L2-cache miss events MCM 0 MCM 1

8. 4th Annual Austin CAS Conference - 21 February 2003 GP X X X X X X X X L2 L3 L2.75 Modified - 88 cycles L2-cache miss events MCM 0 MCM 1

9. 4th Annual Austin CAS Conference - 21 February 2003 GP X X X X X X X X L2 L3 L2.75 Shared - 88 cycles L2-cache miss events MCM 0 MCM 1 L2

10. 4th Annual Austin CAS Conference - 21 February 2003 GP X X X X X X X X L2 L3 L3 - 112 cycles L2-cache miss events MCM 0 MCM 1

11. 4th Annual Austin CAS Conference - 21 February 2003 GP X X X X X X X X L2 L3 L3.5 - 143 cycles L2-cache miss events MCM 0 MCM 1

12. 4th Annual Austin CAS Conference - 21 February 2003 Results

13. 4th Annual Austin CAS Conference - 21 February 2003 Segment Main Memory & L3 Cache Analysis kernel …. text buffer pool data,bss,heap …. ADDRESS SPACE Page Offset/ Cache line PageInstruction

14. 4th Annual Austin CAS Conference - 21 February 2003 Results - Memory Regions Fraction

15. 4th Annual Austin CAS Conference - 21 February 2003 Results - L3 Cache Fraction

16. 4th Annual Austin CAS Conference - 21 February 2003 Results - Segment Fraction

17. 4th Annual Austin CAS Conference - 21 February 2003 Distribution of L3 data load hits across pages of buffer pool segment 0x070000004 Results - Pages

18. 4th Annual Austin CAS Conference - 21 February 2003 Results – Cache Lines

19. 4th Annual Austin CAS Conference - 21 February 2003 Results - Instructions Lock OperationsAtomic Operations simple_lockfetch_and_add simple_lock_ppcfetch_and_add_h simple_unlockfetch_and_addlp disable_lockfetch_and_or unlock_enablefetch_and_orlp simple_unlock_memfetch_and_and unlock_enable_memfetch_and_andlp

20. 4th Annual Austin CAS Conference - 21 February 2003 disable_locksimple_lock L20.0122470.006251 L2.5 Shared0.0157790.005784 L2.5 Modified0.0109020.005858 L2.75 Shared0.0060950.001899 L2.75 Modified0.0014720.000024 L30.0007950.001418 L3.50.0120720.022305 Memory0.0016570.002716 Results - Instructions Fractions of samples corresponding to atomic operations

21. 4th Annual Austin CAS Conference - 21 February 2003 Identified area for improvement Discovered “bottleneck” in the regions of the address space buffer pool data, bss, heap Lock instructions not important Initial 32-way data shows the same reference pattern Conclusions

22. 4th Annual Austin CAS Conference - 21 February 2003 Complete analysis of 32-way data Database Suggestions leading to performance enhancement Future Work

23. 4th Annual Austin CAS Conference - 21 February 2003 Questions?