Presentation is loading. Please wait.

Presentation is loading. Please wait.

Dynamic Representation and Prefetching of Linked Data Structures (or Introspection for Low-Level Data Prefetching) Mark Whitney & John Kubiatowicz ROC.

Published byJeffry Terry Modified over 8 years ago

Similar presentations

Presentation on theme: "Dynamic Representation and Prefetching of Linked Data Structures (or Introspection for Low-Level Data Prefetching) Mark Whitney & John Kubiatowicz ROC."— Presentation transcript:

1 Dynamic Representation and Prefetching of Linked Data Structures (or Introspection for Low-Level Data Prefetching) Mark Whitney & John Kubiatowicz ROC Retreat 2002

2 on-chip data cache cache to execution latency: 1-3 compute cycles main memory main memory to cache latency: ~50 compute cycles Memory Prefetching Problem Want data in cache before requested by execution core But all the data will not fit How to prefetch data in a timely manner w/o wasting cache space? prefetcher

3 struct node_t { int datum1; char *datum2; struct node_t *next; }; struct node_t *node;. for (node = head; node != NULL; node = node->next) { if (key1 == datum1 && strcmp(key2,datum2)) break; } 0x1000 0x1004 0x1008 2500 3000 0x2500 0x2504 0x2508 0x3000 0x3004 0x3008 1000 1 “ford” 2 “trillian” 3 “marvin” Example linked list and its traversal Address accesses: 1000, 1004, 1008, 2500, 2504, 2508, 3000… obvious structure not apparent in simple list of memory locations

4 Prefetching Linked Data Structures In Olden benckmark suite, pointer loads contributed 50-90% to data cache misses Linked data structure elements can occupy non-contiguous portions of memory Prefetching sequential blocks of memory is not effective Solution: prefetch elements of data structure sequentially Use information from element just fetched to find next element to prefetch Split prefetching task into two parts: Picking out the structure of linked data elements and build a representation of the data structure Prefetching this data structure effectively using representation

5 for (node = head; node != NULL; node = node->next) { if (key1 == datum1 && strcmp(key2,datum2)) break; } loop: lw $10,0($16) lw $4,4($16) bne $10,$6,endif jal strcmp bne $7,$0,endif j outloop endif: lw $16,8($16) bne $16,$0,loop outloop: lw $16,8($16) lw $10,0($16) lw $4,4($16) lw $16,8($16) lw $10,0($16) lw $4,4($16) lw $16,8($16) lw $10,0($16) lw $4,4($16) node root datum1 datum2 8 40 How do we get here? ? Loop in CCompiled loop Dynamic memory accesses  3 Linked Data Structure Manifestation

6 Problem: could be up to 50 unrelated load instructions for every load instruction relevant to a particular data structure. Solution: extract load dependencies through register use analysis and analysis of store memory addresses. $16 0x118 register producing instruction address 0x118 lw $16,8($16) $3 0x100 register producing instruction address $16 0x118 0x100 lw $3,0($16) Found dependency between loads 0x118 & 0x100 Register use analysis 0x124 lw $5,64($10) 0x12c lw $2,0($11)

7 Spill table: same idea, keeps track of loads that produced values which are temporarily saved to memory Keep another table of load instruction addresses and the values they produce and we get a chain of dependent loads:. lw $16,8($16) lw $10,0($16) lw $4,4($16) lw $16,8($16) lw $10,0($16) lw $4,4($16) lw $16,8($16) lw $10,0($16) lw $4,4($16). offset:8offset:0offset:4offset:8 … … Dependency arcs Data flow graph of dependent loads

8 Chain of dependencies is still linear in the number of memory accesses Apply compression to dependency chain SEQUITUR compression algorithm Produces hierarchical grammar: offset:8offset:0offset:4offset:8 … … rule1  3 rule1=offset:8 lds back:3 offset:0 lds back:1 offset:4 lds back:2 lw $16,8($16) lw $10,0($16) lw $4,4($16) lw $16,8($16) lw $10,0($16) lw $4,4($16) lw $16,8($16) lw $10,0($16) lw $4,4($16) Compression of data flow graph

9 Data Structure found in Benchmark while (list != NULL) { prefcount++; i = village->hosp.free_personnel; p = list->patient; /* This is a bad load */ if (i > 0) { village->hosp.free_personnel--; p->time_left = 3; p->time += p->time_left; removeList(&(village->hosp.waiting), p); addList(&(village->hosp.assess), p); } else { p->time++; /* so is this */ } list = list->forward; } Loop from “health” benchmark in Olden suiteRepresentation generated rule1  10 rule1=offset:8 lds back:3 offset:0 lds back:1 offset:4 lds back:1

10 Prefetching Dependent load chains with compression provide compact representations of linked data structures. Representations that compress well can be cached and then used to later prefetch the data structure. To prefetch, unroll compressed representation; grab the root pointer value from the stream of incoming load instructions rule1: offset:8 lds back:3 initiating load instruction produces 1000 offset:0 lds back:1 fetch data at memory location 0x1000 (gives 1) offset:4 lds back:2 “ “ 0x1004 (gives“ford”) rule1: offset:8 lds back:3 “ “ 0x1008 (gives 2500) offset:0 lds back:1 “ “ 0x2500 (gives 3000) offset:4 lds back:2 “ “ 0x2504 (gives 2).

11 0x118 lw $16,8($16) 0x100 lw $10,0($16) 0x104 lw $4,4($16) 0x118 lw $16,8($16) 0x100 lw $10,0($16) 0x104 lw $4,4($16) 0x118 lw $16,8($16) 0x100 lw $10,0($16) 0x104 lw $4,4($16) rule1  3 rule1=offset:8 lds back:3 offset:0 lds back:1 offset:4 lds back:2 0x118 Representation cache 0x118 lw $16,8($16) returns value 4000 prefetch mem location 0x4000 prefetch mem location 0x4004 prefetch mem location 0x4008 prefetch mem location 0x4100. When the load instruction with address 0x118 is executed again, initiate prefetch, grab result of initiating prefetch as the first memory location prefetched. Representations are cached on first load instruction address Prefetch flow

12 “Introspective” System Architecture for Prefetcher Representation building and prefetching tasks are rather modular so could split the tasks up on different processors. compute build prefetch 0x118 lw $16,8($16) 0x100 lw $3,0($16) … rule1  3 rule1=offset:8 lds back:3 offset:0 lds back:1 offset:4 lds back:2 start value=1000 prefetch 0x1000 prefetch 0x1004 … prefetch result 1 prefetch result “ford” …

13 Preliminary results: correct, timely prefetches? Most prefetches are later accessed by regular instructions. Great! Unfortuately, prefetched data is usually already there. Too bad!

14 Representations work, prefetches late... Representation phase of the algorithm seems to work reasonably well for linked lists. Works less well on trees & arrays Provides no performance gains due not enough prefetching and late prefetches –More work being done to start prefetching farther along in data structure

Download ppt "Dynamic Representation and Prefetching of Linked Data Structures (or Introspection for Low-Level Data Prefetching) Mark Whitney & John Kubiatowicz ROC."

Similar presentations

Memory.

Memory.
Part IV: Memory Management

Part IV: Memory Management
P3 / 2004 Register Allocation. Kostis Sagonas 2 Spring 2004 Outline What is register allocation Webs Interference Graphs Graph coloring Spilling Live-Range.

P3 / 2004 Register Allocation. Kostis Sagonas 2 Spring 2004 Outline What is register allocation Webs Interference Graphs Graph coloring Spilling Live-Range.
Review of the MIPS Instruction Set Architecture. RISC Instruction Set Basics All operations on data apply to data in registers and typically change the.

Review of the MIPS Instruction Set Architecture. RISC Instruction Set Basics All operations on data apply to data in registers and typically change the.
School of EECS, Peking University “Advanced Compiler Techniques” (Fall 2011) Parallelism & Locality Optimization.

School of EECS, Peking University “Advanced Compiler Techniques” (Fall 2011) Parallelism & Locality Optimization.
Chapter 2: Memory Management, Early Systems

Chapter 2: Memory Management, Early Systems
Analysis of programs with pointers. Simple example What are the dependences in this program? Problem: just looking at variable names will not give you.

Analysis of programs with pointers. Simple example What are the dependences in this program? Problem: just looking at variable names will not give you.
Procedure Calls Prof. Sirer CS 316 Cornell University.

Procedure Calls Prof. Sirer CS 316 Cornell University.
Lecture Objectives: 1)Define pipelining 2)Calculate the speedup achieved by pipelining for a given number of instructions. 3)Define how pipelining improves.

Lecture Objectives: 1)Define pipelining 2)Calculate the speedup achieved by pipelining for a given number of instructions. 3)Define how pipelining improves.
1 Today’s lecture  Last lecture we started talking about control flow in MIPS (branches)  Finish up control-flow (branches) in MIPS —if/then —loops —case/switch.

1 Today’s lecture  Last lecture we started talking about control flow in MIPS (branches)  Finish up control-flow (branches) in MIPS —if/then —loops —case/switch.
Technical University of Lodz Department of Microelectronics and Computer Science Elements of high performance microprocessor architecture Memory system.

Technical University of Lodz Department of Microelectronics and Computer Science Elements of high performance microprocessor architecture Memory system.
Data Structures: A Pseudocode Approach with C

Data Structures: A Pseudocode Approach with C
Week 7 - Friday.  What did we talk about last time?  Allocating 2D arrays.

Week 7 - Friday.  What did we talk about last time?  Allocating 2D arrays.
Chapter 7 Memory Management Operating Systems: Internals and Design Principles, 6/E William Stallings Dave Bremer Otago Polytechnic, N.Z. ©2009, Prentice.

Chapter 7 Memory Management Operating Systems: Internals and Design Principles, 6/E William Stallings Dave Bremer Otago Polytechnic, N.Z. ©2009, Prentice.
Praveen Yedlapalli Emre Kultursay Mahmut Kandemir The Pennsylvania State University.

Praveen Yedlapalli Emre Kultursay Mahmut Kandemir The Pennsylvania State University.
Nov 12, 2009IAT 8001 Hash Table Bucket Sort. Nov 12, 2009IAT 8002  An array in which items are not stored consecutively - their place of storage is calculated.

Nov 12, 2009IAT 8001 Hash Table Bucket Sort. Nov 12, 2009IAT 8002  An array in which items are not stored consecutively - their place of storage is calculated.
1 1 Lecture 4 Structure – Array, Records and Alignment Memory- How to allocate memory to speed up operation Structure – Array, Records and Alignment Memory-

1 1 Lecture 4 Structure – Array, Records and Alignment Memory- How to allocate memory to speed up operation Structure – Array, Records and Alignment Memory-
Computational Astrophysics: Methodology 1.Identify astrophysical problem 2.Write down corresponding equations 3.Identify numerical algorithm 4.Find a computer.

Computational Astrophysics: Methodology 1.Identify astrophysical problem 2.Write down corresponding equations 3.Identify numerical algorithm 4.Find a computer.
Multiprocessing Memory Management

Multiprocessing Memory Management
Cache effective mergesort and quicksort Nir Zepkowitz Based on: “Improving Memory Performance of Sorting Algorithms” by Li Xiao, Xiaodong Zhang, Stefan.

Cache effective mergesort and quicksort Nir Zepkowitz Based on: “Improving Memory Performance of Sorting Algorithms” by Li Xiao, Xiaodong Zhang, Stefan.

Similar presentations

Ads by Google