Presentation is loading. Please wait.

Presentation is loading. Please wait.

Forest Packing: Fast Parallel, Decision Forests

PublishKlaudia Orsósné Modified over 5 years ago

Similar presentations

Presentation on theme: "Forest Packing: Fast Parallel, Decision Forests"— Presentation transcript:

1 Forest Packing: Fast Parallel, Decision Forests
Author: James Browne In Collaboration With: Disa Mhembere, Tyler M. Tomita, Joshua T. Vogelstein, Randal Burns 17/11/2019

2 Agenda Why is forest inference slow? Inference Acceleration
What is Forest Packing? Why is forest inference slow? Inference Acceleration Memory Layout Traversal Methods Results 17/11/2019

3 Why do we need fast decisions?
17/11/2019

4 Forest Inference New Observation  Class A Class B Class A Tree 1
17/11/2019

5 Standard Inference Reality
Internal Node Leaf Node Processed Node Cache Miss Prefetch Instruction Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  17/11/2019

6 Inference Acceleration Methods
Model Structure Make smaller trees Make full trees Use less trees Reduce Mispredictions Assume direction Predication Batching Reduced Accuracy Minimally Affective High Latency 17/11/2019

7 Memory Optimizations BF DF DF- Breadth First (BF) Depth First (DF)
Combined Leaves (DF-) Statistical Layout (Stat) Contiguous Likely Path Bin Contiguous Tree Space Trees Share Leaves 1 1 1 2 3 2 3 α 2 4 5 4 5 3 4 6 7 8 9 6 7 8 9 β α α β 1 2 3 4 5 6 7 8 9 1 3 5 9 8 4 7 6 2 1 2 4 3 α β Stat Bin 1 1A 1B α 2 α 2A 2B 3B 3 4 3A 4A β 4B β α β α α β β α α β β α 1 2 3 4 α β 1A α β 1B 2A 3A 4A 3B 2B 4B 17/11/2019

8 Memory Optimization: Why Bins?
High frequency nodes in single page file Increases cache hits Reduces cache pollution Access Frequency 100% 50% 25% 12.5% 17/11/2019

9 Traversal Optimization: Round-Robin
Internal Node Leaf Node Processed Node Cache Miss Prefetch Instruction w/ 2 Line Fill Buffers Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  17/11/2019

10 Traversal Optimization: Prefetch
Internal Node Leaf Node Processed Node Cache Miss Prefetch Instruction w/ 2 Line Fill Buffers Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  Tree 1 Tree 2 Tree 3 Time  17/11/2019

11 Inference Execution Tree 1 Tree 2 Tree 3 Tree 1 Tree 2 Standard Tree 1
Round-Robin Tree 1 Tree 2 Tree 3 Prefetching 17/11/2019

12 Prediction Method Comparison
17/11/2019

13 Prediction Method Comparison
17/11/2019

14 Memory Optimization Comparisons
FP Forest Packing is 2x-5x faster compared to other optimized methods FP 17/11/2019

15 Forest Packing: Inference Latency Comparison
Forest Packing (FP) 10x faster 17/11/2019

16 Forest Packing: Performance on Varying Forest Size
Trees in Forest Forest Packing has higher throughput than batching Forest Packing R-RerF 17/11/2019

17 Conclusion What is Forest Packing? Why is forest inference slow?
Inference Acceleration Memory Layout Traversal Methods Results Latency reduced by an order of magnitude Efficiently uses additional resources Comparable throughput to batched systems 17/11/2019

18 Questions? Thank You Source Code:
17/11/2019

Download ppt "Forest Packing: Fast Parallel, Decision Forests"

Similar presentations

Analysis of : Operator Scheduling in a Data Stream Manager CS561 – Advanced Database Systems By Eric Bloom.

Analysis of : Operator Scheduling in a Data Stream Manager CS561 – Advanced Database Systems By Eric Bloom.
1 Optimizing compilers Managing Cache Bercovici Sivan.

1 Optimizing compilers Managing Cache Bercovici Sivan.
CPE 731 Advanced Computer Architecture ILP: Part V – Multiple Issue Dr. Gheith Abandah Adapted from the slides of Prof. David Patterson, University of.

CPE 731 Advanced Computer Architecture ILP: Part V – Multiple Issue Dr. Gheith Abandah Adapted from the slides of Prof. David Patterson, University of.
1 Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers By Sreemukha Kandlakunta Phani Shashank.

1 Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers By Sreemukha Kandlakunta Phani Shashank.
1 Adapted from UCB CS252 S01, Revised by Zhao Zhang in IASTATE CPRE 585, 2004 Lecture 14: Hardware Approaches for Cache Optimizations Cache performance.

1 Adapted from UCB CS252 S01, Revised by Zhao Zhang in IASTATE CPRE 585, 2004 Lecture 14: Hardware Approaches for Cache Optimizations Cache performance.
Algorithm Design Techniques: Greedy Algorithms. Introduction Algorithm Design Techniques –Design of algorithms –Algorithms commonly used to solve problems.

Algorithm Design Techniques: Greedy Algorithms. Introduction Algorithm Design Techniques –Design of algorithms –Algorithms commonly used to solve problems.
1 Advanced Computer Architecture Limits to ILP Lecture 3.

1 Advanced Computer Architecture Limits to ILP Lecture 3.
Chapter 8. Pipelining. Instruction Hazards Overview Whenever the stream of instructions supplied by the instruction fetch unit is interrupted, the pipeline.

Chapter 8. Pipelining. Instruction Hazards Overview Whenever the stream of instructions supplied by the instruction fetch unit is interrupted, the pipeline.
Accurately Approximating Superscalar Processor Performance from Traces Kiyeon Lee, Shayne Evans, and Sangyeun Cho Dept. of Computer Science University.

Accurately Approximating Superscalar Processor Performance from Traces Kiyeon Lee, Shayne Evans, and Sangyeun Cho Dept. of Computer Science University.
A Scalable Front-End Architecture for Fast Instruction Delivery Paper by: Glenn Reinman, Todd Austin and Brad Calder Presenter: Alexander Choong.

A Scalable Front-End Architecture for Fast Instruction Delivery Paper by: Glenn Reinman, Todd Austin and Brad Calder Presenter: Alexander Choong.
Arithmetic Expression Consider the expression arithmetic expression: (a – b) + ((c + d) + (e * f)) that can be represented as the following tree.

Arithmetic Expression Consider the expression arithmetic expression: (a – b) + ((c + d) + (e * f)) that can be represented as the following tree.
Performance Potentials of Compiler- directed Data Speculation Author: Youfeng Wu, Li-Ling Chen, Roy Ju, Jesse Fang Programming Systems Research Lab Intel.

Performance Potentials of Compiler- directed Data Speculation Author: Youfeng Wu, Li-Ling Chen, Roy Ju, Jesse Fang Programming Systems Research Lab Intel.
Reducing Cache Misses (Sec. 5.3) Three categories of cache misses: 1.Compulsory –The very first access to a block cannot be in the cache 2.Capacity –Due.

Reducing Cache Misses (Sec. 5.3) Three categories of cache misses: 1.Compulsory –The very first access to a block cannot be in the cache 2.Capacity –Due.
EENG449b/Savvides Lec 18.1 4/13/04 April 13, 2004 Prof. Andreas Savvides Spring 2004 EENG 449bG/CPSC 439bG Computer.

EENG449b/Savvides Lec /13/04 April 13, 2004 Prof. Andreas Savvides Spring EENG 449bG/CPSC 439bG Computer.
Csci4203/ece43631 Review Quiz. 1)It is less expensive 2)It is usually faster 3)Its average CPI is smaller 4)It allows a faster clock rate 5)It has a simpler.

Csci4203/ece43631 Review Quiz. 1)It is less expensive 2)It is usually faster 3)Its average CPI is smaller 4)It allows a faster clock rate 5)It has a simpler.
Techniques for Efficient Processing in Runahead Execution Engines Onur Mutlu Hyesoon Kim Yale N. Patt.

Techniques for Efficient Processing in Runahead Execution Engines Onur Mutlu Hyesoon Kim Yale N. Patt.
Pipelining. Overview Pipelining is widely used in modern processors. Pipelining improves system performance in terms of throughput. Pipelined organization.

Pipelining. Overview Pipelining is widely used in modern processors. Pipelining improves system performance in terms of throughput. Pipelined organization.
Improved results for a memory allocation problem Rob van Stee University of Karlsruhe Germany Leah Epstein University of Haifa Israel WADS 2007 WAOA 2007.

Improved results for a memory allocation problem Rob van Stee University of Karlsruhe Germany Leah Epstein University of Haifa Israel WADS 2007 WAOA 2007.
Optimizing RAM-latency Dominated Applications

Optimizing RAM-latency Dominated Applications
Making B+-Trees Cache Conscious in Main Memory

Making B+-Trees Cache Conscious in Main Memory

Similar presentations

Ads by Google