1. Adaptive Data Refinement for Parallel Dynamic Programming Applications
Shanjiang Tang1,2, Ce Yu1, Bu-Sung Lee2,3, Chao Sun1, Jizhou Sun1
1Tianjin University, China
2Nanyang Technological University, Singapore
3HP Labs, Singapore
25th May 2012

2. OutLine Background & Motivation Adaptive Data Refinement Approach
Evaluation
Conclusion
Tianjin University

3. Dynamic Programming (DP)
What is Dynamic Programming (DP) ?
DP is a popular algorithm design technique for the solution of many decision and optimization problems by decomposing the problem at hand into a sequence of interrelated decision or optimization steps that are solved one after the others.
Generally, if r represents the cost of a solution composed of sub-problems x1, x2…xl, then r can be written as:
r = g(f(x1), f(x2),f(x3), … , f(xl)).
Examples:
Tianjin University

4. DP Applications: Bioinformatics
DP Algorithms in Bioinformatics
Tianjin University

5. a. Block divisions of DP matrix b. DAG mapping
DP Parallelism
Block-based Partitioning vs DAG
a. Block divisions of DP matrix　 　 b. DAG mapping
Figure 1: The one-to-one mapping between the data blocks of DP matrix and its corresponding DAG
Tianjin University

6. DP Parallelism Programming Model
DAG Data Driven Model
S.J. Tang, C. Yu, et al. EasyPDP: An Efficient Parallel Dynamic Programming
Runtime System for Computational Biology, IEEE Transactions on
Parallel and Distributed Systems, vol 23, no 5, pp , May 2012.
Tianjin University

7. DP Parallelism Framework: EasyPDP
EasyPDP Framework
A shared-memory system implementation of DAG Data Driven Model for DP applications
Adopt the dynamic workers pool
Current version works with C/C++ and use P-threads
Source code (open-source now) downloading at:
Tianjin University

8. DP Parallelism: Workload Unbalance
Wavefront Computation
Non-saturated Computing Domain (NCD)
Saturated Computing Domain (SCD)
Workload Unbalance
Not enough tasks evenly shared by
workers during the NCD
NCD becomes serious when there are hundreds
of workers, e.g., cluster environment
Varying Workloads for irregular DP in SCD
Tianjin University

9. OutLine Background & Motivation Adaptive Data Refinement Approach
Evaluation
Conclusion
Tianjin University

10. Adaptive Data Refinement Approach
Dynamic Data Repartitioning
Re-partition the data block into smaller ones whenever detecting workload unbalanced (i.e., there are idle workers)
Challenging Issues:
How to keep the data dependency for repartitioned block data?
How to detect the workload unbalance during runtime?
How many partitions are suitable for a data repartitioning?
Tianjin University

11. Adaptive Data Refinement Approach
Multi-Level DAG Model
Tianjin University

12. Adaptive Data Refinement Approach
Workload Unbalanced Detection
Based on the principle that the load unbalanced is assumed whenever there are idle workers before the whole computation finishes. Our system just need to periodically check the status of workers in the pool.
Size of Partitions for a data block
Give an empirical result by experiment.
It’s fine when set the number of partitions to the square of the number of workers.
Tianjin University

13. Adaptive Data Refinement Approach
Mechanism
Tianjin University

14. OutLine Background & Motivation Adaptive Data Refinement Approach
Evaluation
Conclusion
Tianjin University

15. Analysis of Detailed Execution
Tianjin University

16. Varying Values of Partitions
Figure 8: the exploration for the suitable value of partitions for repartitioning.
Findings: It’s fine to set the divider d to the number of workers.
Tianjin University

17. Performance Evaluation
Figure 9: The improvement rate of the enhanced EasyPDP system with the adaptive data refinement mechanism against the former EasyPDP version with fixed block size.
Tianjin University

18. OutLine Background & Motivation Adaptive Data Refinement Approach
Evaluation
Conclusion
Tianjin University

19. Conclusion Adaptive Data Refinement Mechanism
Currently, we implement it in our EasyPDP framework. In future, we will add it to our distributed framework (EasyHPS).
The idea is not limited to DP applications, it can be used in other applications whose dependency can be modeled as DAG.
Tianjin University

20. Thanks!