Adaptive Data Refinement for Parallel Dynamic Programming Applications Shanjiang Tang1,2, Ce Yu1, Bu-Sung Lee2,3, Chao Sun1, Jizhou Sun1 1Tianjin University, China 2Nanyang Technological University, Singapore 3HP Labs, Singapore 25th May 2012

OutLine Background & Motivation Adaptive Data Refinement Approach Evaluation Conclusion Tianjin University

Dynamic Programming (DP) What is Dynamic Programming (DP) ? DP is a popular algorithm design technique for the solution of many decision and optimization problems by decomposing the problem at hand into a sequence of interrelated decision or optimization steps that are solved one after the others. Generally, if r represents the cost of a solution composed of sub-problems x1, x2…xl, then r can be written as: r = g(f(x1), f(x2),f(x3), … , f(xl)). Examples: Tianjin University

DP Applications: Bioinformatics DP Algorithms in Bioinformatics Tianjin University

a. Block divisions of DP matrix b. DAG mapping DP Parallelism Block-based Partitioning vs DAG a. Block divisions of DP matrix　 　 b. DAG mapping Figure 1: The one-to-one mapping between the data blocks of DP matrix and its corresponding DAG Tianjin University

DP Parallelism Programming Model DAG Data Driven Model S.J. Tang, C. Yu, et al. EasyPDP: An Efficient Parallel Dynamic Programming Runtime System for Computational Biology, IEEE Transactions on Parallel and Distributed Systems, vol 23, no 5, pp. 862-872, May 2012. Tianjin University

DP Parallelism Framework: EasyPDP EasyPDP Framework A shared-memory system implementation of DAG Data Driven Model for DP applications Adopt the dynamic workers pool Current version works with C/C++ and use P-threads Source code (open-source now) downloading at: http://easypdp.sourceforge.net/ http://cs.tju.edu.cn/orgs/hpclab/release/EasyPDP/ Tianjin University

DP Parallelism: Workload Unbalance Wavefront Computation Non-saturated Computing Domain (NCD) Saturated Computing Domain (SCD) Workload Unbalance Not enough tasks evenly shared by workers during the NCD NCD becomes serious when there are hundreds of workers, e.g., cluster environment Varying Workloads for irregular DP in SCD Tianjin University

OutLine Background & Motivation Adaptive Data Refinement Approach Evaluation Conclusion Tianjin University

Adaptive Data Refinement Approach Dynamic Data Repartitioning Re-partition the data block into smaller ones whenever detecting workload unbalanced (i.e., there are idle workers) Challenging Issues: How to keep the data dependency for repartitioned block data? How to detect the workload unbalance during runtime? How many partitions are suitable for a data repartitioning? Tianjin University

Adaptive Data Refinement Approach Multi-Level DAG Model Tianjin University

Adaptive Data Refinement Approach Workload Unbalanced Detection Based on the principle that the load unbalanced is assumed whenever there are idle workers before the whole computation finishes. Our system just need to periodically check the status of workers in the pool. Size of Partitions for a data block Give an empirical result by experiment. It’s fine when set the number of partitions to the square of the number of workers. Tianjin University

Adaptive Data Refinement Approach Mechanism Tianjin University

OutLine Background & Motivation Adaptive Data Refinement Approach Evaluation Conclusion Tianjin University

Analysis of Detailed Execution Tianjin University

Varying Values of Partitions Figure 8: the exploration for the suitable value of partitions for repartitioning. Findings: It’s fine to set the divider d to the number of workers. Tianjin University

Performance Evaluation Figure 9: The improvement rate of the enhanced EasyPDP system with the adaptive data refinement mechanism against the former EasyPDP version with fixed block size. Tianjin University

OutLine Background & Motivation Adaptive Data Refinement Approach Evaluation Conclusion Tianjin University

Conclusion Adaptive Data Refinement Mechanism Currently, we implement it in our EasyPDP framework. In future, we will add it to our distributed framework (EasyHPS). The idea is not limited to DP applications, it can be used in other applications whose dependency can be modeled as DAG. Tianjin University

Thanks!