Focus on Parallel Combinatorial Optimization

Slides:

Advertisements

Similar presentations

Accelerator-based Implementation of the Harris Algorithm International Conference on Image and Signal Processing 2012 (ICISP 2012) June 28-30, Agadir,

Advertisements

Branch prediction Titov Alexander MDSP November, 2009.

Parallel Jacobi Algorithm Steven Dong Applied Mathematics.

1 Finite Constraint Domains. 2 u Constraint satisfaction problems (CSP) u A backtracking solver u Node and arc consistency u Bounds consistency u Generalized.

Dendrogram-based Algorithm Weighted Graphs Flooding Algorithm – Implementation - Parallelization Claude Tadonki and Fernand Meyer Centre de Recherche en.

Algorithms + L. Grewe.

Introductory Lecture. What is Discrete Mathematics? Discrete mathematics is the part of mathematics devoted to the study of discrete (as opposed to continuous)

11Sahalu JunaiduICS 573: High Performance Computing5.1 Analytical Modeling of Parallel Programs Sources of Overhead in Parallel Programs Performance Metrics.

Branch and bound Pasi Fränti Explore all alternatives Solution constructed by stepwise choices Decision tree Guarantees optimal solution Exponential.

Chapter 7 Dynamic Programming 7.

High Performance Computing 1 Parallelization Strategies and Load Balancing Some material borrowed from lectures of J. Demmel, UC Berkeley.

Solving the Protein Threading Problem in Parallel Nocola Yanev, Rumen Andonov Indrajit Bhattacharya CMSC 838T Presentation.

Metaheuristics The idea: search the solution space directly. No math models, only a set of algorithmic steps, iterative method. Find a feasible solution.

Domain decomposition in parallel computing Ashok Srinivasan Florida State University COT 5410 – Spring 2004.

Decentralised Coordination of Mobile Sensors School of Electronics and Computer Science University of Southampton Ruben Stranders,

Distributed Asynchronous Bellman-Ford Algorithm

Parallel Programming Models Jihad El-Sana These slides are based on the book: Introduction to Parallel Computing, Blaise Barney, Lawrence Livermore National.

Claude Tadonki Mines ParisTech – CRI – Mathématiques et Systèmes Laboratoire de l’Accélérateur Linéaire/IN2P3/CNRS France 2nd.

Automatic LQCD Code Generation Quatrièmes Rencontres de la Communauté Française de Compilation December 5-7, 2011 Saint-Hippolyte (France) Claude Tadonki.

Graph Algorithms. Definitions and Representation An undirected graph G is a pair (V,E), where V is a finite set of points called vertices and E is a finite.

8 -1 Dynamic Programming Fibonacci sequence Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, … F i = i if i  1 F i = F i-1 + F i-2 if i  2 Solved.

LATA: A Latency and Throughput- Aware Packet Processing System Author: Jilong Kuang and Laxmi Bhuyan Publisher: DAC 2010 Presenter: Chun-Sheng Hsueh Date:

Synchronization Transformations for Parallel Computing Pedro Diniz and Martin Rinard Department of Computer Science University of California, Santa Barbara.

1 Short Term Scheduling. 2  Planning horizon is short  Multiple unique jobs (tasks) with varying processing times and due dates  Multiple unique jobs.

Graph Algorithms Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar Adapted for 3030 To accompany the text ``Introduction to Parallel Computing'',

1 Solving problems by searching 171, Class 2 Chapter 3.

1 Outline:  Optimization of Timed Systems  TA-Modeling of Scheduling Tasks  Transformation of TA into Mixed-Integer Programs  Tree Search for TA using.

1 Branch and Bound Searching Strategies Updated: 12/27/2010.

ECE 1747H: Parallel Programming Lecture 2: Data Parallelism.

Branch-and-Cut Valid inequality: an inequality satisfied by all feasible solutions Cut: a valid inequality that is not part of the current formulation.

Domain decomposition in parallel computing Ashok Srinivasan Florida State University.

Data Structures and Algorithms in Parallel Computing Lecture 7.

Chapter 7 Dynamic Programming 7.1 Introduction 7.2 The Longest Common Subsequence Problem 7.3 Matrix Chain Multiplication 7.4 The dynamic Programming Paradigm.

The Multicommodity Flow Problem Updated 21 April 2008.

IE 312 Review 1. The Process 2 Problem Model Conclusions Problem Formulation Analysis.

Chapter 3.5 and 3.6 Heuristic Search Continued. Review:Learning Objectives Heuristic search strategies –Best-first search –A* algorithm Heuristic functions.

8 -1 Dynamic Programming Fibonacci sequence Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, … F i = i if i  1 F i = F i-1 + F i-2 if i  2 Solved.

Uses some of the slides for chapters 3 and 5 accompanying “Introduction to Parallel Computing”, Addison Wesley, 2003.

Constraint Programming for the Diameter Constrained Minimum Spanning Tree Problem Thiago F. Noronha Celso C. Ribeiro Andréa C. Santos.

Discrete Optimization MA2827 Fondements de l’optimisation discrète Material from P. Van Hentenryck’s course.

NAME THAT ALGORITHM #2 HERE ARE SOME PROBLEMS. SOLVE THEM. GL HF.

Advanced Algorithms Analysis and Design

Introduction to Parallel Computing: MPI, OpenMP and Hybrid Programming

EMIS 8373: Integer Programming

Courtsey & Copyright: DESIGN AND ANALYSIS OF ALGORITHMS Courtsey & Copyright:

EMGT 6412/MATH 6665 Mathematical Programming Spring 2016

Auburn University COMP8330/7330/7336 Advanced Parallel and Distributed Computing Communication Costs (cont.) Dr. Xiao.

L21: Putting it together: Tree Search (Ch. 6)

Artificial Intelligence

Discrete Mathematics and Its Applications

Chapter 6. Large Scale Optimization

Objective of This Course

Neural Networks for Vertex Covering

Instructors: Fei Fang (This Lecture) and Dave Touretzky

Multi-Objective Optimization

Sungho Kang Yonsei University

Chapter 1. Formulations (BW)

Branch and Bound Searching Strategies

Lecture 2 The Art of Concurrency

NP-Completeness Reference: Computers and Intractability: A Guide to the Theory of NP-Completeness by Garey and Johnson, W.H. Freeman and Company, 1979.

Solving problems by searching

Unit –VII Coping with limitations of algorithm power.

Graph Algorithms: Shortest Path

Major Design Strategies

Major Design Strategies

Chapter 6. Large Scale Optimization

COMPSCI 330 Design and Analysis of Algorithms

Solving problems by searching

Discrete Optimization

Presentation transcript:

Focus on Parallel Combinatorial Optimization Claude Tadonki MINES ParisTech – PSL Research University Centre de Recherche Informatique claude.tadonki@mines-paristech.fr Monthly CRI Seminar MINES ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization Performance Overview of Supercomputers November 2018 ranking 71% 75% 74% 61% 77% Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization Combinatorial Optimization Illustration NP-complete Problem + Subtour breaking constraints Method (Super)Computer TSP heroes: Applegate, Bixby, Chvatal, and Cook Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization Branch-and-Bound Practical instances of discrete (pure or mixed) optimization problems are better solved though a skillfull combination of continuous optimization techniques and branch&bound-like mechanisms. For a pure discrete problem, a relaxation is used. For a mixed formulation, a decomposition approach can be considered. In number of cases, the objective function is (or becomes) non differentiable . We then need a good non differentiable optimization method and solver. Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization Potential Sources of Scalability Issue Load imbalance The real volume of a branch is know after its exploration and cannot be predicted Synchronization There is a strong need of synchronization for the management of the common pool of working items like the gradients and lower/upper bounds. Concurrent and irregular memory accesses From the structural nature of the branch-and-bound, we should expect a significant memory penalty, which will be exacerbated on NUMA architectures. Heavy data exchanges Lot of occurrences of data exchange are expected for the coordination of the process beside ordinary reasons. Resources sharing and weak scalability of node problems Solving the node problems might suffer from weak scalability depending on the Implementation and the required resources are taken from the whole machine. Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

We now focus on directive-based parallelization of Parallelizing common paradigms involved on node problems We now focus on directive-based parallelization of Dynamic Programming and Greedy Algorithms Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization Dynamic Programming Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization Shortest Paths A direct parallelization of a the update is valid because The only dependencies are with the pivot (row and column) The pivot (row and column) is invariant at the corresponding step Pivots sharing is a good point especially with a efficient SM cache protocol Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization Dominated Graph Flooding A direct parallelization of a the update is valid because There is no dependence inside a step The only dependencies are from one step to the next (k axis) & no in-place Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization Dominated Graph Flooding Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization 0-1 Knapsack Problem Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization 0-1 Knapsack Problem All dependencies are of the form The computation along i-axis can be freely parallelized The one-step lifetime of V(i, :) suggests a V(i mod2, :) array compression Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization 0-1 Knapsack Problem Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization Longest Common Subsequence (LCS) The Longest Common Subsequence (LCS) problem is to find the (length of the) longest common contiguous subsequence given two finite sequences. The dependence does not allow a direct parallelization A loop skewing transformation where the computation is done along the hyperplane makes a direct parallelization possible Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization Longest Common Subsequence (LCS) Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization Performance evaluation Intel® Xeon® Processor E5-2699 v4 Released in April 2016 Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization Greedy Algorithms Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)

Focus on Parallel Combinatorial Optimization END END & QUESTIONS Focus on Parallel Combinatorial Optimization Monthly CRI Seminar, Mines ParisTech - CRI January 21, 2019, Fontainebleau (France)