1. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming1 CSV881: Low-Power Design Linear Programming – A Mathematical Optimization Technique Vishwani D. Agrawal James J. Danaher Professor Dept. of Electrical and Computer Engineering Auburn University, Auburn, AL 36849 vagrawal@eng.auburn.edu http://www.eng.auburn.edu/~vagrawal

2. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming2 What is Linear Programming Linear programming (LP) is a mathematical method for selecting the best solution from the available solutions of a problem. Method: State the problem and define variables whose values will be determined. Develop a linear programming model: Write the problem as an optimization formula (a linear expression to be minimized or maximized) Write a set of linear constraints An available LP solver (computer program) gives the values of variables.

3. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming3 Types of LPs LP – all variables are real. ILP – all variables are integers. MILP – some variables are integers, others are real. A reference: S. I. Gass, An Illustrated Guide to Linear Programming, New York: Dover, 1990.

4. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming4 A Single-Variable Problem Consider variable x Problem: find the maximum value of x subject to constraint, 0 ≤ x ≤ 15. Solution: x = 15. 0 15 Constraint satisfied x Solution x = 15

5. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming5 Single Variable Problem (Cont.) Consider more complex constraints: Maximize x, subject to following constraints: x ≥ 0(1) 5x ≤ 75(2) 6x ≤ 30(3) x ≤ 10(4) 051015x (1) (2) (3) (4) All constraints satisfied Solution, x = 5

6. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming6 A Two-Variable Problem Manufacture of chairs and tables: Resources available: Material: 400 boards of wood Labor: 450 man-hours Profit: Chair: $45 Table: $80 Resources needed: Chair 5 boards of wood 10 man-hours Table 20 boards of wood 15 man-hours Problem: How many chairs and how many tables should be manufactured to maximize the total profit?

7. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming7 Formulating Two-Variable Problem Manufacture x 1 chairs and x 2 tables to maximize profit: P = 45x 1 + 80x 2 dollars Subject to given resource constraints: 400 boards of wood,5x 1 + 20x 2 ≤ 400(1) 450 man-hours of labor,10x 1 + 15x 2 ≤ 450(2) x 1 ≥ 0(3) x 2 ≥ 0 (4)

8. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming8 Solution: Two-Variable Problem Chairs, x 1 Tables, x 2 (1) (2) 0 10 20 30 40 50 60 70 80 90 40 30 20 10 0 (24, 14) Profit increasing decresing P = 2200 P = 0 Best solution: 24 chairs, 14 tables Profit = 45×24 + 80×14 = 2200 dollars (3) (4) Material constraint Man-power constraint

9. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming9 Change Profit of Chair to $64/Unit Manufacture x 1 chairs and x 2 tables to maximize profit: P = 64x 1 + 80x 2 dollars Subject to given resource constraints: 400 boards of wood,5x 1 + 20x 2 ≤ 400(1) 450 man-hours of labor,10x 1 + 15x 2 ≤ 450(2) x 1 ≥ 0(3) x 2 ≥ 0 (4)

10. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming10 Solution: $64 Profit/Chair Chairs, x 1 Tables, x 2 (1) (2) Profit increasing decresing P = 2880 P = 0 Best solution: 45 chairs, 0 tables Profit = 64×45 + 80×0 = 2880 dollars 0 10 20 30 40 50 60 70 80 90 (24, 14) 40 30 20 10 0 (3) (4) Material constraint Man-power constraint

11. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming11 A Dual Problem Explore an alternative. Questions: Should we make tables and chairs? Or, auction off the available resources? To answer this question we need to know: What is the minimum price for the resources that will provide us with same amount of revenue from sale as the profits from tables and chairs? This is the dual of the original problem.

12. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming12 Formulating the Dual Problem Revenue received by selling off resources: For each board, w 1 For each man-hour, w 2 Minimize 400w 1 + 450w 2 Subject to constraints: 5w 1 + 10w 2 ≥ 45 20w 1 + 15w 2 ≥ 80 w 1 ≥ 0 w 2 ≥ 0 Resources: Material: 400 boards Labor: 450 man-hrs Profit: Chair: $45 Table: $80 Resources needed: Chair 5 boards of wood 10 man-hours Table 20 boards of wood 15 man-hours

13. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming13 The Duality Theorem If the primal has a finite optimum solution, so does the dual, and the optimum values of the objective functions are equal. If the primal has a finite optimum solution, so does the dual, and the optimum values of the objective functions are equal.

14. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming14 Primal-Dual Problems Primal problem Fixed resources Maximize profit Variables: x 1 (number of chairs) x 2 (number of tables) Maximize profit 45x 1 +80x 2 Subject to: 5x 1 + 20x 2 ≤ 400 10x 1 + 15x 2 ≤ 450 x 1 ≥ 0 x 2 ≥ 0 Solution: x 1 = 24 chairs, x 2 = 14 tables Profit = $2200 Dual Problem Fixed profit Minimize value Variables: w 1 ($ value/board of wood) w 2 ($ value/man-hour) Minimize value 400w 1 +450w 2 Subject to: 5w 1 + 10w 2 ≥ 45 20w 1 + 15w 2 ≥ 80 w 1 ≥ 0 w 2 ≥ 0 Solution: w 1 = $1, w 2 = $4 value = $2200

15. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming15 LP for n Variables n minimize Σ cj xjObjective function j =1 n subject to Σ aij xj ≤ bi, i = 1, 2,..., m j =1 n Σ cij xj = di, i = 1, 2,..., p j =1 Variables: xj Constants: cj, aij, bi, cij, di

16. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming16 Algorithms for Solving LP Simplex method Simplex method G. B. Dantzig, Linear Programming and Extension, Princeton, New Jersey, Princeton University Press, 1963. G. B. Dantzig, Linear Programming and Extension, Princeton, New Jersey, Princeton University Press, 1963. Ellipsoid method Ellipsoid method L. G. Khachiyan, “A Polynomial Algorithm for Linear Programming,” Soviet Math. Dokl., vol. 20, pp. 191-194, 1984. L. G. Khachiyan, “A Polynomial Algorithm for Linear Programming,” Soviet Math. Dokl., vol. 20, pp. 191-194, 1984. Interior-point method Interior-point method N. K. Karmarkar, “A New Polynomial-Time Algorithm for Linear Programming,” Combinatorica, vol. 4, pp. 373-395, 1984. N. K. Karmarkar, “A New Polynomial-Time Algorithm for Linear Programming,” Combinatorica, vol. 4, pp. 373-395, 1984. Course website of Prof. Lieven Vandenberghe (UCLA), http://www.ee.ucla.edu/ee236a/ee236a.html Course website of Prof. Lieven Vandenberghe (UCLA), http://www.ee.ucla.edu/ee236a/ee236a.html http://www.ee.ucla.edu/ee236a/ee236a.html

17. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming17 Basic Ideas of Solution methods Constraints Extreme points Objective function Constraints Extreme points Objective function Simplex: search on extreme points. Complexity: polynomial in n, number of variables Interior-point methods: Successively iterate with interior spaces of analytic convex boundaries. Complexity: O(n 3.5 L), L = no. of int. values

18. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming18 Integer Linear Programming (ILP) Variables are integers. Variables are integers. Complexity is exponential – higher than LP. Complexity is exponential – higher than LP. LP relaxation LP relaxation Convert all variables to real, preserve ranges. Convert all variables to real, preserve ranges. LP solution provides guidance. LP solution provides guidance. Rounding LP solution can provide a non- optimal solution. Rounding LP solution can provide a non- optimal solution.

19. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming19 Traveling Salesperson Problem (TSP) 1 3 2 5 4 12 27 18 10 5 20 12 15 19 6

20. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming20 Solving TSP: Five Cities Distances (dij) in miles (symmetric TSP, general TSP is asymmetric) City j=1 j=1 j=2 j=2j=3j=4j=5 i=1 i=1018101227 i=2 i=218051220 i=3 i=310501519 i=4 i=412121506 i=5 i=527201960

21. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming21 Search Space: No. of Tours Asymmetric TSP tours Asymmetric TSP tours Five-city problem: 4 × 3 × 2 × 1 = 24 tours Five-city problem: 4 × 3 × 2 × 1 = 24 tours Ten-city problem: 362,880 tours Ten-city problem: 362,880 tours 15-city problem: 87,178,291,200 tours 15-city problem: 87,178,291,200 tours 50-city problem: 49! = 6.08×10 tours 50-city problem: 49! = 6.08×10 62 tours Time for enumerative search assuming 1 μs per tour evaluation=1.93×10 years Time for enumerative search assuming 1 μs per tour evaluation=1.93×10 55 years

22. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming22 A Greedy Heuristic Solution City j = 1 j = 2j = 3j = 4j = 5 i = 1 (start) 018101227 i = 218051220 i = 310501519 i = 412 1506 i = 527201960 Tour length = 10 + 5 + 12 + 6 + 27 = 60 miles (non-optimal)

23. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming23 ILP Variables, Constants and Constraints 1 3 2 5 4 d14 = 12 d15 = 27 d12 = 18 d13 = 10 x14 ε [0,1] x15 ε [0,1] x12 ε [0,1] x13 ε [0,1] x12 + x13 + x14 + x15 = 1 four other similar equations Integer variables: xij = 1, travel i to j xij = 0, do not travel i to j Real constants: dij = distance from i to j

24. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming24 Objective Function and ILP Solution 5 i - 1 Minimize ∑ ∑ xij × dij i = 1 j = 1 xij xij j =1 j =12345 i =1 i =100100 210000 301000 400001 500010 5 ∑ xij = 1, for all i, i.e., every node i has exactly one outgoing edge. j = 1 j ≠ i

25. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming25 ILP Solution 1 3 2 5 4 d13 = 10 d45 = 6 Total length = 45 but not a single tour d54 = 6 d21 = 18 d32 = 5

26. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming26 Additional Constraints for Single Tour Following constraints prevent split tours. For any subset S of cities, the tour must enter and exit that subset: Following constraints prevent split tours. For any subset S of cities, the tour must enter and exit that subset: ∑ xij ≥ 2 for all S, |S| < 5 i ε S j ε S Any subset Remaining set At least two arrows must cross this boundary.

27. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming27 ILP Solution 1 3 2 5 4 d13 = 10 d41 = 12 Total length = 53 d54 = 6 d25 = 20 d32 = 5

28. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming28 Characteristics of ILP Worst-case complexity is exponential in number of variables. Worst-case complexity is exponential in number of variables. Linear programming (LP) relaxation, where integer variables are treated as real, gives a lower bound on the objective function. Linear programming (LP) relaxation, where integer variables are treated as real, gives a lower bound on the objective function. Recursive rounding of relaxed LP solution to nearest integers gives an approximate solution to the ILP problem. Recursive rounding of relaxed LP solution to nearest integers gives an approximate solution to the ILP problem. K. R. Kantipudi and V. D. Agrawal, “A Reduced Complexity Algorithm for Minimizing N-Detect Tests,” Proc. 20 th International Conf. VLSI Design, January 2007, pp. 492-497. K. R. Kantipudi and V. D. Agrawal, “A Reduced Complexity Algorithm for Minimizing N-Detect Tests,” Proc. 20 th International Conf. VLSI Design, January 2007, pp. 492-497.

29. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming29 Why ILP Solution is Exponential? LP solution found in polynomial time (bound on ILP solution) Must try all 2 n roundoff points First variable Second variable Constraints Objective (maximize)

30. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming30 ILP Example: Test Minimization A combinational circuit has n test vectors that detect m faults. Each test detects a subset of faults. Find the smallest subset of test vectors that detects all m faults. A combinational circuit has n test vectors that detect m faults. Each test detects a subset of faults. Find the smallest subset of test vectors that detects all m faults. ILP model: ILP model: Assign an integer variable ti ε [0,1] to ith test vector Ti such that ti = 1 means we select Ti, otherwise Assign an integer variable ti ε [0,1] to ith test vector Ti such that ti = 1 means we select Ti, otherwise ti = 0 means we eliminate Ti Define an integer constant fij ε [0,1] such that Define an integer constant fij ε [0,1] such that fij = 1, if ith vector Ti detects jth fault Fj, otherwise fij = 0, if ith vector Ti does not detect jth fault Fj Values of constants fij are determined by fault simulation

31. Test Data T1T2T3T4--Ti--Tn F11100--1--0 F20011--0--1 F30001--0--1 F41000--1--0 ----------- Fj0000--1--1 ----------- Fm1100--0--0 Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming31 n tests m faults fij = 1; vector Ti detects fault Fj Select test Ti if ti = 1

32. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming32 Test Minimization by ILP n minimize Σ ti Objective function i =1 n subject to Σ fij ti ≥ 1, j = 1, 2,..., m i =1

33. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming33 Four-Bit ALU Circuit 74181 Pseudorandom vectors for 100% fault coverage ILP solution Minimized vectors CPU s 285140.65 400131.07 500124.38 1,000124.17 5,0001212.95 10,0001234.61 16,384 (2 14, exhaustive set) 1287.47 14 inputs, 8 outputs

34. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming34 Finding LP/ILP Solvers R. Fourer, D. M. Gay and B. W. Kernighan, AMPL: A Modeling Language for Mathematical Programming, South San Francisco, California: Scientific Press, 1993. Several of programs described in this book are available to Auburn users. R. Fourer, D. M. Gay and B. W. Kernighan, AMPL: A Modeling Language for Mathematical Programming, South San Francisco, California: Scientific Press, 1993. Several of programs described in this book are available to Auburn users. B. R. Hunt, R. L. Lipsman, J. M. Rosenberg, K. R. Coombes, J. E. Osborn and G. J. Stuck, A Guide to MATLAB for Beginners and Experienced Users, Cambridge University Press, 2006. B. R. Hunt, R. L. Lipsman, J. M. Rosenberg, K. R. Coombes, J. E. Osborn and G. J. Stuck, A Guide to MATLAB for Beginners and Experienced Users, Cambridge University Press, 2006. Search the web. Many programs with small number of variables can be downloaded free. Search the web. Many programs with small number of variables can be downloaded free.

35. A Circuit Optimization Problem Given: Given: Circuit netlist Circuit netlist Cell library with multiple versions for each cell Cell library with multiple versions for each cell Select cell versions to optimize a specified characteristic of the circuit. Typical characteristics are: Select cell versions to optimize a specified characteristic of the circuit. Typical characteristics are: Area Area Power Power Delay Delay Example: Minimize power for given delay. Example: Minimize power for given delay. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming35

36. Gate Library: NAND(X), X = 0 or 1 X: an integer variable for each gate. X: an integer variable for each gate. X = 0, choose gate with small delay X = 0, choose gate with small delay Delay = d × fo, where fo = number of fanouts for gate Delay = d × fo, where fo = number of fanouts for gate Power = 3 × p × fo Power = 3 × p × fo d and p are parameters of technology d and p are parameters of technology X = 1, choose gate with low power X = 1, choose gate with low power Delay = 2 × d × fo Delay = 2 × d × fo Power = 0.5 × p × fo Power = 0.5 × p × fo Normalized gate delay = [(1 – X) + 2 X] × fo Normalized gate delay = [(1 – X) + 2 X] × fo Normalized power = [3(1 – X) + 0.5 X] × fo Normalized power = [3(1 – X) + 0.5 X] × fo Normalization: d = 1, p = 1 Normalization: d = 1, p = 1 Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming36

37. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming37 Example: One-Bit Full Adder ABCABC CO SUM 1 1 1 2 3 1 1 3 2 Number of fanouts, fo

38. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming38 Define Arrival Time Variables, Tk ABCABC CO SUM 1 1 1 2 3 1 1 3 2 Number of fanouts, fo T1 = T2 = T3 = 0 T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T12 T11 Tk = Latest signal arrival time at output of gate k

39. Constraint: Gate k in the Circuit Ti = signal arrival time at ith input of gate k Ti = signal arrival time at ith input of gate k Tk = signal arrival time at gate k output Tk = signal arrival time at gate k output Tk ≥ Ti + (1 – Xk) fo(k) + 2 Xk fo(k), for all i Tk ≥ Ti + (1 – Xk) fo(k) + 2 Xk fo(k), for all i Where, fo(k) = fanout number of gate k Where, fo(k) = fanout number of gate k Xk = 0, choose fast cell for k Xk = 1, choose low power cell for k Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming39

40. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming40 Arrival Time Constraints on Gate 7 ABCABC CO SUM 1 1 1 2 3 1 1 3 2 Number of fanouts, fo T1 = T2 = T3 = 0 T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T12 T11 T7 ≥ T5 + (1 – X7) 2 + 2 X7 ✕ 2 T7 ≥ T6 + (1 – X7) 2 + 2 X7 ✕ 2

41. Clock Constraints Ti = 0, for all primary inputs i Ti = 0, for all primary inputs i To ≤ Tc, clock period, for all primary outputs o To ≤ Tc, clock period, for all primary outputs o Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming41 Combinational Logic Register Clock

42. Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming42 Critical Path Constraints ABCABC CO SUM 1 1 1 2 3 1 1 3 2 Number of fanouts, fo T1 = T2 = T3 = 0 T1 T2 T3 T4 T5 T6 T7 T8 T9 ≤ Tc T10 T12 ≤ Tc T11

43. Optimization Function Minimize ∑ 3(1 – Xk) fo(k) + 0.5 Xk fo(k) Minimize ∑ 3(1 – Xk) fo(k) + 0.5 Xk fo(k) all gates k all gates k Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming43

44. Typical Result Copyright Agrawal, 2011Lectures 8 and 9,: Linear Programming44 Normalized delay (Tc) Normalized power 5101520 45 35 25 15 5 (11, 45) (22, 7.5)