1. Basic Feasible Solutions: Recap MS&E 211

2. WILL FOLLOW A CELEBRATED INTELLECTUAL TEACHING TRADITION

6. TEN STEPS TOWARDS UNDERSTANDING BASIC FEASIBLE SOLUTIONS THESE SLIDES: ONLY INTUITION DETAILS ALREADY COVERED

7. Step 0: Notation Assume an LP in the following form Maximize c T x Subject to: Ax ≤ b x ≥ 0 N Variables, M constraints U = Set of all feasible solutions

8. Step 1: Convex Sets and Convex Combinations Convex set: If two points belong to the set, then any point on the line segment joining them also belongs to the set Convex combination: weighted average of two or more points, such that the sum of weights is 1 and all weights are non-negative – Simple example: average

9. Convex Combination Given points x 1, x 2, …, x K, in N dimensions For any scalars a 1, a 2, …, a K such that – Each a i is non-negative – a 1 + a 2 + … + a K = 1 a 1 x 1 + a 2 x 2 + … + a K x K is a convex combination of x 1, x 2, …, x K

10. Example: Convex combination of two points

11. Exercise If w is a convex combination of x and y, and z is a convex combination of x and w, then must z be a convex combination of x and y?

12. Example: Convex combination of three points

13. Exercise Is the set of all convex combinations of N points always convex?

14. Exercise Can any convex set be represented as a convex combination of N points, for finite N?

15. Step 2: Half spaces are convex a T x ≤ b, a T y ≤ b Choose z = (x+y)/2 Then, we must have a T z ≤ b

16. Step 3: Intersection of convex sets is convex Two convex sets S, T Suppose x, y both belong to S ∩T z = (x+y)/2 (1) z belongs to S since S is convex (2) z belongs to T since T is convex Hence, z belongs to S ∩ T Implication: S∩T is convex

17. Step 4: Feasible solutions to LP The set U of feasible solutions to a Linear Program is convex Proof: U is the intersection of Half Spaces

18. Exercise Must every convex set be bounded?

19. Step 5: Basic Feasible Solution x is a basic feasible solution to a LP, if (1)x is a feasible solution (2)There do not exist two other feasible solutions y, z such that x = (y+z)/2 ALSO known as vertex solution, extreme point solution, corner-point solution

21. Step 6: BFS and Bounded Polytopes If U is bounded, then any point in U is the convex combination of its basic feasible solutions Use b 1, b 2, …, b K to denote the K basic feasible solutions

23. Step 7: BFS and Optimality for Bounded Polytopes If U is bounded and non-empty, then there exists an optimal solution that is also basic feasible

24. Step 8: BFS and Optimality for General LPs If an LP has a basic feasible solution and an optimum solution, then there exists an optimal solution that is also basic feasible Will call these solutions “Vertex Optimal” or “Optimum Basic Feasible”

25. Step 9: Two Implications (1)SOLUTION OF LPs: The Simplex method which walks from BFS to BFS is correct for bounded polytopes (2)STRUCTURAL CONSEQUENCES: The Vertex Optimal Solutions often have very interesting properties. Example: For the matching LP, every vertex optimal solution is integral

27. In Practice Most LP Solvers return an optimum basic feasible solution, when one exists. – Either, they use Simplex – Or, they transform the solution that they do find to a basic feasible solution Hence, when we solve a problem using Excel we get an optimum basic feasible solution, when one exists.

28. Step 10: A useful property At every basic feasible solution, at least N constraints are tight More specifically: Exactly N Linearly Independent Constraints are tight Basic solution: Not necessarily feasible, but exactly N Linearly Independent Constraints are tight

29. Exercise You are given a LP with three decision variables where some of the constraints are 0 ≤ x ≤ 1 0 ≤ y ≤ 1 0 ≤ z ≤ 1 Is it possible to add other constraints and an objective function such that the LP has an optimum solution but not an optimum basic feasible solution?

30. LP in standard form Minimize c T x Subject to: Ax = b x ≥ 0 N Variables, M constraints, assume that the M constraints are Linearly Independent At any BFS, at most M variables are strictly positive

31. Recap A feasible solution is basic feasible if it is not the average of two other feasible solutions If the feasibility region U for a LP is bounded and non- empty, then there exists an optimal solution that is also basic feasible If an LP has a basic feasible solution and an optimum solution, then there exists an optimal solution that is also basic feasible Commercial LP solvers return an optimum basic feasible solution by default, where one exists If there are N decision variables, then a basic feasible solution has N linearly independent constraints tight

32. LP Example: Transportation 32 Retailer 1Retailer 2Retailer 3Retailer 4SUPPLY Warehouse 112 (c 11 )1346500 (s 1 ) Warehouse 2641011700 (s 2 ) Warehouse 31091214 (c 34 )800 (s 3 ) DEMAND400 (d 1 )900 (d 2 )200 (d 3 )500 (d 4 ) 1 2 3 1 2 3 4 x 11 x 34 Abstract Model

33. LP Example: Transportation, N warehouses, M retailers 33 At most M + N non-zero variables at a BFS Can we get to M + N -1? 1 2 N 1 2 3 M x 11 x NM

34. LP Example: Transportation, N warehouses, M retailers 34 Demand = -s i on the left, d j on the right Edges directed from left to right Costs c ij Capacities 1 MIN COST FLOW! Hence, BFSs are going to be integral if demands and supplies are integral 1 2 N 1 2 3 M x 11 x NM

35. LP Example: Transportation, N warehouses, M retailers 35 MIN COST FLOW! No cycles at BFS (else can send flow in both directions) => At most M+N-1 non-zero values (the maximum number of edges in a an acyclic graph on M+N vertices) Demand = -s i on the left, d j on the right Edges directed from left to right Costs c ij Capacities 1 1 2 N 1 2 3 M x 11 x NM

36. Exercise What if the demands and supplies don’t match up? For example, supply is bigger than the demand?

37. Exercise What if the demands and supplies don’t match up? For example, supply is bigger than the demand? Solution: Add a fake node with the residual demand, and add 0 cost, infinite capacity edges from all supply nodes to this fake node 1 2 N 1 2 3 M x 11 x NM

38. Exercise What if there is also a cost to disposal of excess supply?

39. Exercise What if there is also a cost to disposal of excess supply? Solution: Add a cost on the edges from the supply nodes to the fake node Same idea works for excess demand 1 2 N 1 2 3 M x 11 x NM

40. THE SIMPLEX METHOD

41. Basic and Basic Feasible Solution 41 In the LP standard form, select m linearly independent columns, denoted by the variable index set B, from A. Solve A B x B = b for the dimension-m vector x B. By setting the variables, x N, of x corresponding to the remaining columns of A equal to zero, we obtain a solution x such that Ax = b. Then, x is said to be a basic solution to (LP) with respect to the basic variable set B. The variables in x B are called basic variables, those in x N are nonbasic variables, and A B is called a basis. If a basic solution x B ≥ 0, then x is called a basic feasible solution, or BFS. Note that A B and x B follow the same index order in B. Two BFS are adjacent if they differ by exactly one basic variable.

42. Simplex Method George B. Dantzig’s Simplex Method for linear programming stands as one of the most significant algorithmic achievements of the 20th century. It is now over 60 years old and still going strong. The basic idea of the simplex method to confine the search to corner points of the feasible region (of which there are only finitely many) in a most intelligent way, so the objective always improves The key for the simplex method is to make computers see corner points; and the key for interior-point methods is to stay in the interior of the feasible region. 42 x1x1 x2x2

43. From Geometry to Algebra How to make computer recognize a corner point? BFS How to make computer terminate and declare optimality? How to make computer identify a better neighboring corner? 43

44. Feasible Directions at a BFS and Optimality Test 44 Recall at a BFS: A B x B +A N x N = b, and x B > 0 and x N = 0. Thus the feasible directions, d, are the ones to satisfy A B d B +A N d N = 0, d N ≥ 0. For the BFS to be optimal, any feasible direction must be an ascent direction, that is, c T d= c T B d B + c T N d N ≥ 0. From d B = -(A B ) -1 A N d N, we must have for all d N ≥ 0, c T d= -c T B (A B ) -1 A N d N +c T N d N =(c T N - c T B (A B ) -1 A N ) d N ≥ 0 Thus, (c T N - c T B (A B ) -1 A N ) ≥ 0 is necessary and sufficient. It is called the reduced cost vector for nonbasic variables.

45. Computing the Reduced Cost Vector 45 We compute shadow prices, y T = c T B (A B ) -1, or y T A B = c T B, by solving a system of linear equations. Then we compute r T =c T -y T A, where r N is the reduced cost vector for nonbasic variables (and r B =0 always). If one of r N is negative, then an improving feasible direction is found by increasing the corresponding nonbasic variable value  Increase along this direction till one of the basic variables becomes 0 and hence non-basic  We are left with The process will always converge and produce an optimal solution if one exists (special care for unbounded optimum and when two basic variables become 0 at the same time) T

46. Summary The theory of Basic Feasible Solutions leads to a solution method The Simplex algorithm is one of the most influential and practical algorithms of all time However, we will not test or assign problems on the Simplex method in this class (a testament to the fact that this method has been so successful that we can use it as a basic technology)