1. Lecture #3; Based on slides by Yinyu Ye
Geometry of LP: Feasible Regions, Feasible Directions, Basic Feasible Solutions
Ashish Goel Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
Lecture #3; Based on slides by Yinyu Ye

2. LP Feasible Region in the Inequality Form
x simultaneously satisfy
This is the intersection of
the m Half-spaces, and it is
a convex (polyhedron) set
Lecture #3; Based on slides by Yinyu Ye

3. Lecture #3; Based on slides by Yinyu Ye

4. Lecture #3; Based on slides by Yinyu Ye

5. Corner or Extreme Points
Convex Hull:
Extreme Points: A point in a set that does not belong to the hull (convex combination) of the other points
For LP in inequality form, an extreme point is the intersection of n hyperplanes associated with the inequality constraints.
Lecture #3; Based on slides by Yinyu Ye

6. Lecture #3; Based on slides by Yinyu Ye
Feasible Direction
Direction: A direction is notated by a vector d
It is always associated with a “location” or point x
Together a point and a direction define a ray:
x + ϵd, for all ϵ > 0
where d and kd are considered the same direction for all k > 0
Feasible Direction: A direction, d, is said to be “feasible” (relative to a given feasible point x) if x + ϵd is feasible for some ϵ> 0
For LP, all feasible directions at a feasible point form a convex (cone) set
Lecture #3; Based on slides by Yinyu Ye

7. Extreme Feasible Direction
A feasible direction d is extreme if d cannot be written as an convex combination of other feasible directions Interior Point is a point x where every direction is feasible
Lecture #3; Based on slides by Yinyu Ye

8. LP Problem in Inequality Form
Lecture #3; Based on slides by Yinyu Ye

9. Recall the Production Problem
Objective contour c
Lecture #3; Based on slides by Yinyu Ye

10. Basic Theorems of Linear Programming
All LP problems fall into one of three cases:
Problem is infeasible: Feasible region is empty.
Problem is unbounded: Feasible region is unbounded towards the optimizing direction.
Problem is feasible and bounded; and in this case:
there exists an optimal solution or optimizer.
There may be a unique optimizer or multiple optimizers.
All optimizers form a convex set and they are on a face of the feasible region.
There is always at least one corner (extreme) optimizer if the feasible region has a corner point.
Moreover, Local optimality implies global optimality
Lecture #3; Based on slides by Yinyu Ye

11. Sketch Proof of Local Optimality Implies Global
(P) minimize f (x)
subject to x ∈ Ω ⊂ Rn,
is a Convex Optimization problem if f (x) is a convex function over a convex feasible region Ω.
Proof by contradiction. Suppose x’ is a local minimizer but not a global minimizer x∗, that is, x’ ∈ Ω and x∗ ∈ Ω but f (x∗) < f (x’). Now the convex combination point αx’ + (1 − α)x∗ must be feasible (why?), and
f (αx’+ (1 − α)x∗) ≤ αf (x’)+ (1 − α)f (x∗) < f (x’)
for any 0 ≤ α < 1. This contradicts the local optimality as α can be arbitrarily close to 1 so that αx’+ (1 − α)x∗ can be arbitrarily close to x’.
Lecture #3; Based on slides by Yinyu Ye

12. Optimality Certification of the Production Problem
a4 a5 a1 a2 a3 a4
Objective contour c
Lecture #3; Based on slides by Yinyu Ye

13. Feasible Directions at a Corner
a4 a5 a1 a2 a3 a4
Objective contour c
Lecture #3; Based on slides by Yinyu Ye

14. How to Certify a Corner being an Optimizer
Every feasible direction at the corner is a worsening (descent in this case) direction, that is, cTd ≤0 in this case.
Or, equivalently, c is a conic combination of the normal directions at the corner point, that is, there are multipliers α1 ≥0 and α2 ≥0, such as c= α1 ai1+ α2 ai2, in the 2-dimensional case.
In the n-dimensional case:
c= α1 ai1+…+ αn ain
where ai1,…, ain are the normal directions associated with the
corner point.
Lecture #3; Based on slides by Yinyu Ye

15. LP in Standard (Equality) Form
Lecture #3; Based on slides by Yinyu Ye

16. Reduction to Standard Form
max cTx to min – cTx
Eliminating ”free” variables: substitute with the difference of two nonnegative variables
x := xl − xll, (xl, xll) ≥ 0.
Eliminating inequalities: add a slack variable
aT x ≤ b = ⇒ aT x + s = b, s ≥ 0
aT x ≥ b = ⇒ aT x − s = b, s ≥ 0
Lecture #3; Based on slides by Yinyu Ye

17. Reduction of the Production Problem
min
-x1
-2x2
s.t. x1
+x3
= 1 x2
+x4
+x2
+x5
= 1.5
(x1,
x2,
x3,
x4,
x5)
≥ 0
x3, x4,and x5 are called slack variables
Lecture #3; Based on slides by Yinyu Ye

18. Basic and Basic Feasible Solution
In the LP standard form, select m linearly independent columns, denoted by the variable index set B, from A. Solve
AB xB = b
for the dimension-m vector xB . By setting the variables, xN , of x corresponding to the remaining columns of A equal to zero, we obtain a solution x such that Ax = b.
Then, x is said to be a basic solution to (LP) with respect to the basic variable set B. The variables in xB are called basic variables, those in xN are nonbasic variables, and AB is called a basis.
If a basic solution xB ≥ 0, then x is called a basic feasible solution, or BFS. Note that AB and xB follow the same index order in B.
Two BFS are adjacent if they differ by exactly one basic variable.
Lecture #3; Based on slides by Yinyu Ye

19. BS of the Production Problem in Equality Formx1 x2 x1
+x3
= 1 x2
+x4
+x2
+x5
= 1.5
(x1,
x2,
x3,
x4,
x5)
≥ 0
Basis
3,4,5
1,4,5
3,4,1
3,2,5
3,4,2
1,2,3
1,2,4
1,2,5
Feasible? √
x1 , x2
0, 0
1, 0
1.5, 0
0, 1
0, 1.5
.5, 1
1, .5
1, 1
Lecture #3; Based on slides by Yinyu Ye

20. BFS and Corner Point Equivalence Theorem
Theorem Consider the feasible region in the standard LP form. Then, a basic feasible solution and a corner (extreme) point are equivalent; the formal is algebraic and the latter is geometric.
Algorithmically, we need only to look at BFSs for finding an optimizer. But still too many to look…
Lecture #3; Based on slides by Yinyu Ye