Optimization Problems

Optimization Problems
Where we learn more fun things we can do with constraints. CSC Simplification, Optimization and Implication

CSC5240 - Simplification, Optimization and Implication
Often given some problem which is modelled by constraints we don’t want just any solution, but a “best” solution This is an optimization problem We need an objective function, a mapping from solutions to reals, so that we can rank solutions By convention, we always try to minimize CSC Simplification, Optimization and Implication

An optimization problem (C,f) consists of a constraint C and objective function f A valuation v1 is preferred to valuation v2 if f(v1) < f(v2) An optimal solution is a solution of C such that no other solution of C is preferred to it Optimization problems may have more than one optimal solution CSC Simplification, Optimization and Implication

Optimization Example An optimization problem Find the closest point to the origin satisfying C. Some solutions and f values Optimal solution CSC Simplification, Optimization and Implication

Optimization Peculiarities
Some optimization problems have no solution. Constraint has no solution For any solution, there is a more preferable one. In other words, problem has no optimum. Can you give an example? CSC Simplification, Optimization and Implication

Optimization for FD CSPs
Domains are finite We can use a solver to build a straightforward optimizer retry_int_opt(C, D, f, best) D2 := int_solv(C,D) if D2 is a false domain then return best let sol be the solution corresponding to D2 return retry_int_opt(C /\ f < sol(f), D, f, sol) CSC Finite Constraint Domains

Retry Optimization Example
Smugglers knapsack problem (optimize profit) First solution found: Corresponding solution CSC Finite Constraint Domains

Smugglers knapsack problem (optimize profit) Next solution found: Corresponding solution CSC Finite Constraint Domains

Smugglers knapsack problem (optimize profit) No next solution! Corresponding solution CSC Finite Constraint Domains

Smugglers knapsack problem (optimize profit) No next solution! Return best solution CSC Finite Constraint Domains

CSC5240 - Finite Constraint Domains
Branch and Bound (BAB) Since the solver may use backtrack search anyway combine it with the optimization At each step in backtracking search, if best is the best solution so far add the constraint f < best(f) CSC Finite Constraint Domains

Branch and Bound (BAB) Eg
Smugglers knapsack problem (whiskey available) Current domain: CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 P = 1 CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 Solution Found: add constraint P = 1 (0,1,3) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 P = 1 (0,1,3) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 P = 1 P = 2 (0,1,3) false CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 P = 1 P = 2 P = 3 (0,1,3) false false CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 W = 1 P = 1 P = 2 P = 3 (1,1,1) (0,1,3) false false CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 W = 1 P = 1 P = 2 P = 3 (1,1,1) (0,1,3) false false Modify constraint CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 W = 1 W = 2 P = 1 P = 2 P = 3 (1,1,1) false (0,1,3) false false Modify constraint CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 W = 1 W = 2 P = 1 P = 2 P = 3 (1,1,1) false (0,1,3) false false Return last sol (1,1,1) CSC Finite Constraint Domains

The Simplex Algorithm - 1
The most widely used optimization algorithm Optimizes a linear function wrt to linear constraints Related to Gauss-Jordan elimination Based on the following observations Linear constraints form convex polygons Optimal solution always occur at a vertex CSC Simplification, Optimization and Implication

The Simplex Form An optimization problem (C, f) is in simplex form: C is the conjunction of CE and CI CE is a conjunction of linear equations CI constrains all variables in C to be non-negative f is a linear expression over variables in C CSC Simplification, Optimization and Implication

Simplex Form Example An optimization problem in simplex form An arbitrary problem can be put in simplex form by replacing unconstrained var X by new vars and inequality by new vars and CSC Simplification, Optimization and Implication

Simplex Solved Form - 1 The simplex method works by repeatedly transforming an optimization problem in “basic feasible solved form” and to another A simplex optimization problem is in basic feasible solved (bfs) form if: The equations are in solved form Each constant on the right hand side is non-negative Only parameters occur in the objective CSC Simplification, Optimization and Implication

Simplex Solved Form - 2 Every problem in bfs form has a corresponding “basic feasible solution” which corresponds to a vertex of the polygon defined by the constraint A basic feasible solution is obtained by setting each parameter to 0 and each non-parameter to the constant in its equation CSC Simplification, Optimization and Implication

Simplex Solved Form Example
An equivalent problem to the previous example in the bfs form We can read off a solution and its objective value CSC Simplification, Optimization and Implication

The simplex algorithm finds the optimum by repeatedly looking for an “adjacent” basic feasible solved form whose basic feasible solution decreases the value of the objective function until no such neighbour can be found By adjacent, we mean that the new bfs form can be reached by a single “pivoting” operation CSC Simplification, Optimization and Implication

starting from a problem in bfs form repeat Choose a variable y with negative coefficient in the obj. func. Find the equation x = b + cy where c<0 and -b/c is minimal Rewrite this equation with y being the subject y = -b/c + 1/c x + ... Substitute -b/c + 1/c x for y in all other eqns and obj. func. until no such variable y exists or no such equation exists if no such y exists then optimum is found else there is no optimum solution CSC Simplification, Optimization and Implication

Simplex Example Choose variable Z, the 2nd eqn is only one with neg. coeff Choose variable Y, the first eqn is only one with neg. coeff No variable can be chosen, optimal value 2 is found CSC Simplification, Optimization and Implication

Another Simplex Example - 1
An equivalent simplex form is: An optimization problem showing contours of the objective function CSC Simplification, Optimization and Implication

Another Simplex Example - 2
Basic feasible solved form: circle Choose S3, replace using 2nd eq Optimal solution: box CSC Simplification, Optimization and Implication

The Missing Part - 1 How do we get the initial bfs form and its basic feasible solution? Solve a different simplex problem Add artificial variables to make equations in basic feasible solved form Minimize the sum of the artificial variables If the sum is zero, we can construct a bfs form for the original problem CSC Simplification, Optimization and Implication

The Missing Part Example - 1
Original simplex form equations With artificial vars in bfs form: Objective function rewritten: minimize CSC Simplification, Optimization and Implication

The Missing Part Example - 2
Problem after minimization of objective function Removing the artificial variables, the original problem is equivalent to CSC Simplification, Optimization and Implication

The Missing Part - 2 Must remember to eliminate the basic variables of the resultant bfs form from the original objective function Finding a bfs form of the original constraint is exactly a constraint satisfaction problem, and thus the first phase of the simplex method is an efficient constraint solver for linear inequalities CSC Simplification, Optimization and Implication

Integer Linear Programming
Unlike Simplex, the previous methods don’t use the objective function to direct search Integer Linear Programming for (C,f) use simplex to find a real optimal, if solution is integer stop otherwise choose a var x with non-integer opt value d and examine the problems use the current best solution to constrain prob CSC Finite Constraint Domains

Integer Linear Programming Eg
Smugglers knapsack problem CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 false CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 false false CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 false false false CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 false false false false CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 Solution (1,1,1) = 32 false false false false CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 Solution (1,1,1) = 32 false Worse than best sol false false false CSC Finite Constraint Domains

Smugglers knapsack problem false Solution (2,0,0) = 30 Solution (1,1,1) = 32 false Worse than best sol false false false CSC Finite Constraint Domains

Smugglers knapsack problem false false Solution (2,0,0) = 30 Solution (1,1,1) = 32 false Worse than best sol false false false CSC Finite Constraint Domains

Optimization Problems

Similar presentations

Presentation on theme: "Optimization Problems"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Optimization Problems

Similar presentations

Presentation on theme: "Optimization Problems"— Presentation transcript:

Similar presentations

About project

Feedback