1. Heuristic Optimization Methods Optimization as Search
Chin-Shiuh Shieh

2. Optimization as Search
Generate and Test
initialize x do {
generate x’ from x
retain x x’ if F(x’) is better than F(x) }
until terminate condition is met

3. Candidate Solutions
x can be a single point in the solution space, or a population of candidate solutions.

4. Initialization Random initialization Incorporation of domain knowledge
Re-initialization to cope with local optima

5. Generation of x’ from x “Direction” and “Step Size”
Random vs. deterministic
Random walk
Gradient decent
x’  x + Δd ; Large step size
x’  x + δd ; Small step size
x’  x + σu ; Random walk
“Fusing” or “Exchange” explored knowledge, as in GA or PSO

6. Retain Best Solution Retain x’ if F(x’) is better than F(x).
Sometimes retain x’ if F(x’) is worse than F(x) for the escaping of local optima.

7. Terminate Conditions No further improvement
Bound of computational resource (time) is reached