Local Search

m = 3h - 8 linear fitness function – optimum at domain boundary

convex fitness function – optimum at one point in domain m = 20h - h 2

complex fitness function – local optima at points in domain m = h 3 mod 101 local optima

Local Search -ideal solution for convex functions -partial solution for complex functions (finds local optimum)

Local search algorithm localSearch(point P in domain) evaluate f P = fitness(P) repeat until no improvement in fitness transform P to a neighbouring point P’ evaluate f P ’ if f P ’ better than f P, P=P’ return P

1 Dimensional Convex space Example localSearch(point h=18) evaluate f h = 20(18)-18 2 =36 repeat until no improvement in fitness transform h to a neighbouring point h’ evaluate f h ’ if f h ’ better than f h, h=h’ return h = 10 m = 20h - h 2 in this domain, neighbouring points of h are (h+1), (h-1) hfhfh

2 Dimensional spaces 4 local search neighbours --- x  --- y 

Local search with SAT what is a “better” point in search space? what is a “neighbouring” point? localSearch(point P in domain) evaluate f P = fitness(P) repeat until no better fitness transform P to neighbour pt P’ evaluate f P ’ if f P ’ better than f P, P=P’ return P localSearch(point P in domain) evaluate f P = fitness(P) repeat until no better fitness transform P to neighbour pt P’ evaluate f P ’ if f P ’ better than f P, P=P’ return P P1P1 P2P2 P3P3 P4P4 (~P 1 \/ P 3 )(P 2 \/ ~P 3 )(P 4 \/ ~P 1 )fitness TFTFTFFF FITNESS: number of TRUE disjuncts 1 1 neighbour has one proposition with different boolean value

SAT representation Any logical expression can be written as a conjunction (/\) of disjunctions (\/) of propositions and negations (~P 1 \/ P 3 ) /\(P 2 \/ ~P 3 ) /\ (P 4 \/ ~P 1 ) (A \/ B) -> C = ~(A \/ B)\/ C = (~A /\ ~B) \/ C

Local search with SAT P1P1 P2P2 P3P3 P4P4 (~P 1 \/ P 3 )(P 2 \/ ~P 3 )(P 4 \/ ~P 1 )fitness TFTFTFF1 localSearch(point P in domain) evaluate f P = fitness(P) repeat until no better fitness transform P to neighbour pt P’ evaluate f P ’ if f P ’ better than f P, P=P’ return P localSearch(point P in domain) evaluate f P = fitness(P) repeat until no better fitness transform P to neighbour pt P’ evaluate f P ’ if f P ’ better than f P, P=P’ return P P1P1 P2P2 P3P3 P4P4 fitness FFTF2 P1P1 P2P2 P3P3 P4P4 TTTF2 P1P1 P2P2 P3P3 P4P4 TFFF1 P1P1 P2P2 P3P3 P4P4 TFTT2

Local search with SAT but SAT space is not convex… procedure GSAT // Michalewicz & Fogel begin for i = 1 to MAX_TRIES do T  random truth assignment for j = 1 to MAX_FLIPS do if T satisfies fitness return (T) flip a proposition truth value end return (“no solution found”) end

Local search with TSP what is a “better” point in search space? EASY – shorter path what is a “neighbouring” point? i.e., what change to transform a path to a similar path? MANY possible definitions of transformation operators. localSearch(point P in domain) evaluate f P = fitness(P) repeat until no better fitness transform P to neighbour pt P’ evaluate f P ’ if f P ’ better than f P, P=P’ return P localSearch(point P in domain) evaluate f P = fitness(P) repeat until no better fitness transform P to neighbour pt P’ evaluate f P ’ if f P ’ better than f P, P=P’ return P

TSP and local search  some neighbourhood definitions depend on the type of TSP e.g., 2-Opt: change two non-adjacent edges: ABCDEFG to ABEDCFG pathlength = pathlength –BC –EF + BE + CF A B C D E FG fails when? TSP with directed edges: DE  ED, CD  DC

Neighbourhood/Search tradeoff  larger neighbourhood  more chance of better optimum but  longer search time e.g. 2-Opt: n edges, n-3 non-adjacent number of neighbours n(n-2)/2 O(n 2 )

TSP neighbourhood definitions swap adjacent cities O(n) 2-Opt: change two non- adjacent edges O(n 2 ) 3-Opt: change among three non adjacent edges  -path (Lin-Kernighan) A B C D E FG A B C D E

3-OptO(n 3 ) THREE possible (2-Opt) neighbours: FOUR possible neighbours: BCDEFGHJKA B EDCHGF JKA FGH HGF CDE EDC B CDEHGF JKA EDC HGF FGH EDC

 -path (Lin-Kernighan) based on 2-Opt with 1 city (A) fixed. remove KA and consider all  -paths (e.g., by adding KE) -record best 2-Opt tour if any (e.g. EF FA) -BUT move to next tour according to best  -path length (e.g. including FA) BCDEFGHJKAA

Lin-Kernighan local search initial tour T, BestT = T start city A, last city L=K added_list =null; // added edges cannot be removed deleted_list=null; // deleted edges can’t be re-added repeat find best 2-Opt tour (may be change BestT) move to tour with best  -path < BestT if any put added edge in added_list (e.g., KE) put deleted edge in deleted_list(e.g., EF) L = F until no moves (all edges added or no short  -path) BCDEFGHJKAA

Variations on local search p, point in search space, fitness f(p) neighbourhood N p = {p 1,…,p k } (1) find first p i with f(p i )> f(p) or (2) find best p j such that f(p j ) ≥ f(p i ) 1≤ i ≤ k … pkpk p2p2 p1p1 f(p) (2) (1)