Heuristic Ordering for Asynchronous Backtracking Amnon Meisels Dept. of Computer Science Ben-Gurion University With Moshe Zazon, Roie Zivan

Retroactive Ordering – Udine 2010 Overview Distributed Constraints Networks Asynchronous BackTracking - ABT Dynamic ordering for ABT Asynchronous ordering heuristics Retroactive ordering Experimental evaluation 1/13/2019 Retroactive Ordering – Udine 2010

Constraints Satisfaction Problems - CSPs {5,6} {1,2,3, } X2 X1 X1 + X2 = 7 X4 < X2 X4 X3 X1 = X4 {a,b,c} {0,1} Variables – X1, X2, X3, X4 Values – {1, 2, 3} are the domain values of X1 Constraints – X3 is constrained with X2 but not with X4 1/13/2019 Meisels and Zivan – IJCAI-09 Tutorial

Meisels and Zivan – IJCAI-09 Tutorial CSPs - Solution {5,6} X2 = 6 X1 + X2 = 7 {1,2,3, } X4 < X2 X1 = 1 X4 = 1 X1 = X4 {0,1} Assignment of a value to each variable Value from the domain of the variable All constraints are satisfied 1/13/2019 Meisels and Zivan – IJCAI-09 Tutorial

Meisels and Zivan – IJCAI-09 Tutorial CSPs – no solution {4,5} {1,2,3} X2 X1 X1 + X2 = 7 X4 < X2 X4 X3 X2 = X3 X3 < X1 X1 = X4 {2,3,4} {0,4} Each constraint can be satisfied Only the value 4 satisfies C(X1,X4) Cannot be satisfied together with C(X2,X4) 1/13/2019 Meisels and Zivan – IJCAI-09 Tutorial

Search Algorithms Backtracking When looking back one can learn conflicting assignments and backtrack to the culprit variable Back Jumping (BJ); Conflict-based Backjumping (CBJ); Dynamic Backtracking (DBT) When looking ahead one can remove incompatible values from future (unassigned) variables or enforce consistency of the remaining variables Forward checking (FC); Directional arc-consistency; Maintaining Arc-Consistency (MAC) 1/13/2019 Meisels and Zivan – IJCAI-09 Tutorial

A Distributed Constraint Network 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 A “realistic” example Three people want to meet: Avi, Beni, and Chen There are three meetings to set: (Beni, Avi) , (Avi, Chen), and (Beni, Chen) at different locations m3 Beni m1 Local Constraint m1 Inter Constraint m3 m2 m2 Avi Chen 1/13/2019 Retroactive Ordering – Udine 2010

Distributed Constraints – the Model Every Agent owns part of the constraints network Agents have no direct access to variables of other Agents To coordinate their value assignments for a solution Agents exchange messages Messages are asynchronous Any search algorithm is a distributed algorithm (since the problem is distributed) 1/13/2019 Retroactive Ordering – Udine 2010

Sequential (synchronous) BT An-1 An [V1=2],[V2=1],[V3=?] Current Partial Assignment (CPA) 1/13/2019 Retroactive Ordering – Udine 2010

Asynchronous Backtracking Motivation: Concurrent computation HighLights: Agents always hold an assignment which is consistent with their view of the system When there is no consistent assignment with an Agent’s view, a backtrack operation is performed On backtrack operations, agents change their views by eliminating inconsistent assignments 1/13/2019 Retroactive Ordering – Udine 2010

Asynchronous BackTracking - ABT Agents are totally ordered Each agent assigns its variable and sends it in ok? messages to agents ordered after it All received assignments are stored in the AgentView Agents check their assignments against their AgentViews An Agent that has no consistent assignment sends a Nogood message to the relevant agent before it Nogood messages cause replacement of assignment, as long as they are consistent with the receiver’s AgentView Yokoo 1998, 2000 1/13/2019 Retroactive Ordering – Udine 2010

Asynchronous Backtrack (ABT) {1,2} A1 A2 {2} ≤ ≠ ≠ [A1=1] A3 {1,2} 1/13/2019 Retroactive Ordering – Udine 2010

Asynchronous Backtrack (ABT) {1,2} A1 A2 {2} ≤ ≠ ≠ [A2=2] A3 {1,2} 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Backtrack in ABT AgentView of Agent A3: A1=1, A2=2 Nogood Current Assignment A2 A3=2 A1=1, A2=2 1/13/2019 Retroactive Ordering – Udine 2010

Asynchronous Backtrack (ABT) {1,2} A1 A2 {2} ≤ ≠ ≠ A3 [A1=1], [A2=2] {1,2} 1/13/2019 Retroactive Ordering – Udine 2010

Cycle: 1 2 Active: 2,3,4 1,2,3,4 Nogood message: A1=1 ^ A2=1 ⇒ A3≠1 OK? A2 OK? Active: A3 2,3,4 1,2,3,4 A4 OK? Nogood message: A1=1 ^ A2=1 ⇒ A3≠1 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Cycle: A1 3 A2 A3 Active: 3 A4 Nogood message: A1=1 ⇒ A2≠3 1/13/2019 Retroactive Ordering – Udine 2010

Cycle: 5 4 Active: 3,4 2 Nogood message: A1=1 ^ A2=4 ⇒A3≠4 1/13/2019 OK? A3 Active: OK? 3,4 2 A4 Nogood message: A1=1 ^ A2=4 ⇒A3≠4 1/13/2019 Retroactive Ordering – Udine 2010

Cycle: 6 Active: 4 Nogood message: A1=1 ^ A2=4 ⇒A3≠2 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Cycle: A1 OK? 8 9 7 A2 A3 Active: 3 2 1 A4 Nogood message: A1=1 ⇒ A2≠4 Nogood message: ⇒ A1≠1 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Cycle: A1 10 11 A2 A3 Active: 3 4 OK? A4 Done! 1/13/2019 Retroactive Ordering – Udine 2010

ABT – Strong Assumption All Agents are ordered by a fixed order of Priorities [Yokoo 92-2000] [BesBrMes 2001 – 2005] [Hammadi 98 – 2002] [Gomez et. al 2002 – 2005] 1/13/2019 Retroactive Ordering – Udine 2010

Motivation for dynamic reordering Sequential assigning algorithms for DisCSPs: Nguyen & Faltings 2004 Brito & Meseguer 2004 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Why is it a problem? [1] [1] V1 ≠ V2 V1 ≠ V2 A2 A1 V1 = 1 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Why is it a problem? A2 A1 V1 = 1 [1] [1] V1 ≠ V2 V1 ≠ V2 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Why is it a problem? [1] [1] V1 ≠ V2 V1 ≠ V2 A1 A2 V1 = 1 V2 = 1 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Why is it a problem? A2 V2 = 1 A1 [1] [1] V1 ≠ V2 V1 ≠ V2 V1 = 1 1/13/2019 Retroactive Ordering – Udine 2010

Dynamic ordering can be a problem [1] [1] V1 ≠ V2 V1 ≠ V2 V2 = 1 V1 = 1 1/13/2019 Retroactive Ordering – Udine 2010

Reordering – in standard backtracking 1 2 3 4 1/13/2019 Retroactive Ordering – Udine 2010

Reordering – sequential assignments algorithms 1 2 3 4 4 3 1/13/2019 Retroactive Ordering – Udine 2010

Reordering – sequential assignments algorithms 1 2 4 3 4 2 4 3 3 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Reordering – standard backtracking Highlights: Moving forward - choose a desirable order Backtracking is done in the same order as the forward moves Same idea can be applied in Asynchronous Backtracking 1/13/2019 Retroactive Ordering – Udine 2010

Dynamic Ordering for ABT Motivation: A general algorithm for dynamic ordering (decoupled from a specific heuristic) Apply known effective heuristics from (centralized) CSP Use polynomial space 1/13/2019 Retroactive Ordering – Udine 2010

Ordering Rules for Agents in ABT Change order only when changing assignment Enforce reordering only on Agents with lower priority Maintain a list of pairs (agent_id, counter) Time-stamps Time-stamps are lists of n counters (one for each agent) During assignment Ai increments its counter and sets to 0 counters of agents with lower priority Zivan & Meisels, 2005 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Agents change order A1 A2 A3 A4 A5 Current Order (time-stamp): A2: A3: (A1,1), (A2,2), (A3,4), (A5,0), (A4,0) (A1,1), (A2,3), (A5,0), (A4,0), (A3,0) (A1,1), (A2,2), (A3,3), (A4,1), (A5,0) 1/13/2019 Retroactive Ordering – Udine 2010

“Time-stamping” Agents’ orderings Changed time-stamp: (A1,1), (A2,3), (A5,0), (A4,0), (A3,0) 13000 > 12400 1/13/2019 Retroactive Ordering – Udine 2010

ABT with Dynamic Ordering [A1,2],[A2,1], [A4,1] A1,A3,A2,A4 A3 ≠ 1 A3 = 1 1/13/2019 Retroactive Ordering – Udine 2010

ABT with Dynamic Ordering [A1,2],[A2,1], [A4,1] A1,A3,A2,A4 A1,A3,A4,A2 A3 = 2 A1,A3,A4,A2 A3 = 2 1/13/2019 Retroactive Ordering – Udine 2010

Another example of Dynamic Ordering [A1,1],[A3,1],[A2,3],[A4,2] A3 ≠ 1 A3 = 1 1/13/2019 Retroactive Ordering – Udine 2010

Another example of Dynamic Ordering (II) [A1,1],[A3,2],[A4,0],[A2,0] [A1,1],[A3,1],[A2,3],[A4,2] [A1,1],[A3,1],[A2,3],[A4,2] [A1,1],[A3,1],[A4,2],[A2,3] [A1,1],[A3,1],[A2,3],[A4,2] A3 = 2 1/13/2019 Retroactive Ordering – Udine 2010

ABT_DO – Exploring Heuristics Random: An agent that changes its value randomly orders lower priority agents Min-Domain: based on current domain sizes (good for centralized CSPs) Nogood Triggered: the Nogood sender is positioned right after the Nogood receiver 1/13/2019 Retroactive Ordering – Udine 2010

Minimal Domain Ordering Reordering agent conflicting values A1 A2 A3 A4 unchecked values assigned value 1/13/2019 Retroactive Ordering – Udine 2010

Minimal Domain Ordering Reordering agent A1 A4 A3 A2 1/13/2019 Retroactive Ordering – Udine 2010

Nogood Triggered Ordering Ordering agent A1 A2 A3 A4 A5 1 1 [A1,1] 1 [A1,1] 1 1 [A1,1] [A3,2] [A1,1] 2 2 2 [A1,1] 2 2 [A3,2] 2 [A2,2] 3 3 3 3 3 3 3 Nogood message: A1=1 ⇒ A2≠2 1/13/2019 Retroactive Ordering – Udine 2010

Nogood Triggered Ordering Ordering agent A1 A2 A5 A3 A4 1 1 [A1,1] 1 [A1,1] 1 [A1,1] 1 2 2 [A1,1] 2 [A1,1] 2 2 3 3 3 3 3 3 1/13/2019 Retroactive Ordering – Udine 2010

Experimental evaluation Random Problems n – number of agents k – domain size p1 –density: percentage of constrained variables p2 – tightness: percentage of conflicting values Performance Measures: NCCCs: number of non-concurrent constraint checks implementation-independent run-time Network Load: total number of messages 1/13/2019 Retroactive Ordering – Udine 2010

Experimental results (2005/6) 1/13/2019 Retroactive Ordering – Udine 2010

Failure of min-domain heuristic Agents remove from their current domains only values that conflict with assignments of higher priority agents When an agent is moved to a higher position, it returns to its current domain values that are no longer conflicting The domain sizes after reordering cannot be anticipated by the ordering agent 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Asynchronous Min-Domain Ordering agent A1 A2 A3 A4 A5 1 1 [A1,1] 1 [A1,1] 1 [A3,2] 1 [A1,1] 2 2 2 [A1,1] 2 2 [A3,2] 2 [A1,1] 3 3 3 3 3 3 3 [A2,2] Nogood message: A1=1 ⇒ A2≠2 1/13/2019 Retroactive Ordering – Udine 2010

Asynchronous Min-Domain swap! Asynchronous Min-Domain Ordering agent A1 A2 A5 A4 A3 1 1 [A1,1] 1 [A1,1] 1 1 [A1,1] 2 2 [A1,1] 2 [A1,1] 2 2 3 3 3 3 3 3 1/13/2019 Retroactive Ordering – Udine 2010

Improved Min-domain ordering Keeping the Min-Domain property Triggered by Nogoods The Nogood generator triggers the move and can check its relative domain size “on its way up” The Nogood generator can be moved to any position in the new order, but only before agents with larger domain sizes 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive ordering for ABT If the Nogood generator is moved to a position which is before the second last in the Nogood, all the agents included in the Nogood must keep the Nogood A predefined Nogood size limit k, defines the ordering flexibility – unlimited moves up are allowed only below k Enabling “deep” moves back (“retroactive”) Keeping Nogoods in memory implies a cost in checking against the kept Nogoods Is it worthwhile ? 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Nogood size limit: 2 A1 A2 A3 A4 A5 1 1 1 1 [A3,1] 1 [A1,1] 2 2 2 2 2 2 [A1,1] 3 3 3 3 3 [A2,1] 4 4 4 4 4 4 [A4,2] [A1,1],[A2,1] Current order: (A1,1), (A2,2), (A3,4), (A4,0), (A5,0) Generated nogood: A1=1 ^ A2=1 ⇒ A4≠2 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Nogood size limit: 2 A1 A2 A5 A3 A4 1 1 1 [A1,1] 1 1 [A3,1] 2 2 2 [A1,1] 2 2 2 3 3 3 [A2,1] 3 3 4 4 4 4 4 [A1,1],[A2,1] New order: (A1,1), (A2,2), (A5,5), (A3,0), (A4,0) 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Nogood messages Nogood size limit: 3 A1 A2 A3 A4 A5 1 1 1 1 [A3,1] 1 [A1,1] 2 2 2 2 2 2 [A1,1] 3 3 3 3 3 [A2,1] 4 4 4 4 4 4 [A4,2] [A1,1],[A2,1] Generated nogood: A1=1, A2=1, A4=2 Current order: (A1,1), (A2,2), (A3,4), (A4,0), (A5,0) 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Nogood size limit: 3 A1 A5 A2 A3 A4 1 1 [A1,1] 1 1 1 [A1,1] 2 2 [A1,1] 2 2 2 2 3 3 3 3 3 4 4 4 4 4 [A1,1],[A2,1] New order: (A1,1), (A5,3), (A2,0), (A3,0), (A4,0) 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Conclusions Moving agents to the highest position possible was found to deteriorate the performance of the algorithm The best heuristic moves to a higher position only agents whose domains are small Avoids the return of values to domains as a result of asynchronous reordering 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Thank You 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 One variable per agent The choice of most studies in the field Simplifies the problem Two major justifications: Single-variable agents can be perceived as virtual agents Multiple-variable assignments (to a local constraint network) can be viewed as a value for a single complex variable 1/13/2019 Retroactive Ordering – Udine 2010

Virtual agents in a local network [0,1,2,3] V6 A6 V4 [0,1] A4 [0,1,2] Agent1 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Single local variable V1 [0,1,2,3] [V1=0,V2=0,V3=0 V1=1,V2=0,V3=0 V1=0,V2=1,V3=0 ………………….. ..………………….] V6 Var1 V4 [0,1] [0,1,2] Agent1 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Pros and Cons of ABT Advantages: Concurrent Computation Backjumping Forward-checking Drawbacks: Computation is done against inconsistent partial assignments Order is static - high priority agents are idle 1/13/2019 Retroactive Ordering – Udine 2010

ABT – Improvements & Versions Read all messages in the mailbox and respond accordingly Resolve Nogoods, to send back shorter inconsistent partial assignments Use dynamic backtracking techniques, to keep only a polynomial number of Nogoods Performance improvements can be substantial Yokoo 2000 , Bessiere & Meseguer 2001, 2005 1/13/2019 Retroactive Ordering – Udine 2010

Which order is most up-to-date? Time-Stamping method of Nguyen & Faltings 2004: Each order is time-stamped A time-stamp is an array of integers of size n, all initialized to 0 Each agent that assigns its variable, updates the time-stamp as follows: Counters of higher priority agents are untouched The counter of the assigning agent is incremented by 1 The counters of lower priority agents are set to zero Time-stamps are compared lexicographically 1/13/2019 Retroactive Ordering – Udine 2010

Choosing the right heuristic Heuristics that work well for sequential assignment algorithms fail for ABT_DO (min-domain) After each change of order, relevant Nogoods are discarded Nogood-triggered heuristic [Ginsberg-93] Agent changes its current order only when it receives a Nogood which causes an assignment change The Nogood sender is moved to a place immediately following that of the assigning agent The only agent that moves up has no conflicts with newly formed lower priority agents  few Nogoods are removed 1/13/2019 Retroactive Ordering – Udine 2010

Yokoo’s comparison ABT vs SBT 1/13/2019 Retroactive Ordering – Udine 2010

Asynchronous Weak Commitment - AWC Motivation: Re-exmination of the queen problem shows that there is no solution as long as the first queen does not change its assignment. A large search space is therefore explored for vain with ABT. HighLights: Agents always hold an assignment which is consistent with their view on higher position agents When agent has no consistent assignment with its view a backtrack operation is performed On backtrack operations, agents change their position in the order. 1/13/2019 Retroactive Ordering – Udine 2010

Asynchronous Weak Commitment -AWC Each agent holds non-negative value which represents its priority order. Agent sends its order priority and current value to its neighbors through ok? messages. Priority value is stored in the AgentView When agent cannot find a consistent value, it sends Nogoods to other agents and increment its priority value. If the same Nogood was already sent the agent just waits for incoming messages Yokoo 2000 1/13/2019 Retroactive Ordering – Udine 2010

Cycle: 1 2 Active: 1,2,3,4 4 (0) (0) (0) (0) (1) Nogood message: A1=1, A2=3, A3=2 1/13/2019 Retroactive Ordering – Udine 2010

Cycle: 3 2 Active: 4 3 (0) (0) (0) (2) (1) Nogood message: A4=3, A1=1 , A2=4 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Cycle: A1 (0) (0) 3 4 A2 (0) A3 Active: (0) (2) 1 3 A4 (1) Done! 1/13/2019 Retroactive Ordering – Udine 2010

Yokoo’s comparison ABT vs AWC 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Pros and Cons of AWC Advantages: Concurrent Computation Forward-checking Ordering is not static Computation is shared more equally by agents Drawbacks: Agents need to store and maintain an exponential number of Nogoods. The algorithm is coupled with specific ordering and value selection. The algorithm performs well for sparse problems but shows poor results for hard instances. 1/13/2019 Retroactive Ordering – Udine 2010

Why does AWC perform well on sparse problems ? Search Space Non-Solution Space Solution Space 1/13/2019 Retroactive Ordering – Udine 2010

Retroactive Ordering – Udine 2010 Why does AWC perform well on sparse problems ? Search Space Solution Space Non-Solution Space 1/13/2019 Retroactive Ordering – Udine 2010

Cycle: 2 1 Active: 1,2,3,4 3,4 (0) (0) (0) (0) (0) (1) Nogood message: A1=1, A2=4, A3=2 1/13/2019 Retroactive Ordering – Udine 2010