1 Cooperative multi-agent systems In cooperative MAS agents strive to reach a common goal and increase the combined utility of their actions Limitations on computational power and bandwidth prohibit the global view of the task structure Exchange of complete local views between agents is prohibitive Need to find a trade-off between the amount of information exchanged and the negotiation’s contribution to increasing combined utility

2 Contribution The proposed negotiation protocol: –Empirically shown to approach the utility of distributed agents’ actions very close to that of their aggregation –Multi-parameter –Uses marginal utility gain and cost to drive the negotiation

3 Assumptions Agents possess local views of the task structures that define their functionality Agents have access to a scheduler that takes a task structure and utility function as input and produces a set of schedules that increase the utility function locally

4 Motivating example: task allocation The agent needs a certain non-local task to be performed by some other agent. The negotiation’s goal is to increase the combined utility of actions of both agents by choosing a certain quality and time range of the non-local task.

5 Vocabulary 1 Contractor – agent that has a non-local task NL that needs to be assigned to another agent Contractee – agent that performs this task for the contractor Utility – weighted sum of quality, cost, and duration of a schedule

6 Vocabulary 2 Marginal utility gain (MUG) – local utility increment for the contractor with NL performed Marginal utility cost (MUC) – local utility decrement for the contractee with NL performed

7 Motivating example: task structures TCR Task1Task2 M1M2M4M5M3 sum q:10 c:10 d:9 q:10 c:10 d:9 q:10 c:10 d:9 q:15 c:0 d:0 q:10 c:10 d:9 deadline:50 enables TCE Task1Task2 B1B2B3B4 sum q:10 c:10 d:9 q:10 c:10 d:9 q:10 c:10 d:9 q:10 c:10 d:9 deadline:47 enables deadline:11deadline:21

8 Contractor’s FSM buildProposal always sndMsgProposal Wait EvalCounterProposal Finish NewProposal rcvMsgCounterProposal evalCounterProposal Reject sndMsgReject Accept&GetNext Accept&HaveEnough sndMsgFinish always generateNewProposal sndMsgProposal rcvMsgAccept& HaveEnough sndMsgFinish rcvMsgAccept& GetNext Start

9 Contractee’s FSM Start EvalProposal Wait Finish rcvMsgProposal evalProposal Accept sndMsgAccept rcvMsgFinish Reject buildCounterProposal sndMsgCounterProposal rcvMsgReject

10 Binary search approach Each next proposal is the midpoint between the contractor’s current proposal and contractee’s current counter proposal. Disadvantages: Finish time of first counter proposal may not be the actual reasonable finish time Negotiation is about multiple issues (time range, quality) while the binary search focuses only on finish time Binary search requires greater certainty about NL’s time range Both agents are likely to cling to their own proposals, greater the likelihood that some solutions are missed.

11 Proposal/counterproposal information EST - Earliest start time of the non-local task NL FT - Finish time minq - Requested quality MUC - Marginal utility cost MUG - Marginal utility gain

12 Protocol functions overview buildProposal: –Creates the initial contractor’s proposal including the time range and quality of NL and MUG –Insists on highest local utility by executing NL according to the contractor’s best local schedule

13 Protocol functions overview generateCounterProposal: –Creates the contractee’s counter proposals including the time range and quality of NL and MUC –Initially insists on highest local utility by executing NL according to the contractee’s best local schedule

14 Protocol functions overview generateCounterProposal: –Tries to increase combined utility by lowering MUC –Successive counter proposals relax the time range and quality of NL until a schedule is found s.t. MUG>MUC

15 Protocol functions overview generateNewProposal: –Creates the next contractor’s proposal based on the preceding proposal or counter proposal

16 Protocol functions overview generateNewProposal: –Tries to increase combined utility by doing multi-parameter heuristic-based search in quality and time range space

17 Protocol functions: buildProposal buildProposal: –produces the initially requested earliest start time and finish time of the non-local task according to the contractor’s best schedule –has limited information about the NL’s quality, cost, and duration –the greater the time range of NL’s execution is the closer to the high bound the requested quality is

18 Protocol functions: generateCounterProposal If no previous Counter Proposal exists: –produces a “what-if” local contractee’s schedule ignoring the time range and quality of NL in the current proposal, but enforcing NL to be in the schedule, thus delivering the minimum marginal utility cost for the NL If there is a previous Counter Proposal : –Alternatively relaxes the NL’s time range and lowers the requested quality from the current proposal until an acceptable schedule is found (MUG>MUC)

19 Protocol functions: generateNewProposal 1 If Previous Proposal is acceptable for the contractee: –If the NL is out of initial time range and its quality below average then increase the quality request and make later the deadline for the NL –If the NL is in initial time range and its quality is above average then reduce quality request in the same time range thus trying to lower MUC –Otherwise shift time range later and reduce quality request thus trying to lower MUC

20 Protocol functions: generateNewProposal 2 If Previous Proposal is not acceptable for the contractee: –If this is first counter proposal: If the initial time range is too short then the deadline is moved later and lower quality is requested Otherwise make the initial range a bit longer than the current estimate of NL’s execution time and request higher than average quality

21 Protocol functions: generateNewProposal 3 –If this is not first counter proposal Assume a previous counter proposal from the contractee as current proposal If requested quality above average then shift the time range later If requested quality below average then request higher quality and shift the NL’s deadline later

22 Modified task structure TCE Task1Task2 B1B2B3B4 sum q:10 c:10 d:9 q:10 c:10 d:9 q:15 c:0 d:0 q:10 c:10 d:9 deadline:47 enables deadline:11deadline:21 New_TCE M4 M41 M42M43 exactly_one sum TCR Task1Task2 M1M2M4M5M3 sum q:10 c:10 d:9 q:10 c:10 d:9 q:10 c:10 d:9 q:15 c:0 d:0 q:10 c:10 d:9 deadline:50 enables

23 Protocol variants Single step: the contractor sends a proposal PC to the contractee which accepts it if MUG(PC)>MUC(PC) or rejects it otherwise Multi-step-Multiple-Try: the agents perform the negotiation process until a predefined number of acceptable solutions are found or iteration limits are exceeded Multi-step-Limited-Effort: the agents perform the negotiation process until iteration limits are exceeded

24 Experiment task structures TCR Task1Task2 M1M2M4M5M3 sum q:10 c:10 d:9 q:10 c:10 d:9 q:15 c:0 d:0 q:10 c:10 d:9 deadline:50 enables q:10 c:10 d:9 enables est:10est:24 TCE Task1Task2 B1B2B3B4 sum q:10 c:10 d:9 q:10 c:10 d:9 q:15 c:0 d:0 q:10 c:10 d:9 deadline:47 enables deadline:11deadline:21 enables

25 Utility function S – schedule Utility(S) = quality_gain(S)*quality_weight + cost_gain(S)*cost_weight + duration_gain(S)*duration_weight

26 Collected data Negotiation outcome: success if the combined utility is increased Utility gain: MUG - MUC Gain percentage: percentage of utility gain relative to the combined utilities without the task allocation Solution quality: utility percentage relative to the utility of the combined tasks Complexity of task structures: function of the number of interrelationships and temporal constraints in the task structures Number of negotiation steps (proposal/counterproposal messages).

27 Protocol comparison SuccessAverage Gain Percentage ANNSGain per Step Solution Quality Single-step Multi-step- One-Try Multi-step- Two-Try Multi-step- Three-Try Multi-step- Limited Effort

28 Complexity ir1 – number of interrelationships in contractor’s task structure tr1 – number of temporal constraints in the contractor’s task structure ir2 – number of interrelationships in contractee’s task structure tr2 – number of temporal constraints in the contractee’s task structure Complexity of task structure = ir1 + tc1 + ir2 + tc2 + (ir1*tc1 + ir2*tc2 + ir1*ir2 + tc1*tc2 + ir1*tc2 + ir2*tc1)/6

29 Dependence of SQ on complexity

30 Net negotiation gain Assume that one negotiation step costs 0.5% of the overall utility of the negotiating agent’s schedules Net negotiation utility gain = negotiation utility gain – negotiation effort

31 Dependence of net gain on complexity

32 Observations The Multi-step-One-Try protocol is much better than Single step in almost all situations(except for very simple ones) The Multi-step-Two-Try and Multi-step-Three- Try protocols are useful in moderately constrained situations The number of constraints can be used to choose a protocol that balances negotiation gain and effort for a certain situation