The Multiplicative Weights Update Method Based on Arora, Hazan & Kale (2005) Mashor Housh Oded Cats Advanced simulation methods Prof. Rubinstein

Outline Weighted Majority Algorithm Binary case Generalized Applications Game Theory Zero-Sum game Linear Programming Fractional Packing problem NP-hard problems Set Cover problem Artificial intelligence (Boosting)

WMA – Binary case N experts give their predictions Our decision rule is a weighted majority of the expert predictions Initially, all experts have the same weight on our decision rule The update rule for incorrect experts is:

WMA – Binary case This procedure will yield in gains/losses that are roughly as good as those of the best of these experts Theorem 1 – The algorithm results in the following bound: Where: - the number of mistakes that expert I after t steps - the number of mistakes that our algorithm made

WMA Binary case – Proof of Theorem 1 I.By induction: II.Define the ‘potential function’: III.Each time we make a mistake, at least half of the total weight decrease by a factor of, so: IV.By induction: V.Using for

WMA – Binary case : Example1 4 analysts give their prediction to the stock exchange: 3 are always wrong and the 4 th is always right 4321 Market Day Expert

WMA – Binary case : Example1 (Cont.) 4321 Market /10.75/11.5/13/4 Balance of powers User Day Expert

WMA – Binary case : Example1 (Cont.) Since our fourth analyst is never wrong:

WMA – Binary case : Example2 100 analysts give their prediction to the stock exchange: 99 predict up with probability 0.05 the 100 th expert predicts up with probability 0.99 The market goes up at 99% of the time.

WMA – Binary case : Example2 (Cont.)

Generalization of the WMA Set of events/outcomes (P) is not bounded is the penalty that expert pays when the outcome is is the distribution associated with the experts The probability to choose an expert is: At every round we choose an expert according to D and follow his advice

Generalization of the WMA (Cont.) The update rule is: The expected penalty of the randomized algorithm is not much worse than that of the best expert Theorem 2 - The algorithm results in: Generalization of the WMA

WMA – Comparison via example The stock market example, using randomized expert instead of majority vote Market

WMA – Comparison via example (Cont.) With penalty only

WMA – Comparison via example (Cont.)

With penalty and reward

Generalization of the WMA - Example 4 weather man give their forecast There are four possible weather conditions The payoff matrix is: SunnyCloudyRainySnowy

Generalization of the WMA – Example (Cont.) The actual weather is sunny and cloudy alternately

Generalization of the WMA – Example (Cont.) The actual weather varies on the four possible weather conditions alternately

Applications Define the following components in order to draw analogy: Experts Events Payoff matrix Weights Update rule

Applications Game theory Zero-Sum games Experts – pure strategies to row player Events – pure strategies to column player Payoff matrix – the payoff to the row player, when the row player plays strategy and the column player plays strategy A distribution on the experts represents a mixed row strategy The game value is (von Neumann’s MinMax theory and Nash equilibrium):

Applications Game theory 1) Initialize. Determine. 2) Random a row strategy according to 3) The column player choose the strategy that maximizes his revenues 4) Update 5) If stop. Otherwise – return to step 2. Algorithm for solving Zero-Sum game:

Applications Game theory – Example /2 1/31/41 1/2 1/31/42 1/2 1/31/43 1/21/31/4 The row player choose minimum of maximum penalty The column player choose maximum of minimum penalty

Applications Game theory – Example1 (Cont.)

Applications Game theory – Example2 (1) The row player chooses a strategy randomly (2) The column player chooses the strategy that yield maximum benefits for him (3) Updating the weighting over row strategies 321 1/3 1/41 3/ /3 1/32/33 1/300

Applications Game theory – Example2 (Cont.)

Applications Artificial Intelligence The objective is to learn an unknown function A sequence of training examples is given: is the fixed unknown distribution on the domain The learning algorithm results in an hypothesis The error is:

Applications Artificial Intelligence (Cont.) Strong learning algorithm – for every and, with probability -weak learning algorithm – for every and, with probability Boosting – combining several moderately accurate rules-of-thumb into a singly highly accurate prediction rule.

Applications Artificial Intelligence (Cont.) Experts – samples in the training set Events – set of all hypotheses that can be generated by the weak learning algorithm Payoff matrix – The final hypothesis is obtained via majority vote among

Applications Linear Programming Finding a feasible solution for a set of m constraints Experts – constraints Events – solution vectors Payoff matrix - the distance from satisfying the constraint: The final solution is Track cases that there is no feasible solution

1) Initialize, and the resulting. 2) Given an oracle which solves the following feasibility problem with a single constraint plus a set of easy constraints (Plotkin, Shmoys and Tardos): Where: If there is no feasible solution – break. Applications Linear Programming (Cont.) Algorithm for finding a feasible solution to a LP problem:

Applications Linear Programming (Cont.) 3) Update Where: 4) Update 5) If stop. Otherwise – return to step 2. Algorithm for finding a feasible solution to a LP problem (Cont.):

Applications Linear Programming - Example Finding a feasible solution to the following problem: Solution:

Applications Fractional Vertex Covering problem Finding a feasible solution for a set of m constraints Experts – constraints Events – solution vectors Payoff matrix - The final solution is Track cases that there is no feasible solution

1) Initialize, and the resulting. 2) Given an oracle which solves the following feasibility problem with a single constraint plus a set of easy constraints (Plotkin, Shmoys and Tardos): Where: If there is no feasible solution – break. Applications Fractional Vertex Covering problem (Cont.) Algorithm for finding a feasible solution to a Fractional Covering problem:

Applications Fractional Vertex Covering problem (Cont.) 3) Update Where: 4) Update 5) If stop. Otherwise – return to step 2. Algorithm for finding a feasible solution to a Fractional Covering problem:

Applications Flow problems Maximum multi-commodity flow problem A set of source-sink pairs and capacity restrained edges

Applications Flow problems (Cont.) Experts – edges Events – a flow of value on the path, where is the minimum capacity of an edge. Payoff matrix: Update rule Terminate rule:

Applications Set Cover problem Find the minimal number of subsets in collection that their union equals the universe Experts – elements in the universe Events – sets in the collection Payoff matrix –

Applications Set Cover problem (Cont.) Update rule: We would search for the set which maximizes (Greedy Set Cover Algorithm)

Applications Vertex covering problem - Example

Applications Vertex covering problem – Example (Cont.) Find the minimum number of nodes (subsets) that cover all of the edges (universal) E set of edges 1,2,41,2,4 2,3,52,3,5 1,31,3 4,5,64,5,6 6

Applications Vertex covering problem – Example (Cont.) The maximum subset which includes edge is: node =1 The selected nodes are : c =1 Iteration: The probability for edge i p = Choose edge following the distribution i = 3 The maximum subset which includes edge is: node = 2 The selected nodes are : c =1 2 Iteration: The probability for edge i p = Choose edge following the distribution i = 6 The maximum subset which includes edge is: node = 4 The selected nodes are : c = 1 2 4

Applications Vertex covering problem – Example (Cont.) Iteration: The probability for edge i p = Choose edge following the distribution i = 6 The maximum subset which includes edge is: node = 5 The selected nodes are : c = 5 Iteration: The probability for edge i p = Choose edge following the distribution i = 3 The maximum subset which includes edge is: node = 2 The selected nodes are : c = 5 2 Iteration: The probability for edge i p = Choose edge following the distribution i = 1 The maximum subset which includes edge is: node = 1 The selected nodes are : c = 5 2 1

Summary This paper presents a comprehensive meta-analysis on the weights update method Various fields developed independently methods that has a common ground which can be generalized into one conceptual procedure The procedure includes the determination of experts, events, penalty matrix, weights and an update rule Additional relevant input: error size, size of and number of iterations.