Learning Market Prices for a Real-time Supply Chain Management Trading Agent David Burke Joint work with Ken Brown, Armagan Tarim and Brahim Hnich David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Outline of Talk  Introduction  Overview of TAC SCM  ‘Foreseer’ – An agent for TAC SCM  A real-time supply chain management pricing model  Pricing strategies for a real-time supply chain  Experiments  Conclusion David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Outline of Talk  Introduction  Overview of TAC SCM  ‘Foreseer’ – An agent for TAC SCM  A real-time supply chain management pricing model  Pricing strategies for a real-time supply chain  Experiments  Conclusion David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Introduction  Today's supply chains are very static and based on long- term relationships between trading partners.  Dynamic supply chains investigate the possibility of introducing more flexible and dynamic mechanisms to allow more efficient relationships between suppliers and customers that respond to changing market conditions. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Introduction  In a typical supply chain environment, the procurement costs and availability of raw material are likely to be changing over time.  The demand is likely to be uncertain, and thus changes over time.  As competitors enter or leave the market and improve their processes, the market price for finished goods will also change over time. An effective supply chain trading agent needs the ability to change prices in response to changing market and supply chain conditions. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Introduction  We propose a model for dynamic pricing that attempts to optimise expected profit by combining:  knowledge of: production capacity; available raw materials; existing customer commitments.  reasoning about uncertainty  learning of market conditions  We evaluate four pricing strategies integrated into a real-time supply chain management agent using the Trading Agent Competition Supply Chain Management game as a test framework. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006 We demonstrate the benefits of incorporating more market data into the dynamic pricing mechanism.

Outline of Talk  Introduction  Overview of TAC SCM  ‘Foreseer’ – An agent for TAC SCM  A real-time supply chain management pricing model  Pricing strategies for a real-time supply chain  Experiments  Conclusion David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Overview of TAC SCM David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Overview of TAC SCM  Three main decisions to be made at each time step:  What offers are made to customers?  What requests/orders are made to suppliers?  How is production scheduled? David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Outline of Talk  Introduction  Overview of TAC SCM  ‘Foreseer’ – An agent for TAC SCM  A real-time supply chain management pricing model  Pricing strategies for a real-time supply chain  Experiments  Conclusion David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Foreseer – An agent for TAC SCM  Scheduling production and delivery.  Production only carried out on confirmed orders.  Produced in order of ascending due date until all available production capacity and supplies are used up.  Orders not produced in the current schedule are provisionally scheduled for being produced the day before they are due to be delivered.  Orders are delivered when ready. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Foreseer – An agent for TAC SCM  Making orders to suppliers.  We can simplify decision on offers if we know price and availability of components, therefore, order components in advance.  We aim to maintain a certain buffer level which is based on our expected customer orders.  Most of supplies ordered with long due dates as cheaper prices can sometimes be got by ordering in advance.  Orders with shorter due dates are used to top up the inventory. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Outline of Talk  Introduction  Overview of TAC SCM  ‘Foreseer’ – An agent for TAC SCM  A real-time supply chain management pricing model  Pricing strategies for a real-time supply chain  Experiments  Conclusion David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

 Each day, agent receives a set of Request For Quotes (RFQs) from customers – each request contains a quantity of demand for a particular product, a reserve price, due date and penalty.  What requests do we bid on, and at what price, such that our profit is maximised?  If we bid too high, we don’t get many orders  If we bid too low, we get too many orders  Sometimes fewer orders at higher prices is better David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006 We define a mixed-integer linear programming model to optimise expected profits – executed in a “rolling horizon framework” A real-time supply chain management pricing model

 The key to the model is a matrix of prices , and a vector of probabilities p, such that  ik is the price for product i that we believe will be accepted with probability p k. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006 p1p1 p2p2 … pkpk … … …850 …………… product p1p1 p2p2 … pkpk …0.9 Example A real-time supply chain management pricing model

 The model also provides the possibility to apply a multiplier, o r, to the price offered for each request. This multiplier, if used, can adjust the prices based on specific request parameters and it use will be described in more detail later.  A decision variable bid rk indicates whether or not an offer is made for request r at the price with acceptance probability p k. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006 A real-time supply chain management pricing model

 Constraints:  Only one offer can be made for each request.  All offers made should meet the customers reserve price.  Schedule products for manufacturing, such that the closing inventory for any day is non-negative and production does not exceed production capacity (expected number of products required for each day, is the number that we bid on with that due- date multiplied by the probability that the bid is successful).  Supply constraints are enforced by ensuring that the closing component inventory for any day is non-negative (the expected component need can be calculated using the bill of materials). David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006 A real-time supply chain management pricing model

 Objective function:  The aim is to maximise the expected profit, where the profit on a product is calculated by subtracting the cost of components from the chosen selling price, and multiplying by the probability that the bid will be accepted. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006 A real-time supply chain management pricing model

Outline of Talk  Introduction  Overview of TAC SCM  ‘Foreseer’ – An agent for TAC SCM  A real-time supply chain management pricing model  Pricing strategies for a real-time supply chain  Experiments  Conclusion David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Pricing strategies for a real-time supply chain  In each time period the agent must determine what prices to offer for the requests. Thus the agent must specify the input to our model, setting appropriate values for the parameters:  available production capacity  component and product inventory  component arrivals  todays customer RFQs  product prices and probabilities David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Pricing strategies for a real-time supply chain  Strategy 1: FixedPrice  A fixed price for each product, set to 75% of base price, is defined and remains constant over the course of the game.  This value is based on observations that this price is generally competitive and provides good margins.  The probability of acceptance of this price is set to 1, as is each request multiplier. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Pricing strategies for a real-time supply chain  Strategy 1: FixedPrice  A fixed price for each product, set to 75% of nominal price, is defined and remains constant over the course of the game.  This value is based on observations that this price is generally competitive and provides good margins.  The probability of acceptance of this price is set to 1, as is each request multiplier. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, Not capable of reacting to changing market prices. - No allowance for cost of components.

Pricing strategies for a real-time supply chain  Strategy 2: FixedMargin  Sets the price of products to have a specific margin, arbitrarily set to 10%, added to the cost price.  Agent can dynamically adapt the price to reflect changing costs by updating the price in each time period.  Again, acceptance probability and request multipliers are set to 1. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Pricing strategies for a real-time supply chain  Strategy 2: FixedMargin  Sets the price of products to have a specific margin, arbitrarily set to 10%, added to the cost price.  Agent can dynamically adapt the price to reflect changing costs by updating the price in each time period.  Again, acceptance probability and request multipliers are set to 1. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, Agent may be undercut in competitive markets. - In high demand markets, agent may lose out on profit. - Assumes all offers are accepted which could leave factory capacity and supplies unused if this is not the case.

Pricing strategies for a real-time supply chain  Strategy 3: DynamicOnline  Maintain a set of prices which will be accepted by customers with a given probability.  We define a weight, w, and use that to calculate our price  price = base * w  We will try and learn a value for this weight for a given set of probabilities. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Pricing strategies for a real-time supply chain p target = 0.5// Target probability w = w default // Default Weight o// Number of offers made a// Number of offers accepted p actual = a/o// Current actual probability Δ = p actual – p target // Diff between actual & target w = w + (α * Δ) // Update the weight by // learning step size α price = base * w// Calculate new price David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Pricing strategies for a real-time supply chain  Strategy 3: DynamicOnline  Weights are learned for each product range and for 3 target probabilities, arbitrarily set to 0.3, 0.6 and 0.9 respectively.  Attempts to learn more detailed weights or a larger number of probabilities proved unsuccessful due to the limited amount of available information, I.e. limited number of customer requests to issue offers to. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Pricing strategies for a real-time supply chain  Strategy 3: DynamicOnline  Weights are learned for each product range and for 3 target probabilities, arbitrarily set to 0.3, 0.6 and 0.9 respectively.  Attempts to learn more detailed weights or a larger number of probabilities proved unsuccessful due to the limited amount of available information, I.e. limited number of customer requests to issue offers to. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, Lack of information about market hinders learning. - Price variations caused by factors such as different lead times, quantities, etc. are ignored.

Pricing strategies for a real-time supply chain  Strategy 4: DynamicHistoric  Complement the online learning with trends or patterns observed from historical data.  By analysing the impact that different parameters have on a product price, we can learn different multipliers that can be applied to the learned prices to tailor them for specific requests.  An analysis of 2005 TAC SCM seeding round was carried out and the effect that individual customer RFQ parameters (product type, quantity, due date, reserve price and penalty) had on sale price was examined. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Pricing strategies for a real-time supply chain  Each table shows the multiplier relative to the first column of the table. To tailor a price for a specific request, the base price of the product is multiplied by both the learned weight and the multipliers that apply to the request. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006 Leadtime Multiplier Quantity Multiplier Quantity Multiplier

Pricing strategies for a real-time supply chain David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006 Reserve Multiplier 75-85%85-95%95-105% % % Product (low) Multiplier Penalty Multiplier 5-8%8-12%12-15% Product (mid) Multiplier Product (high) Multiplier

Outline of Talk  Introduction  Overview of TAC SCM  ‘Foreseer’ – An agent for TAC SCM  A real-time supply chain management pricing model  Pricing strategies for a real-time supply chain  Experiments  Conclusion David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Experiments  We experiment using the TAC SCM framework against four agents which competed in the 2005 TAC SCM competition (downloaded from TAC agent repository).  We perform a set of pair-wise comparisons of our different strategies competing against those four agents.  We ran three sets of 25 games, comparing FixedPrice against FixedMargin, FixedMargin against DynamicOnline, and DynamicOnline against DynamicHistoric. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Experiments  (a) Shows how the average profit changes with each advance in the pricing model.  (b) Shows the average profits obtained in each pair-wise comparison. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Experiments  Revenue, market share and factory utilisation comparisons. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Experiments  Why does the incorporation of historic information not lead to increased average profits?  Analysis of the probability estimates of both models showed that DynamicHistoric has a 8% error rate compared to DynamicOnline’s 35%.  Further investigation reveals that both models are, on average, too cautious, and set prices that are too high to win the expected number of orders.  As DynamicOnline is more inaccurate, its prices are higher – in scenarios with high customer demand, it is able to win enough orders even with its high prices, and these high prices then translate to larger profits from fewer orders. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Experiments  DynamicHistoric is more successful than DynamicOnline in low demand scenarios making more profit on average. The opposite is the case for high demand scenarios. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Outline of Talk  Introduction  Overview of TAC SCM  ‘Foreseer’ – An agent for TAC SCM  A real-time supply chain management pricing model  Pricing strategies for a real-time supply chain  Experiments  Conclusion David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Conclusion  We have proposed an approach to the dynamic pricing problem in real-time supply chain management.  We represent the market conditions as probability distributions over the success rate of different bid prices, and these distributions can be applied without any knowledge of the number of customer or the number, identity or strategy of competitors.  We learn an approximation to these distributions using both online and historical information, and we update our approximation after each day of trading. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006

Conclusion  We show that the incorporation of up-to-date market information gives a significant increase in profits compared to strategies that do not consider market data.  We also show that the incorporation of historical information on the relevance of different parameters in the calls for tenders provides an increase in profits when the market is competitive and when the demand is relatively low. David A. Burke – TADA/AMEC 06, Hakodate – 9 th May, 2006