Online Procurement Auctions for Resource Pooling in Client-Assisted Cloud Storage Systems Jian Zhao, Xiaowen Chu, Hai Liu, Yiu-Wing Leung Department of Computer Science Hong Kong Baptist University Zongpeng Li University of Calgary IEEE INFOCOM 2015, HONG KONG

Outline Motivation: why client-assisted? Background: auctions Model formulation Some technical details (without equations) Conclusions IEEE INFOCOM 2015, HONG KONG

Cloud Storage Services IEEE INFOCOM 2015, HONG KONG

Most of the time, CSPs are IEEE INFOCOM 2015, HONG KONG Durability: Our data will be there without error forever. Availability: Data can be accessed anywhere, anytime, from any device.

Cloud Outages! But occasionally, they become IEEE INFOCOM 2015, HONG KONG

Cloud Outages Cloud services could become unavailable because of IEEE INFOCOM 2015, HONG KONG

Evidence of Cloud Outages “The worst cloud outages of 201X “ by J. R. Raphael Inforworld.com has been tracing cloud outages since 2011 Amazon Google Microsoft Apple Dropbox Facebook Adobe … Big names on the list IEEE INFOCOM 2015, HONG KONG

Plan A: Cloud Federation What can we do? Plan A: Cloud Federation IEEE INFOCOM 2015, HONG KONG

Plan B: Client-Assisted Cloud Storage Alternatively Plan B: Client-Assisted Cloud Storage IEEE INFOCOM 2015, HONG KONG

Client-Assisted Examples in Academia Leverage peer bandwidth to mitigate server bandwidth cost Improve availability and downloading performance FS2You Data is kept at a central cloud and replicated among distributed peers AmazingStore Peer-assisted architecture with a focus on data consistency Triton IEEE INFOCOM 2015, HONG KONG

A Counter Example: IEEE INFOCOM 2015, HONG KONG Wuala was designed to be “client-assisted”. Servers in datacenters Client users casual peers without contribution storage peers trading local storage for increased storage space Wuala abandoned the “client-assisted” design in 2012 Hybrid architecture is very complicated Bandwidth cost is dropping Contribution from peers are marginal More focus on business customers IEEE INFOCOM 2015, HONG KONG

Another Story: Symform Acquired by Quantum in 2014 Observation “cloud providers use extremely inefficient centralized infrastructure to store stuff.” “most users had tons of excess local storage just going to waste” Goal “Creating the World’s Largest Datacenter” “Without Building a Datacenter” Method “Users that contribute get 1 GB free for every 2 GB contributed” IEEE INFOCOM 2015, HONG KONG

Incentive makes a difference Total upload = Total download Public BitTorrent Tit-for-Tat only Free-rider problem Stop seeding after downloading Limit the uploading bandwidth Private BitTorrent (or Darknet) Sharing Ratio Enforcement Users fight to contribute as much as they can to survive Everyone in the community can get high downloading performance People are willing to pay to get into the community IEEE INFOCOM 2015, HONG KONG

Auctions Auctions are economical approaches for allocating resources or trading commodities Participants: auctioneer + bidders Auction mechanism: a set of institutions for the buying/selling of goods or services, such as allocation rules and pricing rules IEEE INFOCOM 2015, HONG KONG

Auctions as Incentive No need to predict users’ demand: users reveal their true information through bids Balance among supply and demand Allocate available resources efficiently Obtain higher revenue IEEE INFOCOM 2015, HONG KONG

How to design the auction mechanism for resource pooling from clients? Research Problem How to design the auction mechanism for resource pooling from clients? For clients: get reasonable monetary return by selling resources (storage space & bandwidth) For CSPs: save cost (and overcome cloud outages) Assumption: the cost of providing service by datacenter herself is expensive than procuring resources from clients IEEE INFOCOM 2015, HONG KONG

Our Approach: Online Procurement Auctions Online auctions Different bidders arrive at different times Auctioneer makes decision about each bid as it is received Different from the traditional case that the auctioneer receives all the bids before determining the allocation In line with asynchronous arrivals of user bids and requests Procurement auctions Ordinary auctions (or forward auctions): one seller, multiple buyers Buyers compete Procurement auctions: one buyer, multiple sellers Sellers compete reverse auctions IEEE INFOCOM 2015, HONG KONG

Our Auction Model (I) Storage & bandwidth are unified as conceptual “resource” Needs further study Clients (i.e., bidders) want to sell resources through bids Bid: (starting time, ending time, amount, unit price) Valuation: the “true” value of the resource, private to the bidder Utility = the received payment - the cost of offering the sold resource Target: maximize utility IEEE INFOCOM 2015, HONG KONG

Our Auction Model (II) CSP (i.e., auctioneer ) wants to buy resources from some clients Set a target S for time period [0, T] For each incoming bid, determine how much to procure (allocation rule) how much to pay (payment rule) IEEE INFOCOM 2015, HONG KONG

Design Objectives Truthful Competitive If ( truthful bidding always maximizes utility ) Then ( rational bidders will report their true valuations ) Truthful The CSP’s total cost is bounded by γ times of the total cost of offline optimal auction Competitive IEEE INFOCOM 2015, HONG KONG

Our Results Truthfulness Competitiveness We derive price-based allocation rule and payment rule that result in truthful auction mechanism. Competitiveness We find a solution to guarantee a target competitive ratio against offline optimal auctions. IEEE INFOCOM 2015, HONG KONG

Our Methodology Start from Myerson’s Principles of Truthfulness (1981) Gives two conditions to satisfy the truthfulness property For one single indivisible good Extend to the case of online procurement auction for divisible goods We first study the conditions for truthfulness of an online auction We then derive allocation and payment equations such that the auction mechanism is truthful. Design marginal pricing function for CSP to achieve a desired competitive ratio IEEE INFOCOM 2015, HONG KONG

Conclusions We consider “client-assisted” a promising approach to addressing cloud outages. We argue that “auctions” can be a good incentive mechanism. We propose an online procurement auction mechanism, and prove its truthfulness and competitiveness. IEEE INFOCOM 2015, HONG KONG

A and IEEE INFOCOM 2015, HONG KONG

Myerson’s Principle of Truthfulness Consider the auction of a single indivisible good The auction mechanism is truthful if and only if The probability of a bidder winning an auction is monotonically non-decreasing in its bid The payment charged to a bidder is independent of its bid IEEE INFOCOM 2015, HONG KONG

Extension to Online Procurement Auctions We first define “allocation monotonicity” for online procurement auctions. Better bids get more allocation If allocation rule is monotone, we can find a payment rule that results in truthful online procurement auction. We design a monotone allocation rule and the corresponding payment rule. IEEE INFOCOM 2015, HONG KONG

Competitive Analysis Our auction mechanism assumes a non-increasing marginal pricing function for the CSP to procure resources from clients. The marginal pricing function is a variable How to minimize the total cost by adjusting the marginal pricing function? An online algorithm design problem We find a solution of setting the marginal pricing function to achieve a target competitive ratio. IEEE INFOCOM 2015, HONG KONG