Appsheet: Crowdsourcing Decision Support Efficiently Alex Quinn, Tom Yeh, Ben Bederson

Ideal

Example Task: Help management choose a city for the new company headquarters. "Anna" (analyst)

Diagram based on Managerial Decision Modeling with Spreadsheets by Balakrishnan, Render, & Stair. © 2007 Pearson Education Interviews to determine problem structure, relevant attributes, weights Define the Problem Develop a Model Acquire Input Data Develop a Solution Test the Solution 1.climate 2.workers 3.affordability 4.health insurance costs 5.employee preference 6.rent 7.transportation 8.labor laws 9.sales tax 10.distance to other offices 11.distance to CEO hometown 12.availability of skilled labor 13.availability of office space … 50. employee satisfaction

Diagram based on Managerial Decision Modeling with Spreadsheets by Balakrishnan, Render, & Stair. © 2007 Pearson Education Define the Problem Develop a Model Acquire Input Data Develop a Solution Test the Solution Select method: Weighted Sums Model

Diagram based on Managerial Decision Modeling with Spreadsheets by Balakrishnan, Render, & Stair. © 2007 Pearson Education Define the Problem Develop a Model Acquire Input Data Develop a Solution Test the Solution Select method: Weighted Sums Model Enter formulas Simplified

Diagram based on Managerial Decision Modeling with Spreadsheets by Balakrishnan, Render, & Stair. © 2007 Pearson Education Define the Problem Develop a Model Acquire Input Data Develop a Solution Test the Solution Strategies: Recruit co-workers Mechanical Turk Acquire selectively Simplified

Diagram based on Managerial Decision Modeling with Spreadsheets by Balakrishnan, Render, & Stair. © 2007 Pearson Education Define the Problem Develop a Model Acquire Input Data Develop a Solution Test the Solution Strategies: Recruit co-workers Mechanical Turk Acquire selectively Simplified Appsheet

Appsheet: Getting started

Mockup

Appsheet: ASK formulas =ASK(1, 1, 10) cost minimum value maximum value Mockup

Appsheet: Starting Input Acquisition Mockup

Appsheet: Data collection Mockup

Appsheet: Savings Mockup

Appsheet: Savings Mockup Conclusive decision with only 5 of the 9 inputs Savings: 44% of the human effort

Prioritization Standard plan: A1, B1, C1, D1 cost minimum value maximum value

Prioritization Standard plan: A1, B1, C1, D1 Appsheet plan: B1, [A1 only if B1 ≤ 4], (C1 or D1) cost minimum value maximum value

Prioritization Standard plan: A1, B1, C1, D1 Appsheet plan: B1, [A1 only if B1 ≤ 4], (C1 or D1) Expected savings: 37% cost minimum value maximum value

Next steps Account for varying confidence in inputs Sharing of models themselves Construct web-based tools using spreadsheets

Take-home lessons Use crowdsourcing to gather data for decisions Human effort is expensive Leverage computation to reduce human effort Blend crowdsourcing with social computing Alex Quinn & Ben Bederson Contact:

Appsheet: Architecture

Diagram: © 2007 Pearson Education. Balakrishnan, Render, and Stair. Managerial Decision Modeling with Spreadsheets Interviews to determine problem structure, relevant attributes, weights 1.business income tax 2.climate 3.labor pool 4.health insurance costs 5.employee preference 6.rent 7.transportation 8.labor laws 9.sales tax 10.distance to other offices 11.distance to CEO hometown 12.availability of skilled labor 13.availability of office space … 50. employee satisfaction

Outline Overview Example Challenges Next steps

Ways to gather data Option 1: Use Mechanical Turk – … but they lack domain knowledge Option 2: Get help from co-workers. – … but their time is valuable Option 3: Gather a little information and narrow the field; then explore top contenders – … but this may lead to suboptimal result Option 4: Appsheet – Combine all three

Overview Problem: Gathering data is work-intensive Goal: Gather less data; get the same result. Strategy: Model-driven data collection with input prioritization for cost minimization Scope: Any data-centric spreadsheet decision model that uses formulas to calculate a result