1. Appsheet: Crowdsourcing Decision Support Efficiently Alex Quinn, Tom Yeh, Ben Bederson

3. Ideal

10. Example Task: Help management choose a city for the new company headquarters. "Anna" (analyst)

11. Diagram based on Managerial Decision Modeling with Spreadsheets by Balakrishnan, Render, & Stair. © 2007 Pearson Education Interviews to determine problem structure, relevant attributes, weights Define the Problem Develop a Model Acquire Input Data Develop a Solution Test the Solution 1.climate 2.workers 3.affordability 4.health insurance costs 5.employee preference 6.rent 7.transportation 8.labor laws 9.sales tax 10.distance to other offices 11.distance to CEO hometown 12.availability of skilled labor 13.availability of office space … 50. employee satisfaction

12. Diagram based on Managerial Decision Modeling with Spreadsheets by Balakrishnan, Render, & Stair. © 2007 Pearson Education Define the Problem Develop a Model Acquire Input Data Develop a Solution Test the Solution Select method: Weighted Sums Model

13. Diagram based on Managerial Decision Modeling with Spreadsheets by Balakrishnan, Render, & Stair. © 2007 Pearson Education Define the Problem Develop a Model Acquire Input Data Develop a Solution Test the Solution Select method: Weighted Sums Model Enter formulas Simplified

14. Diagram based on Managerial Decision Modeling with Spreadsheets by Balakrishnan, Render, & Stair. © 2007 Pearson Education Define the Problem Develop a Model Acquire Input Data Develop a Solution Test the Solution Strategies: Recruit co-workers Mechanical Turk Acquire selectively Simplified

15. Diagram based on Managerial Decision Modeling with Spreadsheets by Balakrishnan, Render, & Stair. © 2007 Pearson Education Define the Problem Develop a Model Acquire Input Data Develop a Solution Test the Solution Strategies: Recruit co-workers Mechanical Turk Acquire selectively Simplified Appsheet

16. Appsheet: Getting started

17. Mockup

18. Appsheet: ASK formulas =ASK(1, 1, 10) cost minimum value maximum value Mockup

19. Appsheet: Starting Input Acquisition Mockup

20. Appsheet: Data collection Mockup

21. Appsheet: Savings Mockup

22. Appsheet: Savings Mockup Conclusive decision with only 5 of the 9 inputs Savings: 44% of the human effort

23. Prioritization Standard plan: A1, B1, C1, D1 cost minimum value maximum value

24. Prioritization Standard plan: A1, B1, C1, D1 Appsheet plan: B1, [A1 only if B1 ≤ 4], (C1 or D1) cost minimum value maximum value

25. Prioritization Standard plan: A1, B1, C1, D1 Appsheet plan: B1, [A1 only if B1 ≤ 4], (C1 or D1) Expected savings: 37% cost minimum value maximum value

26. Next steps Account for varying confidence in inputs Sharing of models themselves Construct web-based tools using spreadsheets

27. Take-home lessons Use crowdsourcing to gather data for decisions Human effort is expensive Leverage computation to reduce human effort Blend crowdsourcing with social computing Alex Quinn & Ben Bederson Contact: aq@cs.umd.edu

28. Appsheet: Architecture

29. Diagram: © 2007 Pearson Education. Balakrishnan, Render, and Stair. Managerial Decision Modeling with Spreadsheets Interviews to determine problem structure, relevant attributes, weights 1.business income tax 2.climate 3.labor pool 4.health insurance costs 5.employee preference 6.rent 7.transportation 8.labor laws 9.sales tax 10.distance to other offices 11.distance to CEO hometown 12.availability of skilled labor 13.availability of office space … 50. employee satisfaction

30. Outline Overview Example Challenges Next steps

31. Ways to gather data Option 1: Use Mechanical Turk – … but they lack domain knowledge Option 2: Get help from co-workers. – … but their time is valuable Option 3: Gather a little information and narrow the field; then explore top contenders – … but this may lead to suboptimal result Option 4: Appsheet – Combine all three

32. Overview Problem: Gathering data is work-intensive Goal: Gather less data; get the same result. Strategy: Model-driven data collection with input prioritization for cost minimization Scope: Any data-centric spreadsheet decision model that uses formulas to calculate a result