1. An Experimental Study of Open Innovation using MASTERMIND®
Benjamin Chiao
School of Information, University of Michigan
1/25/2006

2. Overview
Details

3. What is Open Innovation?
Definition
A method to solving problems with other people by revealing some or the complete history of moves already made.
Examples Today
Primary: Open Source
Secondary: Open Standards
Skipped: Wikipedia
Proxy
A new variant of MASTERMIND I developed

4. Open Source Software and Organizations [From Chiao (2005): Torts In Open Innovation]

5. Standard Setting Organizations [From Benjamin Chiao, Josh Lerner, and Jean Tirole (2005)]

6. Research Questions
Why do we observe the absence of pecuniary price in some open innovation processes? Are they related to modularity?
What kind of invisible hand that actually guides the resource allocation in such processes?

7. Modularity Treatments If one person fails…
Non-modular (O-ring)
Payoff = 0 for all i
E.g. System Software
Modular
Payoff > 0 for some i
E.g. Desktop Software

8. Main Results In a non-modular structure:
Converge to a monotonic Nash with
Zero commission prices
Completely removes a signaling function of price as a measure of difficulty levels of works for coordination
Potentially removes noisy price signals if difficulty can’t be measured easily
Individuals continue to help without commission
Not sufficient to cause the catastrophic outcome of zero payoff

9. Overview
Details

10. What is MASTERMIND?
History of Moves

11. MASTERMIND & Open Source
Take the naïve view that open source means releasing the algorithm so other can see and work on it.

12. The Experiment
To extend the MASTERMIND game into a collaborative version in which the combination breaker can post the games to a public pool.

13. Payoffs Non-Modularity: Modularity: Replace (∏jMgj) with 1
where j=1,...,M, where M is the sum of allotted games for all subjects in the same period.
Qi= potential max payoff for individual i
Non-Modularity:
Payoffi=(∏jMgj)(Qi)
Modularity:
Replace (∏jMgj) with 1

14. Experimental Design Treatment 1: Modularity.
Treatment 2: Non-Modularity
Within-subject design:
Session A: Treatment 2  Treatment 1
Session B: Treatment 1  Treatment 2
5 periods for each treatment
16 students in each 2-hr session
Average payment is $20
Ran at NYU’s Center for Experimental Social Science

15. Experimental Design (To-Do)
Extend the Within-subject design into a Solomon 4-Group Design
Session A: Treatment 2  Treatment 1
Session B: Treatment 1  Treatment 2
Session C: Treatment 1 only
Session D: Treatment 2 only
This takes care of many threats to internal validity such as maturation, instrument decay, history, testing, etc...

16. The Software

17. -- the only non-private area
The Software
Public Pool
-- the only non-private area

18. The Software
Commission
Prices
History of work of
a posted game

19. The Software
Commission
Prices
History of work of
a posted game
clone

20. How Cloning Works
The history of work of a posted game will be duplicated to your private screen when you clone it
Each posted game can be cloned for multiple times
The person who solves it first gets the commission

21. Commission Prices
post & set commission
clone $P
clone

22. When the game is solved $P
Gets $1-$P

23. Payoffs Non-Modularity: Modularity: Replace (∏jMgj) with 1
Maximum Payoff: Qi=∑j∈Igj+∑kN1+N2pk
Non-Modularity:
Payoffi=(∏jMgj)(∑j∈Igj+∑kN1+N2pk)
Modularity:
Replace (∏jMgj) with 1
j=1,...,M, where M is the sum of allotted games for all subjects in the same period.
∑j∈Igj is the number of games in the set of solved games, which were originally allotted to individual i.
pk∈[-$1,$1] is the commission received from and pay to other subjects for the N₁and N₂ transactions, respectively.

24. Literature Stag Rabbit 2,2 0,1 1,0 1,1
About whether people would collectively work more for the bigger payoff
Stag Hunt Game
Studied by Anderson, Goeree, and Holt (1996) using Quantal Response Equilibrium
Table to the right
Other related solutions mentioned by Camerer (2003)
Risk-dominant equilibrium (Harsanyi and Selten, 1988)
Fairness Equilibrium (Rabin, 1993)
Weak-Link Game (or Order-Statistics Game)
Studied by Van Huyck, Battalio, and Beil (1990)
Payoff increases with the minimum in the group, and decreases with the deviation of their own choices from the minimum
Repeated Game PD
Collective efforts collapse more in bigger groups or when efforts are more costly
Stag
Rabbit
2,2
0,1
1,0
1,1

25. Non-modularity begins
Our Data
Non-modularity begins
64%
64%
83%
81%
79%
96%
98%
92%
96%
100%
Posted Games Solved by
Subjects Other than the Poster
The Poster
None—Not Solved

26. 64%
64%
83%
81%
79%
96%
98%
92%
96%
100%
77%
81%
92%
88%
90%
90%
96%
94%
96%
94%

27. Fitting the Data

28. Zero Commission or Not? (Non-Modularity Treatment) r=∑j∈Igj, n=# of subjects minus 1 pi= probability that all games are solved in cell i,
Expert vs. the Rest
Novice vs. the Rest $Y $0
Expert
demands $X
p3r+X, np3r-X
p2r+X, np2r-X
p4r, np4
p1r, np1r $Y $0
Novice demands $X
p3r-Y, np3r+Y
p’2r, np’2r
p’4r-Y, np’4r+Y
p1r, np1r

29. Equilibrium Converged to in the Data if p1≥X/r+p2 , p1 ≥ p4, p1≥p’2 and p1 ≥ Y/nr+p’4
Expert vs. the Rest
Novice vs. the Rest $Y $0
Expert
demands $X
p3r+X, np3r-X
p2r+X, np2r-X
p4r, np4
p1r, np1r $Y $0
Novice demands $X
p3r-Y, np3r+Y
p’2r, np’2r
p’4r-Y, np’4r+Y
p1r, np1r

30. Message Space: Code and Price
post & set commission $P

31. How to Coordinate to Avoid Zero Payoff
If difficulty level can be measured at all:
1. Use price to rank the difficulty of games so resources are directed to solve higher-price games first, and/or
2. Browse the posted games and decide the difficulty of games.
If not, price can actually produce noises, and pricing per se uses up resources.

32. Revisiting Research Questions
Why do we observe the absence of pecuniary price in some OI processes? Are they related to modularity?
Answer: Non-modularity is sufficient to cause such absence.
What kind of invisible hand that actually guides the resource allocation in such processes?
Answer: Perhaps the history of moves can signal difficulty of works.

33. Main Results In a non-modular structure: Why?
Converge to a monotonic Nash with
Zero commission prices
Completely removes a signaling function of price as a measure of difficulty levels.
Potentially removes noisy price signals if difficulty can’t be measured easily
Individuals continue to help without commission
Not sufficient to cause the catastrophic outcome of zero payoff
Why?
Learned to solve games faster
Learned to coordinate through the public pool
Learned strategies used in posted games

34. Implications
This provides a basis for us to hypothesize that open innovation is a key explanation because it allows subjects to directly observe the history of work already done and potentially direct more resources to the more difficult tasks.