Butterfly Effect : An Augmented Reality Puzzle Game Marleigh Norton Blair MacIntyre Steven Dow Maribeth Gandy

Game Overview Spatial puzzle where physical world is not modeled Virtual butterflies in physical environment Catch them! –Butterflies are stationary –Challenge comes from reaching the butterflies Bring butterflies into reach by rotating the virtual world –Virtual world rotates in 90 degree chunks about a player defined axis.

Equipment

Head-Mounted Display

Tornado Stick

Rotations Tornado Stick controls Virtual Axis –First Button – Place / Remove Axis –Second Button – Pause / Unpause Axis –Third Button – Rotate Butterflies

Tornado Stick People had trouble understanding rotations –Not everyone is used to 3D geometry –Though 3D puzzles have worked before Change the metaphor, not the interaction –“Tornado Stick”

Butterfly Effect

Butterfly Effect Video

Design Context Start with medium, then design game –Mapping physical and virtual worlds is hard –AR works best when user is moving slowly Tracking more accurate Safer for player – Cognitive Tunneling

Design Context Home environment –Run on next generation game consoles –Must be easy to set up –Minimize use of special equipment

1st Prototype: 2D Paper

2nd Prototype: 2D Interactive

3rd Prototype: Video

Tuning Two Major Challenges –90 degree rotation constraint –Understanding virtual space and physical space Possible Changes –Loosening rotation constraint –Adding more manipulations (translations) –Adding better depth perception cues

Open Issues: Depth Perception Static Depth Cues

Open Issues: Depth Perception Remove Automatic Depth Cues

Open Issues: Depth Perception Remove Unavailable Depth Cues

Open Issues: Depth Perception Texture probably the best

Without Grid

With Grid

Is Occlusion Really Impossible? Ad hoc modeling –Automatic modeling during calibration –Fiducials Real time depth –Depth from stereo Something to be explored

Virtual Obstacles - Bees No consequences for using poor strategy Bees behave like butterflies “Sting” if caught Penalty –Lose life –Butterflies escape

Questions?

Old Slides from Past Talks (In case people as a question I have slides for.)

Butterfly Effect : An Augmented Reality Puzzle Game Marleigh Norton Technical Advisor: Blair MacIntyre

Person Walks Through Game Area

Derive Walls from Path

Why this is Fun Fun in the same ways Rubik’s Cubes are fun Caillois’s Four Categories of Games [1] –agôn, alea, mimicry, ilinx MacTavish’s “astonishment at visual and auditory technology” [2]

Open Issues Scoring Capture Interaction Depth Perception

Open Issues: Scoring Score based on efficiency, not speed Penalized for moving too much –Accrue score as you walk –High score bad Problem: High Score Traditionally Good

Open Issues: Capture Interaction Location-Based Capture –Capture butterfly by getting close enough –Pros: Technically simple. No additional cost. –Cons: Precise head tracking more important. Less thematically appropriate. Less fun. Butterfly Net Capture –Capture butterfly in a net –Pros: More active, thematically appropriate interaction. –Cons: Requires an additional tracking system, likely expensive.

Open Issues: Depth Perception Judging depth in virtual worlds is difficult –Overcoming this difficulty is part of the game, yet it should not be impossible Problem: What can we do to help the player understand the relative locations of the butterflies? Research on Depth Perception yields several techniques [4] [5] [6] [7]

Questions & Feedback Scoring – Efficiency Capture Interaction –Location-based capture –Butterfly Net Capture Depth Perception –Which cues to use? More ideas? Other Feedback

References: Game Theory [1]Caillois, Roger, “Man, Play, and Games,” trans. Meyer Barash, University of Illiniois Press, 1961 (orig.1958) ch 2. [2]Mactavish, Andrew, “Technological Pleasure: The Performance and Narrative of Technology in Half-Life and other High-Tech Computer Games”, in: Geoff King and Tanja Krzywinska (eds.) Screenplay: Cinema/ Videogames/ Interfaces (London: Wallflower Press, 2002) 33-50

References: Depth Perception 1 [4]Cutting, J. E., and P.M. Vishton, “Perceiving layout and knowing distance: The integration, relative potency and contextual use of different information about depth”, in Perception of Space and Motion, W. Epstein and S. Rogers, Eds. Academic Press, New York, [5]Furmanski, Chris, Ronald Azuma, and Mike Daily. “Augmented-reality visualizations guided by cognition: Perceptual heuristics for combining visible and obscured information”, in Proc. IEEE and ACM Int'l Symp. on Mixed and Augmented Reality (ISMAR 2002) (Darmstadt, Germany, 30 Sept. - 1 Oct. 2002), pp

References: Depth Perception 2 [6]Sinai, M.J., W. K. Krebs, R. P. Darken, J. H. Rowland, and J. S. McCarley, “Egocentric distance perception in a virtual environment using a perceptual matching task”, in Proceedings of the 43 rd Annual Meeting Human Factors and Ergonomics Society, 43, [7]Surdick, R. T., E. T. Davis, R. A. King, and L. F. Hodges, “The perception of distance in simulated visual displays: A comparison of the effectiveness and accuracy of multiple depth cues across viewing distances”, in Presence: Teleoperators and Virtual Environments 6, 5 (October),

References: DART [8]MacIntyre, Blair, Maribeth Gandy, Steven Dow, and Jay David Bolter. "DART: A Toolkit for Rapid Design Exploration of Augmented Reality Experiences." To appear at conference on User Interface Software and Technology (UIST'04), October 24-27, 2004, Sante Fe, New Mexico.

Implementation Status It’s playable!* –Populate the world with a pre-defined set of butterflies –Butterflies can be captured –Tallies of caught and remaining butterflies are displayed –Butterflies can be rotated about the axis –Physical axis interface * When all of the equipment is working.

The Screen

Feedback Rotation Interface Depth Perception Score Aesthetics

Rotation Interface Problem: Rotating about an axis? Huh? –Suggested alternatives: Randomization Jar, Mini-version of gamespace –(Rotation about a point) “Tornado Stick” Changes: Tornado Stick –Axis/Tornado Stick only interface in which all manipulations are valid moves –Arrows pointing to butterflies’ final location

Depth Perception Problem: It’s hard to tell where the butterflies are, especially when they’re “behind walls” –Complication: can’t change appearance of butterfly based on presence of wall (or can we…?) Changes: Add grid with drop shadows –Also, it’s less of a problem when you’re actually playing

Score Problem: for Efficiency-based score, lower was better (confusing players) –Suggestions Timer Subtract from High initial score No Score Changes: No Score –Score isn’t really that important –Was causing more problems than it solved

Aesthetics Problem: What will the game look like? –Collecting the butterflies should be compelling Changes: Aesthetics more fully fleshed out –Childlike graphic design with scanned crayon textures –Player is frog, eating butterflies

Frog has Butterfly in Sights

Frog Catches Butterfly

Frog Eats Butterfly

Future Work: Finish Implementation Sounds Graphics Animations Refine Stick –Are those the right buttons? –Buttons or Switches or ??? –InertiaCube Issues with Ferris Environments

Questions? Thanks to the Augmented Environments Lab, especially: Blair MacIntyre Steven Dow Enylton Machado Coelho