Shoogle Multimodal Excitatory Interfaces on Mobile Devices John Williamson Roderick Murray-Smith Stephen Hughes Shoogle, v., n., (c. 1724) Also shogle, schogle, shougle 1. v. intr. To sway, move unsteadily, to rock, wobble, swing 2. v. tr. To shake, joggle, to cause to totter or rock, to swing backwards and forwards Dictionary of the Scots Language

An Example Scenario Would like to know if any messages have arrived Standard alerts can be intrusive and annoying Visual attention is limited Solution: Simply pick up the device and shake it, hear and feel contents rattling about For example, get a quick overview of the quantity and type of messages

Model-based Interaction Excite – listen – feel Reveal contents of a mobile device through shaking Active perception: output in context of input motion User driven feedback Intuitive display of abstract data Exploit human understanding of natural physical processes Easy to extend interaction in natural ways Physically based but not physically limited

Features Mobile and enclosed : no moving parts Eyes-free : vibration and audio only Rich feedback : communicate as densely as possible Non-disruptive : do not interrupt Respect privacy : do not reveal private information

Concept Shake device to summarise contents Device becomes physical container Holding content balls Model-based audio and haptic display Natural sounding audio Inertial sensing for movement Can be passive or active change in a pocket

Sonification Takes advantage of impact perception Dense communication in extremely short bursts Many attributes can be perceived from a single impact. Mass, material, velocity...

A Video (1:58)

Sensing Hardware Use inertial sensing 3 DoF accelerometers Cheap Small and internal Rich signals Prototypes: iPaq 5550 / MESH inertial sensor pack Could also run on std. phones Bluetooth SHAKE sensor

Physics: Springs, Friction and Collisions Physics Virtual balls anchored via Hooke-law springs Subject to nonlinear frictional forces Collide with walls, losing some energy and rebounding Simple Euler integration model Measured accelerations incorporated directly

Physics: Springs, Friction and Collisions Springs Randomly allocated positions Only term in which ball mass appears Friction Nonlinear stick/slip model Removes lots of annoying low-level activity Collisions Elastic collisions Random direction jitter to remove repetitive bounces No interball collisions Friction Velocity

Audio Synthesis Granular approach Impact sounds triggered on wall collisions Samples of real-world impacts Many impacts recorded for each type to maintain variations Wood, metal, glass, water, ice, gravel, ping-pong balls...

Vibrotactile Feedback Produced on impact But fixed pattern Presence, not identity Greatly increases solidness of feedback Mass decides frequency division Large balls -> loudspeaker + motor >M max

Tap Summary Tap device and balls bounce up Sensed with capacitive sensors Time between impacts linked to relative arrival times

Summary Totally eyes-free interface User driven display – works on demand Physically compact Intuitive and compelling to use Come try it if you don't believe me!

Summary Applicable in many interaction contexts: Messages, files, memory space remaining, battery life, IM contacts online... Several possible extensions of the interaction style Tapping, sieving More sophisticated linking of content to physics

Shoogle Multimodal Excitatory Interfaces on Mobile Devices John Williamson Roderick Murray-Smith Stephen Hughes Shoogle, v., n., Also shogle, schogle, shougle 1. v. intr. To sway, move unsteadily, to rock, wobble, swing 2. v. tr. To shake, joggle, to cause to totter or rock, to swing backwards and forwards Dictionary of the Scots Language