Ken Hinckley Koji Yatani Pen + Touch = New Tools

Handhelds Slates Drafting Boards Tabletops Wall Displays Touch or Pen… DIRECT INPUT A SHIFT TO

ISN’T THE FUTURE ABOUT TOUCH? WHY THE PEN?

To help explain, let me introduce my nemesis.

MR. I.M.A. BIGBODY, CORPORATE DENIZEN, THIRD RATE INC.

“I CAN TYPE FASTER THAN I CAN WRITE. ”

TYPE THIS.

OR THIS.

OR MAYBE THIS.

YOU CAN’T TYPE THE SOUL OF A NEW IDEA.

BUT IT’S NOT ALL ABOUT THE PEN.

FIRST YOU NEED TO GRAB THE NAPKIN.

“JUST TELL ME WHICH IS BEST, TOUCH OR PEN. ”

OF HANDS AND HANDEDNESS

PAPER NOTEBOOK OBSERVATIONS B1: Tuck the pen, interleave stylus and fingers B2: Hold Clippings while Writing B3: Specific Roles for Pen and Fingers B4: Two-handed actions

B5: Drawing along Edges B6: Hold Page while Flipping PAPER NOTEBOOK OBSERVATIONS

BE EVOCATIVE OF PAPER WITHOUT BEING BEHOLDEN TO IT.

Q: Which hand do you write with, Right or Left?

A: Both! Guiard, 1987: NPH defines frame of reference. Right vs. Left is the wrong question to ask.

What is the logic of the division of labor between pen and touch in interface design?

Video – search on “manual deskterity”

Users easily interleaved pen & touch – obvious Staple, Xacto Cut, and Peel off Copy particularly resonated with users The particular tools were not always of interest Some example comments: REACTIONS FROM USERS “The way it works is just like the way I already work in my notebook.” “I wouldn’t have guessed the gestures worked that way, but once I tried it, it felt pretty natural.”

DESIGN PROPERTIES OF PEN + TOUCH

27

Pen + Touch: distinct interaction states Touch + Touch ambiguous : pinch Zero-force, incidental Contact is an inherent property of (soft) Touch – GREAT! Just touch it to do something. – TERRIBLE! Just touch it, and it might do something. WHY NOT ALL MULTITOUCH?

Minimal on-screen UI, widgets, & buttons. Holding touch integrates (1) Object Selection with (2) Pen Mode Switching. And also… (3) provides salient feedback of temporary state. (4) unambiguously phrases together multiple elementary pen and touch inputs. SOME INTRIGUING PROPERTIES…

Differentiated vs. Interchangeable – E.g. Common controls Mobile vs. Stationary – Hand-held use vs. on-desk use, with graceful degradation between the two – One-handed interactions for core actions – Two-handed interactions to take full advantage of expressiveness of pen + touch DESIGN DIMENSIONS

Interleaved vs. Simultaneous – Pen + touch has the most novelty – Touch-pen-touch has tremendous value Elementary Inputs vs. Phrases – Not just assign device/hand to individual interactions – – Workflow across multiple subtasks… DESIGN DIMENSIONS

SUMMARY

IF WE’RE ONLY THINKING ABOUT TOUCH, WE’RE BLOWING IT.

EXPERIENCES TO STAND OUT… IF WE WANT OUR

Not pen. Not touch. Not pen OR touch. Pen AND Touch, designed in concert

“TOUCH OR PEN ARE JUST NEW WAYS TO CONTROL THE MOUSE, SO WHAT’S THE BIG DEAL? ”

YOU’RE FIRED! “TOUCH OR PEN ARE JUST NEW WAYS TO CONTROL THE MOUSE, SO WHAT’S THE BIG DEAL? ”

Pen Writes Touch Manipulates

Pen + Touch Yields New Tools

PEN + TOUCH GESTURES staple straightedgestamp Xacto cut brush peel off copy

Pen Writes Touch Manipulates

fluidity of annotation + manipulation navigation zooming object creation

Pen + Touch Yields New Tools

PROPERTIES SHARED BY PEN AND TOUCH

PEN IS BEST! Mechanical intermediary: familiar tool, lever arm, affords precision!

TOUCH IS BEST! No mechanical intermediary! Nothing to lose! No time to unsheathe!

PEN IS BEST! Point at a specific spot on the coin

TOUCH IS BEST! Grab the whole coin

EVERY COIN HAS TWO SIDES.