Of Apple Patent US 7,479,949 and US 7,469,381 UC Berkeley, Center for Entrepreneurship and Technology IEOR 190G Patent Engineering Abhishek Gupta, BA CS

 Touch Screen technology refers to using fingers to manipulate objects on a touch screen.  Achieved through variety of ways: ◦ heat ◦ finger pressure ◦ infrared light ◦ optic capture

 For invalidity on the two patents, touch screen only refers to single touch methods on the Apple IPhone and Palm Pre.  This includes: ◦ Flicking Left/Right ◦ Locking the screen ◦ Diagonal movements ◦ Document edge in screen auto-display

 Developed a touch tablet capable of sensing an arbitrary number of simultaneous touch inputs, reporting both location and degree of touch for each.  Developed by Bill Buxton at University of Toronto, the video clearly demonstrates multi-touch concepts of sliding finger across: ws.swf

 A manipulation of a desktop display  An early front projection tablet top system that used optical and acoustic techniques to sense fingers and objects  Clearly demonstrated touch concepts such as finger gestures or a finger flick  Demo video: http://video.google.com/videoplay?docid=

 A film produced which displayed the idea of pinching. Although nothing technology related was developed – the film, produced at Sun Microsystems, clearly envisioned the future to include ‘pinching’ and ‘diagonal movements’

 Founded by two University of Delaware academics, John Elias and Wayne Westerman  Product largely based on Westerman’s thesis: Westerman, Wayne (1999). Hand Tracking,Finger Identification, and Chordic Manipulation on a Multi-Touch Surface.  The company was acquired in early 2005 by Apple, where Elias and Westerman moved to Apple.

“state C detects significant motion on all touching fingers and advances to the manipulation state M, the channel selection is locked in. Additional finger touchdowns or liftoffs will not affect the channel selection during manipulation unless they meet the special synchronization sequence” (pg. 269)

 Images placed on a wall, "to advance to the next slide in sequence, one flicked to the right. To go back to the previous image, one flicked left."  "The gestures were much richer than just left-right flicks. One could investigate different behaviors, depending on the direction you moved your finger."  "In this system there were eight options, corresponding to the 8 main points of a compass. For example, a downward gesture over a video meant 'stop'. A gesture up to the right enabled annotation, down to the right launched the application associated with the image, etc."

 The advancing from one slide to the next appears to read directly on claim clause of flipping through items in a list and is the same example as in the Apple Patent.  According to the video, the left/right flick can anticipate the horizontal and vertical IPhone locked scrolling.  Also show’s that other commands can be on the 8 other compass points.

  Developed by Mitsubishi Research Labs, a tabletop similar to Microsoft Surface that could adapt finger touches. Researchers published multiple papers about it.  Videos clearly demonstrate ability for finger to do vertical, horizontal, and diagonal movements which translate into actual actions  Both videos show that scrolling on the screen can occur with a manual finger touch gestures_lr.mov multimodal_gaming.wmv

 ess/2005/ htm ess/2005/ htm  A mobile touch screen display that could detect finger movements.  It could be used to ‘navigate through pages in the same manner as a conventional touch-screen approach’  Similar to scrolling of items through a list with Apple’s touch screen

ElementInterpretationPrior Art (Portfolio Wall, Diamond Touch, Toshiba Mobile Display) “A computing device, comprising:” Portfolio Wall works on a computing device, however is sold independently as software. Diamond Touch is computing device. Toshiba Mobile Display is a computing device. “a touch screen display;”Portfolio Wall software works on a touch-based monitor. Diamond touch has a touch surface. Toshiba Mobile Display has a touch screen display. “one or more processors;”Portfolio Wall requires processor from computing device in order to work. Diamond Touch has a processor. Toshiba Mobile Display has a processor. “memory;”Portfolio Wall requires memory Diamond Touch requires memory. Toshiba Mobile Display requires memory.

ElementInterpretationProduct “one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including:” One or more programs in memory that can be executed by the CPU(s). Portfolio Wall software is executed on a CPU. Diamond touch is a large interface that can execute multiple programs. Toshiba Mobile Display is executed by the device’s CPU.

ElementInterpretationPortfolio Wall Product “instructions for detecting one or more finger contacts with the touch screen display;” It is inherent that if there is a processor and a touch screen display whereby a user can use his finger, that there must be instruction for detecting finger contacts with the touch screen.

ElementInterpretationDiamond Touch Product “instructions for detecting one or more finger contacts with the touch screen display;” It is inherent that if there is a processor and a touch screen display whereby a user can use his finger, that there must be instruction for detecting finger contacts with the touch screen.

ElementInterpretationToshiba Mobile Display Product “instructions for detecting one or more finger contacts with the touch screen display;” It is inherent that if there is a processor and a touch screen display whereby a user can use his finger, that there must be instruction for detecting finger contacts with the touch screen.

ElementInterpretationPortfolio Wall Product “instructions for applying one or more heuristics to the one or more finger contacts to determine a command for the device;” ("heuristics [programs] are used to translate imprecise finger gestures into actions desired by the user." [column 109, line 50-51]) Heuristics are generally inherent in computer programs processing imprecise input, such as the movement of a touch on a touch screen. There must be instructions for the computer to apply the one or more heuristics and determine the desired command for the device from the finger contacts.

ElementInterpretationDiamond Touch Product “instructions for applying one or more heuristics to the one or more finger contacts to determine a command for the device;” ("heuristics [programs] are used to translate imprecise finger gestures into actions desired by the user." [column 109, line 50-51]) Heuristics are generally inherent in computer programs processing imprecise input, such as the movement of a touch on a touch screen. There must be instructions for the computer to apply the one or more heuristics and determine the desired command for the device from the finger contacts.

ElementInterpretationPortfolio Wall Product “and instructions for processing the command;” Executes the commands. Inherently a processor- driven device uses instructions to process commands. “wherein the one or more heuristics comprise:”

ElementInterpretationDiamond Touch Product “and instructions for processing the command;” Executes the commands. Inherently a processor- driven device uses instructions to process commands. “wherein the one or more heuristics comprise:”

ElementInterpretationToshiba Mobile Display Product “and instructions for processing the command;” Executes the commands. Inherently a processor- driven device uses instructions to process commands. “wherein the one or more heuristics comprise:”

ElementInterpretationPortfolio Wall Product “a vertical screen scrolling heuristic for determining that the one or more finger contacts correspond to a one- dimensional vertical screen scrolling command rather than a two- dimensional screen translation command based on an angle of initial movement of a finger contact with respect to the touch screen display;” A heuristic (implemented as a program) that looks for initial vertical movement of the finger(s) and decides that strictly vertical scrolling is desired even though the finger may move off a vertical path after initial contact. According to the video, there is a horizontal screen scrolling heuristic, which can anticipate a vertical screen scrolling heuristic for determining that the one or more finger contacts correspond to a one- dimensional vertical screen scrolling command rather than a two-dimensional screen translation command based on the angle of initial movement of a finger contact with respect to the touch screen display. Hence this is due to obviousness.

ElementInterpretationDiamond Touch Product “a vertical screen scrolling heuristic for determining that the one or more finger contacts correspond to a one- dimensional vertical screen scrolling command rather than a two- dimensional screen translation command based on an angle of initial movement of a finger contact with respect to the touch screen display;” A heuristic (implemented as a program) that looks for initial vertical movement of the finger(s) and decides that strictly vertical scrolling is desired even though the finger may move off a vertical path after initial contact. According to the videos, there is a vertical screen scrolling heuristic, where the user moves the finger and the screen locked into the movement of the finger.

ElementInterpretationPortfolio Wall Product “a two-dimensional screen translation heuristic for determining that the one or more finger contacts correspond to the two- dimensional screen translation command rather than the one- dimensional vertical screen scrolling command based on the angle of initial movement of the finger contact with respect to the touch screen display;” A heuristic (implemented as a program) that looks for initial movement of the finger(s) not close to vertical and decides that some degree of diagonal (vertical and horizontal) scrolling is desired rather than strictly vertical scrolling. The Portfolio wall uses a two-dimensional screen translation heuristic for determining that the one or more finger contacts correspond to the two- dimensional screen translation command rather than the one-dimensional vertical screen scrolling command based on the angle of initial movement of the finger contact with respect to the touch screen display. “A gesture up to the right enabled annotation. Down to the right launched the application associated with the image. ”

ElementInterpretationPortfolio Wall Product “and a next item heuristic for determining that the one or more finger contacts correspond to a command to transition from displaying a respective item in a set of items to displaying a next item in the set of items.” A heuristic (implemented as a program) that looks for an unspecified finger(s) contact to move stepwise from item to item in a set rather than scroll through the set. An example reciting the text of this clam clause refers to an image in an album of images. The Portfolio Wall uses a next item heuristic for determining that the one or more finger contacts correspond to a command to transition from displaying a respective item in a set of items to displaying a next item in the set of items. -The video makes this apparent

ElementInterpretationToshiba Mobile Display Product “and a next item heuristic for determining that the one or more finger contacts correspond to a command to transition from displaying a respective item in a set of items to displaying a next item in the set of items.” A heuristic (implemented as a program) that looks for an unspecified finger(s) contact to move stepwise from item to item in a set rather than scroll through the set. An example reciting the text of this clam clause refers to an image in an album of images. Toshiba Mobile Display could clearly be used to navigate through a set of items. – The description and image show this feature.

 Based on the prior art of Portfolio Wall, Diamond Touch and Toshiba Mobile Display – yes.  Although Portfolio Wall is not a piece of hardware, when implemented on a computing device, the software interacts with the hardware to make a finger(s) touch-based device.  Portfolio Wall’s heuristics were similar to touch- based mobile devices available today.  The combination of the vertical scrolling from Diamond Touch and slideshow from Toshiba Mobile Display also contribute to the heuristics in today’s touch-based mobile devices.

 Finding prior art for the ‘381 patent has been difficult to find: document edge in screen auto- displays to configuration.

 Questions & Answers  References:  analysis/#continuedengadget.com/2009/01/28/apple-vs-palm-the-in- depth-analysis/%23continued analysis/#continuedengadget.com/2009/01/28/apple-vs-palm-the-in- depth-analysis/%23continued 