1. Supporting Multiple Pointing Devices in Microsoft Windows
Michael Westergaard
Department of Computer Science
University of Aarhus
Denmark
September 10, 2002

2. Outline Why multiple pointing devices? Requirements Architecture
Framework
High-level API
Low-level API
Device driver
Example
September 10, 2002

3. Why Multiple Pointing Devices
Demonstration
September 10, 2002

4. Why Multiple Pointing Devices (2)
Advanced interaction techniques
Different mice for different tasks
Multiple hand interactions
Two-handed resize + move/zoom + move
Two handed move of 3D objects
Three handed zoom + move + rotate of 3D objects
Floating palettes
Tool glasses
Move work from one hand to another  reduce risk of cumulative trauma disorders
Cooperative work (for example on a wall PC)
September 10, 2002

5. Requirements
Earlier: External application directly polling hardware. This requires exclusive access to the (serial) port
Support arbitrarily many mice
Treat USB, serial, and PS/2 mice alike
Wish to use mouse in “old” Windows applications
Generic solution not tied to a specific application or framework
September 10, 2002

6. Architecture CPN Tools Hardware dependent driver Device driver
Low-level API
High-level API
Framework
Windows mouse subsystem
Application
Octopus
Main topic of article
September 10, 2002

7. Architecture Framework
Hardware dependent driver
Device driver
Low-level API
High-level API
Framework
Windows mouse subsystem
Application
September 10, 2002

8. Framework
Provide widget-like interface (for example one can override the OnMouseDown event of a Button )
Must also provide information about what mouse was clicked (this can be obtained by adding a Mouse property to the MouseEventArgs along with information about all mice on the system )
Must also provide information about other mice working with an element (just another property of MouseEventArgs ) in order to allow exclusive access to an element and to allow multiple hand actions
Give mouse back to Windows when it leaves application window (only when no native support is available, of course)
September 10, 2002

9. Architecture Device Driver
Hardware dependent driver
Device driver
Low-level API
High-level API
Framework
Windows mouse subsystem
Application
September 10, 2002

10. Device Driver Windows does not distinguish between different mice
Install filter between hardware dependent driver and Windows Mouse subsystem
We do not have to deal with hardware differences
We can select whether to send a mouse event to Windows
Keep driver as simple as possible to avoid bugs in kernel-mode code
Use callback to send data to user-mode
Communicate by creating files in the drivers namespace
September 10, 2002

11. Device Driver (2) Interface A finite state machine is implemented Name
Description
Get
Hook up a mouse for use by an application, register a callback, stop sending Windows mouse events
UnGet
Release a mouse, drop the callback and start sending Windows events again
Suspend
Temporarily suspend a mouse, keep the callback but start sending Windows events
UnSuspend
Stop sending Windows events
September 10, 2002

12. Device Driver (3)
September 10, 2002

13. Architecture Low-level API
Hardware dependent driver
Device driver
Low-level API
High-level API
Framework
Windows mouse subsystem
Application
September 10, 2002

14. Low-level API
Hide operating system specific support (for multiple mice) from higher levels
Simple C interface rather than operating system specific calls to communicate with device driver
If generic Windows support is developed only the low-level API needs to be changed
Simplify identification of mice
Add simple error handling
Calls are not thread-safe
Designed to be statically linked to a specific application
September 10, 2002

15. Low-level API (2) Interface
The provided callback does not have to be thread-safe
Name
Description
RegisterCallback
Register a callback
GetMice
Hook up a specified number of mice
HasMouse
Query if we have hooked a mouse with a specified number
UnGetMouse
Release a specified mouse
SuspendMouse
Passed to the device driver
UnSuspendMouse
UnRegisterCallback
Drop a previously registered callback
UnGetAllMice
Release all hooked mice
September 10, 2002

16. Architecture High-level API
Hardware dependent driver
Device driver
Low-level API
High-level API
Framework
Windows mouse subsystem
Application
September 10, 2002

17. High-level API Hide the underlying operating system from the framework
Build on top of low-level API
Spawn a thread to listen for callbacks from the low-level API
Send “mouse events” to application
Calls are thread-safe
Draw mouse cursors
Accelerate mouse
Offer polling interface
Like low-level API designed to be statically linked to an application
September 10, 2002

18. High-level API (2) Interface Name Description Initialise
Start up the listener thread and do some other initialising
Cleanup
Kill the listener thread and cleanup other allocated resources
SuspendMouse
Thread-safe version of corresponding low-level functions
UnsuspendMouse
GetRelativePosition
Get mouse movement since last call
GetAbsolutePosition
Get the position of the mouse
SetAbsolutePosition
Set the position of the mouse
SetCursor
Set the cursor for a specified mouse
LockCanvas
Used for drawing cursors
UpdateCursors
September 10, 2002

19. Architecture Example -- Startup
Application
Framework
H-API
L-API
Driver
User
Start
Start
Initialise
GetMice
Get
n times n
spawn n ok
September 10, 2002

20. Architecture Example -- Suspend
Application
Framework
H-API
L-API
Driver
User
Move mouse
Mouse moved callback
Mouse moved callback
Mouse moved event
Mouse is moved out of window
Suspend
Suspend
Suspend
September 10, 2002

21. Future Work
More control of what mice are hooked by GetMice function (by number, name, position on bus, or capabilities)
Be able to handle multiple applications sharing mice
Higher-level API: DLL/System service
Port the needed layers to also support Windows 9x/ME/NT, Unix, and Mac
September 10, 2002

22. Conclusions During this talk we have…
seen how multiple mice can be useful (advanced interaction techniques, cooperative work)
seen requirements for an architecture supporting multiple mice (work with any kind of mouse, backwards compatibility, not tied to an application)
seen requirements for a framework supporting multiple mice
seen a device driver and two supporting APIs that can satisfy the needs of a framework supporting multiple mice
September 10, 2002

23. Framework Widgets With Events
public class MyButton: Button { protected override void OnMouseDown(MouseEventArgs e) { base.OnMouseDown(e);
if (e.Button == left) { MessageBox.Show(“Left button pressed", ""); } } }
September 10, 2002

24. Framework Knowledge of Active Mouse
public class MyButton: Button { protected override void OnMouseDown(MyMouseEventArgs e) { base.OnMouseDown(e);
if (e.Mouse == Mice[1]) { MessageBox.Show("Mouse 1 pressed", ""); } } }
September 10, 2002

25. Framework Knowledge of Other Mice
public class MyButton: Button { protected override void OnMouseDown(MyMouseEventArgs e) { base.OnMouseDown(e);
if (e.OtherMice.Length == 0) { MessageBox.Show("Only one mouse pressed the button", ""); } } }
September 10, 2002