Supporting Multiple Pointing Devices in Microsoft Windows Michael Westergaard Department of Computer Science University of Aarhus Denmark September 10, 2002

Outline Why multiple pointing devices? Requirements Architecture Framework High-level API Low-level API Device driver Example September 10, 2002

Why Multiple Pointing Devices Demonstration September 10, 2002

Why Multiple Pointing Devices (2) Advanced interaction techniques Different mice for different tasks Multiple hand interactions Two-handed resize + move/zoom + move Two handed move of 3D objects Three handed zoom + move + rotate of 3D objects Floating palettes Tool glasses Move work from one hand to another  reduce risk of cumulative trauma disorders Cooperative work (for example on a wall PC) September 10, 2002

Requirements Earlier: External application directly polling hardware. This requires exclusive access to the (serial) port Support arbitrarily many mice Treat USB, serial, and PS/2 mice alike Wish to use mouse in “old” Windows applications Generic solution not tied to a specific application or framework September 10, 2002

Architecture CPN Tools Hardware dependent driver Device driver Low-level API High-level API Framework Windows mouse subsystem Application Octopus Main topic of article September 10, 2002

Architecture Framework Hardware dependent driver Device driver Low-level API High-level API Framework Windows mouse subsystem Application September 10, 2002

Framework Provide widget-like interface (for example one can override the OnMouseDown event of a Button ) Must also provide information about what mouse was clicked (this can be obtained by adding a Mouse property to the MouseEventArgs along with information about all mice on the system ) Must also provide information about other mice working with an element (just another property of MouseEventArgs ) in order to allow exclusive access to an element and to allow multiple hand actions Give mouse back to Windows when it leaves application window (only when no native support is available, of course) September 10, 2002

Architecture Device Driver Hardware dependent driver Device driver Low-level API High-level API Framework Windows mouse subsystem Application September 10, 2002

Device Driver Windows does not distinguish between different mice Install filter between hardware dependent driver and Windows Mouse subsystem We do not have to deal with hardware differences We can select whether to send a mouse event to Windows Keep driver as simple as possible to avoid bugs in kernel-mode code Use callback to send data to user-mode Communicate by creating files in the drivers namespace September 10, 2002

Device Driver (2) Interface A finite state machine is implemented Name Description Get Hook up a mouse for use by an application, register a callback, stop sending Windows mouse events UnGet Release a mouse, drop the callback and start sending Windows events again Suspend Temporarily suspend a mouse, keep the callback but start sending Windows events UnSuspend Stop sending Windows events September 10, 2002

Device Driver (3) September 10, 2002

Architecture Low-level API Hardware dependent driver Device driver Low-level API High-level API Framework Windows mouse subsystem Application September 10, 2002

Low-level API Hide operating system specific support (for multiple mice) from higher levels Simple C interface rather than operating system specific calls to communicate with device driver If generic Windows support is developed only the low-level API needs to be changed Simplify identification of mice Add simple error handling Calls are not thread-safe Designed to be statically linked to a specific application September 10, 2002

Low-level API (2) Interface The provided callback does not have to be thread-safe Name Description RegisterCallback Register a callback GetMice Hook up a specified number of mice HasMouse Query if we have hooked a mouse with a specified number UnGetMouse Release a specified mouse SuspendMouse Passed to the device driver UnSuspendMouse UnRegisterCallback Drop a previously registered callback UnGetAllMice Release all hooked mice September 10, 2002

Architecture High-level API Hardware dependent driver Device driver Low-level API High-level API Framework Windows mouse subsystem Application September 10, 2002

High-level API Hide the underlying operating system from the framework Build on top of low-level API Spawn a thread to listen for callbacks from the low-level API Send “mouse events” to application Calls are thread-safe Draw mouse cursors Accelerate mouse Offer polling interface Like low-level API designed to be statically linked to an application September 10, 2002

High-level API (2) Interface Name Description Initialise Start up the listener thread and do some other initialising Cleanup Kill the listener thread and cleanup other allocated resources SuspendMouse Thread-safe version of corresponding low-level functions UnsuspendMouse GetRelativePosition Get mouse movement since last call GetAbsolutePosition Get the position of the mouse SetAbsolutePosition Set the position of the mouse SetCursor Set the cursor for a specified mouse LockCanvas Used for drawing cursors UpdateCursors September 10, 2002

Architecture Example -- Startup Application Framework H-API L-API Driver User Start Start Initialise GetMice Get n times n spawn n ok September 10, 2002

Architecture Example -- Suspend Application Framework H-API L-API Driver User Move mouse Mouse moved callback Mouse moved callback Mouse moved event Mouse is moved out of window Suspend Suspend Suspend September 10, 2002

Future Work More control of what mice are hooked by GetMice function (by number, name, position on bus, or capabilities) Be able to handle multiple applications sharing mice Higher-level API: DLL/System service Port the needed layers to also support Windows 9x/ME/NT, Unix, and Mac September 10, 2002

Conclusions During this talk we have… seen how multiple mice can be useful (advanced interaction techniques, cooperative work) seen requirements for an architecture supporting multiple mice (work with any kind of mouse, backwards compatibility, not tied to an application) seen requirements for a framework supporting multiple mice seen a device driver and two supporting APIs that can satisfy the needs of a framework supporting multiple mice September 10, 2002

Framework Widgets With Events public class MyButton: Button { protected override void OnMouseDown(MouseEventArgs e) { base.OnMouseDown(e); if (e.Button == left) { MessageBox.Show(“Left button pressed", ""); } } } September 10, 2002

Framework Knowledge of Active Mouse public class MyButton: Button { protected override void OnMouseDown(MyMouseEventArgs e) { base.OnMouseDown(e); if (e.Mouse == Mice[1]) { MessageBox.Show("Mouse 1 pressed", ""); } } } September 10, 2002

Framework Knowledge of Other Mice public class MyButton: Button { protected override void OnMouseDown(MyMouseEventArgs e) { base.OnMouseDown(e); if (e.OtherMice.Length == 0) { MessageBox.Show("Only one mouse pressed the button", ""); } } } September 10, 2002