1. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 1/22 September 10, 2002 Supporting Multiple Pointing Devices in Microsoft Windows Michael Westergaard Department of Computer Science University of Aarhus Denmark

2. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 2/22September 10, 2002 Outline Why multiple pointing devices? Requirements Architecture –Framework –High-level API –Low-level API –Device driver –Example

3. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 3/22September 10, 2002 Why Multiple Pointing Devices Demonstration

4. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 4/22September 10, 2002 Why Multiple Pointing Devices (2) Advanced interaction techniques –Multiple hand interactions Two-handed resize/zoom Two handed rotation of 3D objects Three handed zoom + move + rotate –Different mice for different tasks –Floating palettes –Tool glasses Move work from one hand to another  reduce risk of cumulative trauma disorders Cooperative work (for example on a wall PC)

5. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 5/22September 10, 2002 Requirements Earlier: External application directly polling hardware. This requires exclusive access to the (serial) port Support arbitrarily many mice Treat USB, serial, and PS/2 mice alike Wish to use mouse in “old” Windows applications Generic solution not tied to a specific application or framework

6. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 6/22September 10, 2002 Architecture Hardware dependent driver Device driver Low-level API High-level API Framework Windows mouse subsystem Framework ApplicationLow-level API Main topic of article Octopus CPN Tools

7. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 7/22September 10, 2002 Architecture Framework Hardware dependent driver Device driver Low-level API High-level API Framework Windows mouse subsystem Framework ApplicationLow-level API

8. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 8/22September 10, 2002 Framework Provide widget-like interface (for example one can override the OnMouseDown event of a Button ) Must also provide information about what mouse was clicked (this can be obtained by adding a Mouse property to the MouseEventArgs along with information about all mice on the system ) Must also provide information about other mice working with an element (just another property of MouseEventArgs ) in order to allow exclusive access to an element and to allow multiple hand actions Give mouse back to Windows when it leaves application window (only when no native support is available, of course)

9. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 9/22September 10, 2002 Architecture Device Driver Hardware dependent driver Device driver Low-level API High-level API Framework Windows mouse subsystem Framework ApplicationLow-level API

10. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 10/22September 10, 2002 Device Driver Windows does not distinguish between different mice Install filter between hardware dependent driver and Windows Mouse subsystem –We do not have to deal with hardware differences –We can select whether to send a mouse event to Windows Keep driver as simple as possible to avoid bugs in kernel-mode code –Use callback to send data to user-mode –Communicate by creating files in the drivers namespace

11. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 11/22September 10, 2002 Device Driver (2) Interface A finite state machine is implemented NameDescription GetHook up a mouse for use by an application, register a callback, stop sending Windows mouse events UnGetRelease a mouse, drop the callback and start sending Windows events again SuspendTemporarily suspend a mouse, keep the callback but start sending Windows events UnSuspendStop sending Windows events

12. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 12/22September 10, 2002 Device Driver (3)

13. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 13/22September 10, 2002 Architecture Low-level API Hardware dependent driver Device driver Low-level API High-level API Framework Windows mouse subsystem Framework ApplicationLow-level API

14. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 14/22September 10, 2002 Low-level API Hide operating system specific support (for multiple mice) from higher levels –Simple C interface rather than operating system specific calls to communicate with device driver –If generic Windows support is developed only the low-level API needs to be changed Simplify identification of mice Add simple error handling Calls are not thread-safe Designed to be statically linked to a specific application

15. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 15/22September 10, 2002 Low-level API (2) Interface The provided callback does not have to be thread-safe NameDescription RegisterCallbackRegister a callback GetMiceHook up a specified number of mice HasMouseQuery if we have hooked a mouse with a specified number UnGetMouseRelease a specified mouse SuspendMousePassed to the device driver UnSuspendMousePassed to the device driver UnRegisterCallbackDrop a previously registered callback UnGetAllMiceRelease all hooked mice

16. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 16/22September 10, 2002 Architecture High-level API Hardware dependent driver Device driver Low-level API High-level API Framework Windows mouse subsystem Framework ApplicationLow-level API

17. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 17/22September 10, 2002 High-level API Hide the underlying operating system from the framework Build on top of low-level API Spawn a thread to listen for callbacks from the low- level API Send “mouse events” to application Calls are thread-safe Draw mouse cursors Accelerate mouse Offer polling interface Like low-level API designed to be statically linked to an application

18. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 18/22September 10, 2002 High-level API (2) Interface NameDescription InitialiseStart up the listener thread and do some other initialising CleanupKill the listener thread and cleanup other allocated resources SuspendMouse Thread-safe version of corresponding low-level functions UnsuspendMouse GetRelativePositionGet mouse movement since last call GetAbsolutePositionGet the position of the mouse SetAbsolutePositionSet the position of the mouse SetCursorSet the cursor for a specified mouse LockCanvasUsed for drawing cursors UpdateCursorsUsed for drawing cursors

19. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 19/22September 10, 2002 Architecture Example -- Startup ApplicationFrameworkH-APIL-APIDriverUser Start Initialise GetMice Get n times n spawn n ok

20. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 20/22September 10, 2002 Architecture Example -- Suspend ApplicationFrameworkH-APIL-APIDriverUser Move mouse Mouse moved callback Mouse moved event Mouse is moved out of window Suspend

21. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 21/22September 10, 2002 Future Work More control of what mice are hooked by GetMice function (by number, name, position on bus, or capabilities) Be able to handle multiple applications sharing mice Higher-level API: DLL/System service Port the needed layers to also support Windows 9x/ME/NT, Unix, and Mac

22. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 22/22September 10, 2002 Conclusions During this talk we have… seen how multiple mice can be useful (advanced interaction techniques, cooperative work) seen requirements for an architecture supporting multiple mice (work with any kind of mouse, backwards compatibility, not tied to an application) seen requirements for a framework supporting multiple mice seen a device driver and two supporting APIs that can satisfy the needs of a framework supporting multiple mice

23. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 23/22September 10, 2002 Framework Widgets With Events public class MyButton: Button { protected override void OnMouseDown(MouseEventArgs e) { base.OnMouseDown(e); if (e.Button == left) { MessageBox.Show("Button pressed", ""); } } }

24. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 24/22September 10, 2002 Framework Knowledge of Active Mouse public class MyButton: Button { protected override void OnMouseDown(MyMouseEventArgs e) { base.OnMouseDown(e); if (e.Mouse == Mice[1]) { MessageBox.Show("Mouse 1 pressed", ""); } } }

25. Supporting Multiple Pointing Devices in Microsoft WindowsSummer Research Workshop 2002 25/22September 10, 2002 Framework Knowledge of Other Mice public class MyButton: Button { protected override void OnMouseDown(MyMouseEventArgs e) { base.OnMouseDown(e); if (e.OtherMice.Length == 0) { MessageBox.Show(“Only one mouse pressed the button", ""); } } }