1. copyright 2011 controltrix corpwww. controltrix.com Hand held motion tracking using MEMS gyros and accelerometer for gaming applications www.controltrix.com

2. copyright 2011 controltrix corpwww. controltrix.com Accelerometers (acc) measure acceleration Gyroscopes (gyro) measure angular velocity Integrated MEMS may have 3 axis gyro + 3 axis acc MEMS have low cost compared to other types of gyro /acc The MEMS device is Clamped to the object (strap down) (Unlike gyro stabilized system which give direct values) Measurements are with respect to object and not with earth Complex/Vector /coordinate computation for absolute values Intro

3. copyright 2011 controltrix corpwww. controltrix.com Objective : Measure angles(orientation) in 3D in real time 3D Angle (orientation) is mapped to screen object motion Integrating(accumulating) angular velocity gives angular displacement Integration causes drift Accumulation errors  diverging results to ∞  loss of sync Example MPU 6000 has angular velocity error of 20 degrees/s. After 9 sec, the object may point opposite !!! Intro.

4. copyright 2011 controltrix corpwww. controltrix.com Essentially an inertial measurement system Attitude Heading Reference systems (AHRS) used in aircraft Best systems drift ~ 1Km /hr and few degrees/hr Cost ~ US$ 100K,weight ~ few Kg Aircraft has auxillary systems like GPS, magnetometer Augment inertial measurements (keep drift negligible) Objective : emulate AHRS in a few US$, < 100 gm Intro..

5. copyright 2011 controltrix corpwww. controltrix.com To overcome drift filtering is used Filtering removes DC offset in measurement but…. Creates a side effect of homing A Stationary object the measured angles drift towards 0 with time. (still better than drifting to ∞ ) To fix homing some thresholding is done but…… It causes slow movements not accurately tracked….. Present system approach and limits

6. copyright 2011 controltrix corpwww. controltrix.com Only relative motion tracked..screen object and handheld. Example The directionality of motion is correct, but A 90 degree counter clockwise followed by 90 degree clockwise is never initial position. Cannot Track pure translation motion Slow movements are not properly tracked. Below a certain limit the system essentially rejects data as noise Present system limits

7. copyright 2011 controltrix corpwww. controltrix.com Auxiliary angular position data to periodically recalibrate (accelerometer and magnetometer) Remove unbounded drift Even noisy, jumpy, low bandwidth, low sample rate data is good Real time data fusion Sensor data fusion algorithm to compute best estimate Kalman filter or Modified Kalman filter What is required ?

8. copyright 2011 controltrix corpwww. controltrix.com Accelerometer measures gravity ‘g’ (always down) when stationary Gravity is absolute reference direction and magnitude Accelerometer features Fig: The accelerometer measures the component of the acceleration due to gravity acting on each of the three axes. These components are trigonometrically related to the angle of inclination

9. copyright 2011 controltrix corpwww. controltrix.com 3 components provide crude estimate for roll and pitch Simple vector math required Doesn’t help with Yaw Example North and east pointing is indistinguishable / give same readings Accelerometer features.

10. copyright 2011 controltrix corpwww. controltrix.com Magnetometer (mag) measures axial magnetic field strength (B) 3 axis magnetometer measures in all 3 dimensions Absolute reference is local earth magnetic field 2 Angles (pitch and yaw) can be measured (assuming 0 magnetic dip/perfectly horizontal) Doesn’t help with roll e.g. any roll about the magnetic line axis will give same readings Magnetometer features and utility

11. copyright 2011 controltrix corpwww. controltrix.com Combining both acc and mag all 3 angles can be found but… Earths magnetic field is rarely horizontal dip is non 0 More computation required to account for dip Calibration of magnetometer to get local dip initially Magnetometer features and utility.

12. copyright 2011 controltrix corpwww. controltrix.com One to one mapping of all rotational motion Extremely intuitive gaming experience for role playing game Perfect synch /small tracking error (ref: simulation) Accurate tracking of slowest possible movements (No drift) Inspite of noise/ jumpy acc /mag based angle sensing Very smooth operation (limited by display frame rate) Proposed method advantages

13. copyright 2011 controltrix corpwww. controltrix.com Future of gaming Auto calibration for acc and mag Unlike filtering, method acts like a filter but without the lag Virtually 0 lag filter Performance can be easily tweaked (ref. appendix) Minimal tuning/ trial and error Tracking Pure translation is still not possible but….. Hand movements are seldom pure translation Proposed method advantages.

14. copyright 2011 controltrix corpwww. controltrix.com Acceleration and velocity are measured using noisy sensor Direct velocity measurement is noisy (  v  m/s) Acceleration is measured with  a = 0.1 m/s 2 offset = 0.2 m/s 2 (DRIFT) Superposed sine wave drive Amplitude A = 3 m/s 2, frequency f = 0.05 Hz Sample time Ts = 0.1 s

15. copyright 2011 controltrix corpwww. controltrix.com Example from a different problem, but math is same Replace Velocity with angle (from acc and mag )in deg Replace Acceleration with angular velocity (gyro data in deg/s) Sample time is 0.01 s  timescale units change to 0.1s Total simulated time  20 s (instead of 200 as shown)

16. copyright 2011 controltrix corpwww. controltrix.com Measured velocity noisy data (True velocity is smooth sine wave of amp 10, period 20 s/ 10 cycles  (2 s for our handheld system)

17. copyright 2011 controltrix corpwww. controltrix.com velocity estimation error (v^ - v) vs time

18. copyright 2011 controltrix corpwww. controltrix.com error = v^ – v vs time

19. copyright 2011 controltrix corpwww. controltrix.com 3 Man months including rigorous testing Excluding hardware/firmware mods for adding mag Timeline

20. copyright 2011 controltrix corpwww. controltrix.com Velocity estimation techniques using sensor fusion MEMS -Accelerometer MEMS gyro Appendix

21. copyright 2011 controltrix corpwww. controltrix.com

22. copyright 2011 controltrix corpwww. controltrix.com Consider a vehicle moving Desired to measure the velocity accurately Velocity is directly measured but is noisy Acceleration also measured using onboard accelerometers Integrating acceleration data gives velocity Offset errors in acc./random walk cause drift in velocity Standard solution Kalman filter with optimal gain K for sensor data fusion Estimate by combining velocity and acc. Measurement Objective

23. copyright 2011 controltrix corpwww. controltrix.com Acceleration and velocity are measured using noisy sensor Direct velocity measurement is noisy (  v  m/s) Acceleration is measured with  a = 0.1 m/s 2 offset = 0.2 m/s 2 (DRIFT) Superposed sine wave drive Amplitude A = 3 m/s 2, frequency f = 0.05 Hz Sample time Ts = 0.1 s Simulated time = 200s - 400s

24. copyright 2011 controltrix corpwww. controltrix.com Measured velocity noisy data (True velocity is smooth sine wave of amp 10, period 20 s)

25. copyright 2011 controltrix corpwww. controltrix.com No matrix calculations Easier computation, can be easily scaled Equivalent to Kalman filter structure (easily proven) No drift (the error converges to 0) Estimate accelerometer drift in the system by default Drift est. for calib. and real time comp. of accelerometers

26. copyright 2011 controltrix corpwww. controltrix.com Can be modified easily to make tradeoff between drift performance (convergence) and noise reduction Systematic technique for parameter calculations No trial and error Advantages.

27. copyright 2011 controltrix corpwww. controltrix.com Sl NoMetricKalman FilterModified Filter 1.Drift Drift is a major problem (depends inversely on K) Needs considerable characterization.(Offset, temperature calibration etc). Guaranteed automatic convergence. No prior measurement of offset and characterization required. Not sensitive to temperature induced variable drift etc. 2.Convergence Non-Zero measurement and process noise covariance required else leads to singularity Always converges No assumptions on variances required Never leads to a singular solution 3.Method Two distinct phases: Predict and update. Can be implemented in a few single difference equation or even in continuum.

28. copyright 2011 controltrix corpwww. controltrix.com Note: The right column filter is a super set of a standard Kalman filter Sl NoMetricKalman FilterModified Filter 4.Computation Need separate state variables for position, velocity, etc which adds more computation. Highly optimized computation. Only single state variable required 5. Gain value /performance In one dimension, K = process noise / measurement noise. dt ‘termed as optimal’ Gains based on systematic design choices. The gains are good though suboptimal (based on tradeoff) 6.Processor req. Needs 32 Bit floating point computation for accuracy and plenty of MIPS/ computation Easily implementable in 16 bit fixed point processor 40 MIPS/computation is sufficient

29. copyright 2011 controltrix corpwww. controltrix.com velocity estimation error (v^ - v) vs time

30. copyright 2011 controltrix corpwww. controltrix.com error = v^ – v vs time

31. copyright 2011 controltrix corpwww. controltrix.com MEMS - Micro electro-mechanical systems Simplest MEMS devices possible, consisting of little more than a cantilever beam with a proof mass (also known as seismic mass). Under the influence of external accelerations the proof mass deflects from its neutral position. This deflection is measured in an analog or digital manner. MEMS ACCELEROMETER

32. copyright 2011 controltrix corpwww. controltrix.com MEMS ACCELEROMETER.

33. copyright 2011 controltrix corpwww. controltrix.com Most commonly, the capacitance between a set of fixed beams and a set of beams attached to the proof mass is measured. This method is simple, reliable, and inexpensive. Integrating piezo-resistors in the springs to detect spring deformation, and thus deflection, is a good alternative For very high sensitivities Quantum tunneling is also used; this requires a dedicated process making it very expensive. MEMS ACCELEROMETER..

34. copyright 2011 controltrix corpwww. controltrix.com Most micromechanical accelerometers operate in-plane, i.e. they are designed to be sensitive only to a direction in the plane of the die. By integrating two devices perpendicularly on a single die a 2-axis accelerometer can be made By adding an additional out-of-plane device 3-axes can be measured. Such a combination may have much lower misalignment error than 3 discrete models combined after packaging. MEMS ACCELEROMETER...

35. copyright 2011 controltrix corpwww. controltrix.com MEMS GYROSCOPE Almost all reported micro machined gyroscopes use vibrating mechanical elements (proof-mass) to sense rotation They have no rotating parts that require bearings, and hence they can be easily miniaturized and batch fabricated using micromachining techniques All vibratory gyroscopes are based on the transfer of energy between two vibration modes of a structure caused by Coriolis acceleration

36. copyright 2011 controltrix corpwww. controltrix.com Coriolis acceleration is an apparent acceleration that arises in a rotating reference frame and is proportional to the rate of rotation MEMS GYROSCOPE.

37. copyright 2011 controltrix corpwww. controltrix.com MEMS GYROSCOPE.. In general, gyroscopes can be classified into three different categories based on their performance: inertial grade, tactical - grade, and rate-grade devices. Tuning fork gyroscopes contain a pair of masses that are driven to oscillate with equal amplitude but in opposite directions. When rotated, the Coriolis force creates an orthogonal vibration that can be sensed by a variety of mechanisms.

38. copyright 2011 controltrix corpwww. controltrix.com The Draper Lab gyro, figure 2, uses comb-type structures to drive the tuning fork into resonance, and rotation about either in- plane axis results in the moving masses to lift, a change that can be detected with capacitive electrodes under the mass. MEMS GYROSCOPE...

39. copyright 2011 controltrix corpwww. controltrix.com MEMS GYROSCOPE…. Vibrating-Wheel Gyroscopes have a wheel that is driven to vibrate about its axis of symmetry, and rotation about either in-plane axis results in the wheel’ s tilting, a change that can be detected with capacitive electrodes under the wheel, Figure 3. It is possible to sense two axes of rotation with a single vibrating wheel.

40. copyright 2011 controltrix corpwww. controltrix.com Wine Glass Resonator Gyroscopes. A third type of gyro is the wine glass resonator. Fabricated from fused silica, this device is also known as a hemispherical resonant gyro. Researchers at the University of Michigan have fabricated resonant-ring gyros in planar form. In a wine glass gyro, the resonant ring is driven to resonance and the positions of the nodal points indicate the rotation angle. The input and output modes are nominally degenerate, but due to imperfect machining some tuning is required. MEMS GYROSCOPE.....

41. copyright 2011 controltrix corpwww. controltrix.com