Using height sensors for biometric identification in multi-resident homes Vijay Srinivasan, John Stankovic, Kamin Whitehouse University of Virginia

Identifying (and locating) residents in smart homes Important problem for smart homes Elderly monitoring Home energy conservation Bedroom #2 Kitchen Cooking Bathroom Toileting Who is cooking? Who is toileting? Living Room The problem that we are addressing in this paper is how to identify and locate multiple residents living in smart homes. To illustrate, we show a smart home with an elderly husband and a wife, with the wife toileting in the bathroom, and the husband cooking in the kitchen. Assume we have instrumented the home with motion sensors, that tell us if someone is currently in a room, and with contact sensors on daily objects, such as the flush and the microwave that tell us when they are being used. The problem we are trying to solve in this paper is to identify who is currently cooking, and who is currently toileting. This is an important problem for smart home applications. For example, elderly monitoring applications need to know who is visiting the toilet more, or who is sleeping more? Home energy conservation applications need to know who is currently in a room, or who is using more energy in order to respond appropriately. And there are numerous other smart home applications that will inevitably need this information in the future… Bedroom #1 Front Door

Existing solution space Tag and Track Strong Biometrics Floor sensing Cost, effort, aesthetics Inconvenient Activity patterns Before we go on to describe our proposed approach, we take a minute to discuss existing approaches and their shortcomings. Tag and track approaches track a resident by asking them to wear a tracking device, and use radio ranging or inertial sensing to track them. People also use strong biometrics such as fingerprint readers or retina scanners. However, these sensors are inconvenient to the user because they require the user to manually wear a device or identify themselves. Cameras can also be used, along with vision techniques, but this sensor is perceived to be invasive by residents for long term monitoring in homes. Floor sensors can be used to identify residents using their gait or weight, but this sensor requires high setup effort, cost, and also impacts home aesthetics Past studies have also used residents’ activity patterns based on passive sensors such as motion or switch sensors, but we observe that these techniques have too low an accuracy for our applications. Cameras Perceived to be Invasive Low accuracy

Height as a biometric Go Motion Ultrasonic ranger Ultrasonic distance sensor mounted above doorways to measure height Cheap, convenient, non-invasive Height as a biometric Generally considered weak Especially for large populations (school, restaurant …) In this work, we explore the use of resident height as a biometric. We use an ultrasonic sensor mounted above the doorway to measure the height of a resident as they pass through. Our solution is cheap … However, height is generally considered a weak biometric, since many residents in a large population can have the same height. We found that … Such a large height difference means that our solution will not work …

However, in homes … Height sensing is an excellent biometric A home Has few residents (2-4) Allows for improved biometric accuracy using spatio-temporal continuity of resident movement 95% wife 5% husband Bedroom #2 Kitchen Cooking Bathroom 40% wife 60% husband Toileting Living Room Biometric error Filtered by tracking However, in a home with only a few residents, it is more likely that our approach will perform very accurately, since it is more likely that all residents Also, in a home, residents move through rooms in predictable ways constrained by the floor layout. Suppose the initial state is … Suppose then that … But we see a misclassification … However, suppose then that … Now, we see an event that suggests that it is resident … Bedroom #1 Front Door 3.25 cm difference gives 99% accuracy 7 cm difference gives 99% accuracy

Contributions Biometric solution with ultrasonic height sensors Improve accuracy with in-home tracking Perform comprehensive evaluation Controlled studies, In-situ studies, National applicability study Potential to achieve 95% identification accuracy in 95% multi-resident elderly homes in US Enable convenient, non-invasive smart homes

Rest of the talk Height sensor setup Evaluation of biometric accuracy Improvement using track history Conclusion and Future work

Height sensor setup Ultrasonic range finder above doorway Go Motion Ultrasonic ranger Ultrasonic range finder above doorway Distance from travel time of echo Reports only minimal distance (to head rather than shoulder) 50 KHz pulses cone divergence

Biometric identification Temporal clustering Height = default - min(Cluster) Given n resident heights, and probabilistic error models Resident identity = MLE default

Rest of the talk Height sensor setup Evaluation of biometric accuracy Improvement using track history Conclusion and Future work

Public height data from ~2100 multi-resident homes Evaluation Overview Goals: Evaluate biometric accuracy Across a large sample of real subjects In-situ environments At a nationwide scale 20 subjects in the lab 3 homes for 5 days each Public height data from ~2100 multi-resident homes

Evaluation in the lab 20 graduate student subjects True heights measured Stand Walk 42 times. Vary Width of doorway (75 , 90 cm) Direction of walking Speed 90 cm

Biometric accuracy For walking Log normal error distribution High mean error due to apparent height (3.1 cm) Low standard error (1.45) Log normal error distribution Accuracy sufficient? For homes? 2-4 residents For offices, schools? More residents … Maximum likelihood labeling Extrapolate to national height data set Homes Office/School 100 Random samples of subjects for each n

In-situ evaluation Deploy Ground truth resident location Height sensors on doorways 3 homes for 5 days Motion sensors in rooms A few contact sensors Ground truth resident location Radio ranging with Motetrack tags Height sensor Contact switch Motetrack beacon Motion sensor

In-situ evaluation Compute room visits by temporal clustering Room visit labeling accuracy for Height sensing – ML labeling Activity patterns from binary sensing (STAR particle filter) Discuss results Height sensing very accurate in our test homes Activity patterns based on binary sensing not as accurate

Nationwide evaluation Public data set with heights and weights of residents 2107 multi-resident homes (US) For each resident in each home, extrapolate error distribution for Height (log normal) from empirical study Weight (normal) from Jenkins et al, Pervasive 2006 Analytically compute probability of correct Maximum Likelihood labeling 85% homes at 95% accuracy!

Rest of the talk Height sensor setup Evaluation of biometric accuracy Improvement using track history Conclusion and Future work

Using track history Simulation study of home with n residents HMM to generate room transitions Height sensor noise from empirical error distribution 1000 height events generated Use multi-hypothesis tracking approach to compute best sequence of height sensor assignments 95% wife 5% husband Bedroom #2 Kitchen Cooking Bathroom 40% wife 60% husband Toileting Living Room Biometric error Filtered by tracking Bedroom #1 Front Door 3.25 cm difference gives 99% accuracy 7 cm difference gives 99% accuracy

Improvements 95% homes at 95% accuracy! Apply simulation study to 2107 multi-resident homes

Rest of the talk Height sensor setup Evaluation of biometric accuracy Improvement using track history Conclusion and Future work

Conclusions Biometric height sensor solution for identification in homes Potential to achieve 95% identification accuracy in 95% multi-resident elderly homes in US Enable new generation of convenient, non-invasive smart home apps Energy saving, entertainment, passive elderly health …

Open issues and future work Location granularity Activities in same room? Multi-path noise from adjacent height sensors Multi-modal identification Laser rangefinders for height, width Color sensors, … Power efficient design Trigger ultrasonic using PIR? Breakdown scenarios? Guests, crutches …

Current status Time Distance reading in mm In Out Direction classification – 100% accuracy for this example

Thank you Questions, feedback?