 Fall Detection Nicholas Chan (EE) Abhishek Chandrasekhar (EE) Hahnming Lee (EE) Akshay Patel (CmpE) 1

Elderly Fall Statistics  16,000 elderly Americans die from falling each year (CDC, 2005)  300,000 elderly Americans have hip fractures each year  90% of hip fractures result from falls  24% of elderly Americans who suffer hip fractures die within one year  40% of elderly women with hip fractures never walk unassisted again (National Osteoporosis Foundation) 2

Proposed Solution  Two camera system executing custom algorithm: 1. Detect person in room 2. Perform statistical analysis of person’s motion 3. Determine if a fall has occurred 4. Send an alarm for help  Projected cost of $500 per room 3

Target Market 4 Smart Hospital RoomsNursing Homes & Clinics  Our solution offers to reduce injuries arising from falls and to improve safety records at nursing homes and hospitals.

Alternative Solutions  Pressure-sensitive mats by the bed  Camera detection with optical flow algorithm  RFID Solutions  Accelerometers (e.g., iLife ™) 5

Alternative Solution Problems  Pressure sensitive mats have unavoidable edges that can cause falls  Optical flow analysis prone to errors arising from shadow artifacts  Potential EMI interference from RFID readers; RFID readers also very expensive (over $1000)  Accelerometer results in many false positives (e.g. a person sitting down quickly) 6

Technical Specifications  Two webcams (Microsoft VX 6000)  Resolution of 160x120 pixels  Video recorded at 15 frames per second  Personal Computer to run algorithm:  Intel Pentium Dual Core 2.5GHz Processor  3GB RAM  Standard Keyboard and Mouse 7

Camera Positioning  Privacy is a major concern  Gaining maximal coverage from camera position is also critical  A balance between these two must be achieved 8

Camera Positioning Maximal Coverage Head-level Camera High-level camera Coverage Area 9

Camera Positioning Maximal Privacy High-level camera Knee-level camera Coverage Area 10

Algorithm Overview 1. Identify the region of an image occupied by the person 2. Ascertain the velocity of the person’s motion 3. Fit an ellipse to the person 4. Analyze the changes in the ellipses’ properties 5. Determine if a fall has occurred 11

Foreground Segmentation  The background of every frame is subtracted  Statistical Gaussian model is generated for each pixel  HSV color space is used to minimize shadow effect  Pixels are labeled as either foreground or background based on a preset threshold  A binary foreground image is thus generated 12

Foreground Segmentation 13 Foreground Segmentation

14 Foreground Segmentation

Largest Blob Detection  Additional filtering is performed on the foreground- segmented image  The largest continuous cluster of pixels is detected and then isolated from the smaller clusters of noise 15

Largest Blob Detection Blob Detection 16

Motion History Imaging 17  Filtered foreground-segmented image data used to form Motion History Image (MHI)  MHI used to quantify the velocity of the person’s motion  0 (zero velocity) ≤ C motion ≤ 1 (extreme velocity)

Motion History Imaging Swiftly Walking ( Medium C motion ) Turning Around ( Low C motion ) Falling ( High C motion ) 18

Elliptical Approximation 19 Frame 1 Normal Walking Frame 150 Mid-Fall Change in Ellipse Angle

Frame 120 Normal Walking Frame 150 Mid-Fall Change in Eccentricity 20 Elliptical Approximation

High Frequency Noise Possible Fall Elliptical Approximation

Statistical Analysis  Falls result in: 1) high-velocity motion (high C motion values) and 2) large statistical variance in elliptical orientation/eccentricity  Numerically, we define a fall is defined by: C motion > 0.65 and σ θ > 0.60  These thresholds may vary slightly with camera position 22

Statistical Analysis 23 C motion > 0.65 σ θ > 0.60

Call for Assistance  Computer connected to Ethernet network  When fall happens a picture is taken  A fuzzy picture is stored to a local server  An updating intranet page is displayed at the nurse station  The page incorporates archiving features  Nurse analyzes picture and determines if a response is necessary 24

Call for Assistance UI 25 Page refreshes every 5 seconds to check for screenshot on the server

Call for Assistance UI 26 When a fall occurs a flashing red message along with a screenshot is displayed

Archiving Falls 27  The shot can be archived with a date stamp onto the local server  The detected fall log shows a queue of falls that happened  On archiving and reloading the system shows normal status again

Results  Results are based on evaluation of 30 falls and 20 non-falls 28 Category% Success% Failure Falls %16.66 % Non-Falls75 %25 %

Problems and Solutions  Hardware and Software Problems:  MATLAB requires substantial memory to execute programs  Algorithm has difficulty accounting for auto-light adjustments by the webcam  Solutions Proposed:  Port existing algorithm to C++ in order to run it more efficiently; using C++ also removes the licensing hassles required with MATLAB  Light intensity can be normalized with histogram equalization techniques; alternatively use a webcam without light adjustment 29

Real-Time Analysis  Existing Problems:  MATLAB is incapable of running threaded applications  Analysis and recording of video simultaneously is almost impossible as a result  Solution:  Use C++; Supports threading and memory management  Real time analysis is available via OpenCV library  Many MATLAB functions are implemented in the library 30

Privacy Concerns  Use of cameras brings in a major privacy concern  Different configurations are necessary for concealment  Terms & Conditions have to be included in hospital paperwork  The picture taken of the patient upon a fall is blurred  An option of not having the system on should be implemented if requested by the patient 31

Cost Analysis  Assuming a rate of $28/hr, Engineer salaries would amount to $44,800 for 4 engineers during a 10 week development phase  Equipment Cost:  $60 for two cameras  $270 for a modern Dell Inspiron 530  $170 Installation and Software Costs  Total Cost per Room = $500 32

Future Improvements  Enable support for multiple people  Improve speed of algorithm  Reduce false positives by making a self-learning system  Make the program standalone for easy deployment  Enable mainframe support for hospital with servers 33

Questions?  16,000 Americans die from falling each year  300,000 elderly Americans have hip fractures each year  24% elderly Americans who suffer hip fractures die within one year 34 Category% Success% Failure Falls %16.66 % Non-Falls75 %25 %