Bilge Mutlu, Andreas Krause, Jodi Forlizzi, Carlos Guestrin, and Jessica Hodgins Human-Computer Interaction Institute, Carnegie Mellon University Robust, Low-cost, Non-Intrusive Sensing and Recognition of Seated Postures

Why seated postures? Automobile Classroom Wheelchair Home Office

Using posture information Today’s talk

Pellegrini and Iocchi., 2006 Kinesthetic Motion-capture markers or conductive- elastomer-embedded fabrics Existing approaches

Kinesthetic Motion-capture markers or conductive- elastomer-embedded fabrics Vision-based Image sequences from a single camera or multiple cameras Tognetti et al., 2005 Existing approaches

Kinesthetic Motion-capture markers or conductive- elastomer-embedded fabrics Vision-based Image sequences from a single camera or multiple cameras Pressure-sensing-based Pressure readings from the seating surfaces Existing approaches Han et al., 2001

Poor generalization Good performance in classifying “familiar” subjects, poor performance with “unfamiliar” subjects due to high dimensionality. High cost High-fidelity pressure sensors are expensive. Slow performance Processing high-fidelity sensor data demands computational power, which leads to slow processing. Challenges Robust generalization Low-cost Near-real-time performance

Our solution Robust generalization Up to 87% accuracy in classifying 10 postures with new subjects. Low cost Using 19 pressure sensors instead of Reducing sensor cost from $3K to ~$100. Near-real-time performance 10Hz on a standard desktop computer Novel methodology Using domain knowledge and near- optimal sensor placement.

Methodology

Learning Algorithm Logistic Regression Sparse representation Cross-validation 10-fold, gender-balanced training and testing samples from different subjects Separate sets Training, testing, and reporting samples from 52 people in 5 trials Implementation in Java ✴ We would like to thank Hong Tan and Lynne Slivovsky for providing their data set for comparison. ✴

Understanding pressure data Modeling

Understanding pressure data Modeling

Understanding our data Modeling

Domain knowledge Modeling

Features Modeling Size and position of bounding boxes Distances to the edges of the seat Distance and angle to between bounding boxes Parameters of the ellipses that fit the bottom area Pressure applied to the bottom area

Features Modeling Classification accuracy

Separability test Modeling

Feature elimination Modeling

Methodology

Dimensionality Reduction Sensor granularity

Dimensionality Reduction Sensor granularity

How to place sensors? F, feature variables V, locations and granularities A subset A of V that maximizes information gain about F where H is entropy NP-Hard optimization problem We use near-optimal approximation algorithm Dimensionality Reduction IG(A;F) = H(F) - H(F | A) F V A ⊆ V

Near-optimal placement Dimensionality Reduction

Sensor placements Dimensionality Reduction

Near-optimal placement Dimensionality Reduction Classification accuracy

Methodology

Prototyping

Evaluation of prototype 20 naive participants 10-fold cross validation testing with %5 of the data 78% accuracy In classifying 10 postures 10 Hz real-time performance On a standard desktop computer

Methodology

Conclusions Generalizability Up to 87% (with a base rate of 10%) achieved with unfamiliar subjects. Low cost Higher classification accuracy than existing systems using less than 1% of the sensors. ~ $100 sensor cost compared to the commercial sensor for $3K (33 times reduction in price). Near-real-time performance At 10Hz on a standard desktop computer.

Applications Automobile Classroom Wheelchair Home Office

Future challenges Transferring learning across chairs A “transformation map” could be created Only static postures Temporal dimension needs to be considered The set of ten postures The set of postures should come from the activity Next Steps

Summary of Contributions A non-intrusive, robust, low-cost system that recognizes seated postures with generalizable, near-real-time performance. A novel methodology that uses domain-knowledge and near- optimal sensor placement strategy for classification. This work was supported by NSF grants IIS , DGE , CNS , Intel Corporation and Ford Motor Company.

From Postures to Activities Reading the paper Watching TV Reading paperwork Watching TV + eating Sleeping Talking on the phone Reading a book Craftwork Reading the paper + watching TV Reading the paper + eating Next Steps