1. Bilge Mutlu, Andreas Krause, Jodi Forlizzi, Carlos Guestrin, and Jessica Hodgins Human-Computer Interaction Institute, Carnegie Mellon University Robust, Low-cost, Non-Intrusive Sensing and Recognition of Seated Postures

2. Why seated postures? Automobile Classroom Wheelchair Home Office

3. Using posture information Today’s talk

4. Pellegrini and Iocchi., 2006 Kinesthetic Motion-capture markers or conductive- elastomer-embedded fabrics Existing approaches

5. Kinesthetic Motion-capture markers or conductive- elastomer-embedded fabrics Vision-based Image sequences from a single camera or multiple cameras Tognetti et al., 2005 Existing approaches

6. Kinesthetic Motion-capture markers or conductive- elastomer-embedded fabrics Vision-based Image sequences from a single camera or multiple cameras Pressure-sensing-based Pressure readings from the seating surfaces Existing approaches Han et al., 2001

7. Poor generalization Good performance in classifying “familiar” subjects, poor performance with “unfamiliar” subjects due to high dimensionality. High cost High-fidelity pressure sensors are expensive. Slow performance Processing high-fidelity sensor data demands computational power, which leads to slow processing. Challenges Robust generalization Low-cost Near-real-time performance

8. Our solution Robust generalization Up to 87% accuracy in classifying 10 postures with new subjects. Low cost Using 19 pressure sensors instead of 4032. Reducing sensor cost from $3K to ~$100. Near-real-time performance 10Hz on a standard desktop computer Novel methodology Using domain knowledge and near- optimal sensor placement.

9. Methodology

10. Learning Algorithm Logistic Regression Sparse representation Cross-validation 10-fold, gender-balanced training and testing samples from different subjects Separate sets Training, testing, and reporting samples from 52 people in 5 trials Implementation in Java ✴ We would like to thank Hong Tan and Lynne Slivovsky for providing their data set for comparison. ✴

11. Understanding pressure data Modeling

12. Understanding pressure data Modeling

13. Understanding our data Modeling

14. Domain knowledge Modeling

15. Features Modeling Size and position of bounding boxes Distances to the edges of the seat Distance and angle to between bounding boxes Parameters of the ellipses that fit the bottom area Pressure applied to the bottom area

16. Features Modeling Classification accuracy

17. Separability test Modeling

18. Feature elimination Modeling

19. Methodology

20. Dimensionality Reduction Sensor granularity

21. Dimensionality Reduction Sensor granularity

22. How to place sensors? F, feature variables V, locations and granularities A subset A of V that maximizes information gain about F where H is entropy NP-Hard optimization problem We use near-optimal approximation algorithm Dimensionality Reduction IG(A;F) = H(F) - H(F | A) F V A ⊆ V

23. Near-optimal placement Dimensionality Reduction

24. Sensor placements Dimensionality Reduction

25. Near-optimal placement Dimensionality Reduction Classification accuracy

26. Methodology

27. Prototyping

28. Evaluation of prototype 20 naive participants 10-fold cross validation testing with %5 of the data 78% accuracy In classifying 10 postures 10 Hz real-time performance On a standard desktop computer

29. Methodology

30. Conclusions Generalizability Up to 87% (with a base rate of 10%) achieved with unfamiliar subjects. Low cost Higher classification accuracy than existing systems using less than 1% of the sensors. ~ $100 sensor cost compared to the commercial sensor for $3K (33 times reduction in price). Near-real-time performance At 10Hz on a standard desktop computer.

31. Applications Automobile Classroom Wheelchair Home Office

32. Future challenges Transferring learning across chairs A “transformation map” could be created Only static postures Temporal dimension needs to be considered The set of ten postures The set of postures should come from the activity Next Steps

33. Summary of Contributions A non-intrusive, robust, low-cost system that recognizes seated postures with generalizable, near-real-time performance. A novel methodology that uses domain-knowledge and near- optimal sensor placement strategy for classification. This work was supported by NSF grants IIS-0121426, DGE- 0333420, CNS-0509383, Intel Corporation and Ford Motor Company.

34. From Postures to Activities Reading the paper Watching TV Reading paperwork Watching TV + eating Sleeping Talking on the phone Reading a book Craftwork Reading the paper + watching TV Reading the paper + eating Next Steps