1. Pose Recognition using Kinect for Home Care System
Zihang Huang
CS2310
Project presentation

2. Goal Detect human being in a room using Kinect
Segment each parts of the body, such as torso, hands, head
Diagnose symptoms based on video stream
Falling detection
Identify hand gestures to decide data flow

3. System Overview 3D skeleton Diagnose Symptom frames OpenPose status
…….
…….
…….
OpenPose

4. Component Diagram Falling Detection Index Cell M2 M3
Pose Estimation index cell
Status Index Cell M0 M1 M4
Diagnose Symptoms Index Cell M5
Frame message(M0) is sent to pose estimation index cell generating 3D skeleton message(M1). Status index cell sends initial status(M2 and M5) to both falling detection index cell and diagnose symptoms index cell. Falling detection index cell will return detection result(M3) back to status index cell. When diagnose symptoms index cell detects symptom, it will send symptom message(M5) to status index cell updating current status.

5. Emergency Officer(EO)
Diagnose Symptoms T1 T2
Emergency Officer(EO) T3 P0 P5 P1 P2 P3 T4
T1: Headache
T2: Heart Attack
T3: Stomachache
T4: Left Knee Pain
T5: Right Knee Pain
P0: initial status
P1:Symptom diagnosed
P2: Show symptom text
P3: Send message to EO
P4: Confirm gesture
P5: message fetching P4 T5

6. Emergency Officer(EO)
Falling Detection T1
Emergency Officer(EO) P0 P5 P1 P2 P3 T2
T1: Movement velocity per 10 frames
T2: The distance between the middle point of the body and ground P4
P0: initial status
P1: Falling detected
P2: Show “falling detected” text
P3: Send message to EO
P4: Confirm gesture
P5: message fetching

7. Headache demo
-headache

8. Headache Demo
-confirm

9. Headache Demo
-cancel

10. Falling Demo

11. Falling Demo

12. Demo

13. Conclusion
So far, this system can detect five symptoms, headache, heart attack, stomachache, left and right knee pain. And each of components are able to run individually.
A valid symptom detection requires a specific hand confirmation gesture which is a designed fist gesture.
Walking around or jumping in the room will not affect falling detection.

14. Future Work Explore multiview application
Improve detection accuracy and precision
Integrate face emotion detection to diagnose symptoms
Record symptom times, duration and frequency and apply deep learning to predict potential risk of diseases

15. Reference
Paper:
[1] Cao, Z., Simon, T., Wei, S. E., & Sheikh, Y. (2016). Realtime multi-person 2d pose estimation using part affinity fields. arXiv preprint arXiv:
[2] Simon, T., Joo, H., Matthews, I., & Sheikh, Y. (2017). Hand Keypoint Detection in Single Images using Multiview Bootstrapping. arXiv preprint arXiv:
GitHub:
OpenPose
OpenKinect
OpenCV
My project repository