3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia and Networking Lab The University of Texas at Dallas

Contents Motivation Previous Work Current Work – Extracting 2D feature data (MHI) – Classifying human motions

Motivation Cleaned Semantic Data Easy to get, but Quite noisy 3D MOCAP data Video Human Motion data Recognizing Video Human Motion +

HMM Modeling T … t MHI K-means (WEKA) 2D Motion Shape data 3D Motion Capture data Observation Sequence data Hidden state-transition Sequence data Quantization Which Motion? Forehand, Backhand, Smash, Left kick, Right Kick, Left punch, Right punch Test data 3D Motion Capture Assisted Video Human Motion Recognition Enhancement

Current Work The system for falling-down detection of elderly or patient at home Lower layered HMMs with 3D motion capture data are to estimate one of atomic activities (e.g. movement of human hip portion) Higher layer recognizes exactly the falling- down motion with much longer time granularity

Layered HMM HMM (A) HMM (B) (Baum-Welch) 2D Feature Vector Horizontal direction Up direction Down direction Normal Action Abnormal Action (Falling down) Classification Results Position of human hip Movement directions

Background Techniques Extracting 2D feature (Computer Vision) – Motion History Images (MHI) Classification (Machine Learning) – Hidden Markov Model (HMM) – Layered Hidden Markov Model (LHMM)

Motion History Images Keywords: – Motion Energy Image (MEI) – Motion History Image (MHI) – 2D Image feature data with suggestion of possible actions.

Motion History Images Motion Energy Image :- – Describes the motion energy for a given view of action – Spatial distribution of motion – WHERE

Motion History Images Motion History Image :- – Pixel intensity – HOW the spatial distribution has occurred

Motion History Images MEI MHI WHERE HOW 2D Image Feature Date Suggestion of Possible actions

Motion History Images Reference Paper :- – Hierarchical Motion History Images for recognizing Human Motion.

Project - Face detection using HAAR like Features & AdaBoost algorithm deals with the application of one of the four AdaBoost algorithms in boosting the classifiers based on the paper "Robust Real Time Face detection by viola & jones“ OpenCV Visual C++ Available Source files: Face detection using available HAAR like Features. PreRequisites: Basic knowledge of using OpenCV library. Knowledge on AdaBoost(Adaptive Boosting) – A Machine Learning Algorithm. Other References:

Project – Contour detection using Background Subtraction and Edge Detection Techniques OpenCV Visual C++ Available Source files: Reading a video file. PreRequisites: Basic knowledge of using OpenCV library. Other References: Introduction to opencv programming - notes/opencv-intro/opencv-intro.html

Thank you Myunghoon Suk Ashok Ramadass