1. “Mapping while walking”
Pose Invariant Activity Classification
for Multi-floor Indoor Localization
Saehoon Yi1, Piotr Mirowski2,3, Tin Kam Ho2,4, Vladimir Pavlovic1
1Computer Science Department, Rutgers University
2Statistics and Learning Research Department, Bell Labs, Alcatel-Lucent 3Now at Microsoft Bing
4Now at IBM Watson
“Mapping while walking”

2. Outline Related work Indoor localization Pedestrian Dead Reckoning
GraphSLAM
Motivation
“Mapping while walking”
Methods
Pose-invariant sensor features
SVM classification of actions
HMM temporal smoothing
Results

3. Indoor Localization Indoor localization Various practical use
Radio Frequency (WiFi, 4G cell) maps “where am I in this building?”
Network deployment optimization: “where should we place that 4G cell in the building?”
No GPS
Smartphone sensors
Accelerometer
Gyroscope
Magnetometer
Barometer
WiFi, cell

4. Pedestrian Dead Reckoning
Step detection
on vertical acceleration
[Steinhoff et al., Pervasive Computing and Communications 2010]
3-axis orientation
[Madgwick et al., Rehabilitation Robotics 2011]
Trajectory updates
step length d, offset angle β
Subject to drift
due to orientation error
Sensor measurement noise
Magnetic field perturbations
xt-5
xt-4
xt-3
xt-2
xt-1
error at position
reset
xt+1 xt
direction of walk
angle offset beta β
Yaw of the phone w.r.t. horizontal X axis
longitude or human/horizontal X (“towards East”)

5. GraphSLAM Modify trajectory to minimize estimation error xj
[Grisetti et al., Transportation Systems Magazine 2010]
Challenges
Requires landmark detection
How do we know that two different observations are actually taken at the same location?
Hand-placed landmarks, e.g.,: QR code or NFC tag
Manually installed and maintained xj xi
𝒛 𝒊𝒋 ( 𝒙 𝒊 , 𝒙 𝒋 )
𝒆 𝒊𝒋 ( 𝒙 𝒊 , 𝒙 𝒋 )
xj*
zij

6. Motivation Detect and provide natural landmark for GraphSLAM
Stairs and elevators are accurately detected
They are non moving, distinct structures, which is ideal for landmarks
Classify human activities using a smartphone in the pocket
Pose invariant features extracted from smartphone sensors
Jointly infer activity and floor information
Focus on activities that incur floor change
“Mapping while walking”
Facilitate radio-frequency map building for network engineers

7. Methods SVM: Classify activity at each time point
HMM: Smoothing SVM activity classification and jointly infering floor
Activities
walking
stair up
stair down
stand still
elevator up
elevator down

8. Pose invariant features for IMU sensors
Pose invariant features from A
[Kobayashi et al., ICASSP, 2011]
Invariant to rotation R

9. Statistical features for barometer
Rotation does not affect air pressure
Fluctuate over time
Depend on weather and temperature
Detects ascending/descending air pressure

10. SVM classification
Features are extracted from sliding window of sensor observations
Linear SVM for 6 activity classes
Fast and storage efficient
Linear classification
Able to implement real time classification in Android OS
Provide activity probability
Platt’s scaling algorithm
Required to obtain HMM observation probability

11. HMM smoothing 6 activities for each floor
State transit for strong evidence
Smooth sporadic brief misclassification
Augment inference of activity with floor from Viterbi algorithm
Observation probability
from SVM confidence level(Platt’s scaling)
from mixture of Gaussian

12. HMM smoothing Transition probability
Manually design transition probabilities
Higher probability of transition to the same state
Floor changes only for stair and elevator

13. Experiment set up Input data Sensor data recorded at 50Hz
Feature extraction
Sliding window
IMU sensor features
Length: 64 frames
Step size: 35 frames
Barometer features
Length: 192 frames
Train data: seconds of each class repeatedly performed
Test data: 6160 seconds of 12 natural sequence

14. Activity classification result
For SVM,
Walking is confused to taking stairs
Standing still is confused to taking elevators
Leg dynamics are similar
Air pressure does not change over short period of time
Each sliding window is considered independently

15. Rectified HMM inference
Post rectification
HMM allows: walking → taking stair → walking on the same floor
Taking stairs w/o floor change → walking
Taking elevators w/o floor change → stand still

16. Activity classification result
HMM removes sporadic misclassification between walking and taking stairs
Rectification: rectifies stairs to walking when it does not incur floor change

17. Stability of HMM transition model
p: Probability of state transition
Fairly stable with p
HMM slightly degrades with high p
Lower probability of staying at the same state

18. Landmarks match Types of landmark Stair Elevator
GraphSLAM requires matching of the same landmark along the trajectory
Training phase
Obtain information from reference landmarks
WiFi access point visibility
Testing phase
Landmarks matching
Get current WiFi AP visibility
Calculate distance to reference landmarks
Take the closest corresponding landmark

19. Initial PDR trajectory
Initial trajectory obtained from PDR
Rotation angle underestimated for every turn
Need to be modified using GraphSLAM

20. Multi-floor GraphSLAM result

21. Conclusion
Rotation invariant features were able to capture different dynamics of motion activities
Our approach improves classification accuracy and jointly infers activity and corresponding floor information
GraphSLAM successfully modifies multi-floor trajectory using natural landmarks detected by our framework.

22. Q & A
Thank you