1. Mixture Modeling and Inference for Recognition of Multiple Recurring Unknown Patterns
Zeyu You, Raviv Raich,
Yonghong Huang (presenter)
School of Electrical Engineering and Computer Science
Oregon State University
Corvallis, Oregon
Intel Labs, 2111 Northeast 25th Avenue,
Hillsboro, Oregon 97124

2. Overview Motivation Single Pattern recognition
Multiple Pattern recognition
Results
Conclusions / future work

3. Motivation Previous work on Single template recognition
Target: Multiple template
recognition

4. Single Pattern Recognition
Approach:
Statistical Model
MLE for s:

5. Approach cont. Solve MLE by Robust Initialization for s*:
Finding position containing s
(Complete graph -> Bi-partite graph approximation)
Finding the estimated s
Refinement: MM solver for iterative updates on s

6. Multiple Pattern Recognition
MLE Approach:
Statistical Model
S->Sk
Unknown parameter:
Vector of template probabilities α
Templates {s1, s2 ,…,sk}
Hidden parameter:
Template indicator vector [K1,…,KN]
Position vector [J1,…,JN]

7. Solution Complete Data Likelihood function: Expectation Maximization
G(Xi/sk) is the ith bag PDF which has the same form as single pattern model PDF
Expectation Maximization
Robust Initialization
Majorization Minimization for Refinement

8. Expectation Maximization
E-step:
M-step:
Single template ML

9. Challenges Non-convex optimization for each M-step
No guarantees on global optimal solution
Sensitive to the initialization

10. Learning Signatures
Consider each activation template as a bag containing all possible delay windows
Map an item in each bag to one of K unknown patterns

11. Solution on Inner Iteration
Robust Initialization( ): where weight wik is
Find the k’th template
Refinement using Majorization-Minimization:
For each k, find delays:

12. Experiment Results Synthetic Data: Real World Data: MSE Analysis
Detection Error
Real World Data:
Similar AUC on the single pattern task
Increased AUC on the multiple pattern task

13. Synthetic Data: Template Estimation MSE
No. bags
Length
No of instances
K=3

14. Data Resources Appliance activation ground truth: Pecan Street Project
A research and development organization
Developing and testing
advanced technology,
business model,
and customer behavior
for advanced energy management system
Voltage measurement:
Intel’s BEST device
Source: Pecan Street Project

15. Experiments on Real-world Data
AUC of a single template model [ICASSP’14] vs. mixture model with K=1 [WCCI’14]
Note the proposed model for K=1 is trained on the same filter dataset.
Maintained the same performance

16. Real-world Data Detection
ROC changes significantly as K increases for some device, but not all devices:
Air conditioning ROC
Oven ROC

17. Test with different K
AUCs of mixture model by varying K
K-pattern model captures the variations in patterns.
AUC significantly increased as K=3 vs. single pattern model.
Over-fitting can be overcome using Cross-Validation.

18. Conclusions Single pattern Model: Mixture pattern Model:
Statistical single pattern model
Robust initialization
Refinement using MM
Mixture pattern Model:
Statistical K-pattern model
Problem and solutions
Experiment results: significant improvements over the single pattern model

19. Future Work Learning in the presence of outliers
Learning from large datasets
Trade-off: training time complexity for accurate templates
Transfer learning
Test on homes which are not included in the training data
Classification framework

20. Thanks !
Questions?

21. Gradient Descent
Steepest Descent
Newton’s