1. Sufficient Markov Decision Processes with Alternating Deep Neural Networks
Marshall Wang
Dept. of Statistics, NC State University
Advisor: Eric Laber
Aug 4, 2017
Reassure audience that the talk is self-contained. 30 sec.

2. Motivation
Want to apply mobile intervention to students with heavy drinking/smoking behavior
Hard to identify an optimal strategy when data are high dimensional and noisy
Need a dimension reduction that retains useful information
30 sec.

3. Contributions
Provided a criterion to measure the quality of a dimension reduction
Designed a deep learning model to produce a dimension reduction with no information loss
Demonstrated the method on a mobile intervention study
30 sec.

4. Sufficient Markov Decision Process Alternating Deep Neural Network
Outline
Sufficient Markov Decision Process
Alternating Deep Neural Network
Simulation Study
Application on Mobile Health
20 sec.

5. Sufficient Markov Decision Process
Alternating Deep Neural Network
Simulation Study
Application on Mobile Health

6. Markov Decision Process
1 min.

7. Markov Decision Process
2 min.

8. Sufficient Markov Decision Process
Mention why PCA is bad in this case. 2 min.

9. Sufficient Markov Decision Process
Alternating Deep Neural Network
Simulation Study
Application on Mobile Health

10. Deep Neural Networks (DNN)
30 sec.

11. Naive Dimension Reduction with DNN
Recall our criterion for a sufficient dimension reduction:
1.5 min.

12. Alternating Deep Neural Networks
Recall our criterion for a sufficient dimension reduction:
Remember to mention variable selection. 2 min.

13. Sufficient Markov Decision Process
Alternating Deep Neural Network
Simulation Study
Application on Mobile Health

14. Setup
1 min.
Additionally, add 200 noise variables including constants, white noise, and dependent noise.

15. Results
1 min.

16. Sufficient Markov Decision Process
Alternating Deep Neural Network
Simulation Study
Application on Mobile Health

17. Data
BASICS-Mobile is a mobile intervention targeting heavy drinking and smoking among college students.
Enrolled 30 students and lasted for 14 days.
On each afternoon and evening, the student is asked to complete a list of self-report questions. Then either an informational module or a treatment module is provided.
15 variables are collected, including baseline information, answers to self-report questions, weekend indicator and so on.
We’ll focus on smoking: Find an optimal intervention strategy to minimize cigarettes smoked.
1 min.

18. Dimension Reduction with ADNN
30 sec.

19. Dimension Reduction with ADNN
30 sec.

20. Questions? Thank you! lwang31@ncsu.edu
Main References:
Learning Deep architectures for AI, Y. Bengio, 2009.
Reducing the dimensionality of data with neural networks, G. E. Hinton & R. Salakhutdinov, 2006.
Neural networks and deep learning, M. A. Nielsen, 2015.
Reinforcement learning: an introduction, R. Sutton & A. Barto.
Brownian distance covariance, G. Szekely & M. Rizzo.
Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion, P. Vincent et al.
Conditional distance correlation, X. Wang et al.
Development and evaluation of a mobile intervention for heavy drinking and smoking among college students, K. Witkiewitz et al.
Thank you!