Sufficient Markov Decision Processes with Alternating Deep Neural Networks Marshall Wang (lwang31@ncsu.edu) Dept. of Statistics, NC State University Advisor: Eric Laber Aug 4, 2017 Reassure audience that the talk is self-contained. 30 sec.

Motivation Want to apply mobile intervention to students with heavy drinking/smoking behavior Hard to identify an optimal strategy when data are high dimensional and noisy Need a dimension reduction that retains useful information 30 sec.

Contributions Provided a criterion to measure the quality of a dimension reduction Designed a deep learning model to produce a dimension reduction with no information loss Demonstrated the method on a mobile intervention study 30 sec.

Sufficient Markov Decision Process Alternating Deep Neural Network Outline Sufficient Markov Decision Process Alternating Deep Neural Network Simulation Study Application on Mobile Health 20 sec.

Sufficient Markov Decision Process Alternating Deep Neural Network Simulation Study Application on Mobile Health

Markov Decision Process 1 min.

Markov Decision Process 2 min.

Sufficient Markov Decision Process Mention why PCA is bad in this case. 2 min.

Sufficient Markov Decision Process Alternating Deep Neural Network Simulation Study Application on Mobile Health

Deep Neural Networks (DNN) 30 sec.

Naive Dimension Reduction with DNN Recall our criterion for a sufficient dimension reduction: 1.5 min.

Alternating Deep Neural Networks Recall our criterion for a sufficient dimension reduction: Remember to mention variable selection. 2 min.

Sufficient Markov Decision Process Alternating Deep Neural Network Simulation Study Application on Mobile Health

Setup 1 min. Additionally, add 200 noise variables including constants, white noise, and dependent noise.

Results 1 min.

Sufficient Markov Decision Process Alternating Deep Neural Network Simulation Study Application on Mobile Health

Data BASICS-Mobile is a mobile intervention targeting heavy drinking and smoking among college students. Enrolled 30 students and lasted for 14 days. On each afternoon and evening, the student is asked to complete a list of self-report questions. Then either an informational module or a treatment module is provided. 15 variables are collected, including baseline information, answers to self-report questions, weekend indicator and so on. We’ll focus on smoking: Find an optimal intervention strategy to minimize cigarettes smoked. 1 min.

Dimension Reduction with ADNN 30 sec.

Dimension Reduction with ADNN 30 sec.

Questions? Thank you! lwang31@ncsu.edu Main References: Learning Deep architectures for AI, Y. Bengio, 2009. Reducing the dimensionality of data with neural networks, G. E. Hinton & R. Salakhutdinov, 2006. Neural networks and deep learning, M. A. Nielsen, 2015. Reinforcement learning: an introduction, R. Sutton & A. Barto. Brownian distance covariance, G. Szekely & M. Rizzo. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion, P. Vincent et al. Conditional distance correlation, X. Wang et al. Development and evaluation of a mobile intervention for heavy drinking and smoking among college students, K. Witkiewitz et al. Thank you! lwang31@ncsu.edu