Designing JITAI for Skylar Maria Mayorga Associate Professor North Carolina State University NSF International Workshop on Dynamic Modeling of Health Behavior Change and Maintenance, Sept 8-9, 2015, London, UK

Baseline Data Physical information Weight, measurements, % body fat, BP Food consumption (food diary) Calories, nutrients, when Exercise/physical activity (diary, pedometer, wearable such as fitbit, heart rate monitor) Water intake Supplements or cigarette use? Fitness test

Baseline Data Reinforcements Sleep patterns Location (GPS) Stress (potentially through a wearable) Social media interactions Type of event/activity engaged in (e.g. dinner for work) Dieting history

Baseline Data Results from validated survey instrument on Self-confidence, body image Mood (depression) Belief regarding potential success Time should allow one to Detect patterns (day-to-day, weekly, monthly) Observe reaction to stressful events My thought is 1-3 months, but with very detailed data this may be too much Potentially repeated week long collections

Prediction Model (Long term) Several layers, or hybrid Agent-based model would allow one to capture interactions, especially if a cohort of individuals with connections Depending on the level of detail, some sort of “natural history” or biological model for calorie balance At a slightly higher level a Markov or partially observable Markov model Requires other models to extract information from data (potentially machine learning)

External factor or triggers, can be modeled as events Biological model Input- calorie intake, physical activity Output-expected weight loss/gain Self-efficacy Input- state in theoretical model Output- extract transition probabilities “Health Status” model Can we categorize skylar’s current behavior into some trajectory, e.g. (low physical activity but eating well and confident) and extract transitions between states? L,L H,L L,H H,H Agent-based or discrete event Simulation model

Stochastic model of Skylar’s progression To model skylar’s progression, we define a “state” as skylar’s condition (e.g. Moderately active and eating well) and a “state transition” is a change in the his condition. The “state” focuses on the features of the Skylar’s condition, while the “state transition” provides the mechanism for structuring the patient's condition’s dynamic behavior. We can model the patient’s health using a continuous time semi-Markov process where we define health-state conditional Markov transition model as shown before. Transitions can depend on external factors, such as level of stress or confidence, at a population level, could depend on age, gender, race, etc.

Identifying Dynamic Interventions – Notification Setting A Markov Decision Process (MDP) model provides a framework for decision making under uncertainty defined by <S, A, P, R>: a finite set of states S, a finite set of actions A, the transition probability, P, characterized by a Markov chain defining movement between states as a function of the current state and action, and an immediate reward, R(s,a), associated with performing the action a in state s. In an MDP, a decision maker can influence the behavior of a probabilistic system through his/her actions as the system evolves through time The decision maker’s goal is to choose a sequence of actions which allow the system of interest to perform optimally by predetermined success criteria, such as minimizing the expected time for Skylar to reach his goal. An SMDP is an extension of an MDP in which the state transitions depend not only on the current state and action but also on how long the system has been in the current state

Side Notes

Types of Models/Questions to Answer Pre-intervention: Statistical models – what causes skylar to binge eat? Or what factors are associated with binge eating? Post-intervention: Need to test hypotheses, to build a model for the effect of the intervention on sever

Questions Are we interested in Skylar’s weight loss and other physical measurements or in his “journey” What time scale are we interested in? Can we gain from using information from several individuals like skylar, to incorporate variability uncertainty