1. Generating and sharing large datasets: Moving out of our measurement comfort Rita Kukafka and Pamela M. Kato October 16-17, 2012 Bruxelles, Belgique

2. Why this is important Takes advantage of technological capabilities to capture and store and analyze large amounts of health behavior data Takes advantage of technological capabilities to capture and store and analyze large amounts of health behavior data From sensors, mobile technology, etc. From sensors, mobile technology, etc. Cloud computing Cloud computing Capture and store a multitude of data streams to represent simultaneously contextual factors, as well as individual level factors Capture and store a multitude of data streams to represent simultaneously contextual factors, as well as individual level factors Behavior change interventions can be adaptive in response to emerging patterns and contexts Behavior change interventions can be adaptive in response to emerging patterns and contexts

3. Examples Ecological Momentary Assessment Data Ecological Momentary Assessment Data Data automatically connected via blood glucose monitors, blood pressure monitors, scales Data automatically connected via blood glucose monitors, blood pressure monitors, scales Web data collected daily Web data collected daily Data collected semiannually in extended longitudinal studies Data collected semiannually in extended longitudinal studies Thank you, Runze Li: http://methodology.psu.edu/media/2012_SRNT/Li_SRNT.pdf

5. Statistical Analysis Challenges Complex data structure Complex data structure Data collected at irregular time points within and between subjects Data collected at irregular time points within and between subjects Covariates can vary over time (negative affect) and/or be constant (gender) Covariates can vary over time (negative affect) and/or be constant (gender) Ordinary linear statistical approaches are not appropriate Ordinary linear statistical approaches are not appropriate

6. Practical Challenges Expertise Expertise Inadequate knowledge to plan data collection and ability to analyze the data Inadequate knowledge to plan data collection and ability to analyze the data Not knowing where to find appropriate expertise (not knowing you need to work with one) Not knowing where to find appropriate expertise (not knowing you need to work with one)

7. Research Challenges Causality Causality correlational, non-experimental, post-hoc analyses, atheoretical correlational, non-experimental, post-hoc analyses, atheoretical Reliability and validity Reliability and validity Were data collected in the same way at each site? Were data collected in the same way at each site? Is the data clean or noisy? How can we tell? Is the data clean or noisy? How can we tell? Some principles may be ignored Some principles may be ignored such as choosing a representative sample such as choosing a representative sample Selecting data that is driven by behavior change theory and models Rita-What is meant by “data” Selecting data that is driven by behavior change theory and models Rita-What is meant by “data” Need theory and specialists in behavior change to contextualize and offer insights into data Need theory and specialists in behavior change to contextualize and offer insights into data Integration across heterogeneous data resources Integration across heterogeneous data resources logistical as well as analytical challenges logistical as well as analytical challenges

8. Addressing Challenges Need for psychometricians and experts in analyzing complex data Need for psychometricians and experts in analyzing complex data Need for collaboration across disciplines and distances Need for collaboration across disciplines and distances Need the right metrics to measure outcomes Need the right metrics to measure outcomes Focusing on what matters to the end users Focusing on what matters to the end users patient oriented outcomes patient oriented outcomes Usability issues Usability issues

9. Exploring Opportunities I Expertise Expertise Sharing experts and expertise Sharing experts and expertise Promoting the role that behavioral scientists play Promoting the role that behavioral scientists play Directory of experts??? Where? Directory of experts??? Where? Use framework/manual for non-experts Use framework/manual for non-experts End Users End Users Sharing with end users – require models that can be opened up for inspection so that the user can see how the data collected has represented his or her progress and misconceptions Sharing with end users – require models that can be opened up for inspection so that the user can see how the data collected has represented his or her progress and misconceptions Listening to the end user (patient/consumer) and meeting their needs (what do patients value and want) Listening to the end user (patient/consumer) and meeting their needs (what do patients value and want)

10. Exploring Opportunities II Linking data, people, technologies Linking data, people, technologies Cloud capabilities Cloud capabilities Creating a community site where standards are debated, agreed on, shared between researchers (behavioral scientists, statisticians, etc.), end-users, care providers and technology specialists Creating a community site where standards are debated, agreed on, shared between researchers (behavioral scientists, statisticians, etc.), end-users, care providers and technology specialists Use of communication technologies (video conferencing, Google docs) Use of communication technologies (video conferencing, Google docs) Ensuring interoperability of technologies across platforms and devices Ensuring interoperability of technologies across platforms and devices

11. Any other ideas?? Thank you! Thank you!