The NIH Toolbox: Assessment of function across the lifespan Nathan Fox, Ph.D University of Maryland This study is funded in whole or in part with Federal funds from the Blueprint for Neuroscience Research, National Institutes of Health under Contract No. HHS-N C

PURPOSE OF THE NIH TOOLBOX The NIH Toolbox, part of the NIH Neuroscience Blueprint initiative, seeks to develop brief yet comprehensive assessment tools measuring motor, cognitive, sensory, and emotional function. Upon completion, the Toolbox will be available for use in longitudinal epidemiologic studies and prevention or intervention trials for people ages 3-85.

ADDITIONAL OBJECTIVES OF THE TOOLBOX Enable cross-study comparisons and integration of data from multiple studies by providing a standard set of brief, well-validated measures Dynamic/adaptable over time Utilize state-of-the-art psychometric and technological approaches including computerized adaptive testing (CAT) and computer assisted evaluation

ADDITIONAL OBJECTIVES OF TOOLBOX Cover the full range of normal function (not disease states) Be used as the basis for detecting at-risk populations Be minimally burdensome to subjects and investigators

COGNITIVE DOMAIN Executive Function Episodic Memory Working Memory Processing Speed Language Attention

SOCIO-EMOTIONAL HEALTH DOMAIN Negative Affect Psychological Well-Being Stress & Self-Efficacy Social Relationships

MOTOR DOMAIN Locomotion Strength Non-Vestibular Balance Endurance Dexterity

SENSORY DOMAIN Vision Audition Vestibular Balance Somatosensation Taste Olfaction

SUB-DOMAINS OF COGNITION AND INSTRUMENTS Episodic MemoryPicture Sequence Memory Executive Function- Cognitive FlexibilityDimensional Change Card Sort Executive Function –Inhibitory ControlFlanker Language-Vocabulary ComprehensionPicture Vocabulary Comp Adap Processing SpeedPattern Comparison Working MemoryList Sorting Language- Reading Oral Reading Recognition

10 Working Memory: Complex Span Task

DOG HORSE

RABBIT

THREE ITEM FLASHING PRACTICE

RABBIT

SHEEP

ELEPHANT

Language: Reading

cucurbitaceous forisfamiliate prevarications circumlocutory imparidigitate

is

art

dodecahedral

Attention/Executive Function: Choice Reaction Time (Flanker)

Look at all the fish!

Let’s practice! Try to focus on the + sign in the middle of the screen. Press the green light when you’re ready to start

Processing Speed: Pattern Comparison

28 Executive Function: Dimensional Change Card Sort (DCCS)

Executive Function: Self-Ordered Pointing Task

Now look at these pictures. Press the one you want to pick

36 Language: Vocabulary

37 Executive Function: 3 to 6 Years

38 Executive Function: 8 to 85 Years

The initial NIH toolbox contract included assessment of children (starting at age 3) Identification of domains and measures included those that could be used with children as young as age 3 (for the most part) NIH directors approved the Pediatric Option---additional funding for the norming and validation of measures with children Domain managers and consultants identified measures that can be used with young children (for the most part) Committee organized to identify principles for pediatric assessment (chaired by Nathan Fox) Pediatric Option: A Brief Review

Marilyn Jager Adams, Ph.D. Brown University Patricia Bauer, Ph.D. Emory University David Blitz, M.A. CORE Helena Correia CORE Scott Debb, M.A. CORE Nathan Fox, Ph.D. University of Maryland Richard Gershon, Ph.D. CORE Jean Berko Gleason, Ph.D. Boston University Roberta Golinkoff, Ph.D. University of Delaware Kathy Hirsh-Pasek, Ph.D. Temple University Jin-Shei Lai, Ph.D. CORE GiglianaMelzi, Ph.D. New York University Kathleen Wallner-Allen, Ph.D. Westat Phil Zelazo, Ph.D. University of Minnesota Nick Zill, Ph.D. Westat Pediatric Working Group Members CORE: Center on Outcomes, Research and Education

Importance of having an examiner present Clarity of instructions Importance of practice trials to insure that children understand task Importance of feedback after practice trials and during task performance Importance of engaging child and motivating performance Pace of assessment to be monitored Principles of Pediatric Assessment

Age-Appropriateness of Instrument Construct Appropriateness Instructions Equipment Child’s Perspective Flexibility Cross-domain Issues with Pediatric Assessment

Age-Appropriateness of Instrument – Can a 3-year-old do it? If not, are there minor adaptations to make it appropriate? Construct Appropriateness – Will it be valid? Will it measure the same construct that the task measures for an adult? Is it a meaningful construct for children? Instructions – Are the instructions clear for young children? – Can they be repeated? Should they be administered automatically (via voice or on screen) or by examiner? Cross-domain Issues with Pediatric Assessment

Equipment – Appropriately-sized and comfortable to perform best – Child-size chairs, tables Feet flat on the ground when sitting back in chair Can reach comfortably to respond – Non-threatening Cross-domain Issues in Pediatric Assessment

Child’s Perspective – Will it be fun, scary? – Will I be able to do it? – Will it be boring? – What do I get out of it? Cross-domain Issues in Pediatric Assessment

Flexibility:Tasks, equipment, computer-interfaces must be flexible ---by design. – One size does not fit all. Children are Less predictable than adults Less capable of regulating behavior and attention to task Less able to remember or understand instructions – Be prepared, anticipate behavior, but expect the unexpected. Schedule adaptations will be necessary. Children may need to go back to a particular screen or task They may need breaks during the testing session. Cross-domain Issues in Pediatric Assessment

Flexibility:Tasks, equipment, computer-interfaces must be flexible ---by design. – To optimize a child’s performance Must actively create an environment that supports the child Must be sensitive to the needs of the child Must keep the child engaged and motivated Cross-domain Issues in Pediatric Assessment

– Flexibility Schedule, interface, and examiner need to be flexible to be sensitive to the child and to try to optimize performance Obviously, flexibility must occur within pre-defined boundaries Balance between child’s needs, practical considerations and needs for standardization of administration. Flexibility in testing

Ergonomic Design – Type of equipment – “Workstation” (appropriately sized table, chair, screen at comfortable angle, comfortable reach to respond) Human-Computer Interface

Software Design -- Flexibility Important – Even with (or especially with) extremely complex and expensive tasks there is a need to balance practical and child considerations. – Need repeat or clarify instructions, pause in the middle of a task (as appropriate), easily move from one task to another, modify task order Human-Computer Interface

Software Design -- Flexibility Important – Consider: What is ideal on-task behavior? Vs. What is acceptable on-task behavior? – Design should allow for both, if possible. For example, ideally the subject should complete the battery of tasks in one sitting but it should be acceptable to take breaks. Children may be momentarily distracted---they may need to be re-oriented to the task instructions. Administrators may need to skip ahead or go back. Software should be flexible for these encounters. Human-Computer Interface

Voice – Standardized, prerecorded voice with appropriate reflection good, but children may need human interaction to keep them engaged and elicit optimal performance. – Build system so both are possible. – If prerecorded is not working/holding attention/problem for some other reason, then shift to human during administration. – Instructions presented with prerecorded voice should be available to the examiner if the examiner needs to repeat/clarify. Pacing – Needs to be adjusted to the needs of the child. Some want to move faster, others need slower, some need extra clarification in instructions Human-Computer Interface

Role of the Examiner – Need clear guidelines for how to interact with the participant— what kind of feedback/encouragement; when to intervene/not to intervene. When examiner can say something, what they can say, how often can they say it. – Trouble-shooting section to standardize the way the examiner interacts with the participant Role of the Examiner

Training – Crucial for standardizing administration, minimizing across-site differences in administration – Certification to ensure standardized implementation of protocols. – What materials will be provided? Manuals? Videos? Web-based? – Working with children is a skill. Training Examiners

Human interaction and eye contact Establishing rapport Smiling, enthusiasm, praise (appropriately placed) Make it fun Give the child some control Be respectful of child Provide breaks and snacks Child should feel success, not failure Incentives – Type Stickers, toys, visual and audio computer display – Frequency Along with way (points, stars) After finishing a task After finishing the visit Ways to Engage and Motivate

– Must standardize assessment Precisely define by task, how the examiner can interact with the participant (what can say, when, how often) Documentation, documentation, documentation (e.g., problem-solving or trouble-shooting section to provide guidance for handling common problems that arise). Training – need to minimize across site variation in administration. This is particularly vital while documentation is not fully developed. Standardizing Assessments

Consent form need to be clear – Different versions for different ages IRB review – There will be differences in requirements across institutions Who gives consent? – Parent/guardian – At what age can a child? Assent --- verbal vs. written? – At what age is this required? – Is verbal assent allowed? Until what age? Issues of Consent/Assent

The End