1. Ethical and Epistemic Values in Assistive Technology Evidence (EVALUATE)
Leah McClimans
Associate Professor, University of South Carolina
ASSISTID Marie Curie Fellow, University College Cork

2. Aim: Provide an ethical-epistemic analysis of (some of) the evidence for assistive technologies (AT)
Justification: EBM represents established methodologies to improve decision-making about health care, some areas of health care thrive under EMB, others find it a more challenging fit. EBM is a challenging fit for the AT of intellectual disabilities (ID) and autism spectrum disorder (ASD)
Thesis: Some have argued that for AT we need to balance rigor with relevance. This project adds a science and values perspective to this argument: all evidence involves the use and application of values. Rather than ”balance” rigor with relevance, we need to understand rigor in terms of relevance.
EVALUATE

3. Assistive Technology (AT): products whose primary purpose is to maintain or improve individual function, independence and well- being
Examples for ASD and ID:
Assistive dogs
The Picture Exchange Communication System (PECS)
Voice Output Communication Aids (VOCAs)
Virtual reality (VR)
Robots
Multisensory rooms
Some Terminology

4. Some Terminology: Autism Spectrum Disorder
Autism Spectrum Disorder: umbrella term used to group a range of brain development disorders
Diagnosis:
Persistent social communication and interaction deficit in multiple context
Restricted, repetitive patterns of behavior, interests, or activities
Prevalence: 1-1.5% of the population
Currently can be reliably diagnosed by 2 years old
Some Terminology: Autism Spectrum Disorder

5. Some Terminology: Intellectual Disability
Intellectual “learning” Disability (ID): Deficits in intellectual and adaptive functioning
E.g. difficulties with reasoning, problem solving, planning, exercising judgment, learning, social skills, language development, etc., activities of daily living
Diagnosis
IQ of 70 or below
Observations of the child
Interviews with parents
Prevalence: 1-2% of population
Currently diagnosed by 18 years old
ASD and ID occur together in 56% of cases
Some Terminology: Intellectual Disability

6. Some Important Facts ASD and ID AT Evidence of AT
are “low incidence” disorders
have very broad diagnostic spectrums
often exist with other co-morbidities
populations are stigmatized even as other disabled populations are targeted for inclusion AT
5-15% of the people who need AT have access
access is a human right (WHO)
limited access in high income countries because ATs are rationed or very often not included in health and welfare schemes
Evidence of AT
poor evidence base: what works and how to implement successfully
E.g. What Works Clearinghouse doesn’t include Evidence Based Policies for ID
Some Important Facts

7. Evidence Based Challenges
Research Design
Experiments: random allocation or control group
Random allocation may not be possible/ethical, may not have control group, e.g. assistance dogs
Quantity of Research
Evidence from multiple studies
Low incidence disorder + broad diagnostic spectrum
Methodological Quality
Eliminating bias, employing measures with sound psychometric properties
Some kinds of “bias” are part of the phenomena to be studied
Magnitude of Effect
Effect size
How big of an effect size is needed for meaningful change?
Evidence Based Challenges

8. Case Study: Assistance Dogs

9. Assistance Dogs for ASD
Study Design
Cannot randomize
No natural control group
Quality of Research
Some children/families will not meet ”prescription” requirements
Significant barriers to eligibility
Methodological Quality
Need to want a dog
For the ”technology” to work, dog and handler need to form epistemic partnership
Almost infinite number of idiosyncratic factors can affect this formation
Magnitude of Effect
Successful outcomes vary widely
Assistance Dogs for ASD

10. Evidence Base for Assistive Dogs for ASD
Thus far:
Qualitative studies
Cross-sectional study
Quasi-experimental study
Systematic review indicates preliminary “proof of concept” but need more rigorous research to establish convincing evidence base (O’Haire, ME, 2013)
A critical review of the evidence argues for better research designs and larger sample sizes (Berry A, Borgi M, Francia, N, et al. 2013)
Proposal for an observational cohort study at UCC does not get funded by HRB because “no control group”
Evidence Base for Assistive Dogs for ASD

11. Evidentiary Marginalization
Need better evidence
ASD and ID Populations are problematic
Evidence base is inferior
AT are often not included in national health and welfare schemes
Access to AT is limited
Evidentiary Marginalization

12. Shifting focus Too much focus on internal validity of methodologies
Too much focus on eliminating certain kinds of bias from measurement
Still an emphasis on “off the shelf” results
What if we als0 focus on the epistemic and ethical values that methodologies employ?
Use values in conjunction with the, e.g. technologies, and their context to justify use of specific methods
Pay attention to value laden methodological decisions as decision points
Shifting focus

13. Values in EBM All methods employ values
Experimental designs and mainstream psychometric measures:
Standardisation (of bodies, of information)
Generalisability
Impartiality
Distributive or economic justice
“High quality” evidence is ableist
AT methods are often characterized by:
Individualisation
Diversity
Relationship
Social justice
Values in EBM

14. How do values affect evidence?
Maya Goldenberg, “On evidence and evidence-based medicine”
Ross Upshur, “A call to integrate ethics and evidence”; Upshur et al “Meaning and measurement: an inclusive model of evidence in health care”
Mike Kelly et al “The Importance of values in evidence- based medicine”
The role of values in selecting methods for identifying and appraising research evidence.
Nicholl’s et al “The need for ethics as well as evidence in evidence-based medicine”
Values in the selection and evaluation of evidence
Analyses that explore values in guideline development
How do values affect evidence?

15. How do we implement values in evidence construction?
What does this look like?
Clinical ethics?
Implementation science?
The study of questions, methods and x to promote the systematic inclusion of values into routine practice of evidence construction and application to improve the quality and effectiveness of health care?
How do we implement values in evidence construction?
the scientific study of methods to promote the systematic uptake of research findings and other EBPs into routine practice, and, hence, to improve the quality and effectiveness of health services.”