1. Disseminating, Tracking and Evaluating New Knowledge in P&O
Jennifer L. Flagg
Center on Knowledge Translation for Technology Transfer
School of Public Health & Health Professions
University at Buffalo (SUNY)
Funded by NIDRR, US Department of Education, PR# H133A060028

2. Three KT Tools
Need to Knowledge (NtK Model) Knowledge Value Mapping (KVM) Level Of Knowledge Use Survey (LOKUS)

3. Focus of Need to Knowledge Model
The NtK Model is relevant to government sponsors and grantees of Research & Development projects which are expected to create technology-based Innovations, capable of generating beneficial socio-economic impacts, and do so in the near term future. The NtK Model is not relevant to government sponsors or grantees of basic or inquiry-driven “R&D” projects, with no explicit intent to generate socio-economic impacts, nor expectations for application in any specific field or in any defined timeframe.

4. Three Methods & States

5. R&D for Innovation Each Method has own rigor and jargon.
Actors are trained and operate in one method and over-value that method.
Academic & Government sectors dominate policy at expense of Industry.
Methods are actually inter-dependent.

6. NtK Model Assumptions
Socio-economic impacts start with a validated need, recognized by stakeholders, addressed through delivery of innovations via market mechanisms.
Industry is customer for R&D outputs due to ability to design & deploy market innovations in short term.
Three different methods (R/D/P) create knowledge outputs in three different states (Discovery, Invention, Innovation), each with unique value.
Decision to adopt/implement knowledge rests with recipient stakeholders not with the producers.

7. Schematic.

8. “Gamification” of Technological Innovation
Progress through three Methods of Knowledge Generation, and the effective Communication of three Knowledge States, may be circuitous and iterative, punctuated and prolonged, risky and unpredictable, yet still be planned, implemented and accomplished through the deliberate and systematic efforts of key stakeholders.

9. NtK Model Value Technology Grantees: Program Sponsors: Organizations:
Proposal structure – Review Panel liked.
RERC Tech Transfer/ SBIR Phase II Plans.
Program Sponsors:
Assess proposals; Track progress.
Compliance enforced – Funding continuation?
Organizations:
PDMA’s “The Source”; Tech Transfer Tactics;
CIHR; CEUD; DIT; ATIA; AAATE.

10. Knowledge Value Mapping Questionnaire
Reaching Target Audiences efficiently and effectively

11. New Requirements Sponsors & Grantees tasked with:
Communicating findings to non-traditional audiences.
Demonstrating evidence of knowledge use.
New unfunded mandates to:
Translate findings into appropriate language and formats.
Identify channels for communication.

12. Rationale for KVM Reach diverse and non-traditional audiences.
Communicate findings efficiently and effectively under current constraints.
Employ broker organizations with appropriate membership.
Understand how each values research to properly tailor message.

13. AT: Six Stakeholder Groups
Researchers (Scientist & Engineer)
Clinicians (Therapist/Educator/Counselor)
Consumers (PWD & Family Member)
Manufacturers (OEM & VAR)
Policy Implementers (government/agency/ program administrator)?
Brokers (attorney/employer/consultant)

14. Knowledge Value Mapping Study
Multiple Comparative Case Studies
National Organizations -AAC Stakeholders
ATIA – Manufacturers
ASHA – Clinicians
ISAAC – Consumers
AHEAD – Brokers
OSERS – Policy Implementers
RESNA – Cross-Stakeholder (Pilot)
Assistive Technology Industry Association
American Speech-Language Hearing Association
International Society for Augmentative and Alternative Communication
Association of Higher Education and Disability
Office of Special Education and Rehabilitative Services
Rehab Engineering and AT Society of North America

15. The KVM Questionnaire explores six ways in which national organizations may interact with new knowledge generated through scientific research:
1) Creating Knowledge: Conducting research internally or funding others to do research for your organization;
2) Identifying Knowledge: searching for research findings that have already been produced by others;
3) Translating Knowledge: paraphrasing research findings to make them more relevant and understandable;
4) Adapting Knowledge: interpreting research findings to improve their fit within your organization’s context;
5) Communicating Knowledge: disseminating or demonstrating research findings through various media;
6) Using Knowledge: applying research findings to situations within your organization or membership;

16. Question #1. Relative to other activities, how frequently does your organization engage in Creating Knowledge through Research activity? That is, conduct or perform your own research or pay/fund others to do research for you?
For what purpose are you conducting research or funding research performed by others?
Who conducts the research?
Who are the main intended users of the research knowledge your organization creates?

17. Probing Questions regarding KVM:
Question #7. Please describe any incentives that your organization uses to encourage your internal associates or members to become aware of, or apply new research-based knowledge.
Question #8. How does your organization measure the levels of awareness, interest or application of new knowledge among your memberships? What is being measured in each case?
Question #9. What percentage of your members have education/training in a research field equivalent to a Masters or Doctoral degree?
Question #10. Can you identify or suggest any ways in which researchers could help your organization facilitate the flow of knowledge from them as the sources, through your organization and out to your members?

18. KVM Results
All surveyed national organizations seek, review and use research results internally.
All communicate research via electronic means, while some use formal journal/conference outlets.
Those with internal expertise adapt findings to context – but all respect author’s original intent.
All use incentives to attract member attention – webcasts, CEU’s, certificates, content advisors.
National organizations can serve as effective mediators and translation/dissemination networks.

19. Ranking importance across various types of knowledge use
Important
Very
Moderately
important
importance
Of little
Unimportant
Not applicable
To create or revise industry standards or clinical protocols is …
AHEAD
ASHA
OSERS
RESNA
ATIA
ISAAC
To build laboratory instruments or clinical tools is …
To create freeware (hardware, software) for free download or access is …
Designing new or improved commercial devices or services is …
For other purposes is …
–Promote the AT field
–Inform policy or practice

20. KVM Results

21. KVM Results Recommendations for researchers Increase engagement!
“Translate” from research jargon to practical terms
Explain the findngs and implications, and give them a call to action
What? So what? Now what?
Distribution ready formats

22. LOKUS Level Of Knowledge Use Survey
No existing instrument fit study purpose.
Created LOKUS Questionnaire for web-based self-report (VOVICI).
Five Levels; each containing multiple types, dimensions and activities.
Psychometric analysis shows LOKUS to be valid and reliable for measuring change in level of knowledge use.

23. Purpose of the KT intervention studies
Problem: Sub-optimal level of demonstrated impact from R&D investment, so OMB mandates Federal programs demonstrate evidence of uptake & use.
Solution: NIDRR selected Knowledge Translation as model and method to generate evidence.
Challenge: Identify KT best practice models that are :
Effective: increase K use by relevant stakeholders;
Feasible : easy to implement; and
Useful: K producers (technology grantees) can document evidence of impact from their project outputs
Purpose: Develop and evaluate KT intervention strategies that are feasible for use by technology R&D projects and effective in increasing use of new knowledge by potential users.

24. Relevance of LOKUS
Sponsors & Grantees seeking to demonstrate evidence of knowledge use by stakeholders.
Compare strategies for communicating knowledge.
Differentiate between “Levels” of knowledge use:
Non-awareness to Awareness (Conceptual)
Awareness to Interest (Motivational)
Interest to Use (Action)
As intended As Modified
Appropriate for All Stakeholders.

25. LOKUS Survey – 4 Levels/5 Types
Non-Awareness
Awareness
Interest
(Orientation & Preparation)
Intended Use
(Initial & Routine Use)
Modified Use
(Collaboration, Expansion, Integration, Modification)

26. Method
Map values of user categories for tailoring material to their needs and interests.
Measure baseline awareness and use of all innovations among a sample of knowledge users from each of six categories.
Divide sample of user into three conditions: 1) Full KT intervention, 2) Standard KDU, 3) Control.
Measure post-intervention awareness and use of all innovations among sample.

27. Research Design T1 O X1a X1b T2 X2 C Five Stake- Holder Groups
Baseline Assess-ment
Intervention Delivery
(4 Mo.)
Follow/up Test 1
Follow/ up Test 2
Five Stake- Holder Groups T1 O
X1a
X1b
Five Stake-Holder Groups T2 X2 C
Where T1 = group exposed to TTDK; T2 = group exposed to TDK; C = Control group;
O = Observation (via LOKUS); X1a and X1b are components of TTDK method; &
X2 = TDK method.

28. KT Intervention Results LOKUS Use Types 1 - 5
Treatment
Pre-Test Mean
(S.D.)
Post 1 Mean
Post 2 Mean
Difference
α ≤ .05
x2 (p)
Post-hoc Test
α ≤ .0167
Z (p)
T1 – KT
(N = 72)
1.22
(.68)
1.79
(1.16)
1.69
(1.03)
(<.001)
Pre vs Post 1
3.826 (<.001)
Pre vs Post 2
4.297 (<.001)
T2 – KD
1.26
(.77)
1.76
(1.19)
1.74
(.001)
3.330 (.001)
3.206 (.001)
Control
(N = 63)
1.38
(.97)
1.51
(1.05)
1.63
(1.22)
6.484 (.079)

29. Conclusions T1 and T2 strategies effective. Generalizable?
Can lead a horse to water, but…
When they are ready, they will sip~
Ensure tailored information is available
Ensure information is easy to access
Reminders!

30. Key Take Aways
NtK Model useful for planning R&D projects when socio-economic impact is the goal.
KVM provides insights regarding how to reach stakeholder groups.
LOKUS can be used to determine uptake and use of new knowledge.
Information about all tools and projects available at:

31. Questions?

32. U.S. Department of Education.
ACKNOWLEDGEMENT This is a presentation of the Center on Knowledge Translation for Technology Transfer, which is funded by the National Institute on Disability and Rehabilitation Research, U.S. Department of Education, under grant #H133A
The opinions contained in this presentation are those of the grantee and do not necessarily reflect those of the
U.S. Department of Education.