1. Implementing PROMIS CAT
Michael Bass
Northwestern University

2. Disclosure
I disclose that I have no relevant financial relationships with commercial interests.

3. Settings Turnkey solutions (services, applications) Assessment Center
PROMIS iPad apps
Integration solutions
Assessment Center API
Research Setting
Clinical Setting (EHR Integration)
EHR Vendors (i.e. EPIC)
mHealth
HealthMeasures demo app
In preparing for this talk, I thought what story or analogy I could create in order to highlight the connection between PROMIS content and the software that is used to administer and score the content.
So I will group the software that administers CAT into three categories based on the resources and software development need in order to implement, will assign a traffic light.
First I’ll talk about the software that is ready to use of off the shelf software or services. GREEN
Then we have the Integrated Solutions that need some resources in order to implement, but there is a defined workflow in place to implement YELLOW
Finally I’ll discuss mobile applications, but focus on the creation of customizated of mHealth solutions. RED

4. Turnkey solutions (services, apps) Assessment Center PROMIS iPad apps
No Assembly Required
Turnkey solutions (services, apps)
Assessment Center
PROMIS iPad apps
I will focus on 2 Off the shelf software applications or services that are available to researchers that can be used with no extra technical/ IT resources.

5. Assessment Center
Web-based data collection platform for self- reported measures (PROMIS, NIH Toolbox, Neuro- Qol, Ascq-me)
Fees for use will be assessed starting as early as Jan. 2017
nformation_ pdf
Our story begins with Assessment Center. This software service was created hand-in-hand with the development of PROMIS content.
PROMIS is based on a strong psychometric foundation tied to Item Response Theory.
In order to accomplish this we need software to perform the IRT-based algorithms during selection and scoring of PROMIS assessments.
So the question becomes, at what level do we provide the software.
--Do we provide just the algorithms and ask researchers to code them
--Do we provide software that only administers and scores PROMIS content and stop there
--Or do we provide a complete protocol solutions.
We came to the conclusion that we needed to provide as much software support as possible to potential users in order for this new measurement system to gain traction.
During the initial PROMIS grant submission I remember specifically creating the appendix for the software detailing how researchers would be able to
log-in and create a study,
Create or select instruments
Define study protocols
Administer assessments to participants
Export the assessment results.
This is the back story of Assessment Center.
Assessment Center has been funded by the NIH for 12 years, but the subsidized are ending so we have to start charging fees for cost recovery purposes.

6. PROMIS iPad app
Designed for Research and Clinical use. Assisted data collection.
Administration of Performance measures and self report instruments.
Functionality
Participant Management
On the fly instrument selection and administration
Export Results
Assessment center is an online tool, There are some situations, usually in clinics where online connectivity is not available.
This is where our second off the shelf product comes into play. The PROMIS iPad App. It was released about a month ago and can be downloaded from iTunes.
It is designed for research and clinical use with the assistance of research staff. You need someone to register the participant and select the instruments you want before handing the iPad over to the participants.

7. PROMIS iPad app workflow
Participant management
Selecting instruments
Immediate data collection
Data exports
So for the PROMIS iPad app, the workflow is the following

8. Some Assembly Required
Integration solutions
Assessment Center API
Research Setting
Clinical Setting (EHR Integration)
EHR Vendors (i.e. Epic)
Some assembly required. The words that strike fear in all parents. But don’t worry. There are people who have done most of the heavy lifting already.

9. Banks and Calibrations
Assessment Center API
PROMIS
Banks and Calibrations
2. Get Next Item
1. Load Assessment
(Fatigue CAT)
3. Process Item
4. Get Results
PROMIS CAT Algorithm
In general, an API is a about taking a process a packaging in such a way as to simplify and hide the details so that other software could run that process without needed to worry about all the details inside.
For us, the process is the administration of a PROMIS CAT. So we packaged all the PROMIS content including the banks and calibrations with the algorithms for selecting and scoring and exposed them in to 4 calls.

10. Research Setting - REDCap
REDCap – Research Electronic Data Capture in 1,700 installations around the globe
The first example I will discuss is REDCap.
Vanderbilt, approached us around 2012 and were interested in including PROMIS content within REDCap.
We had just started this development and they were able to complete the coding with very few issues.

11. REDCap - Shared Instrument Library
Here is an example of how instruments are selected from their shared library.
Once an instrument is selected, it is then available to be included into your study protocol.

12. REDCap - PROMIS Item
Here is an example of how REDCap displays PROMIS items.

13. API - Clinical Setting (EHR Integration)
Oberd – Software as a Service (SaaS)
Through custom development – at University medical Utah, at U Washington, at Rochester etc…
The next examples of API users are the clinical settings which then send data to electronic health records.

14. EHR Vendors for PROMIS CAT
Architecture Overview
Fees/Licensing
The last API example would be EHR vendors like Epic.
We recently collaborated with them to embed the ability to administer CATS directly from their tool set.
This will be available in their 2016 release scheduled for later this year.

15. Architecture Overview
Tools
Cache Database
Browser
PROMIS
CAT Algorithm
The only point I want to make regarding their architecture approach is that as you can see, the CAT algorithms are behind their clinical and patient tools.
With this approach they can use PROMIS CATS in any of their tools, whether it be My Chart tools, kiosk, or bedside tools.
MyChart
Web Server

16. Annual Fee per Epic Licensee
Epic Integration and Consulting
Unlimited use of all measures (no per measure, per patient, per doctor or per staff fees)
API support
Support for as a doctor and patient resource
PROMIS will be available to EPIC customers through annual fees to licensee.
It will cover costs for the following services.

17. HealthMeasures demo app
Construction Zone
mHealth
HealthMeasures demo app
Finally we get to the collaborative solutions which I think fit well with mobile Health solutions.
In order to highlight how I see PROMIS CAT assessments within mobile applications, I created a demo app called HealthMeasures which available on both Google Play and iTunes.

18. HealthMeasures Demo App
System Requirement
Communicate with server-based content authoring tool
Communicate with server-based data storage
Administer PROMIS
Provide user feedback and assessment reminders
User can control data access

19. Protocol authoring tool & server-based data storage
System Requirements
Server-based system
Expose content through an API
Availability of PROMIS
Exposes data transmission through an API

20. Participant workflow Here are some screen shots from the app.
A user first selects the assessment they would like to take.
In this app, there are two brief protocols available . An emotional distress assessment that consists of the PROMIS depression and anxiety CATs the other protocol consists of a fatigue and physical functioning CAT.
Once you select the protocol, you are presented with the CATS.
From there you can review you assessment results.

21. Researcher Workflow Select CAT instruments from REDCap library
Generate REDCap API Token
1. Create PROMIS CAT Assessment protocol
Distribute QRCode to participants
2. Generate QRCode for protocol
Scan QRCode
Available on iTunes and Google Play
3. Health Measure mobile app
For researchers to customize the app. The workflow is as follows.
Create a study protocol in REDCap.
Ask participants to download the app onto their mobile phone.
Distribute a QRCode .
The purpose of this app is to highlight the components needed to administer mobile CAT assessments.
What I would hope for is to find collaborators who maybe have interventions or Ecological Momentary Assessment studies that would like to use CATS as their tool for measuring the effectiveness of their interventions,

22. Thank you
Questions?
Comments