MER Analytics & Visualizations: Clinical Cascade Breakout 2017 PEPFAR Data and Systems Applied Learning Summit MER Analytics & Visualizations: Clinical Cascade Breakout Good afternoon, everyone, and welcome to the Clinical cascade breakout of the MER Analytics and Visualization session. My name is …. If you made a mistake and intended to join the KP session, you can find it in room X. And the DREAMS/Prevention/OVC session can be found in room X. September 19th, 2017

Agenda Introduction ……………………………..10 minutes Clinical Cascade in Panorama………..20 minutes Additional platforms for review………..15 minutes Exercise Breakouts ……………………60 minutes HIV Testing Services 2nd 90 TB/HIV PMTCT For those of you who are in the right place, during this 2-hour session we are going to do a quick overview of clinical cascades, highlight existing platforms for their review, and then, most importantly and interestingly, break into smaller groups for an exercise in one of four program areas: Testing, Treatment, TB-HIV, or PMTCT.

Session Learning Objectives Increase participants’ understanding of existing MER clinical cascade analytics Strengthen participants’ skills in interpreting cascade results through use of Panorama, DATIM, or ICPI tools Develop follow-up analytic questions and identify programmatic actions based on data review The objectives of the session are to highlight existing clinical cascade analytics, strengthen interpretation of results through use of Panorama, DATIM, or ICPI tools, and to develop additional follow-up analytic questions or programmatic actions based on this review.

1. Overview and Analytic Considerations for Clinical Cascade So, a quick review of clinical cascades…

Section 1: Clinical Cascade Introduction Sequence of services needed to achieve the desired program impacts Clinical cascades are simply a sequence of services needed to achieve desired program impact. This graph, which you are likely familiar with, depicts our estimated global progress towards our 90-90-90 goals. To achieve epidemic control we must identify PLHIV, link them to treatment, ensure they adhere to treatment, and that they achieve and maintain viral suppression. Each of the steps in this cascade is critical to that outcome. PLHIV Diagnosed On ART Viral Suppression

Section 1: Clinical Cascade Introduction Not limited to traditional 90-90-90 Applies to cascade service delivery for variety of program areas Examples: TB, PMTCT, Key Pops But, of course, our clinical cascades are not limited to this very traditional, overarching 90-90-90 cascade, and applies to many of our program areas, including PMTCT, TB, and Key Pops. These programs (and their cascades) contribute to the attainment of PEPFAR and national targets and epidemic control through both prevention and treatment services. I’ll now go through several examples:

Section 1: Clinical Cascade Introduction TB-HIV Cascade This TB-HIV cascade shows a cascade of TB services provided to patients on ART, from TB screening to TB diagnosis to TB treatment start and completion. In the TB breakout session later, the exercise focuses on the PEPFAR TB/HIV cascade which includes both the TB prevention and TB treatment cascades within our treatment program.

Section 1: Clinical Cascade Introduction PMTCT-EID Cascade HIV - % of pregnant/BF women who know their status Among HIV +, % on ART % Virally Suppressed % of HEI who have been tested Mother Infant Our PMTCT cascades and programs try to ensure that all pregnant women are screened for HIV, positives are initiated and retained on treatment, they achieve viral suppression, and that their exposed infants are tested and monitored until the risk period has passed. The full PMTCT cascade is particularly challenging and interesting because it follows not only one but two individuals (mother and exposed infant) across multiple service delivery points, frequently across multiple registers, and over a period of 1-2 years.

Section 1: Clinical Cascade Introduction KP Cascade – Prevention and Treatment The KP cascade displayed here incorporates both a prevention and treatment cascade. Prevention is on top in blue and treatment is on the bottom in yellow. Once KPs are reached and tested, those who are HIV negative are provided with services to help them remain negative, while those who are HIV positive are provided with treatment services and monitored for adherence and viral suppression. Many of you may also be analyzing alternative prevention cascades through your DREAMS, OVC, VMMC and other prevention programs.

Section 1: Clinical Cascade Analysis Indicators associated with steps of cascade are reviewed together Allows us to: Measure HIV service system performance Identify program gaps and missed opportunities Target interventions to address these gaps Monitor burden and service coverage by age, sex, location, key pop, and other sub-populations May identify additional gaps Cascade analysis allows us to better understand the relationship between indicators and activities to strengthen interpretation and use. This integrated review can provide more information than the review of indicators independently. For example, reviewing new on treatment results and testing results together can provide a much more meaningful analysis than if we reviewed new on treatment alone. Clinical cascade analysis allows us to: Measure HIV service system performance Identify program gaps and missed opportunities Target interventions And monitor burden and service coverage by age, sex, location, key pop, etc. This is critical to help identify any additional gaps in program coverage and can trigger further analyses.

Section 1: National vs PEPFAR Cascades National Cascade: All PLHIV Provides full picture of program coverage Can be constructed at national or sub- national levels In PEPFAR, we often mention the “National” and “PEPFAR” cascades so I wanted to briefly review the difference between the two. National cascades refer to a comprehensive or complete coverage of the targeted population. These include ALL PLHIV and all services provided to them, not just those supported by PEPFAR. They can be constructed at both the national and SNU levels and often include a mix of both program and surveillance data. An example of this is the 2010 US HIV treatment cascade shown in the graph here. The PLHIV estimate is for the entire US and the cascade results are for the entire country, not just results supported by one partner. In contrast..

Section 1: National vs PEPFAR Cascades Built with MER or PEPFAR produced indicators only Limited to PEPFAR- supported geographic and program areas In contrast, the PEPFAR cascade is built only with MER or PEPFAR produced indicators and is limited to PEPFAR-supported geographic and programmatic areas. For example, in a district in a Rwanda, PEPFAR may only support 50% of facilities; therefore, a district review of ART coverage would be severely limited by the review of PEPFAR-supported results only. For today’s session, we will be focusing on PEPFAR cascades constructed by MER indicators because, unsurprisingly, that is what is routinely reported at HQ and we have access to. Within your own programs you may be able to analyze and use much more robust cascades depending on the data collected by your partners.

Section 1: Considerations for Interpretation Critical to understand value and limitations of data in cascade during analysis and use. Potential limitations: - Geographic or program coverage Data quality Client-level data may not be linked across steps ie MER Linkage Proxy - Note: HTS_TST does not represent ever diagnosed Of course, we cannot talk about data analysis and use without talking about limitations! It is critical to understand the data: what does it actually represent? There are several potential limitations with cascade data depending on a variety of factors. For example, the geographic or activity coverage of your program: Does the program provide support for only a portion of the cascade (for example only community testing)? This can make it difficult to effectively monitor the entire cascade from testing to treatment to viral suppression. Data quality, patient movement between facilities and across sub-national units, and our M&E systems all affect interpretation and use of results. Despite this, cascade analysis can still provide a very strong indication of performance and trigger additional analysis. There are a lot of nuances and we hope that the exercise breakout sessions will provide a great opportunity for valuable discussion on these points, so please ask questions and share your experiences during the upcoming breakouts. Can still provide a strong indication of program performance and trigger additional analysis

Section 1: Culture of Monitoring & Analysis Routine & timely reviews of cascade data Disaggregated by age, sex, geography, KPs, etc Monitor trends over time Trigger further analyses (“deep dives”) With all of that in mind, we – the collective PEPFAR “we” - are continuing to build and support a culture of monitoring and analysis, which should include: Routine and timely reviews, disaggregated by age, sex, geography, and sub-population. Should monitor trends over time to understand patterns, effects of interventions, etc. And trigger further analyses to help identify both best practices and gaps. These additional analyses may require reviewing data streams and sources outside of the routine cascade data.

Section 1: Review Questions Key questions to ask during your cascade reviews: Where are the leaks? Geographic & Programmatic Who is most affected by the leaks? Why are there leaks? How do we best address the leaks? What resulted from the actions taken? As you review cascade data both this afternoon and back home, consider the following questions: Where are the leaks? Both geographically and programmatically Who is most affected by the leaks? Adolescents? Female sex workers? Older men? Young women in urban areas? Why are there leaks? Is it related to training? Commodity availability? Stigma? Policies? How do we best address the leaks? And, once you do respond, what resulted from the actions taken? Did you see the expected results? Or do you need to try something else? What worked? What didn’t? While there may be leaks throughout the cascade, some areas may be more important or more feasible to address than others. Deciding where to focus quality improvement efforts is a critical step in interpreting and responding to cascade data.

2. Clinical Cascade Visualizations in Panorama Now, X will review the clinical cascade analytics that are available in Panorama. Thank you. Speaker Notes Provide detailed notes on the topic you will cover and how it fits in to the overall structure of the course. When writing speaker notes, please consider the following: What would someone need to know about this content if they are teaching it for the first time? What concepts are most important for the audience to understand on each slide? Where might the learners have some confusion about the content, and can you anticipate what that would be and how to remedy it? Are there any case studies, personal examples, or stories you’ve heard of certain experiences with the topic? If there is an exercise associated with this topic, how can you set it up so the learners know what to expect (i.e. “You will have the opportunity to practice [skill y] in an activity we will do later in the course.”) Where can learners go for further information about the topic? This will be expanded upon at the end of the deck, but feel free to mention resources throughout the session as reminders. Remember: repetition is not a bad thing!

Panorama Session Goals Understand where you can go in Panorama to find data and visualizations related to cascades Review the different components of the clinical cascade in Panorama Compare targets and quarterly results for indicators related to one another using available visualizations Notes Consider the questions below as you craft and present your content: How much do learners already know about what you are presenting? What are the key terms they need to know in order to understand a concept? Are you trying to teach them too much in the amount of time you have? Where can you provide real-world examples/case studies/story from your experience to highlight a learning concept? What kind of questions might learners ask about this content? Are you going to provide time for Q&A during or after your course, or both? How much text do you need on this slide to communicate the main idea without overwhelming the learners? How can you use visuals to support the learning concept? Can you use visual metaphors?

The data are visualized for you Why Use Panorama? The data are visualized for you Panorama is useful for showing recent trends over time You can find details about testing, treatment, TB, PMTCT, and the clinical cascade as a whole all on one site. Notes Consider the questions below as you craft and present your content: How much do learners already know about what you are presenting? What are the key terms they need to know in order to understand a concept? Are you trying to teach them too much in the amount of time you have? Where can you provide real-world examples/case studies/story from your experience to highlight a learning concept? What kind of questions might learners ask about this content? Are you going to provide time for Q&A during or after your course, or both? How much text do you need on this slide to communicate the main idea without overwhelming the learners? How can you use visuals to support the learning concept? Can you use visual metaphors?

Panorama calculations Panorama: Where Does the Data Come From? Data entry close HQ Data storage Panorama calculations View Pano! First, some very brief systems talk! You may remember from the Day 1 MER session that data flows through data entry in DATIM and is synchronized with the PEPFAR Data Hub (PDH). The PDH is a high-capacity data-storing warehouse that allows us to manage, and even apply calculations to, PEPFAR data. Twice per quarter, the systems team takes a “snapshot” of the data at its current state. That snapshot—or “frozen”– dataset is what feeds Panorama. Therefore, the data that you see in Panorama is: Sourced entirely from the PDH “Frozen” twice per quarter Approx. 1 week

Panorama: Home Page & Orientation Hopefully, you opened up Panorama yesterday in your MER session so you already understand the purpose and overall functionality of this tool. As a quick recap, you start on this homepage, where you get a quick snapshot of some high-level targets and results for certain indicators. You can also select “download files/links” for excel versions of the data found in Panorama (in the form of ICPI Fact View datasets).

Panorama: Clinical Cascade Visual

Panorama: Exercise 1 Visual Using the graph, how do the latest TX_NEW results compare to this time period last year in Kenya? Note that this is for the OU level! An OU analysis may hide ups and downs in program between SNUs. Discuss with your neighbor why you might be seeing this change—test and start? New partners?

Panorama: Clinical Cascade Age/Sex Disaggs This looks very similar on the onset, but note that the values you see depend on the filters you select. To make sure you’re selecting filters that make sense, look at the Disaggregate Completeness as Percentage of Numerator table. Notice that you have to select HTS and TX separately to find whether fine or coarse disaggs make more sense for the cascade as a whole.

Clinical Cascade: Key Pop Disaggs What do you notice about this tab? There is no 2016 data—Key Pop became a required disagg for clinical cascade indicators in 2017 under MER 2.0 There is no TX_CURR data—Key Pop is not a required disagg for TX_CURR We don’t have a reporting completeness chart for this disagg: That is because KP is not a required disagg for clinical cascade indicators: it is optional for country teams to fill out if it is safe to do so. Therefore, we do not include overall completeness for this indicator You may select multiple of the disaggs on the top selection menu to view these disaggs grouped in different ways. Mention concurrent KP session meeting at this time. If that’s your focus, get up and go to that room.

TX_NET_NEW Analysis Visual We navigated all the way back up away from the “clinical cascade” visuals to discuss another important part of the clinical cascade: the number of patients put net new on treatment.

TX Net New for Q3 is defined as: (Q3 TX_CURR) – (Q2 TX_CURR) Knowledge Check: What is Net New? TX Net New for Q3 is defined as: (Q3 TX_CURR) – (Q2 TX_CURR) Discussion for why we should use TX_NET_NEW, and why we should compare it to TX_NEW Note: You can see this example of how to calculate NET_NEW in the online reference sheet “Calculated Indicators” In our example, we have: Q3 TX_CURR = 1,030,207 Q2 TX_CURR = 1,011,712 Q3 – Q2 = 18,495

How Can I Find Partner/SNU Data? You can view cascade data by… FY and Quarter DSD/TA SNU 1-3 SNU Prioritization Agency Partner Mechanism …in the Partner/Mech Analysis Tab!

What’s Coming for the Clinical Cascade? Viral Load and Retention—part of the clinical cascade in Q4 You can find FY16 data for these indicators in the “Coverage Indicators” section of Panorama

Testing Visuals in Panorama There are many other cascades available beyond the clinical cascade analysis in Panorama! Here you see testing cascades. Note that this is testing by modality. There are many other visuals here, including age/sex and trend analyses, and Shabeen will walk through these in further detail during the Testing mini-breakout.

PMTCT in Panorama This is a different kind of cascade than the example we just viewed. This also has multiple visual options. Victoria will go into more detail in the PMTCT mini-breakout.

TB/HIV in Panorama The TB cascade is different than the others: it starts in the middle and splits out to the right and left. The middle bar shows the number of ART patients screened for TB. Then the left-hand cascade is for TB prevention—starting on TB preventative therapy, etc.—and the cascade that stretches to the right is the TB treatment cascade—starting patients on TB therapy.

3. Additional Platforms to View Clinical Cascades (that you can adapt!) Speaker Notes Provide detailed notes on the topic you will cover and how it fits in to the overall structure of the course. When writing speaker notes, please consider the following: What would someone need to know about this content if they are teaching it for the first time? What concepts are most important for the audience to understand on each slide? Where might the learners have some confusion about the content, and can you anticipate what that would be and how to remedy it? Are there any case studies, personal examples, or stories you’ve heard of certain experiences with the topic? If there is an exercise associated with this topic, how can you set it up so the learners know what to expect (i.e. “You will have the opportunity to practice [skill y] in an activity we will do later in the course.”) Where can learners go for further information about the topic? This will be expanded upon at the end of the deck, but feel free to mention resources throughout the session as reminders. Remember: repetition is not a bad thing!

Analytic Tools for Clinical Cascade Analysis POART COP Planning Program Monitoring Program Performance Program Planning

Excel-based Analytic Tools for Clinical Cascades Wide variety of tools are available through ICPI across program areas Many visuals (HTS, TB and TX) incorporated into Panorama Using MER data or other data submitted to HQ (IMPATT etc) Released in alignment with the DATIM submission cycle (but allow 2-3 weeks for availability of dataset) Notes Consider the questions below as you craft and present your content: How much do learners already know about what you are presenting? What are the key terms they need to know in order to understand a concept? Are you trying to teach them too much in the amount of time you have? Where can you provide real-world examples/case studies/story from your experience to highlight a learning concept? What kind of questions might learners ask about this content? Are you going to provide time for Q&A during or after your course, or both? How much text do you need on this slide to communicate the main idea without overwhelming the learners? How can you use visuals to support the learning concept? Can you use visual metaphors?

Data Availability Lifecycle for ICPI Tools Approx. 2 weeks Approx. 1 week Intermediate Processing

ICPI Huddle Board

Partner Progress Report (PPR)

HIV Testing Services (HTS)

Care and Treatment – Site Level Analysis

Laboratory

DATIM for Clinical Cascades Refer to Day 1 Level 1 and Level 2 Exercises as a guide (Results only, Results vs Targets) Remember to use Group Sets Cheat Sheet (especially for HTS) to pull your data from DATIM

Selecting the ‘Right’ Platform for Your Analysis

Non-MER Platforms to Assess Cascades Country-level web-based or other platforms for analysis May include patient level data May include other surveillance, patient tracking data Implementing partner/other NGO analytic platforms Format of these platforms may differ (web-based, apps, excel etc)

Remember: Key Questions for Cascade Review Where are the leaks (geographic & programmatic)? Who is most affected by the leaks? Why are there leaks? How do we best address the leaks? What resulted from the actions taken?

Other Considerations… What other data or program information are needed to fully understand the relationships between and amongst clinical indicators? Policies affecting sites Commodities Epidemiologic context Socio-economic context Are interventions being implemented as planned? Status of site or SNU level program strategies? Program Quality?

4. Breakout Sessions: HTS Second 90 PMTCT TB/HIV

I am not familiar with Panorama, or have not routinely used it For 2nd 90 and HTS mini-breakouts I am not familiar with Panorama, or have not routinely used it Please do the Panorama exercises for HTS OR 2nd 90

Summary Notes You have the opportunity to summarize the main concepts learned throughout the course so learners can feel a sense of accomplishment for the work they’ve done, in addition to triggering what they’ve learned so they can ask any remaining questions about elements of the session that might have been unclear to them.

Summary During today’s session we discussed how to: Increase participants’ understanding of existing MER clinical cascade analytics Strengthen participants’ skills in interpreting cascade results through use of Panorama, DATIM, or ICPI tools Develop follow-up analytic questions and identify programmatic actions based on data review Notes Provide a summary of what learners achieved by reviewing the objectives from the beginning of the course. Please see further examples of how to write learning objectives in the Training Development Resources section of this deck.

Questions? Notes Provide an opportunity for learners to ask any questions they may have about the content before you move on to providing information about further learning resources and key contacts. You may also encourage learners to ask questions throughout the course, but determine how you want to handle this given the amount of time you have to present and the amount of content you need to get through.

Thank You! Speaker Notes Don’t forget to that everyone for their time, participation, and attention during the session! Leave on a high note!