1. AAMC Building Better Curriculum Webinar Series We will begin our presentation shortly. Topic: Update from the AAMC’s Curriculum Inventory Presenter from AAMC: Angela Blood Director, Curricular Resources PLEASE NOTE: All users will be muted during the webinar but should use the chat feature to send questions to Angela Blood during the presentation. We will try to answer as many questions as possible at the end of the presentation.

2. CI Three-Year Plan Two priorities: Fortify the existing product
Envision and create an evolved product
Phase 1: July 2018 – Dec 2018
Phase 2: Jan 2019 – June 2019
Phase 3: July 2019 – June 2020
Phase 4: July 2020 – June 2021
AAMC has a rigorous process for vetting existing and new programs, and for investing further support in existing programs. The CI completed the phases of that process in July and October This process allows us to map out all the specific ideas and projects we feel are necessary to fortify the CI and help it grow into its future. Evolving the CI was the equivalent of about 50 projects. Vetting these projects with senior leadership at AAMC allows us to ask ourselves why are we doing this, what are the intended outcomes, is this a priority compared to other priorities, is it feasible.
In addition to this rigorous process which the CI passed, the CI was also made an institutional goal for AAMC in , further raising its visibility and confirming its investment as valuable. We’ve also been given resources to increase staffing. I’m sharing this with you to show that the AAMC leadership is fully supportive of the CI, and continuing to dedicate resources to its success.

3. Agenda Curriculum reports Vendor retreat and CI Vendor Portal
Communications
On the horizon

4. Curriculum Reports
As many of you know, the Verification Report is the report that you as a school receive when you successfully submit your CI data. We made a number of improvements to the Verification Report for 2018, last season, and have some new improvements to share with you for 2019.

5. CI Verification Report Resources
Before we jump into improvements, I wanted to share this website with you. All of the resources we’re talking about today are available to you on the CI website, specifically under “Resources to Use Your CI.” You can find samples of Verification Reports, and a Guide explaining how the previous version of the Verification Report differs from the new version.

6. Verification Report Improvements for 2017-18
Readability: we changed the font, increased the size slightly, and tried to make better use of white space
Explanations: we added guide information for every section of the report, explaining what data is and is not included in given tables, how amounts are calculated, etc. 
Wording: we added clarity in the glossary, retitled tables, retitled columns, and added columns within tables to more accurately represent which data is provided
OLD
NEW
Here on the left is a reminder of what the Verification Report used to look like, and what the major aims for the Verification Report improvements in 2018 were. We focused on the readability, using the white space on the pages thoughtfully, adjusting the font and size of text. We added explanations for each table, and adjusted wording where needed for clarity.
These changes were made in a matter of months, with a very aggressive delivery timeline. Now we’ve had additional time to improve the Verification Report for 2019 so I’m excited to show you the further improvements we’ve built upon here.

7. OLD
NEW
The table above is the previous version. The table below is the updated 2018 version.
We received a very positive reception to the changes we made for The improvements included:
Numbering of all tables
Underlines within tables as clickable links so you can jump to other parts of the PDF.
Links to resources on the CI website from within the report
Explicit wording to show which tables are designed to align with the LCME DCI.
Relabeling tables. It might have said “all” but it really was primary.
Adding explanations so its clear how formulas are calculated, what data makes it into the data.

8. Navigating the Verification Report
Another key improvement for 2018 was adding the bookmarks feature, so that the reader of the Verification Report could jump around from one section of the report to another. I’m showing you this especially here because we’re planning additional uses for the bookmarks feature for 2019.

9. Previous Sequence Block (10) Details Table
As a reminder, this is the table, table 10, which had the most details about sequence blocks (typically thought of as courses). It had information like the dates and duration of the course, the learning objectives (sequence block expectations), links to program objectives (program expectation ID).

10. Previous Event (11) Details Table
As a reminder, this is the table, table 11, which had the most details about events (or sessions). It had information like the instructional methods, assessment methods, and resources for a given event, as well as sequence block and program expectation (learning objective) links.

11. Verification Report 2019 Goal: Make the report easier to share
We recognize that for some schools, the Verification Report can be very lengthy.
If the goal of reviewing the Verification Report is to make sure its accurate, asking one curricular dean to know the details or each event or course is probably not realistic. The faculty directly involved in the leadership of specific sections of the curriculum (like a neurology clerkship director, or dean of basic science) are likely to know their sections of the curriculum more intimately. It would also help not only help check for accuracy in the Verification Report, but use the report for program evaluation if the report were easier to share.
Currently, all sequence blocks/courses were listed in one large table (and the same was true for events, across courses) so parsing out pieces of the Verification Report for specific faculty or specific purposes was not really possible.
If we could at least insert page breaks among sequence blocks/courses, the report would be easier to share with other key faculty and curriculum leaders.

12. New Sequence Block/Course Summary Table
This is all sample data, so we’re sharing this with you not so much to read the specific content, but to see the new formatting.
We will have a new summary table for your courses, so as the curricular dean you can see some big picture information about your sequence blocks/courses. Having this high level data would allow the dean to see if the number of events seem equitable (or not) across courses, if the links to course objectives look right, etc.

13. New Table on Learning Objectives and Links
For the details about sequence blocks/courses, we will separate those from the big picture items related to courses. With a piece of the report packaged around a specific course, and bookmarked, it will be easier for you to potentially share this data with your faculty leaders (directing them, to say, pages 23-26, as an example) so that they can review their curricular data which they’re most familiar with, and use the information to inform curricular adjustments.
As an example, you might ask your faculty to make sure all course objectives have a link to at least one program objective (table above), or you might ask your faculty to do the same with event and course objectives (table below). You might ask your faculty to not only check that there are objective links, but to check that they make sense (as an example, you have an event objective about a specific pharmacokinetic pathway that is linked to a course objective about professionalism. That doesn’t seem quite right, there’s probably a better link that can be made).

14. New Event Details Table
Again this is using sample made up data, and the point of this slide is more to show the look and format. This table has very similar content to the “old” event details table, but we were able to take out links to objectives as we’re putting that into a new table on the previous slide. This gives you more room to see the items on this table. You could use a table like this to not only verify that the data is accurate, but also judge if the match among event expectations (learning objectives), instructional methods, assessment methods, and resources are as they should be.

15. CI School Portal: Curriculum Reports
One really exciting new feature we have for 2019 for you is a section for “curriculum reports” in the CI School Portal. The CI School Portal is normally the system you (as the curricular deans and primary curriculum administrators for medical schools) use to upload and send your annual curriculum inventory to the AAMC. It’s also the site you use to access your Verification Report. In the past the CI School Portal has typically been a site that is used during the August-September CI upload season, but there hasn’t been much reason to access it beyond that.

16. Past Verification Reports on the CI Portal
Now, the CI School Portal will have a new tab (in the light blue towards the top of the page) for “Your Curriculum Reports.” These curriculum reports will include your past Verification Reports. In the past, previous Verification Reports (from years , , etc.) have been erased when the new season opens. Now you’ll be able to find your past Verification Reports up to a number of previous. We hope this will help schools who have changes in leadership roles and cannot find important curricular documents, schools who are bringing on new curriculum leaders who they want to have access to CI data (e.g., you want your Assistant Dean of Basic Science to have CI Viewership capability), and more easily make comparisons from year to year.

17. Curriculum Reports on the CI Portal
Your Curriculum Reports will also house the curriculum reports your school requests. As a participating member of the CI, faculty from your school are able to request data reports on whatever curricular topic they’re interested in. In the past we’ve shared those via . It’s not the most secure sharing pathway, and doesn’t help faculty at your school see collectively the reports that were created for you. We also have a category for “Other” which will include reports that the AAMC thinks are important to share with our members, but which we do not want to post on the public website. The types of reports you might find in that tab are things like a curricular report on an LCME “hot topic” area.

18. Curriculum Reports on the CI Website
We’re going to walk through one example of finding a curriculum report here. If you’re on the AAMC homepage, or another AAMC webpage, you can navigate to Services. Then choose “curriculum.” From the CI homepage, choose the left-hand button for “Curriculum Reports.” You can either search through the table of available public reports, or you can enter your keyword(s). In this case, I’m searching for genetics. Once you choose the report you’re interested in, you’ll notice the reports are dynamic and interactive. You can click on the reports or choose different drop-down menu options to see different data. You’ll notice a “download data” button – this gives you the raw data in Excel so that you can perform your own analyses. You’ll also notice a related reports section – this shows you any other public reports that are related to your topic, MedEdPORTAL, Academic Medicine, and Pub Med resources that are available to you. These results are prefiltered to be those relevant to the topic of the curriculum report, so it is more than just a general link to MedEdPORTAL. Once you’ve explored the publicly available reports, if you still need more information, you can submit a data request. As an participating CI school, you’re able to request these reports for free.
We’re in the process of updating all the publicly available reports on the CI website. We’ve checked for things like best visual display, updating with 2018 data, ensuring data accuracy, etc. We’re now working on updating all the related resource links.

19. Curriculum Report Requests
As a CI participant, you and faculty from your medical school are eligible to submit requests for free curriculum reports. The kinds of question we get are: what kinds of instructional methods are people using in pediatric clerkships, or how many hours of anatomy content was in the curriculum this year compared to last year? In the past, when schools would ask for these kinds of reports, all the communication occurred via including the AAMC sending the report. Now, all reports that you request will be stored on your CI portal, so you won’t have to dig through or try to locate a report if a faculty member leaves and a new person has a role in CI. You’ll also be able to access your previous verification report submissions, again so you don’t have to dig through and to help you if there is staff turnover.

20. Questions About Curriculum Reports?

21. Vendor Retreat and CI Vendor Portal

22. Vendor Retreat January 2019
Really liked the interactive discussion approach, was very productive
It was an excellent meeting; well worth our time and effort
16 attendees representing 11 companies
Topics included
MedBiquitous Technical Specifications
AAMC Business Rules
Communications and technical support
CI season upload
It was very reassuring to hear about the commitment from AAMC to the CI & MedBiquitous
The Vendor Portal was an excellent surprise
It would have been helpful to have the agenda and materials early
Super receptive to feedback…impressed with extra time on important conversations
We held a two day retreat for vendors in mid-January, Vendors from across the U.S. and Canada were flown in to discuss pressing CI issues. This is in addition to every-other-month web meetings we now have with vendors.
We had really great engagement from the group. It was a very collaborative, discussion and interaction focused meeting. Vendors gave us really positive feedback and appreciated the format and content. We walked away with a lot of great ideas, and it also gave us an opportunity to showcase a new feature we have, called the CI vendor portal, which we can show you on the next slide.
Minor group activities were good for pacing instead of solid lecture

23. New Vendor Portal
One of the major improvements this year to support vendors in helping their clients is a new Vendor Portal. This was one of the major asks we heard from vendors during our needs assessment which began in April last year, Vendors wanted a better idea of which of their clients was where in the CI process, and this portal will give them a password-protected view of which of their clients might need the most support. The schools above are just sample data, again. Vendors can only see the schools who assign them as a vendor, so there is no way a vendor can access a school’s verification report who hasn’t designated them as having permission. Through the different tabs across the top of this image, vendors can see which of their clients have completed the process, how many times schools have attempted to submit (and perhaps target their customer service better to you). It also shows the vendor the Verification Report so they can in real time see the same potential error messages if there’s an issue, and the same results that you see. Instead of you having to send your vendor your verification report or take screen shots of errors, they can view these things for themselves.

24. Vendor Portal Status Reports
Vendors can also pull reports on their clients along with their contact information (based on the CI faculty you as the school provide to us), so vendors can more easily get in touch with schools who need assistance.

25. Communications
Now we’d like to get into the methods by which we share resources and information with you, and the communication methods we have made some changes to.

26. CI Website
The CI Website is organized according to resources to establish a curriculum inventory, resources to use your curriculum inventory effectively, and resources for developers and vendors. Some of the communications that are especially helpful and can be found here include:
The CI School Portal user guide
The AAMC Building Better Curriculum webinar series
The CI Standardized Vocabulary on instructional methods, assessment methods, and resources
The CI Newsletter
The CI Staging environment
And the CI listesrv

27. Some of our past topics have included using your curriculum inventory as part of your program evaluation process, tagging your clinical and clerkships experiences in your curriculum inventory, and writing learning objectives. If you’re interested in submitting a topic or becoming a presenter, please get in touch at
We hit a record high of registries and attendees for the February 2019 webinar with more than 70 attendees.

29. CI in Context Publication
We now have author instructions to make sure each publication is consistent. We’re actively recruiting a few topics: opioid and addiction medicine curriculum, humanities in medical curriculum, away and elective time in the fourth year. We are averaging 300+ users per CI in Context publication.

30. Questions About Communications?

31. On the Horizon AAMC Committee
New mission, vision, and value statements
MedBiquitous CI Specifications review
Data flags for quality assurance
New staff member: Linh Le, Project Specialist

32. Thank you and we will see you next month!
Please register for next month’s Building Better Curriculum Webinar Series on Wednesday, June 12, 2019 at 1:00 pm EST. We will post May’s series on AAMC’s website here: