1. A Direct Observation Process & Electronic Tool that Supports Milestones Reporting

2. Our Team Nancy Newman, MD Praveen Tupakula, MD Lourdes Pira, MD Susan Hasti, MD James Anderson, PhD Jerome Potts, MD Rajasekhar Tupakula, MS WE HAVE NO RELEVANT FINANCIAL RELATIONSHIPS TO DISCLOSE

3. Presentation Outline Description of Direct observation process Demo of Milestone recording tool Review of preliminary findings in 2014 Findings after full implementation Lessons learned (and how to get the free software if you want)

4. Direct Observation Process Identified needs – for more feedback to residents in clinic – to document milestone progress – to design something efficient and learner- centered

5. Clinic Direct Observation Process More faculty observation of encounters – Adapted from “Field Notes” Assign one preceptor faculty to observe at least one resident encounter per session Develop a computer-based tool

6. Software Tool

9. Pilot Study December-February 2013-14 (8 weeks) 5 Faculty and 15 residents participated 30 direct observations

10. Outcomes Resident work flow – Visit length did not change Faculty precepting process – Numbers of encounters precepted did not change Faculty and resident satisfaction – “Satisfying to give on the spot feedback” – “Enables…more active role in teaching” – Concerns about efficiency and clarity of task

11. Moving to full implementation Involve all faculty in clinic direct observation Expand software tool – to facilitate documentation of performance in other settings – to allow residents to directly view their own performance information Encourage faculty to document observations in other teaching situations

12. Quantity of Data points gathered

13. Software Tool Technical information: – Used Microsoft Visual Studio 2013 V11 (with ASP.net 4.5 framework, C# language) – Ready to use with: MS SQL/Access capable online web hosting site, or IIS (Internet Information Services) enabled Local Area Network Free tool available at: FMDRL STFM Contact us- TupakulaTool@gmail.com@gmail.com

14. Faculty Satisfaction Faculty Survey March 2016 – 16/19 respondents report satisfied/ very satisfied – Things they like Easy to use Allows tracking of specific milestones Facilitates timely feedback to residents – Things to improve Make login in process easier Fix software glitch – sometimes "times out” too fast

15. Resident satisfaction Resident survey March 2016 – 14/23 - satisfied or very satisfied. – 7/23 - neutral – 2/23 - don’t know what it is – Things they like “Good way to get feedback” “Great tool” – Things to improve “wish more faculty used it.” “get a reminder email” when new feedback entered

16. Lessons learned This process and software tool produces Valuable data for milestones reporting High satisfaction from faculty and residents Online software management ~ 4 hrs/month In clinic precepting, co-faculty support is vital

17. Next Steps Maintain the new process in clinic Continue to improve the tool Encourage more faculty to document when on call and in other teaching settings Remind faculty to notify residents when they enter new information.

18. Questions? ‘How to install&user manual’ and the tool available at: FMDRL STFM (tip-search ‘Tupakula’)