1. What’s New at The CITI Program Paul Braunschweiger Ph.D. Professor Radiation Oncology University of Miami Miller School of Medicine www.citiprogram.org

2. Talking points Historical perspectiveHistorical perspective Organizational structureOrganizational structure New materials through CITINew materials through CITI New curriculum structure opportunities.New curriculum structure opportunities. New registration, enrollment and navigation aidsNew registration, enrollment and navigation aids New administrator utilitiesNew administrator utilities PRIM&R Member AccessPRIM&R Member Access New look for 2009New look for 2009

3. CITI Founders Karen Hansen, Fred Hutchinson Cancer Research Center, Seattle, WA., USA Paul Braunschweiger Ph.D. University of Miami Miami, Fl. USA CITI Program is a web based bioethics education program designed to promote the responsible conduct of research.

4. June 2000 DHHS Mandates Training in Human Subjects Protection 12. Educational Training OHRP strongly recommends that the Institution and the designated IRB(s) establish educational training and oversight mechanisms (appropriate to the nature and volume of its research) to ensure that research investigators, IRB members and staff, and other appropriate personnel maintain continuing knowledge of, and comply with, the following: relevant ethical principles; relevant federal regulations; written IRB procedures; OHRP guidance; other applicable guidance, state and local laws; and institutional policies for the protection of human subjects. Furthermore, OHRP recommends that a) IRB members and staff complete relevant educational training before reviewing human subjects research; and b) research investigators complete appropriate institutional educational training before conducting human subjects research. A. TERMS OF THE FEDERALWIDE ASSURANCE (FWA) FOR INSTITUTIONS WITHIN THE UNITED STATES. Announced Dec. 2000; implemented, 2005

5. CITI – Program 11-2008 CITI Administration – Office of Research Education, University of Miami Participating Institutions and Organizations (~1150) CITI Developer Group (~60) CITI Editorial Board (15) Intl. GCP RCR CITI Executive Advisory Committee HSRP Founders L.A.W. HIPS www.citiprogram.org

6. CITI Support Staff

7. CITI Program Milestones 10 Participating Institutions in 2000.10 Participating Institutions in 2000. –CITI - Collaborative IRB Training Initiative New Courses added 2004.New Courses added 2004. ~ 1150 Participating Institutions in 11- 2008.~ 1150 Participating Institutions in 11- 2008. –CITI - Collaborative Institutional Training Initiative Lab Animal Welfare Course – 5-2007Lab Animal Welfare Course – 5-2007 ~ 35,000 Learners per month. 40- 60% students~ 35,000 Learners per month. 40- 60% students >105,000 Learners since 1-1-2009.>105,000 Learners since 1-1-2009. > 12,000 Lab Animal Course learners since 1-1-2009> 12,000 Lab Animal Course learners since 1-1-2009 >1,000,00 people have completed a CITI course since 9-1-2000.>1,000,00 people have completed a CITI course since 9-1-2000. Subscriber ProfileSubscriber Profile –Universities, colleges, medical centers, community hospitals, societies, government, commercial IRBs, industry

8. CITI Program Milestones International InitiativesInternational Initiatives –CITI - Collaborative International Training Initiative Multi-language Course Site. 11-2007 updated 2-2009Multi-language Course Site. 11-2007 updated 2-2009 –Spanish –Portuguese –Chinese, Japanese, Russian, Mongolian, French, Thai

9. CITI Worldwide Footprint World wide n > 1150 International n = 49sites / 35 countries

10. CITI Program Courses Human Subject ProtectionHuman Subject Protection –Good Clinical Practice (15) –Refresher Courses Health Information Privacy and Security (HIPS)Health Information Privacy and Security (HIPS) –Basic –Refresher The Responsible Conduct of Research.The Responsible Conduct of Research. –5 Disciplines –Text, cases, quizzes. Laboratory Animal Welfare (L.A.W.)Laboratory Animal Welfare (L.A.W.) –Basic –Refresher

11. Lab Animal Welfare Program 2004 - Collaboration between CITI and Dr, M. Fallon at the Atlanta, VA.2004 - Collaboration between CITI and Dr, M. Fallon at the Atlanta, VA. 2007 – VA decided to close the www.ResearchTraining.org LAW course site.2007 – VA decided to close the www.ResearchTraining.org LAW course site.www.ResearchTraining.org 5-2007 CITI offered the CITI Version of RT.org to our participating institutions.5-2007 CITI offered the CITI Version of RT.org to our participating institutions. 10-2007 – CITI Lab Animal Welfare Working Group.10-2007 – CITI Lab Animal Welfare Working Group. 2008 - 2009 - Comprehensive review and updating of the course content.2008 - 2009 - Comprehensive review and updating of the course content. 2009 CITI International Collaborating Centers2009 CITI International Collaborating Centers RCR 12-29-07

12. CITI L.A.W. Course Library CITI Basic Courses.CITI Basic Courses. –Working with the IACUC –Essentials for IACUC Members Special CoursesSpecial Courses –Aseptic Technique –Minimizing Pain and Distress in Rodents. RCR 12-29-07

13. CITI L.A.W. Course Library Species / model Specific Courses Working With Amphibians In The Research SettingWorking With Amphibians In The Research Setting Working With Mice In The Research SettingWorking With Mice In The Research Setting Working With Rats In The Research SettingWorking With Rats In The Research Setting Working With Guinea Pigs In The Research SettingWorking With Guinea Pigs In The Research Setting Working With Hamsters In The Research SettingWorking With Hamsters In The Research Setting Working With Gerbils In The Research SettingWorking With Gerbils In The Research Setting Working With Rabbits In The Research SettingWorking With Rabbits In The Research Setting Working With Cats In The Research SettingWorking With Cats In The Research Setting Working With Dogs In The Research SettingWorking With Dogs In The Research Setting Working With Swine In The Research SettingWorking With Swine In The Research Setting Working With NHP In The Research SettingWorking With NHP In The Research Setting RCR 12-29-07

14. Establishing an Institutional Curriculum Each institution can establish their own curriculum from the L.A.W. Content Library.Each institution can establish their own curriculum from the L.A.W. Content Library. Curriculum can be customized according to:Curriculum can be customized according to: –Role in LAW research. –Species / models used. Basic courses and refresher courses.Basic courses and refresher courses. –Reminder notifications RCR 12-29-07

15. CITI L.A.W Course site usage 384 institutions

16. Top 10 Institutional Users Institution / Organization Total for 2009* Veterans Affairs**3533 Emory University681 University of Miami / Jackson Memorial Hospital739 Wayne State University-Detroit MI255 Georgia Institute of Technology501 Binghamton University559 Clemson University495 University of Maryland492 West Virginia University591 University of Texas Health Science Center San Antonio587 University of California Irvine551 *First 3 months of 2009 ** 151 VA sites have access to the CITI Program.

17. 2008 CITI L.A.W. Welfare Course Usage. n=384 institutions Course # Completed Working with the IACUC 11909 Working with Mice 8180 Working with Rats 5022 Reducing Pain and distress 4699 Essentials for IACUC Members 1954 Aseptic surgery 1001 Working with Rabbits 930 Working with Swine 849 Working with NHP 803 Working with Amphibians 490 Working with Dogs 470 Working with Guinea pigs 412 Working with Hamsters 341 Working with Cats 266 Working with Gerbils 139 RCR 12-29-07

18. CITI L.A.W. Working Group NameInstitution Tracy Arwood, MS Clemson University Paul Braunschweiger, Ph.D. University of Miami Mark Christensen, MA Lourdes College Mike Fallon DVM, Ph.D. Veterans Affairs Cynthia Gillett, DVM University of Minnesota Molly Greene Michigan State University Susan Harper, DVM, MS Veterans Affairs Mike Mann, Ph.D.* Emeritus, University of Nebraska Medical Center Randall J. Nelson, DVM University of Memphis Gwenn Oki, MS City of Hope Medical Center Ernie Prentice, Ph.D. University of Nebraska Medical Center Susan Silk, MS Office of Laboratory Animal Welfare (OLAW) Kathy Wadsworth, MS University of California at Los Angeles Thomas Zimmerman, DVM SUNY Stony Brook *CITI L.A.W. Working group leader

19. L.A.W. Working Group Review learner commentsReview learner comments –Modify the content if appropriate. New ContentNew Content –Susan Silk MS, Director of the Division of Policy and Education, OLAW “Using Animals in Research: Information for PHS-funded Investigators.” will provide a comprehensive overview of the regulations and guidelines for PHS- funded researchers including institutional and investigator responsibilities, applying for funding, receiving an award and post award responsibilities.“Using Animals in Research: Information for PHS-funded Investigators.” will provide a comprehensive overview of the regulations and guidelines for PHS- funded researchers including institutional and investigator responsibilities, applying for funding, receiving an award and post award responsibilities. –Mark Christensen, Prof. of Ethics, Lourdes College 5 module course on the Ethics of conducting research with Lab animals.5 module course on the Ethics of conducting research with Lab animals. Ethical considerations related to the use of live animal subjects in scientific research. In this module we will briefly highlight some philosophical theories about ethics and moral standing and their relation to the use of animals as research subjects.Ethical considerations related to the use of live animal subjects in scientific research. In this module we will briefly highlight some philosophical theories about ethics and moral standing and their relation to the use of animals as research subjects. –Conducting Field Studies Responsibly. –Case based refresher course. RCR 12-29-07

20. L.A.W. Working Group Recently completed updating and editing course content for:Recently completed updating and editing course content for: –Working With the IACUC –Essentials for IACUC Members Quiz questions Reviewed and updated.Quiz questions Reviewed and updated. –Feedback for all of the questions in all of the modules. –Posting in June 2009. Antibody productionAntibody production –New Course on antibody production RCR 12-29-07

21. Course Site and Software Update

22. Multilanguage Course Site

23. Curriculum design Curriculum design is institutional responsibilityCurriculum design is institutional responsibility Stratified Curriculum to meet educational needs.Stratified Curriculum to meet educational needs. –Research Focus Model / species specific modules.Model / species specific modules. –Role in research IO, IACUC Member / staff, PI, Investigators, research staff, students.IO, IACUC Member / staff, PI, Investigators, research staff, students. –Institution specific modules Policies and Procedures.Policies and Procedures. Materials not covered by CITI CoursesMaterials not covered by CITI Courses –Special species or model systems –Field studies. –Lab Animal Research Ethics. RCR 12-29-07

24. Curriculum Development Required vs. Optional vs. Elective RequiredRequired –Learner is required to complete modules in this category. ElectiveElective –Institution can designate several “Elective” modules (e.g., 10) and require that the learner complete a specific number (e.g., 8) that are of interest. OptionalOptional –Optional modules are made available to the learner after all required and elective modules are completed. Passing score set by the institutionPassing score set by the institution –Learner must attain an aggregate score that meets the institutional requirement to get a Completion Report RCR 12-29-07

25. Curriculum design CITI Course order forms are available at www.citiprogram.orgCITI Course order forms are available at www.citiprogram.orgwww.citiprogram.org Institutional Administrator menuInstitutional Administrator menu

26. Curriculum Modifications CITI Course order forms are available at www.citiprogram.orgCITI Course order forms are available at www.citiprogram.orgwww.citiprogram.org Institutional Administrator menuInstitutional Administrator menu

27. Quiz Questions Utilities Integrity Assurance.Integrity Assurance. New presentation paradigms.New presentation paradigms. –Randomized questions from a pool –Using a case study as a stem. New psychometric tools to evaluate quiz questions.New psychometric tools to evaluate quiz questions. RCR 12-29-07

28. Institutional Modules Developed by organizations specifically for their research community.

29. Examples of Institutional L.A.W. Modules Medical Surveillance ProgramMedical Surveillance Program General Zoonotic TrainingGeneral Zoonotic Training Farm AnimalsFarm Animals Wild AnimalsWild Animals Fish, Amphibians and ReptilesFish, Amphibians and Reptiles Introduction to Surgical ResearchIntroduction to Surgical Research Antibody ProductionAntibody Production Occupational Health and SafetyOccupational Health and Safety Other models (e.g., Birds)Other models (e.g., Birds) CU Policy on the Care and Use of Animals in Research and Teaching.CU Policy on the Care and Use of Animals in Research and Teaching. Conducting Potentially Painful/Distressful Procedures in Lab Animals At The University of MiamiConducting Potentially Painful/Distressful Procedures in Lab Animals At The University of Miami RCR 12-29-07

30. Registration, Enrollment and Navigation

31. REGISTRATION

33. Add Customizable fields

34. Enrollment Questions Curriculum Selection New page to page branching logic enrollment processNew page to page branching logic enrollment process –Learner self-identifies research areas. –Chooses topics Choices determine the next series of choicesChoices determine the next series of choices

35. Enrollment Questions Choose the most appropriate research focus or role in researchChoose the most appropriate research focus or role in research Software then shows the next series of questions appropriate for the learner.Software then shows the next series of questions appropriate for the learner.

36. Choose the Animal Models

37. Special Invasive Techniques

38. Special Course Enrollment

39. Course Summary

40. Learner Menu

41. Grade Book Integrity Statement

42. Integrity Statement X

44. Utilities for Learners and Administrators

45. CITI Program Knowledge Base http://citiprogram.supportcenterpro.com/knowledgebase/ or CITI Home Page, Contact Us http://citiprogram.supportcenterpro.com/knowledgebase/ The CITI Program Knowledge Base provides helpful information to assist in the use of the CITI Program website.The CITI Program Knowledge Base provides helpful information to assist in the use of the CITI Program website. Provide support, resources, hints, tips, links and answers to frequently asked questions about the CITI Program, all in one place.Provide support, resources, hints, tips, links and answers to frequently asked questions about the CITI Program, all in one place. We welcome contributions and feedback from learners and CITI institutional administrators.We welcome contributions and feedback from learners and CITI institutional administrators. Please use the “Submit New Ticket” ticket link to the right of the screen to share comments, questions, and feedback with the CITI Support team.Please use the “Submit New Ticket” ticket link to the right of the screen to share comments, questions, and feedback with the CITI Support team.

46. CITI Program Knowledge Base Please use the “Submit New Ticket” ticket link to the right of the screen to share comments, questions, and feedback with the CITI Support team.

47. Admin Access to User Data All CITI Institutional administrators have access to user dataAll CITI Institutional administrators have access to user data Can be restricted according to admin role.Can be restricted according to admin role. –Receive completion reports –Responsible for segments of the training program Human subjectsHuman subjects RCRRCR Lab AnimalLab Animal HIPSHIPS

48. Institutional Administrator Menu

49. Course listing Review a completion ReportReview a completion Report

50. Training Data Retrieval

52. View the data Links to download the data in various formats

53. Look up a specific learner

54. Learner’s Records

55. Other data management options available Bulk data downloadBulk data download –CITI will provide institution specific URL to download the institution’s training data. –Username and password protected. –Data for each course (HSR, lab Animal) can be downloaded together or separately –The data fields to be included in the download is customizable. –Suitable format to upload to data management system. –Contact CITI Helpdesk - 305 243-7970; citisupport@med.miami.edu citisupport@med.miami.edu https://www.citiprogram.org/members/siteadministrators/manageinstitutions/downloadaccesscodes.asp?st rKeyID=XXXXXXXXXXXXXXXXXXXXXXXXX73

56. Other data management options CITI can now be placed behind your institutional Portal System.CITI can now be placed behind your institutional Portal System. –User authentication –Username and password conservation. –Increased data and password security. –Data can be automatically added to training database. Contact CITI helpdeskContact CITI helpdesk 1 time cost for programming and implementation.1 time cost for programming and implementation.

57. PRIM&R Member Access

58. Go to Main MenuGo to Main Menu “Affiliate with another institution”“Affiliate with another institution” “Register as PRIM&R Member”“Register as PRIM&R Member”

59. Contact Information for PRIM&R Member X

61. New initiatives for 2009 Courses and modulesCourses and modules –Software updates “Look back” Utility“Look back” Utility Non-English authoring utilities.Non-English authoring utilities. Knowledge Base.Knowledge Base. Username and Password securityUsername and Password security Verification and Portal pass through.Verification and Portal pass through. –Human subjects (Summer 2009) ESCRO (1 modules)ESCRO (1 modules) New Refresher ModulesNew Refresher Modules Community Member ModuleCommunity Member Module Research CoordinatorsResearch Coordinators CBPRCBPR

62. New initiatives for 2009 Courses and modulesCourses and modules –RCR RCR for Engineers.RCR for Engineers. RCR for Arts and Humanities. (Fall 2009)RCR for Arts and Humanities. (Fall 2009) –Biosafety and Bio-security (Summer 2009) 15 module course15 module course –Lab animal Ethics and Lab Animal ResearchEthics and Lab Animal Research Field StudiesField Studies Zebra FishZebra Fish

63. Download this Slide Show Slide show available back

64. Announcements and FAQ

66. CITI L.A.W. Satisfaction Survey

67. The current course site presents the materials in a way that is easy for me to understand and learn Strongly Disagree Strongly Agree

69. If you have previously completed a Lab Animal Welfare course, how does this CITI Course compare with others you have completed? Strongly Disagree Strongly Agree 61%

70. Although training in Laboratory Animal Welfare issues is mandated by the Federal government and by my institution, I believe that this training is really not very useful for new faculty and students? Strongly Disagree Strongly Agree

71. It was easy to register and enroll in the appropriate course. Strongly Disagree Strongly Agree

72. The quiz questions addressed the relevant issues in the modules Strongly Disagree Strongly Agree

73. Now that I have completed the CITI Lab Animal Welfare Course, I feel that the time I invested was time well spent. Strongly Disagree Strongly Agree

74. After completing this instruction, I now have a better understanding of how to apply the "3Rs" to insure that my lab animal studies are conducted to the highest ethical standards? Strongly Disagree Strongly Agree 71%

75. Now that I have completed the course, I am more confident in my ability to advise a student or a colleague on how to apply the "3Rs" to insure that their lab animal studies are conducted to the highest ethical standards? Strongly Disagree Strongly Agree 68%

76. Now that I have completed this course on the ethical conduct of studies with laboratory animals, I intend to take a more active role in assuring that lab animal research at my institution is conducted to the highest ethical standards by pursuing professional certification, by joining an ethics committee or an Institutional Animal Care and Use Committee (IACUC) Strongly Disagree Strongly Agree 60%

77. The "Essentials for IACUC Members" Course has provided we with the knowledge base and confidence to take a more active role in the business of the IACUC. Strongly Disagree Strongly Agree 64%

78. About how many hours did it take you to complete your Lab Animal Welfare Course requirements?

79. I DO NOT like the current presentation model another approach should be implemented. 2% I prefer the "page to page" approach with interactive games or activities to break up the text. 5% The "page to page" is better because I could pause and reflect on important points in the content. 6% I DO NOT recommend the "Page to Page" model as it would increase the time to complete the course. 10% The materials presented in the course are new to me. I like the current presentation model. 15% I am familiar with the content. The current approach permits an efficient review of the material. 22% The current approach enabled me to read through the content as if I where reading a book or journal. 40% The presentation model used by CITI permits you to scroll through the content as if you were reading a book or a journal. Other web based models require that the learner click from "page to page” in order to read the material in smaller segments.

80. Please visit the CITI exhibit