1. Semantic MediaWiki in Action: CTCAE Revision Project Traci St. Martin, Ann Setser and Lewis Frey Architecture/VCDE Face-to-Face Meeting Northwestern University, Evanston, IL Oct. 28 th, 2008

2. Agenda Introduction to Semantic MediaWiki (2 min)-Traci St. Martin Use of Semantic MediaWiki: CTCAE Revision Project (20 min) Background of CTCAE Revision Project-Ann Setser CTCAE Community Requests for Additional Functionality-Traci St. Martin VCDE Input into CTCAE Revision Project-Lewis Frey Lessons Learned from CTCAE Revision Project-All (5 min)

3. Introduction to Semantic MediaWiki Semantic MediaWiki (SMW) utilized for BiomedGT The Biomedical Grid Terminology is an open, collaboratively developed terminology system for translational research, initially based on NCI Thesaurus Goal: Evolve BiomedGT into a set of sub-terminologies that can be: Federated Collaboratively developed Maintained by experts in relevant research communities Integrated, where appropriate With re-use and integration of existing high quality terminologies Additional tools: Protégé OWL Collaborative Protégé LexGrid/LexBIG

4. Why SMW? Traditional approach using NCI Thesaurus didn’t scale Desire to move to federated terminology development Include external groups for wider amount of expertise Create better acceptance and maintenance of ontologies The infrastructure and culture of the semantic wiki is a promising workbench for collaborative ontology engineering. Easy to learn tool Open content development platform Change history management Formal output rendering (RDF export) With some extensions, the SMW can be useful for publishing, annotating and cross-referencing other terminological resources SMW offers: Browsing support (hierarchical view, data elements, etc.) Authoring support (workflow for proposals, etc.) Ability to load multiple terminologies

5. BiomedGT Wiki: Current & Future Current Projects within BiomedGT Wiki Environment BiomedGT Nanotechnology terminology Pathology related terminology updating and expansion (interested) CTCAE4 Revision project Enhancements via collaboration between NCI and Mayo Clinic for BiomedGT Wiki Enabling Wiki features Mayo developed for ICD11 Adding additional namespaces (RadLex, etc.) Supporting BiomedGT development Maintaining and developing Wiki for CTCAE Revision project

6. Agenda Introduction to Semantic MediaWiki (2 min)-Traci St. Martin Use of Semantic MediaWiki: CTCAE Revision Project (20 min) Background of CTCAE Revision Project-Ann Setser CTCAE Community Requests for additional functionality-Traci St. Martin VCDE Input into CTCAE Revision Project-Lewis Frey Lessons Learned from CTCAE Revision Project-All (5 min)

7. CTCAE Background The Cancer Therapy Evaluation Program (CTEP) of the NCI developed the original Common Toxicity Criteria (CTC) in 1983 to aid in the documentation and analysis of adverse effects of chemotherapy. The CTC is a list of Adverse Event (AE) terms commonly encountered in oncology accompanied by a grading (severity) scale for each AE. 1983CTC v1 ~ 50 terms 1998CTC v2 ~ 300 terms 2003 CTCAE v3 ~ 1000 terms 2008Draft 1 ~ 820 terms – MedDRA terms

9. Medical Dictionary for Regulatory Activities – MedDRA Is a clinically validated international medical terminology used by regulatory authorities and the regulated biopharmaceutical industry throughout the entire regulatory process, from pre- marketing to post-marketing activities, and for data entry, retrieval, evaluation, and presentation.medical terminology Is the adverse event classification dictionary endorsed by the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH).adverse event International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use Is used in the US, European Union, and Japan. Its use is currently mandated in Europe and Japan for safety reporting.USEuropean UnionJapan

10. MedDRA Structure System, Organ, ClassSOCHighest level of the terminology, and distinguished by anatomical or physiological system, etiology, or purpose 26 High Level Group TermHLGTSubordinate to SOC, supraordinate descriptor for one or more HLTs 332 High Level TermHLTSubordinate to HLGT, superaordinate descriptor for one or more PTs 1,688 Preferred TermPTRepresents a single medical concept18,075 Lowest Level TermLLTLowest level of the terminology, related to a single PT as a synonym, lexical variant, or quasi-synonym (Note: All PTs have an identical LLT) 66,135 MedDRA translations CzechDutchEnglish DutchFrenchGerman EnglishItalianJapanese PortugueseSpanish MedDRA includes Standardised MedDRA Queries (SMQs) -groupings of terms that relate to a defined medical condition or area of interest. http://www.meddramsso.com

11. CTCAE v3.0 & MedDRA Use CTCAE & MedDRA MedDRA is used by the biopharmaceutical industry and regulatory agencies within the ICH regions CTCAE use by Industry CTCAE is widely used in oncology and HIV clinical research 11 MedDRA List of terms >86,000 Content: Comprehensive --- Hierarchy: 5 levels Medically validated CTCAE v3.0 List of terms 1,059 Content: Oncology Severity Scale Hierarchy: 2 levels ---

12. CTCAE v3.0 Issues & MedDRA Oncology community reported concerns/challenges using CTCAE vs/and MedDRA 2006MedDRA MSSO International Meeting 2006 +CTEP, NCI & FDA discussions 2007caBIG VCDE Review of CTCAE CTCAE – MedDRA mapping imprecise CTCAE terms too general – loss of granularity Critical concepts in Grades Lack of CTCAE maintenance Recommendations To facilitate data exchange within internal databases using MedDRA and with regulatory authorities for the purpose of SAE reporting, must establish a mechanism to ‘translate’ or ‘convert’ CTCAE terms from investigators to MedDRA terms.

13. Rationale for CTCAE v3.0 Revision CTCAE v3.0 Terms Multiple concepts One/many element of Grade description is critical AE concept Not all MedDRA terms CTCAE v3.0 Revision Goals Harmonize with MedDRA 100% MedDRA single concepts Revise and extend AE terms and severity indicators to be machine interpretable; conformance with caBIG Vocabulary Standards Establish a formal, ongoing governance for future versions Understandability, Reproducibility, Usability

14. CTCAE v3.0 Multiple Concepts in one AE Term 14 Mapped to MedDRA: Fatigue 10016256 CTCAE v4.0/MedDRA –Fatigue –Asthenia –Lethargy –Malaise

15. MedDRA Preferred Term Fatigue + 20 LLTs

16. 16 CTCAE v3.0 Revision Critical concept listed in Grade only – not as AE Term CTCAE v4.0/MedDRA –Allergic reaction –Anaphylaxis

17. CTCAE Governance Group FDA,CBIIT,CTEP, PhRMA Steering committee CBIIT, CTEP,VCDE, WG Leads CTCAE v3.0 Revision Project Semantic Media Wiki CTCAE Editing/modeling Establish long term governance of CTCAE Develop the strategic vision of CTCAE Drive the development of CTCAE Defines work; prioritizes and schedules the content Review/ Comments CTCAE Version 4.0 Internal review WG CTCAE draft version

18. Working Groups / SOCs

19. CTCAE v3.0 Revision Project Methods – Wiki

26. Agenda Introduction to Semantic MediaWiki (2 min)-Traci St. Martin Use of Semantic MediaWiki: CTCAE Revision Project (20 min) Background of CTCAE Revision Project-Ann Setser CTCAE Community Requests for additional functionality-Traci St. Martin VCDE Input into CTCAE Revision Project-Lewis Frey Lessons Learned from CTCAE Revision Project-All (5 min)

27. CTCAE Iteration 1 Publishing CTCAE v3.0: CTCAE v3.0 existed as an Excel spreadsheet. Created a tool to map the spreadsheet to LexGrid model Once in the model, the tool generated XML in LexGrid format This XML was loaded into LexGrid database using LexBig API Used existing LexWiki tab plug-in to Protégé to get the LexGrid contents to load into Protégé database and publish to Wiki CTCAE3 spreadsheet Map to LexGrid model XML Generated XML loaded to LexGrid DB Publish to Wiki LexGrid to Protégé DB Protégé Plug-in

28. CTCAE Iteration 1

29. Utilized additional SMW extensions, created new Semantic Forms, and created new Templates specifically for CTCAE.

30. CTCAE Iteration 1 Functionality Initial requirement - to display the data in a tabular format. Utilized Dynamic Page List extension to dynamically show all the Adverse Event terms of a Category in a table. Query looks for all pages in a Category and displays them in a table. The Adverse Event (AE) name displays as a link instead of the URL

31. CTCAE Iteration 1 Functionality Initial requirement continued: Created templates to display original content and proposal in a tabular format.

32. CTCAE Iteration 1 Functionality Using features of Semantic Forms, created a proposal mechanism with a drop down menu for users to select Proposal status of Edit, Split or New Term.

33. CTCAE Iteration 1 Functionality Utilized Header tabs extension in SMW to easily toggle between the Tabular format and the Factbox.

34. CTCAE Iteration 1: Processing proposals Created mappings based on rules for Edit, New, and Split Term status. Used RDF feed to read the proposals and render to LexGrid model. Used RDF feed and created a script to generate spreadsheets for Ann to review proposals, MedDRA review and reporting to Steering Committee. Published to Wiki using same initial process. Published to a new CTCAE SOC Draft 1 namespace for 2 nd Iteration of CTCAE revisions. Enabled referencing CTCAE 3 proposals.

35. CTCAE Iteration 1: Usability enhancements Based on user difficulties with standard Talk/Discussion page, utilized Article comments to change to a more user-friendly format. Old formatNew format

36. CTCAE Iteration 1: Usability enhancements Users experienced difficulties with using the Edit tab instead of creating a proposal. PHP code modified to hide the Edit tab for non-Sysop users. Edit tab hidden

37. CTCAE Iteration 2: Changes Iteration 2 organizes data by SOC instead of Category Established mapping between Category, Adverse Event term, and SOC. Allowed publishing by SOC but maintains Category information in database.

38. CTCAE Iteration 2: Changes Created a new Header tab and link back to CTCAE 3.0 Adverse Event term and Proposal (from which the CTCAE Draft term originated). Link opens in a new tab or window CTCAE Category (v3) is ‘locked’(via a change to the CTCAE form). Users cannot edit 1 st iteration proposals.

39. CTCAE Iteration 2: Changes Modified Status Allowed for separate status for Adverse Event & Grades Created new options: No Change, New, Edit, Delete

40. CTCAE Iteration 2: Additional Enhancement Customized Auto-complete feature Use Ctrl-Space or enable in My Preferences Easier to find specific article (AE term)

41. Agenda Introduction to Semantic MediaWiki (2 min)-Traci St. Martin Use of Semantic MediaWiki: CTCAE Revision Project (20 min) Background of CTCAE Revision Project-Ann Setser CTCAE Community Requests for additional functionality-Traci St. Martin Demonstration of Media Wiki-Traci St. Martin VCDE Input into CTCAE Revision Project-Lewis Frey Lessons Learned from CTCAE Revision Project-All (5 min)

42. CTCAE VCDE Subgroup Scope Propose governance and structure for the CTCAE caBIG terminology. Proposals that are designed to help CTCAE pass the VCDE vocabulary review for terminologies that support value domains within the caBIG infrastructure Support CTEP’s need for multiple formats of CTCAE such as Excel and PDF as well as a paper version of the terminology support for use in clinical settings consistent with the way it is currently used Subgroup: Salvatore Mungal, Mike Riben, Stuart Turner, Lewis Frey & Larry Wright

43. CTCAE V3.0 PDF Format

44. Agenda Introduction to Semantic MediaWiki (2 min)-Traci St. Martin Use of Semantic MediaWiki: CTCAE Revision Project (20 min) Background of CTCAE Revision Project-Ann Setser CTCAE Community Requests for additional functionality-Traci St. martin Demonstration of Media Wiki-Traci St. Martin VCDE Input into CTCAE Revision Project-Lewis Frey Lessons Learned from CTCAE Revision Project-All (5 min)

45. Lessons Learned Overall – good learning experience, great user community leadership! Modified Development process important for quicker deployments to react to user needs Stage is “Production” Developed build script to make deployments easier Quick response important for new users Encountered slowness due to maintenance by infrastructure team Maintenance schedule modified to accommodate user availability needs Workflow Fully understanding workflow early on is important for determining features needed for users and for processing proposals Workflow functionality not developed for CTCAE which necessitated developing scripts to generate spreadsheets for reviewing data

46. Lessons Learned Requirements defined before NCI clinical team understood Wiki ‘Maintain paper format’ Reluctant users – clinicians Performance – very slow at times Data in – nothing out No summary data No analysis of complete vs. to-do Data quality poor Misspellings; Upper / lower case errors Non-MedDRA terms Abbreviations Inconsistent use of indicators – Edit, Split, New Identified need for additional indicator – Delete Lack of ‘checks’ associated with indicators

47. Next Phase of Wiki Work Training for WG members Well-defined instructions Strive for consistent use of indicators – Edit, Delete, New Accurate data entry (spelling, case, no abbreviations, etc.) New terms must be pre-approved with MedDRA Performance – ? Data in – Wiki team working on provision for summary/report data