AMIA 2008 Monday, Nov :15-1:30 National Library of Medicine National Institutes of Health U.S. Dept. of Health & Human Services UMLS ® Users’ Meeting

2 Welcome & Agenda NLM Overview and Update UMLS Basics Tutorial New DVD Packaging Webcasts & UMLS News page Content Views UMLS Products & Services Group UMLSKS Usability Testing Discussion Questions

3 UMLS Basics Online Tutorial First of many courses, next topic is Customizing the Metathesaurus with MetamorphoSys For entry level users Linked from Home page, New Users’ Guide, etc.

4 New DVD Packaging & Procedures Environmentally friendly self mailers, less waste Announcements and What’s New inside Low inventory prior to next release, users advised to download

5 Webcasts & News Page Webcast schedule to be announced soon. Topics will include: SNOMED & the UMLS New Source Documentation pages MMSYS Unified Object Model & API Lexical Tools and the new LVG Metathesaurus Sources UMLS News page – View news on UMLS Knowledge Sources and related resources, including upcoming Webcasts. UMLS News RSS Feed – Subscribe to receive items from the News page via RSS Feed. UMLS Webcast RSS Feed disabled, these items will appear in News Feed.

UMLS & SNOMED CT® Reporting Requirements UMLS Reporting Requirements Annual Report due January 31st Complete via online form Required to maintain UMLS License SNOMED CT Affiliate Reporting Twice yearly Due January 31 & July 15 Online form coming in 2009, details via in December

7 Content Views A purpose-specific subset of content (atoms, relations, attributes) Anticipated uses –Vocabulary standards involving multiple vocabularies e.g. public health vocabulary, allergy vocabulary –Subsets to support specific use cases e.g. natural language processing, acyclic ontology view –Distribution/sharing of value sets e.g. HITSP designated value sets, user-contributed value sets Representation in the UMLS –Members flagged by CVF field (bit string) in most RRF files Row level member identification Computationally efficient (intersection, union) 3 content views in 2008AA: MetaMap NLP view, VA/KP problem list with and without MedDRA

8 Content View Tools Recently added content view tools in MMSYS Subsetting by CVF Content view highlight in RRF browser Future tools being considered Content view selector Help users find a specific content view User-specified content view Users can specify and share their own content view by a list of UMLS identifiers based on the latest release

9 UMLS Products & Services (UPS) Workgroup Purpose: to develop a list of proposed priorities for UMLS development and customer service for the next several years to promote and support more effective use of UMLS resources Formed in April, biweekly meetings Sources of input – UMLS users’ feedback – annual report, customer service enquiries, listserv postings, usage statistics – Solicited input from knowledgeable users/critics – Published literature

10 UPS Workgroup Cast a wide net Identify a comprehensive list of suggestions Discuss, prioritize and flesh out: – High priority – implement within 3 years – Medium priority – implementation subject to resources – Low priority – will not be considered in the next 3 years Examples of high priority items: – Improve entry level documentation – Enhance documentation of source vocabularies – Simplify MMSys interface – Provide easier access to commonly used vocabularies – Increase availability of UMLS Knowledge Source Server (UMLSKS)

UMLSKS Usability Testing Baseline Usability Testing By Deborah Bennett and Sarah Ward of CSB. 18 participants Licensed UMLS users, Washington DC area. From UMLS mailing list Varying degrees of familiarity with UMLSKS. Overall System Usability Scale Results >80 likes the new system 60 to 79 accepts the new system 0 to 59 dislikes the new system [corrected 2008/12/08]

UMLSKS Usability Testing Initial Impressions Think Out-Loud 8 Tasks: Login Create an Account 6 tasks (rotated) Score for each task failed 1 needed help 2 succeeded Average: 11/16 [corrected 2008/12/08]

UMLSKS Usability Testing Better System Stability (slow responses, down time etc) Better Navigation Orient new users Contextual help (on mouse over) Pointers to Documentation New Default Page Design (concept view) Lexical View vs. Atom (RRF) view.

14 Discussion Questions Do you use co-occurrence data (MRCOC.RRF)? Is the CVF feature useful? Does it need to be changed? What subsets do you use or create? Should we reproduce them? What enhancements would you make to MetamorphoSys? Do you use the MRCXT Builder feature? Request for more suggestions and feedback on the new UMLSKS How can we better support you?

15 Thank you