Strategies LLC Taxonomy November 2, 2005Copyright 2005 Taxonomy Strategies LLC. All rights reserved. Information Classification in the Workplace Joseph A. Busch

2 Taxonomy Strategies LLC The business of organized information Agenda v Why information classification is needed v What the research says v Agricultural economics case study

3 Taxonomy Strategies LLC The business of organized information Information access issues, problems, and concerns v Enormous volumes of information within organizations v Diversity of assets  Content and technology v Complex and IT-oriented standards .NET, SOAP, WSDL, etc. v Limited (if any) integration with applications:  Search engines  Information management applications  Back office transaction-based systems  Analytical systems  …

4 Taxonomy Strategies LLC The business of organized information Finding information should not be about “Feeling Lucky”

5 Taxonomy Strategies LLC The business of organized information Finding information requires multiple approaches

6 Taxonomy Strategies LLC The business of organized information Wine shopping by facets

7 Taxonomy Strategies LLC The business of organized information

8 Taxonomy Strategies LLC The business of organized information Powered by

9 Taxonomy Strategies LLC The business of organized information Facets on FirstGov.gov site Organization Content Type Frequency Audience

10 Taxonomy Strategies LLC The business of organized information about 3,890,000 results

11 Taxonomy Strategies LLC The business of organized information 2,199 results

12 Taxonomy Strategies LLC The business of organized information More than 1,000 relevant results

13 Taxonomy Strategies LLC The business of organized information

14 Taxonomy Strategies LLC The business of organized information

15 Taxonomy Strategies LLC The business of organized information

16 Taxonomy Strategies LLC The business of organized information Agenda v Why information classification is needed v What the research says v Agricultural economics case study

17 Taxonomy Strategies LLC The business of organized information Usability research— Taxonomy compared to search results lists v “We found that users preferred a browsing oriented interface for a browsing task, and a direct search interface when they knew precisely what they wanted.” Marti Hearst (and others) v “The category interface is superior to the list interface in both subjective and objective measures.” Hao Chen & Susan Dumais

18 Taxonomy Strategies LLC The business of organized information Taxonomy compared to search result lists Median Search Time in Seconds In top 20 results Not in top 20 results Category is 36% faster Category is 48% faster Source: Chen & Dumais

19 Taxonomy Strategies LLC The business of organized information Time saved— Taxonomy compared to search result lists v 1 hour per day searching x 36% faster = 22 minutes each day v 22 minutes x 250 working days per year = 5500 minutes or 92 hours per year

20 Taxonomy Strategies LLC The business of organized information User interface survey— Which search UI is ‘better’? v Criteria  User satisfaction  Success completing tasks  Confidence in results  Fewer dead ends v Methodology  Design tasks from specific to general  Time performance  Calculate success rates  Survey subjective criteria  Pay attention to survey hygiene: – Participant selection – Counterbalancing – T-scores Source: Yee, Swearingen, Li, & Hearst

21 Taxonomy Strategies LLC The business of organized information User interface survey — Results (1) Which Interface would you rather use for these tasks? Google-like Baseline Faceted Category Find images of roses1516 Find all works from a certain period230 Find pictures by 2 artists in the same media129 … Overall assessment: Google-like Baseline Faceted Category More useful for your usual tasks428 Easiest to use823 Most flexible624 More likely to result in dead-ends283 Helped you learn more131 Overall preference229 … Source: Yee, Swearingen, Li, & Hearst

22 Taxonomy Strategies LLC The business of organized information User interface survey — Results (2) Faceted Category Google-like Baseline Source: Yee, Swearingen, Li, & Hearst

23 Taxonomy Strategies LLC The business of organized information Trusted advisers— Taxonomy avoids costs v “The amount of time wasted in futile searching for vital information is enormous, leading to staggering costs …” Sue Feldman, v Poor classification costs a 10,000 user organization $10M each year—about $1,000 per employee. Jakob Nielsen, useit.com

24 Taxonomy Strategies LLC The business of organized information Knowledge workers spend up to 2.5 hours each day looking for information … … But find what they are looking for only 40% of the time. Source: Kit Sims Taylor

25 Taxonomy Strategies LLC The business of organized information 25% 8% Knowledge workers spend more time re-creating existing content than creating new content Source: Kit Sims Taylor (cited by Sue Feldman in her original article)

26 Taxonomy Strategies LLC The business of organized information Agenda v Why information classification is needed v What the research says v Agricultural economics case study

27 Taxonomy Strategies LLC The business of organized information Task-based test* v 15 representative questions were selected  Perspective of various organizational units  Most frequent website searches  Most frequently accessed website content  Correct answers to the questions were agreed in advance by team. v 15 users were tested  Did not work for the organization  Represented target audiences v Testers were asked “where would you look for …”  “under which facet… Topic, Commodity, or Geography?”  Then, “… under which category?”  Then, “…under which sub-category?”  Tester choices were recorded v Testers were asked to “think aloud”  Notes were taken on what they said v Pre- and post questions were asked  Tester answers were recorded * Based on Donna Maurer’s usability work with the Australian government

28 Taxonomy Strategies LLC The business of organized information Task-based testing: Representative questions 1. How much cotton is imported from China? 2. What are the impacts of “mad cow" disease on U.S. meat production, sales? 3. What is the average farm income level in your state? 4. How much of our diet comes from fast food? 5. How many people receive WIC benefits (Special Supplemental Nutrition Program for Women, Infants, and Children)? 6. How much acreage is planted to genetically engineered corn? 7. What is the cost of foodborne illness in the United States? 8. What part of food costs go to farmers, retailers? 9. Which States produce the most tobacco? 10. What percentage of farms in the United States are small farms? 11. What are the costs and benefits associated with providing more traceability in the U.S. food supply? 12. How many people in America don’t get enough to eat? 13. What is behind the trade balance (surplus or deficit) in agricultural goods? 14. What is the extent of conservation compliance? How does that impact farmer's decisions? 15. What are the impacts of foreign trade restrictions on U.S. farmers, U.S. food prices?

29 Taxonomy Strategies LLC The business of organized information Task-based testing: Closed card sorting 3. What is the average farm income level in your state? 1.Topics 2.Commodities 3. Geographic Coverage 1.Topics 1.1 Agricultural Economy 1.2Agriculture-Related Policy 1.3Diet, Health & Safety 1.4Farm Financial Conditions 1.5Farm Practices & Management 1.6Food & Agricultural Industries 1.7Food & Nutrition Assistance 1.8Natural Resources & Environment 1.9Rural Economy 1.10Trade & International Markets 1.4Farm Financial Conditions 1.4.1Costs of Production 1.4.2Commodity Outlook 1.4.3Farm Financial Management & Performance 1.4.4Farm Income 1.4.5Farm Household Financial Well-being 1.4.6Lenders & Financial Markets 1.4.7Taxes

30 Taxonomy Strategies LLC The business of organized information Task based testing: Card sort analysis Find-it TasksUser 1User 2User 3User 4User 5 1. CottonCotton AsiaCotton 2. Mad cowCattleFood SafetyCattle 3. Farm incomeFarm Income US StatesFarm Income 4. Fast food Food Consumption Diet Quality & Nutrition Food Expenditures Diet Quality & Nutrition 5. WICWIC Program 6. GE CornCorn 7. Foodborne illness Foodborne Disease Consumer Food Safety Foodborne Disease 8. Food costsFood PricesMarket StructureMarket Analysis Food Expenditures Retailing & Wholesaling 9. TobaccoTobacco 10. Small FarmsFarm Structure 11. TraceabilityFood SystemLabeling Policy Food Safety Innovations Food Safety PolicyFood Prices 12. HungerFood Security 13. Trade balance Commodity Trade Trade & Intl Markets Commodity TradeMarket Analysis Commodity Trade 14. Conservations Cropping Practices Conservation Policy 15. Trade restrictionsTrade Policy Food Safety & TradeWTOMarket Analysis Commodity Trade

31 Taxonomy Strategies LLC The business of organized information Task based testing: Card sort results v In 80% of the trials users looked for information under the categories that we expected them to look for it. v Breaking-up topics into facets makes it easier to find information, especially information related to commodities.

32 Taxonomy Strategies LLC The business of organized information Task based testing: Card sort results Test Questions % Correct % Agree 1. Cotton91%82% 2. Mad cow73%64% 3. Farm income100%55% 4. Fast food91%73% 5. WIC100% 6. GE corn100% 7. Foodborne illness82% 8. Food costs55%27% 9. Tobacco100% 10. Small farms91% 11. Traceability36%18% 12. Hunger100%73% 13. Trade balance36%64% 14. Conservation91% 15. Trade restrictions55%36% Possible change required. Change required. Possible error in categorization of this question because 64% thought the answer should be “Commodity Trade.” On these trials, only 50% looked in the right category, & only 27-36% agreed on the category. Policy of “Traceability” needs to be clarified. Use quasi-synonyms.

33 Taxonomy Strategies LLC The business of organized information Task-based testing: User satisfaction survey v Was it easy, medium or difficult to choose the appropriate Topic? – Easy – Medium – Difficult v Was it easy, medium or difficult to choose the appropriate Commodity? – Easy – Medium – Difficult v Was it easy, medium or difficult to choose the appropriate Geographic Coverage? – Easy – Medium – Difficult

34 Taxonomy Strategies LLC The business of organized information User satisfaction survey: Results EasierMore Difficult

35 Taxonomy Strategies LLC The business of organized information Summary 1. Classifications and classification-like schemes are being used to facilitate information seeking in the workplace. 2. Users take advantage (and prefer) this type of scheme (faceted navigation) when it is made available in the user interface. 3. Hierarchical navigation is guided by the UI. 4. It is best combined with keyword searching. E.g., keyword search followed by faceted navigation of results. 5. What are the challenges they face in navigating classificatory structures? 6. What are the differences between their use of traditional classification schemes such as the Dewey Decimal Classification and navigation of newly developed online directories?

Strategies LLC Taxonomy November 2, 2005Copyright 2005 Taxonomy Strategies LLC. All rights reserved. Questions Joseph A. Busch