1. Help-Desk Systems Stephen Lee-Urban November 17, 2006

2. 2 References Experience Management: Foundations, Development Methodology, and Internet- Based Applications, Ralph Bergmann. –Chapter 9: Developing and Maintaining Experience Management Applications –Chapter 11: Experience Management for Self- Service and Help-Desk Support

3. 3 Outline Motivation / Goals / Background The HOMER architecture Developing & Managing EM* Applications Evaluation of HOMER *EM = Experience Management

4. 4 Motivation Complexity of technology increasing –Operation, Maintenance, Repair –Probability of failure grows exponentially with complexity –Expertise in all features unreasonable Problem solving experience distributed –Experience overlap is variable  Experience Management System (EMS)

5. 5 Goals of EMS Create knowledge repository –Store problem solving experience –Simplify trouble-shooting complex technical domain –Domain is likely to change over time Use knowledge repository –By varying levels of expertise –In time-critical situations

6. 6 Experience Management vs CBR Experience Management CBR (Organization) (IDSS) 2. Reuse 3. Revise 4. Retain Case Library 1. Retrieve Background Knowledge Experience base Reuse- related knowledge Problem acquisition Experience evaluation and retrieval Experience adaptation Experience presentation Complex problem solving Development and Management Methodologies BOOK Slide: Dr. Munoz-Avila

7. 7 Help Desk Support Levels Key-user Specific technician Vendor Help-Desk sys. Bottlenecks at each level. –Problems recur. –Frustration for all parties, and wasteful of resources! “Managerial” Aids to Help-Desk: –Inventory systems, trouble-ticket tools, call-tracking software –Don’t support diagnosis nor serve as knowledge repository http://www.simpsonstrivia.com.ar/simpsons-photos/wallpapers/homer-simpson-wallpaper-brain-1024.jpg CBR

8. 8 Outline Motivation / Goals / Background The HOMER architecture Developing & Managing EM* Applications Evaluation of HOMER

9. 9 The HOMER System German: “HOtline Mit ERfahrung” Developed as part of INRECA-II project For CAD/CAM help-desk at DaimlerChrysler in Sindelfingen Generic experience management architecture & set of related tools –Stores experience in CB for access, reuse and extension –Drastically decreases problem solving time http://www.aeria.phil.uni-erlangen.de/photo_html/portraet/griechisch/dichter/homer/homer3.JPG http://gattaca.com.ar/weblog/wp-content/homer.gif

10. 10 Structure and Representation Determines experience management approach HOMER uses an object oriented approach to model experience. Advantages of OO approach: –Structure of system to diagnose is representable in detail –Symptoms (attr.) easily related to owning object –Semantics of prob. description can be captured and used for selecting appropriate prior exp. –High retrieval accuracy can be achieved –Particularly suited for diagnosis help-desks w/ complex equipment  Can show many hard to diagnose faults –Can be used to help guide help-desk operator while describing and entering cases –Better domain model = easier maintenance and use Disadvantages of OO approach: –Harder to create model –Additional effort in beginning of knowledge acquisition Representation should be discussed with system admin. Shallow?

11. 11 Case Structure Modeled according to approach used by help-desk operators to solve problems Feasible for most complex help-desks Topic: problem area (e.g. hardware) Subject: failing object (e.g. printer) Behavior: way (miss-) behaves (e.g crashes) Minimum set of attribute- value pairs describing symptoms necessary to fault diagnosis. In OO, all attributes are related to certain object. Fault: problem cause Remedy: problem fix Represented as (hyper-) text Case: Complete path from prob. to solution

12. 12 Experience Base Partitioning Domain size & complexity + demanded accuracy & consistency = partitioning Two kinds of cases: –Approved: experience reviewed by top-level experts. “Best Practice” for problem solving –Open: recently captured, pending validation (correctness, completeness, & utility) Case base separated into: –Case Buffer – all open cases –Main Case Base – all approved cases All available to help- desk operators

13. 13 HOMER Architecture Client-Server –Shared domain model (DM) + case base (CB) –Eases maintenance of DM + CB –Server accessible via intranet / internet –“Fat” client reduces network traffic Object-Oriented experience model –Not “shallow”  users are experienced –For non-trivial problems

14. 14 Lowest access rights Daily user Retrieval & acquisition Middle rights Case maint. & case approval Redundancy & consistency checks Buffer to base May modify value ranges of attributes Highest rights Creates & maint. domain and case model +/- attr & concepts Admin users and rights

15. 15 The Server CBR-Works server stores model & CB, CBR-Works modeling tools provided by server for domain modeling, case & model maintenance, & initial case acquisition Domain model snapshot…

16. 16 Hierarchical domain concepts Help Desk Case Attributes Problem, holds failure Situation, holds symptoms - Hierarchical, sub-concepts - Structure helps maint. & retrieval Loesung, holds solution Administrativa, holds organizational & statistical information

17. 17 The Client Interface to server for case retrieval, case acquisition, & case browsing Written in Java “Hotline” component for help-desk operator –Only shows relevant information –Assists via two modes: user/system driven “Case Browser” component for experience author –For access to CB and case buffer –Revision, extension, approval of “open” cases –Removal of outdated cases

18. 18 Hotline Component (HC) Used mainly during hotline operations Supports help-desk operator during problem-solving process Four main modes of execution –Problem description (initialization/refinement) –Situation description (user/system driven) –Solution retrieval (manual/automatic) –Retain w/ feedback questions answers to questions solutions (cases) problems

19. 19 HC: Problem Description Gives initial info. on user’s problem Answer following questions consecutively: –Problem’s topic? (e.g. output, software, …) –Topic’s subject? (e.g. output: plotter or printer) –Behavior? (e.g. “no output at all”) problems question (free text) values value rangeunits

20. 20 HC: Situation Description Problem determines relevant case structure (i.e. situation template) –Contains possible symptoms for prob. –Each symptom associated w/ question Symptom attribute values complex/simple Two modes of operation –User-driven: (no guidance, direct entry) –System-driven: Shows sorted view of most relevant questions Based on attributes with highest info gain

21. 21 HC: Solution Retrieval Experience retrieved based on problem and situation descriptions Manual (batch) / automatic (per question) Each row a solution, sorted by relevance (similarity) Solutions viewed (read-only)

22. 22 HC: Retain/Feedback Retain case when it –Has new experience –Requires case model extension (e.g. new value to type) Case entry interface allows –Final modifications (e.g. finish case descr.) –Annotating why op. thinks case should be retained

23. 23 Case Browser Component Used by experience author to manage CB Works on both approved and open cases Can perform following operations: –Case creation –Case copy –Case deletion –Case approval Problem, situation, solution, administrativa question (free text) values value rangeunits

24. 24 Outline Motivation / Goals / Background The HOMER architecture Developing & Managing EM* Applications Evaluation of HOMER

25. 25 Developing & Maintaining EM Applications IT companies require efficient, effective application development –Guidelines/Methods of implementing apps –Also seek to preserve past project experience INRECA methodology –Targeted at industrial EM applications –Developed in the INCRECA-II European ESPRIT project (1996-9)

26. 26 EM Model Knowledge Kernel Problem Solving Cycle Knowledge Acquisition & Knowledge Maintenance Technical, organizational & managerial aspects

27. 27 Three Process Types Technical –Describe development of system & required documentation –E.g. requirements analysis, system design, implementation and testing Organizational –Address parts of business process in which software will be embedded –E.g. training end users, archiving request records, updating & maintaining the help-desk system Managerial –Provides environment and services for developing software that meets product requirements and project goals –E.g. project planning, monitoring, quality assurance

28. 28 Methodologies Makes development an engineering activity, rather than an art Use of methodology provides benefits: –Productivity, Quality, Communication, Management decision making INRECA methodology based on software engineering principles, covering: –Project management (cost assessment, schedules, project plans, etc.) –Product/Deliverable specification –Product development and maintenance –Analysis/Organization of target env. for CBR system

29. 29 INRECA Basic philosophy: experience based construction of experience management applications Self-application of principles of experience reuse Experience modeled as structured text documents, hyper-text linked. –Origin: Software process modeling

30. 30 Software Process Modeling Well defined terminology to describe the engineering of a product –Process, “what”: basic step to carry out, transforming input into output. Defined by a goal, a set of alternative methods, input/output/modified products, & required resources (agent/tool) –Method (simple/complex) “how”: Detailed specification of a way to achieve a process’ goal –Product: goal of the process INRECA notation

31. 31 INRECA Process Models Make explicit all processes, products, methods, resources and interactions Diagrams insufficient  Each element must be described in detail –Called a process model –Solid basis for project planning (e.g. effort calculable based on processes involved) General vs. Specific descriptions –Common Generic/Cookbook vs. Project level

32. 32 The INRECA Experience Base Collection of processes, products, & methods common across EM apps. Processes, products, & methods tailored to class of applications (e.g. help-desk). Each class has recipe Recipe: has process models describing how app. of that class be developed & maintained Describes experience of an instance of a completed project Project-specific info. (e.g. processes carried out, effort of processes, etc.)

33. 33 The Common Generic Level (CGL) Overview of top-level processes and products in Bergmann, chapter 9 –General enough to occur in many projects –Problem statement  “vision document”  goal checklist  feasibility study  detailed analysis of organization  project plan  software development  evaluation –Meanwhile, experience base can be consulted for all of the above processes Consult book for thorough exposition

34. 34 Go / No-go (analysis & elicitation) Organizational (planning) Implementation Maintenance

35. 35 CGL: The Three Processes Managing an AI / EM project differs from other IT projects –Concepts & technologies altogether new –Emphasize early awareness training –Ensure user-participation in iterative prototyping process Technical processes very typical of software development projects Organizational includes identifying perceived problems and opportunities at the human & organizational levels

36. 36 Documenting in INRECA Processes, products, & methods documented and stored in experience base “Sheets”  structured page w/ all relevant information in a predefined format –Standardize documentation –Created as web pages for easy access & use –CGL has more than 150 linked sheets –8 type of description sheets: [ generic | specific ] [process | product | simple method | complex method] –Contains references to respective input, output, and modified products of the process

37. 37 Process Description Sheets Recipe/Project Name Process Name Process Goal Input/Output/Modified Products Set of Applicable Methods (names) Agents (e.g. personnel) Required Tools Administrative Information (e.g. sheet author, version, last modified)

38. 38 Product Description Sheets Module/Project Name Product Name (e.g. “requirements doc”) Product Description Administrative Information

39. 39 Simple Method Description Sheets Module/Project Name Method Name Method Description Administrative Information

40. 40 Complex Method Description Sheets Module/Project Name Method Name Method Description Details –Links to a product flow description which contains the relevant sub-processes (byproducts) Administrative Information

41. 41 The INRECA Reuse Procedure Recipes at cookbook level are most useful for building a new application Even projects not fitting a recipe can use processes described in CGL

42. 42 Don’t Forget to Remember! After completing project, include it in the experience base  continuous improvement of the EM software development process

43. 43 Tool Support for INRECA Experience modeling methodology tool implemented in Visio® –Natural choice: shapes, hierarchical, HTML, VBA, database access Knowledge modeling tools –Integrated in the CBR-Works tool (tec:inno GmbH 1999) –CBR-Works Concept & Type Hierarchy Editor for vocabulary modeling –Similarity modeling tools integrated in the Concept & Type editors Also has an editor for adaptation and completion rules

44. 44 Outline Motivation / Goals / Background The HOMER architecture Developing & Managing EM* Applications Evaluation of HOMER

45. 45 Evaluation of HOMER Evaluation performed by INCRECA-II project partners to identify benefits of EMS Incoming calls monitored. Help-desk operator 1 st solves conventionally, then with HOMER –No solution  new case created –Two month test period: 102 calls; 45 unsuitable for HOMER

46. 46 Evaluation (II) Of the 57 (/102) suitable problems –18 solved (i.e. 32%) Ave. resolution time w/out HOMER: 141 min. Ave. resolution time w/ HOMER: 9 min. Correct result a limited bias  HOMER sys. Mode HOMER + CB transferred to a new site –Operators have no experience with the process chain so the cases are of great value –Initial knowledge acquisition gave operators insight into domain.

47. 47 Evaluation (III) Methodology recipe created and used –Impact: productivity, quality, communication, & management decision making –Customer and developer both benefited Creation of project definition from scratch –Took three months –New development team reused recipe to define three new projects, each in < one week (12x speedup!) Were sure all relevant aspects taken into account Basic recipe available  developers focus on domain peculiarities  quality/detail of descriptions greatly enhanced

48. 48 Evaluation (IV) Development & testing of 1 st prototype –About 6 months –Subsequent efforts: 2 weeks (13x speedup) –Less qualified personnel w/out sacrifice of quality Useful as training tool for maintenance & use Message: development of EM system a science, not art –Each process describable –Validity of approach defensible –Realistic expectations of effort by whom and when –Measurable project process

49. 49 Thank You! (Questions?)(Comments?)