SYSTEM TESTING AND DEPLOYMENT Lecture Seven (Chapter 7, Notes; Chapter 8, Textbook)

System Testing & Deployment after Knowledge Codification Capture Tools Programs, books, articles, experts Intelligence gathering Shells, tables, tools, frames maps, rules KNOWLEDGE CAPTURE (Creation) KNOWLEDGE CODIFICATION DATABASES TESTING AND DEPLOYMENT Explicit Knowledge KNOWLEDGE SHARING KNOWLEDGE TRANSFER KNOWLEDGE BASE

Stages of KMSLC Iterative Rapid Prototyping Evaluate Existing Infrastructure Knowledge Capture Design KM Blueprint Verify and validate the KM System Implement the KM System Manage Change and Rewards Structure Form the KM Team Post-system evaluation Iterative Rapid Prototyping KM system development life cycle is largely composed of 8 stages, which we will briefly discuss in the remaining slides. You should obtain description of their details in both the lecture notes and the prescribed text.

Quality Assurance KM system should meet expectations of members represented in the KM team. Performance depend highly on quality of explicit/tacit knowledge stored in KB. For expert, quality relates to reasoning process that produces reliable and accurate solutions. For user, quality relates to system’s ease of use and efficiency. For knowledge developer, quality relates to how valid the knowledge sources are and how well they are codified into KB. Today, we will learn about the KMSLC, that is the actual stages involved in the development of a working KMS: First, I will highlight several major challenges one might face when constructing KM systems. This will set the stage for the remaining discussion on KMSLC. For some of you, you might be familiar with the system development life cycle for conventional systems. But how about KMSLC? What are the main similarities and differences between the conventional life cycle and the KMSLC that we will be learning today? Next, we will attempt to understand some characteristics of a user (or person using a system) as compared to an expert who are knowledgeable in an area where we are attempting to apply KM. Altogether, we can identify 8 distinct stages in a KMSLC (at least as outlined in our textbook by Awad and Ghaziri). For the most part, the stages are sequential. But, there are certain stages that are iterative because of the methodology applied in developing KMS.

KMS Testing To control quality, performance and efficiency of both KB and KMS. Types of Testing Logical Testing, make sure that the system produces correct results. User Acceptance Testing, follows logical testing and check system’s behaviour in a realistic environment. Today, we will learn about the KMSLC, that is the actual stages involved in the development of a working KMS: First, I will highlight several major challenges one might face when constructing KM systems. This will set the stage for the remaining discussion on KMSLC. For some of you, you might be familiar with the system development life cycle for conventional systems. But how about KMSLC? What are the main similarities and differences between the conventional life cycle and the KMSLC that we will be learning today? Next, we will attempt to understand some characteristics of a user (or person using a system) as compared to an expert who are knowledgeable in an area where we are attempting to apply KM. Altogether, we can identify 8 distinct stages in a KMSLC (at least as outlined in our textbook by Awad and Ghaziri). For the most part, the stages are sequential. But, there are certain stages that are iterative because of the methodology applied in developing KMS.

Hurdles in KMS Testing Subjective nature of tacit knowledge. Lack of reliable specifications to guide testing How to decide whether knowledge is complete Human errors Lack of time User interface un-friendly

Logical Testing Approaches Knowledge base formation Structure of knowledge as it relates to circular or redundant errors Consistency, correctness, and completeness of knowledge base rules Knowledge base functionality Confidence Reliability of knowledge base

Logical Errors Circular errors - contradictory in meaning or logic Redundancy errors - offer different approaches to the same problem Unusable knowledge - comes up no matter conditions succeed or fail Subsumption errors - if one rule is true, one knows the second rule is always true Inconsistent knowledge - the same inputs yield different results

User Acceptance Testing Approach Select a person or a team for testing Decide on user acceptance test criteria Develop a set of test cases unique to the system Maintain a log on various versions of the tests and test results Field-test the system

Requirements for Test Team Independent of the design or codification of the system Understand the knowledge problem and computer technology Well versed in the organization’s goals Know how the system being tested will fit into the designated area of operation

Content of Test Plan Items of KM system that require testing Schedule of user acceptance testing Description of test methods that are approved by user and company Guidelines for documenting test results

User Acceptance Criteria Accuracy and correctness of outcome for test items Adequacy of the solutions Adaptability to novel input Ease of use Face validity or credibility Robustness Technical/operational test

User Acceptance Test Techniques Face validation Test Cases Sub-systems validation

Factors affecting System Deployment Technical Organizational Procedural Behavioural Political Economical

Steps in KMS Deployment Transfer of KM System from knowledge developer to the organization’s operating unit. Transfer of KM’s skills from knowledge developer to the organization’s operators.

Successful KM System Deployment User has prior experience with computer applications User has been involved in the building of the KM system Benefits of the KM system is immediate and measurable First impression Management support the system from start Adequate user training

Factors affecting User Training Primary The user’s knowledge of knowledge-based system Complexity of KM system and user interface’s ease of use Secondary Trainer’s technical experience/ communication skills Environment of training venue

Pre User Training Session Define how the KM system agrees with organizational mission Demonstrate how the system can support organizational goals Allocate adequate resources for the training project Advocate positive effects of the system Explain cost-benefit analysis of the KM system

Overcome Resistances to Change User attitude survey Communication sessions Role negotiation

Post-Implementation Review How KM System has changed accuracy/timeliness of decision making How KM System has affected attitude of end users Whether the system has caused constructive organizational changes? Whether the system has benefited the operation cost?

Security Issues The new system should provide password / protocol protection. Security procedures should be consistently observed. Access should be restricted regarding update of the KB.

Circular Errors Rule 1: Rule 2: IF John and Rob are co-owners of company A THEN John and Rob own the same company Rule 2: IF John and Rob own the same company THEN John and Rob are co-owners

Redundancy Errors Rule 1: Rule 2: IF (Salary > 50,000 OR HOME > 100,000) THEN Loan = ‘yes’ Rule 2: IF (Salary > 60,000)

Unusable Knowledge IF humidity is 5% AND heavy_rain = ‘yes’ THEN some actions

Subsumption Errors Rule 1: Rule 2: IF A AND B, THEN D IF A AND B AND C, THEN D