1. Knowledge and Learning in Complex Business Systems Zuobing Xu University of California, Santa Cruz (Silicon Valley Center) Ram Akella, Kristin Fridgeirsdottir, Eric Wang, Arvind Vidyarthi INFORMS Pittsburgh, November 6, 2006

2. 1 Overview: Perspective on Learning and Knowledge - Semiconductor process learning (KLA, AMD, INTEL) - Automotive product development learning (Delphi) - IT service center knowledge mining and learning (IBM, Cisco)

3. 2 Perspective on Learning and Knowledge - 1 Semiconductor process yield management and learning. The models were new, but still in the OR framework

4. 3 Perspective on Learning and Knowledge - 2 - In the automotive environment, OR modeling ran into a roadblock - Forced to learn ontology to speed up learning in new product development - Subsequently, discovered that learning and knowledge management are closely related to text mining and information retrieval, Applicable in airlines, health care, service/call centers

5. 4 Perspective on Learning and Knowledge - 3 IT Service Center - Millions of documents (services call logs, Technical documents) - We use text mining and information retrieval to manage and extract knowledge efficiently.

6. 5 Is There a Unifying Framework for Knowledge and Learning? Do not know, but.. Need to combine Operations Management Knowledge Management Text Mining Specific research examples follow

7. Process Learning in the Semiconductor Industry

8. 7 In-line Monitoring for Semiconductor Product- Process Learning and Manufacturing IsolationMetal 2Poly 1Poly2Metal 1 Wafer Probe Þ 2110 Þ KLA Photo Depo Etch Wafer processingInspectionClassification Off-line Review In-line ADC or 10 days 30 days Yield

9. 8 Business Need Methodology: For rapid product and process learning – How to use inspection machines to improve yield per machine

10. 9 7 209 Detection Delay Event Occurring Event Detected Source Isolated Fix Validated Source Isolation Root Cause Analysis Corrective Actions 17 2 Hours 50 In- Control Defect level REDEFINED GOAL: MINIMIZING TIME TO DETECT AND FIX YIELD EXCURSION ( AN EXCURSION @ ETCHER IN FAB A) Goal Optimize procedures and inspection-review machine usage to reduce delay to detect and fix yield excursion Using defects as surrogates ( linking defects to yield is a technology problem in electrical/computer engineering)

11. 10 Resources & strategies IsolationPoly1Poly2Metal1Metal2 Market Review & classification Review & classification Source identification Source identification Data/information flow Root-cause analysis Root-cause analysis Corrective actions Corrective actions Inspection Validation Goal Detect killer defect excursion faster through efficient integrated inspection-review cycles Trade-off: Time versus benefit and cost Conversion Of Defect Data To Yield Information And Action

12. 11 Analytics Developed generalized Neyman-Pearson Lemma for multi-dimensional control charts Model queuing effects of inspection-review-isolation-fix-validation cycle

13. Business and Knowledge Management in the Automotive Industry

14. 13 Shift in Learning Paradigm: From Numerical to Text Knowledge and learning are now associated with text Traditional OR models cannot be directly used! What do we do?

15. 14 Triggered Learning Process: “Dynamic’ Ontology Usage Step #2 – Update Step #3 – Communicate Step #4 – React Step #1 – Feedback GMDealer Customer TLP is a structured approach that 1.Feeds back lessons learned created by downstream organizational personnel 2.To a staff that condenses these lessons learned into an ontology and, 3.Communicates these items to NPD 4.NPD personnel reacts to this information as it arrives by incorporating it into the new product or process under development First time quality Safety and ergonomics Formal OEM complaints WarrantyLong-term durability NPD ProductionOEM Field <3 yrs Field >3 yrs DelphiGMDealerCustomer Lessons Learned Ontology

16. 15 Step 2: Update Ontology Staff used To summarize the Lesson Learned in the ontology “Attach” the Lesson Learned documents provided

17. Business and Knowledge Management in IT Services

18. 17 Service Center Work Flow Search or browse for solution (self service) Call service center Solve the problem by their own knowledge Search or browse internal database for solution Customer have trouble with product Deskside Expert help

19. 18 Improve IT Service Performance by Text Mining and Information Retrieval - Text Mining : Organize Technical Documentation Automatically - Information Retrieval : Retrieve and Find the Correct Solution Can we trust computer algorithm? - We need human guidance ( Relevance Feedback). How can we reduce human effort ? - Active learning ( Actively selecting document for human evaluation )

20. 19 Query Retrieval Engine Retrieval Engine’s Ranking d1=3.4 d2=2.0 … d10=1.0 Judgments d1(Non-relevant) d2(Non-relevant) … d10 (Relevant) Feedback Query Update Document Collection Active Learning in Information Retrieval Service Engineer or Customer

21. 20 Active Learning to Choose Feedback Documents Goal : choose a set of documents containing most information for user to evaluate. Relevance Select highly ranked retrieval documents given by search engine Diversity Increase distance between selected documents. Density Choose document in high density region.

22. 21 Experimental Results Mean Average Precision (MAP) - Measure of overall ranking accuracy. Precision at 10 documents(Pr@10) - Measure of the precision for the first 10 documents.

23. 22 Organize Documents - Technical documents and case logs always have some meta data indicating its product category. - But problem description, root cause, solution are not well categorized. - Design a new active semi-supervised text clustering algorithm to group documents.

24. 23 Text Mining: Semi-supervised clustering with pair-wise constraints (Organize Trouble Tickets) Can not be linked!!!!

25. 24 Future Research for Knowledge Management of Service Center - Using OR to model the benefit and cost for human interaction. - Feed the lessons learned from service center to New product develop process. - Combine text analytics with call center scheduling.

26. 25 Conclusions: Unified Model for Knowledge and Learning Data (Defect signal, Documents) Information Knowledge Management Reduce operations cost + Increase overall efficiency Statistical Detection Building Ontology Text Clustering Information Retrieval … Queuing Model Resource Allocation … Goa l