Meta Learning and Active Learning: Meta Learning and Active Learning: Collaborative Knowledge Discovery in Distributed Systems Dr Yonghong Peng Department of Computing School of Informatics University of Bradford A+B=?

Meta Learning and Active Learning: Meta Learning and Active Learning: Collaborative Knowledge Discovery in Distributed Systems  Knowledge Communication in Distributed Systems;  Knowledge Discovery/Management.  Meta-Learning and Active Learning;  CKD framework and Key Techniques. Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Systems are toward distributed Systems are toward distributed Centralized Systems: Their actions are coordinated based on the communicating with one control centre. This kind of centralized communication is usually less efficient. Collaborative Knowledge Discovery in Distributed Systems Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Systems are toward distributed Systems are toward distributed Collaborative Knowledge Discovery in Distributed Systems Decentralized Systems: all component are autonomous, and their actions are coordinated based on their communication. Free of central control. Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Characteristics of Distributed Systems Characteristics of Distributed Systems Their effectiveness and efficiency rely on the capability of collaboration among the components/agents. The capability of collaboration comes from the ability of communication between all the components. Collaborative Knowledge Discovery in Distributed Systems Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Issues in Distributed Systems Issues in Distributed Systems Collaborative Knowledge Discovery in Distributed Systems Instead of using data communication, knowledge sharing is the key for a successful distributed system. An new concept is called knowledge mobility. Data Communication:  Data communication is time-consuming and expensive.  Data collected from a variety of sources are always heterogeneous. Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Communication via Data or Knowledge Communication via Data or Knowledge Collaborative Knowledge Discovery in Distributed Systems This is what I have! I do not know what it is. This is what you need! It is A not B, and A is bigger than B 20%. Communication via Data Communication via Knowledge Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Knowledge is not easy to obtain Knowledge is not easy to obtain Collaborative Knowledge Discovery in Distributed Systems Data is available everywhere but is difficult to use;  Data is easy to collect: availability of large amount of data. Knowledge is easy to use but is difficult to obtain.  Shortage of domain experts;  Knowledge obtained from different experts may be Inconsistent. Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Knowledge Discovery and Management Knowledge Discovery and Management Collaborative Knowledge Discovery in Distributed Systems knowledge management: to Make use of knowledge effectively Knowledge Verification Knowledge Updating; Knowledge Reuse. Strategies Knowledge Discovery: To extract knowledge from data using machine learning and data mining techniques. Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Knowledge Discovery and Management Knowledge Discovery and Management Collaborative Knowledge Discovery in Distributed Systems Drawback of the current techniques: -- They are inefficient as they are working in a passive style! Data Mining: we do not know what we will get. No interaction between the existing knowledge of upcoming mining activities. Knowledge management are performed after the knowledge has been extracted, not within the process of learning. Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Collaborative Knowledge Discovery (CKD) Collaborative Knowledge Discovery (CKD) Collaborative Knowledge Discovery in Distributed Systems New Strategies: Managing the knowledge when learning. Collaborative Learning: To learn how to work with others Learn according to what I want to know. Learn according to what the partners want to know. Idea: Meta-Learning and Active Learning based Collaborative Knowledge Discovery. Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Meta-Learning and Active Learning Meta-Learning and Active Learning Collaborative Knowledge Discovery in Distributed Systems What is Meta-Learning?  Meta-learning is to learn how a learner works. Applications:  To select the suitable algorithm(s) (Current);  MetaL European Project (  To learn the collaborative knowledge (NEW). Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Meta-Learning Process Meta-Learning Process Collaborative Knowledge Discovery in Distributed Systems Data Collection Knowledge generation Machine Learner Data pre- processing Expected Performances Application Objectives Meta- Learner Meta- Knowledge Data Characterisation Model/Algorithm Characterisation End-users Meta-Data Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Meta-Learning for Algorithm Selection Meta-Learning for Algorithm Selection Collaborative Knowledge Discovery in Distributed Systems New Data Post- processing Machine Learner Pre- processing [LA1,f1,f2,……….., Acc1, Time1] …………. [LAn,f1,f2,……….., Acc_n, Time_n] …………. Ranker Data Characterisations Ranking ALs LA1; LA3; …… Ranking LAs AL2; AL1; …… Meta-Knowledge Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Active Learning Active Learning Collaborative Knowledge Discovery in Distributed Systems What is Active-Learning? Data Knowledge Data DM Data Miner Active Learner Knowledge Passive Learning: Data-Driven learning: Active Learning: Objective-Driven learning: Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Collaborative Knowledge Discovery in Distributed Systems Active Learning Active Learning Tasks of Active Learning: keep on ansowering 1)What can I learn? 2)How to Learn? Active Learning Process: Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Collaborative Knowledge Discovery in Distributed Systems Meta-Learning based Active Learning Meta-Learning based Active Learning Approaches: 1)what I can learn from the data: Data Characteristic Techniques (DCT); Perform the rough mining (RM) with the sampled data; 2)Select appropriate methods: Select the target models according to the objectives; Using Meta-Learning to select the learning methods. Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Data Characterisation Techniques (DCT) Data Characterisation Techniques (DCT)  StatLog type DCT:  Simple Measures (e.g. number of attributes, classes et al.)  Statistical Measures (e.g. mean of numerical attributes)  Information-based measures (e.g. entropy of classes)  Histograms based DCT  information regarding the distribution of values of attributes with relational nature (e.g. mutual information between symbolic attributes and class)  Landmarking  use the performance of simple (fast) learners to predict the performance of candidate algorithms Collaborative Knowledge Discovery in Distributed Systems Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

New Data Characterisation Techniques New Data Characterisation Techniques  Idea:  Capturing information from the Standard Decision Tree model (or other models) [Peng, IDDM2002, DS2002].  Approach:  using standard decision tree method: C5.0  measuring the size, structure and shape of tree. Collaborative Knowledge Discovery in Distributed Systems x1 x2x3 x2x4 x5 C1C3 C1C2C3C1 C4 Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Meta-Learning for Algorithm Selection Meta-Learning for Algorithm Selection Collaborative Knowledge Discovery in Distributed Systems 4. Rank LAs according to the accuracy and time C5.0  13%, 30s Ltree  8%, 35s … Adversor System: Given new data set 1. characterize it general (# attributes, # examples,...) statistical (skewness, kurtosis,...) information-theoretic (class entropy,...) 2. select k neighbors 3. retrieve performance information accuracy + time

Collaborative Knowledge Discovery in Distributed Systems Collaborative Knowledge Discovery Framework Collaborative Knowledge Discovery Framework Application of Meta-Learning and Active Learning: 1) Control the process of each learning agent actively. 2) Coordinate activities of multiple learning agents. 3) Synthesize the outcome of distributed learning agents. Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Collaborative Knowledge Discovery in Distributed Systems Collaborative Knowledge Discovery Framework Collaborative Knowledge Discovery Framework Key techniques in collaborative learning: 1)To improve my situation 1)What do I need? 2)Where can I get it? 3)How to get it? 2)To improve the partners’ situation 1)What do other need? 2)Do I possibly have? 3)How can I get it efficiently? – Objective driven problem solving; – On-line information retrieval; – Meta-learning and Meta-knowledge; – Knowledge communication; – On-line DCT or data probe; – Meta-learning and meta-knowledge. Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Collaborative Knowledge Discovery in Distributed Systems Collaborative Knowledge Discovery- Summary Collaborative Knowledge Discovery- Summary -Knolwedge discovery deals with extracting interesting associations, classifiers, clusters and patterns, which are previous unknown, from data; -The emergence of network-based computing has introduced a new dimension to this problem, i.e., the distributed sources of data and computing. -Advanced analysis of distributed data for extracting useful knowledge is the next natural step. -The existing data mining algorithms are designed to work for centralized data, and they often do not pay attention to the distributed resource. -Collaborative Knowledge Discovery is a new strategy and approach. Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.

Meta Learning and Active Learning: Meta Learning and Active Learning: Collaborative Knowledge Discovery in Distributed Systems Thanks! Dr Yonghong Peng, Department of Computing, School of Informatics, University of Bradford.