Comp 538 Course Presentation Discrete Factor Analysis Learning Hidden Variables in Bayesian Network Calvin Hua & Lily Tian Computer Science Dep, HKUST

Objectives and Outlines Objective: - Present a space of BN topologies with hidden variables(or factors) and a method for rapidly learning an appropriate topology from data. Outline: -Motivating example -Methodology * Finding the topology * Constructing the factors - Some Results & Evaluation Q&A

Motivating Example N H S C Observable Variables: H- Hives N-Nasal Congestion C-Cough S-Sore Throat Question: 1.What independencies are encoded? 2.Is the direction of each edge unique?

Motivating Example (Cond.) VA H NC S More compact Inference easier Hidden variables used to explain dependencies and independencies

Our Work DATA SET MODEL DO INFERENCE OUR WORK Task: 1.How to find the topology given data(structure)? 2.How to construct the factors(parameters)?

Learning Factor Structure Finding the topology –Decide which observable variables each factor should cover –Decide what factors to use Constructing the factors –Determine highly probable number of values per factor –Determine highly probable conditional dependencies between factors and observable variables

Algorithm - Finding the topology Step1: –Introduce a link between two variables when they are dependent –Label each link with the probability that those two variables are dependent Step2: –Extract cliques in the graph Step3: –Perform a greedy search for the cliques

Algorithm - Step1 (Cond.) How to test whether two variables are dependent or not? –Using Chi-Squared Test

Algorithm - Step2(Cond.) Principles of extracting cliques Iterating through the variables, in each iteration we do the following : –Adding a variable to an existing clique if the variable is dependent on all other variables in that clique.

Algorithm- Step3(Cond.) Perform a greedy search for cliques –By maximizing the sum of the labels represented in the set of cliques. –Labels: the probability that those variables are dependent.

Algorithm- Constructing the factors Initialization Calculate the most probable assignment of the nth instance, I, to the values of each factor given the first n-1 instances: (1)Choose a random order of factors (2)Iterate over the factors (details later)

Algorithm- Constructing the factors (Cond.) Task: –Choose the number of values for each factor –Choose the conditional probabilities Note : –FL(Factor Learning) can do so rapidly by approximating the normative Bayesian method for learning hidden variables –The normative way should consider all possible numbers of values and all possible assignments of hidden variable values to the instances in the data set(Cooper & Herskovits, 1992; Cooper, 1994)

Algorithm- Step 2 (Cond.) 1.Compute for each value,, of the ith factor 2.Calculate the probability of a new value for the ith factor, 3.- Label the instance with the factor value with the maximum probability - Update the estimated prior probabilities of the ith factor’s values and the estimated conditional probabilities of the observable values given the factor’s value Note: In all cases where probabilities must be estimated from frequencies we use the following formula:

Some Results and Evaluation M P G C Association tested by FL M-Math P-Physics C-Chemistry G-Geology Note: In this figure, the arc’s denote the dependencies between any pair of variables

Some Results and Evaluation AM2 MP CG A-Analytic ability M2-Memory M-Math P-Physics C-Chemistry G-Geology

Some Results and Evaluation Characteristics of factor structure 1.There are hidden variables, called factors. 2.Hidden variables can interact to influence observable variables. 3.It can support polynomial time probabilistic inference. 4.The resulting network captures conditional independencies among the observable variables.