The Automatic Explanation of Multivariate Time Series (MTS) Allan Tucker

The Problem - Data Datasets which are Characteristically: –High Dimensional MTS –Large Time Lags –Changing Dependencies –Little or No Available Expert Knowledge

Lack of Algorithms to Assist Users in Explaining Events where: –Model Complex MTS Data –Learnable from Data with Little or No User Intervention –Transparency Throughout the Learning and Explaining Process is Vital The Problem - Requirement

Contribution to Knowledge Using a Combination of Evolutionary Programming (EP) and Bayesian Networks (BNs) to Overcome Issues Outlined Extending Learning Algorithms for BNs to Dynamic Bayesian Networks (DBNs) with Comparison of Efficiency Introduction of an Algorithm for Decomposing High Dimensional MTS into Several Lower Dimensional MTS

Contribution to Knowledge (Continued) Introduction of New EP-Seeded GA Algorithm Incorporating Changing Dependencies Application to Synthetic and Real-World Chemical Process Data Transparency Retained Throughout Each Stage

Real Data Data Preparation Search Methods Variable Groupings Synthetic Data Explanation Model Building Evaluation Changing Dependencies Framework Pre-processing

Key Technical Points 1 Comparing Adapted Algorithms New Representation K2/K3 [Cooper and Herskovitz] Genetic Algorithm [Larranaga] Evolutionary Algorithm [Wong] Branch and Bound [Bouckaert] Log Likelihood / Description Length Publications: –International Journal of Intelligent Systems, 2001

Key Technical Points 2 Grouping A Number of Correlation Searches A Number of Grouping Algorithms Designed Metrics Comparison of All Combinations Synthetic and Real Data Publications: –IDA99 –IEEE Trans System Man and Cybernetics 2001 –Expert Systems 2000

Key Technical Points 3 EP-Seeded GA Approximate Correlation Search Based on the One Used in Grouping Strategy Results Used to Seed Initial Population of GA Uniform Crossover Specific Lag Mutation Publications: –Genetic Algorithms and Evolutionary Computation Conference 1999 (GECCO99) –International Journal of Intelligent Systems, 2001 –IDA2001

Key Technical Points 4 Changing Dependencies Dynamic Cross Correlation Function for Analysing MTS Extend Representation Introduce a Heuristic Search - Hidden Controller Hill Climb (HCHC) –Hidden Variables to Model State of the System –Search for Structure and Hidden States Iteratively

Future Work Parameter Estimation Discretisation Changing Dependencies Efficiency New Datasets –Gene Expression Data –Visual Field Data

DBN Representation t-4 t-3 t-2 t-1 t a 0 (t) a 1 (t) a 2 (t) a 3 (t) a 4 (t) a 2 (t-2) a 3 (t-2) a 4 (t-3) a 3 (t-4) (3,1,4) (4,2,3) (2,3,2) (3,0,2) (3,4,2)

Sample DBN Search Results N = 5, MaxT = 10N = 10, MaxT = 60

Grouping One High Dimensional MTS (A) 1. Correlation Search (EP) 2. Grouping Algorithm (GGA) Several Lower Dimensional MTS List (a, b, lag) 12R12R G {0,3} {1,4,5} {2}

Sample Grouping Results Original Synthetic MTS Groupings Groupings Discovered from Synthetic Data Sample of Variables from a Discovered Oil Refinery Data Group

Parameter Estimation Simulate Random Bag (Vary R, s and c, e) Calculate Mean and SD for Each Distribution (the Probability of Selecting e from s) Test for Normality (Lilliefors’ Test) Symbolic Regression (GP) to Determine the Function for Mean and SD from R, s and c (e will be Unknown) Place Confidence Limits on the P(Number of Correlations Found  e)

0: (a,b,l) 1:(a,b,l) 2:(a,b,l) EPListSize: (a,b,l) Final EPList EP 0: ((a,b,l),(a,b,l)…(a,b,l)) 1: ((a,b,l),(a,b,l)…(a,b,l)) 2: ((a,b,l),(a,b,l)…(a,b,l)) GAPopsize: ((a,b,l) … (a,b,l)) GA Initial GAPopulation DBN EP-Seeded GA

EP-Seeded GA Results N = 10, MaxT = 60N = 20, MaxT = 60

Varying the value of c

P(TGF instate_0) = 1.0 t t-1 t-11 t-13 t-16 t-20 t-60 P(TT instate_0) = 1.0P(BPF instate_3) = 1.0 P(TT instate_1) = P(TGF instate_3) = 1.0 P(SOT instate_0) = P(C2% instate_0) = P(T6T instate_0) = P(RinT instate_0) = TimeExplanation

Changing Dependencies Time (Minutes) Variable Magnitude A/M_GB TGF

Dynamic Cross- Correlation Function

Hidden Variable - OpState t-4 t-3 t-2 t-1 t a 2 (t)OpState 2 a 2 (t-1) a 3 (t-2) a 0 (t-4)

Hidden Controller Hill Climb Update Segment_Lists through Op_State Parameter Estimation Update DBN_List through DBN Structure Search Score

HCHC Results - Oil Refinery Data

HCHC Results - Synthetic Data Generate Data from Several DBNs Append each Section of Data Together to Form One MTS with Changing Dependencies Run HCHC

t t-1 t-3 t-5 t-6 t-9 Time Explanation P(OpState 1 is 0) = 1.0P(a 1 is 0) = 1.0P(a 0 is 0) = 1.0 P(a 2 is 1) = 1.0 P(OpState 1 is 0) = 1.0P(a 1 is 1) = 1.0P(a 0 is 0) = 1.0 P(a 2 is 1) = 1.0 P(a 2 is 0) = P(a 2 is 0) = P(a 0 is 0) = P(a 0 is 1) = P(OpState 0 is 0) = P(a 0 is 1) = 0.778P(OpState 0 is 0) = 0.720

t t-1 t-3 t-5 t-6 t-7 t-9 Time Explanation P(OpState 1 is 4) = 1.0P(a 1 is 0) = 1.0P(a 0 is 0) = 1.0 P(a 2 is 1) = 1.0 P(OpState 1 is 4) = 1.0P(a 1 is 1) = 1.0P(a 0 is 0) = 1.0 P(a 2 is 1) = 1.0 P(a 2 is 1) = P(a 2 is 1) = P(a 0 is 0) = P(a 0 is 1) = P(OpState 2 is 3) = P(a 2 is 0) = P(OpState 2 is 4) = 0.222

Process Diagram TT T6T T36T RBT SOTT11 SOFT13 TGF BPF %C3 %C2 RINT FF PGM PGB AFT C11/3T

Typical Discovered Relationships TT T6T T36T RBT SOTT11 SOFT13 AFT TGF BPF %C3 %C2 RINT FF C11/3T PGM PGB

Parameters DBN SearchGAEP PopSize10010 MR CR GenBased on FCBased on FC Correlation Search c - Approx. 20% of s R - Approx. 2.5% of s Grouping GA Synth. 1Synth. 2-6 Oil PopSize CR MR Gen (1000 for GPV) 150

Parameters EP-Seeded GA c- Approx. 20% of s EPListSize- Approx. 2.5% of s GAPopSize - 10 MR- 0.1 CR- 0.8 LMR- 0.1 Gen- Based on FC HCHC OilSynthetic DBN_Iterations1× Win len Win jump 50050