Process Model Realism Measuring Implicit Realism 8/09/2014dr. Benoît Depaire

Research “Trigger” 8/09/2014dr. Benoît Depaire Number of possible execution paths explode with AND-construct with n activities in parallel: X = n! nX

Research “Trigger” 8/09/2014dr. Benoît Depaire ABC ACB BAC BCA CAB CBA

Research “Trigger” 8/09/2014dr. Benoît Depaire ABC ACB CAB CBA

Research “Trigger” 8/09/2014dr. Benoît Depaire ABC ACB CAB CBA R1: All activities must occur

Research “Trigger” 8/09/2014dr. Benoît Depaire ABC ACB CAB CBA R1: All activities must occur R2: A must occur before B, unless we start with C

Research Trigger  When are we generalizing too much?  Context: Capture the true underlying process  Is the discovered model realistic? 8/09/2014dr. Benoît Depaire

Process Realism  Explicit Realism: All observed behavior should be in the model  Implicit Realism: Only the realistic unobserved behavior should be in the model 8/09/2014dr. Benoît Depaire

Assumptions  There are no measurement errors in the log  There are no infinite loops possible in the process  The fitness of a discovered model = 1  (All execution paths are equiprobable) 8/09/2014dr. Benoît Depaire

Process Realism  Implicit Realism: Only the realistic unobserved behavior should be in the model. 8/09/2014dr. Benoît Depaire

Process Realism  Implicit Realism: Only the realistic unobserved behavior should be in the model.  Implicit Realism Measure: How confident can we be that the unobserved behavior is realistic? 8/09/2014dr. Benoît Depaire

Implicit Realism Measure  m = number of paths in process M  n = number of paths in log L  x i = frequency of path i in log L  P i = Probability of path i occurring in L  u = # unobserved paths of M in L  T M (L) = statistic to determine u 8/09/2014dr. Benoît Depaire

Implicit Realism Measure  IR(L,M) = P[T M (L) >= u | M, n]  IR(L,M):  Probability that a model M would generate a log L with at least u missing paths (given n)  The lower IR(L,M), the less confident we can be that M actually produced L  because M contains too much unobserved behavior! (for a given n) 8/09/2014dr. Benoît Depaire

Implicit Realism Measure 8/09/2014dr. Benoît Depaire

Empirical Illustration 8/09/2014dr. Benoît Depaire

Assumptions  There are no measurement errors in the log  There are no infinite loops possible in the process  The fitness of a discovered model = 1  (All execution paths are equiprobable) 8/09/2014dr. Benoît Depaire

Conclusions  IR Measure has a very precise and intuitive interpretation  Current IR Measure should be used for evaluation, not comparison! 8/09/2014dr. Benoît Depaire

Process Model Realism Q&A 8/09/2014dr. Benoît Depaire

Implicit Realism  Precision  To what extent does the model NOT contain too much behavior (no underfitting)  Generalization  To what extent does the model NOT contain too little behavior (no overfitting) 8/09/2014dr. Benoît Depaire

Implicit Realism  Precision  To what extent does the model NOT contain too much behavior (no underfitting)  To what extent does the model ONLY contain observed behavior  Generalization  To what extent does the model NOT contain too little behavior (no overfitting)  To what extent does the model contain unobserved behavior 8/09/2014dr. Benoît Depaire