Jorge Muñoz-Gama Josep Carmona A fresh look at Precision in Process Conformance Jorge Muñoz-Gama Josep Carmona Universitat Politècnica de Catalunya (Barcelona, Spain)

Outline Process Mining and Process Conformance Motivation Approach General Approach Implementation Results Extensions Future work Conclusions Precision in Process Conformance 15 Sep 2010

Process Mining * www.processmining.org Precision in Process Conformance 15 Sep 2010

Conformance Dimensions Fitness How much of the observed behavior is captured by the model Precision Models with minimal behavior to represent accurately the log Generalization Overly precise models which overfit the log Structure Minimal structure which clearly reflect the behavior Precision in Process Conformance 15 Sep 2010

Outline Motivation Process Mining and Process Conformance Approach General Approach Implementation Results Extensions Future work Conclusions Precision in Process Conformance 15 Sep 2010

Related Work Precision in the literature Most related work Rozinat et al. Information System 33 (2008) Metric for Precision in Petri Nets Computation of Follows and Precedes relations (Always, Never, Sometimes) of Model and Log. Measurement based on discrepancies in Sometimes relations Model relations require a model space state exploration Coverability Graph Precision in Process Conformance 15 Sep 2010

Motivation Goals and Requirements Precision Dimension Petri Nets Avoid the complete state space exploration Effort needed to obtain an accurate model Fine-level precision Locate the precision inconsistencies Precision in Process Conformance 15 Sep 2010

Process Conformance and Refinement Locate the inconsistencies Petri Net A C B D Conformance (Precision) MDT ETC Precision Metric A B D A C D Measure the inconsistencies Event Log Precision in Process Conformance 15 Sep 2010

Outline Approach Process Mining and Process Conformance Motivation General Approach Implementation Results Extensions Future work Conclusions Precision in Process Conformance 15 Sep 2010

General Idea: Escaping Edges Model Behavior Escaping Edges Log Behavior Model Behavior Precision in Process Conformance 15 Sep 2010

Conformance Route Map Petri Net Event Log B A D C Model States MDT Traversal MDT Metric A B D A C D Event Log Precision in Process Conformance 15 Sep 2010

Log and Model States Log Model Incorporate state information in the log (Aalst et al. Software and Systems Modeling, 2009) Past, Unlimited and Sequence Model Markings of the Petri Net Precision in Process Conformance 15 Sep 2010

Model States and Mapping Not all the reachable markings (could be infinite) Only Markings with a Log State mapped on Log and Model States Mapping i.e., reached marking after replay state prefix A E D C B p1 p2 p3 p4 s1 p1 s2 p2 s3 p3 s4 p4 p1 p2 p3 p4 p5 A B E 0 1 0 0 1 … 0 1 0 0 n p5 Markings not explored Precision in Process Conformance 15 Sep 2010

Log-guided Traversal Log-guided Traversal of Model Behavior Allowed Tasks : i.e., actions enabled in that moment Reflected Tasks : i.e., actions really executed (thus, annotated in the log) B C D <p2> A E D C B p1 p2 p3 p4 A B E A C E B C A E D C B p1 p2 p3 p4 A B E A C E Precision in Process Conformance 15 Sep 2010

Traversal (2) Escaping Edges : i.e., enabled actions not executed Precision discrepancies B C D B p1 p2 p3 p4 A C E A B E A C E D Precision in Process Conformance 15 Sep 2010

Precision Metric Take into account the Escaping Edges Between 0 (imprecise) and 1 (precise) More frequent traces, more weight in the metric Independent of Structural dimension Global precision Localizability A P H Z Q I A H I Z A P Q Z Precision in Process Conformance 15 Sep 2010

Minimal Disconformant Traces (MDT) Localizability of precision inconsistencies i.e., Minimal traces indicating where the model starts to deviate from the log Algorithm to compute all MDT using Escaping Edges Refinement Analysis Precision MDT A E A B E C D P Q A D C B Refined Petri Net Precision in Process Conformance 15 Sep 2010

Outline Implementation Process Mining and Process Conformance Motivation Approach General Approach Implementation Results Extensions Future work Conclusions Precision in Process Conformance 15 Sep 2010

Implementation ProM 6 Framework ETConformance Plug-In Precision in Process Conformance 15 Sep 2010

Outline Results Process Mining and Process Conformance Motivation Approach General Approach Implementation Results Extensions Future work Conclusions Precision in Process Conformance 15 Sep 2010

Results Precision in Process Conformance 15 Sep 2010 1 Table: ab’ VS etcp. Small benchmarks. Parikh, less 1 second. When model is precise, both metrics return 1. However, when model is not precise, the value differ, given the way they consider precision. 2 Table: Large benchmarks, Conformance Checker not handle. 3 miners: Parikh, RBMiner and toy miner only transitions without places and arcs (really overaproximation, few precise). Results show reasonable time even for prototype version. It can be seen that the results for the toy miner are really low (close to 0) where the precision of the other miners is greater. Notice also, that when the log gets complex, the miners make more abstractions, and therefore precision is lost. Precision in Process Conformance 15 Sep 2010

Results (2) Precision in Process Conformance 15 Sep 2010 This graphic is used to show the linear dependency between the time of the method and the size of the log, as it was expected. Precision in Process Conformance 15 Sep 2010

Outline Extensions Process Mining and Process Conformance Motivation Approach General Approach Implementation Results Extensions Future work Conclusions Invisible Tasks Duplicate Tasks States as Markings Non fitting done in progress Some special cases and some extension to the general case Precision in Process Conformance 15 Sep 2010

Invisible Tasks B A C A C Which Sequence? INDETERMINISM I H A H C ? (Transitions associated with no event) p3 I B Which Sequence? A H C ? A I C ? A H C p4 INDETERMINISM A C Precision in Process Conformance 15 Sep 2010

Invisible Tasks (2) Invisible Coverability Graph Solutions Union of Enabled Lazy Invisibles * One path only Shortest Invisible Path * A,B A B <1, 0, 0> Inv2 Inv1 <1, ω, 0> <0, 0, 1> Inv3 C C X A,D D <0, ω, 1> A,C X X *Rozinat et al. Information System 33 (2008) Precision in Process Conformance 15 Sep 2010

Duplicate Tasks B C A B D ... A B C ... Which Task? INDETERMINISM (Several Transitions associated with the same event) Which Task? B ? INDETERMINISM Solutions e.g. Look-ahead B C A B D ... A B C ... Precision in Process Conformance 15 Sep 2010

Variant: States as Markings States as Prefix 2 Escaping Edges B C B A C A B C States as Markings B A B C A C NO Escaping Edges p1 p2 p3 <p1> <p2> <p3> Precision in Process Conformance 15 Sep 2010

Variant: Non fitting models Symmetric to the Escaping Edges (Ee) Log Escaping Edges (LEe): The points where the log deviates from the model Fitness instead of Precision Escaping Edges Model Behavior Log Behavior Log Escaping Edges Model Behavior Precision in Process Conformance 15 Sep 2010

Outline Future work Process Mining and Process Conformance Motivation Approach General Approach Implementation Results Extensions Future work Conclusions Precision in Process Conformance 15 Sep 2010

Future Work: Refinement Breaking Concurrencies Supervisory Control A C B D Petri Net Refined Petri Net Event Log A E A B E MDT B H J G Precision in Process Conformance 15 Sep 2010

Future Work: Breaking Concurrencies Concurrencies in the model but not in the log Break the model concurrency with a restriction, e.g. a place Structural Concurrency Best effort overapproximation for general Petri Nets Exact for live and bounded Free Choice systems Polynomial Algorithm Kovalyov and Esparza , Proc. Intl. Workshop on Discrete Event Sytems, 1996 B A D A B C D C Precision in Process Conformance 15 Sep 2010

Supervisory Control Refined Model MDT MDT Abstraction Supervisory Control in Process Mining Santos et al. Supervisory Control Service (2010) Precision in Process Conformance 15 Sep 2010

Conclusions New technique for precision between Petri nets and Log. Avoids complete models state space exploration. Metric based on the effort needed to obtain a precise model. MDT, indicating the points where the model starts to deviates from the log. Approach implemented as plug-in of ProM 6. Precision in Process Conformance 15 Sep 2010

Thank You for Your Attention Precision in Process Conformance 15 Sep 2010