1. Jorge Muñoz-Gama Josep Carmona
A fresh look at Precision in Process Conformance
Jorge Muñoz-Gama Josep Carmona
Universitat Politècnica de Catalunya
(Barcelona, Spain)

2. Outline Process Mining and Process Conformance Motivation Approach
General Approach
Implementation
Results
Extensions
Future work
Conclusions
Precision in Process Conformance
15 Sep 2010

3. Process Mining * www.processmining.org
Precision in Process Conformance
15 Sep 2010

4. Conformance Dimensions
Fitness
How much of the observed behavior is captured by the model
Precision
Models with minimal behavior to represent accurately the log
Generalization
Overly precise models which overfit the log
Structure
Minimal structure which clearly reflect the behavior
Precision in Process Conformance
15 Sep 2010

5. Outline Motivation Process Mining and Process Conformance Approach
General Approach
Implementation
Results
Extensions
Future work
Conclusions
Precision in Process Conformance
15 Sep 2010

6. Related Work Precision in the literature
Most related work Rozinat et al. Information System 33 (2008)
Metric for Precision in Petri Nets
Computation of Follows and Precedes relations (Always, Never, Sometimes) of Model and Log.
Measurement based on discrepancies in Sometimes relations
Model relations require a model space state exploration
Coverability
Graph
Precision in Process Conformance
15 Sep 2010

7. Motivation Goals and Requirements Precision Dimension Petri Nets
Avoid the complete state space exploration
Effort needed to obtain an accurate model
Fine-level precision
Locate the precision inconsistencies
Precision in Process Conformance
15 Sep 2010

8. Process Conformance and Refinement
Locate the inconsistencies
Petri Net A C B D
Conformance
(Precision)
MDT
ETC Precision
Metric
A B D
A C D
Measure the inconsistencies
Event Log
Precision in Process Conformance
15 Sep 2010

9. Outline Approach Process Mining and Process Conformance Motivation
General Approach
Implementation
Results
Extensions
Future work
Conclusions
Precision in Process Conformance
15 Sep 2010

10. General Idea: Escaping Edges
Model
Behavior
Escaping
Edges
Log
Behavior
Model
Behavior
Precision in Process Conformance
15 Sep 2010

11. Conformance Route Map Petri Net Event Log B A D C Model States MDT
Traversal
MDT
Metric
A B D
A C D
Event Log
Precision in Process Conformance
15 Sep 2010

12. Log and Model States Log Model
Incorporate state information in the log
(Aalst et al. Software and Systems Modeling, 2009)
Past, Unlimited and Sequence
Model
Markings of the Petri Net
Precision in Process Conformance
15 Sep 2010

13. Model States and Mapping
Not all the reachable markings (could be infinite)
Only Markings with a Log State mapped on
Log and Model States Mapping
i.e., reached marking after replay state prefix A E D C B p1 p2 p3 p4 s1 p1 s2 p2 s3 p3 s4 p4
p1 p2 p3 p4 p5
A B E … n p5
Markings not explored
Precision in Process Conformance
15 Sep 2010

14. Log-guided Traversal Log-guided Traversal of Model Behavior
Allowed Tasks :
i.e., actions enabled in that moment
Reflected Tasks :
i.e., actions really executed (thus, annotated in the log) B C D
<p2> A E D C B p1 p2 p3 p4
A B E
A C E B C A E D C B p1 p2 p3 p4
A B E
A C E
Precision in Process Conformance
15 Sep 2010

15. Traversal (2) Escaping Edges : i.e., enabled actions not executed
Precision discrepancies B C D B p1 p2 p3 p4 A C E
A B E
A C E D
Precision in Process Conformance
15 Sep 2010

16. Precision Metric Take into account the Escaping Edges
Between 0 (imprecise) and 1 (precise)
More frequent traces, more weight in the metric
Independent of Structural dimension
Global precision
Localizability A P H Z Q I
A H I Z
A P Q Z
Precision in Process Conformance
15 Sep 2010

17. Minimal Disconformant Traces (MDT)
Localizability of precision inconsistencies
i.e., Minimal traces indicating where the model starts to deviate from the log
Algorithm to compute all MDT using Escaping Edges
Refinement
Analysis
Precision
MDT
A E
A B E
C D P Q A D C B
Refined Petri Net
Precision in Process Conformance
15 Sep 2010

18. Outline Implementation Process Mining and Process Conformance
Motivation
Approach
General Approach
Implementation
Results
Extensions
Future work
Conclusions
Precision in Process Conformance
15 Sep 2010

19. Implementation ProM 6 Framework ETConformance Plug-In
Precision in Process Conformance
15 Sep 2010

20. Outline Results Process Mining and Process Conformance Motivation
Approach
General Approach
Implementation
Results
Extensions
Future work
Conclusions
Precision in Process Conformance
15 Sep 2010

21. Results Precision in Process Conformance 15 Sep 2010
1 Table: ab’ VS etcp. Small benchmarks. Parikh, less 1 second.
When model is precise, both metrics return 1.
However, when model is not precise, the value differ, given the way they consider precision.
2 Table: Large benchmarks, Conformance Checker not handle. 3 miners: Parikh, RBMiner and toy miner only transitions without places and arcs (really overaproximation, few precise).
Results show reasonable time even for prototype version. It can be seen that the results for the toy miner are really low (close to 0) where the precision of the other miners is greater. Notice also, that when the log gets complex, the miners make more abstractions, and therefore precision is lost.
Precision in Process Conformance
15 Sep 2010

22. Results (2) Precision in Process Conformance 15 Sep 2010
This graphic is used to show the linear dependency between the time of the method and the size of the log, as it was expected.
Precision in Process Conformance
15 Sep 2010

23. Outline Extensions Process Mining and Process Conformance Motivation
Approach
General Approach
Implementation
Results
Extensions
Future work
Conclusions
Invisible Tasks
Duplicate Tasks
States as Markings
Non fitting
done
in progress
Some special cases and some extension to the general case
Precision in Process Conformance
15 Sep 2010

24. Invisible Tasks B A C A C Which Sequence? INDETERMINISM I H A H C ?
(Transitions associated with no event) p3 I B
Which Sequence?
A H C ?
A I C ? A H C p4
INDETERMINISM
A C
Precision in Process Conformance
15 Sep 2010

25. Invisible Tasks (2) Invisible Coverability Graph Solutions
Union of Enabled
Lazy Invisibles *
One path only
Shortest Invisible Path *
A,B A B
<1, 0, 0>
Inv2
Inv1
<1, ω, 0>
<0, 0, 1>
Inv3 C C X
A,D D
<0, ω, 1>
A,C X X
*Rozinat et al.
Information System 33 (2008)
Precision in Process Conformance
15 Sep 2010

26. Duplicate Tasks B C A B D ... A B C ... Which Task? INDETERMINISM
(Several Transitions associated
with the same event)
Which Task?
B ?
INDETERMINISM
Solutions
e.g. Look-ahead B C A B D
... A B C ...
Precision in Process Conformance
15 Sep 2010

27. Variant: States as Markings
States as Prefix 2
Escaping Edges B C B A C
A B C
States as Markings B
A B C A C
NO Escaping Edges p1 p2 p3
<p1>
<p2>
<p3>
Precision in Process Conformance
15 Sep 2010

28. Variant: Non fitting models
Symmetric to the Escaping Edges (Ee)
Log Escaping Edges (LEe): The points where the log deviates from the model
Fitness instead of Precision
Escaping
Edges
Model
Behavior
Log
Behavior
Log
Escaping
Edges
Model
Behavior
Precision in Process Conformance
15 Sep 2010

29. Outline Future work Process Mining and Process Conformance Motivation
Approach
General Approach
Implementation
Results
Extensions
Future work
Conclusions
Precision in Process Conformance
15 Sep 2010

30. Future Work: Refinement
Breaking Concurrencies
Supervisory Control A C B D
Petri Net
Refined Petri Net
Event Log
A E
A B E
MDT
B H J G
Precision in Process Conformance
15 Sep 2010

31. Future Work: Breaking Concurrencies
Concurrencies in the model but not in the log
Break the model concurrency with a restriction, e.g. a place
Structural Concurrency
Best effort overapproximation for general Petri Nets
Exact for live and bounded Free Choice systems
Polynomial Algorithm
Kovalyov and Esparza , Proc. Intl. Workshop on Discrete Event Sytems, 1996 B A D
A B C D C
Precision in Process Conformance
15 Sep 2010

32. Supervisory Control Refined Model MDT MDT Abstraction
Supervisory Control in Process Mining
Santos et al. Supervisory Control Service (2010)
Precision in Process Conformance
15 Sep 2010

33. Conclusions New technique for precision between Petri nets and Log.
Avoids complete models state space exploration.
Metric based on the effort needed to obtain a precise model.
MDT, indicating the points where the model starts to deviates from the log.
Approach implemented as plug-in of ProM 6.
Precision in Process Conformance
15 Sep 2010

34. Thank You for Your Attention
Precision in Process Conformance
15 Sep 2010