Neo4j for Process Mining Big Data and Hadoop Users Group of West Michigan Gordon Food Service 3/9/2016

Agenda Introduction What is a Process What is a Process Model What is Process Mining Data Prospecting with Neo4j Tool Demo

Introduction In the domain of information systems it is very difficult to see a product let alone determine how efficiently it was developed. In most cases we are left with textual information to understand and make assessments of a process.

*Better ways to capture, visualize, and analyze processes* Introduction The purpose of this proof of concept is to explore the application of graph database techniques in conjunction with process mining to improve the assessment of software development processes. *Better ways to capture, visualize, and analyze processes*

What is a Process?

*There are as many definitions of process as there are people* What is a Process? A series of actions or steps taken in order to achieve a particular end. A collection of linked tasks which find their end in the delivery of a service or product to a client. A set of activities and tasks that, once completed, will accomplish an organizational goal. *There are as many definitions of process as there are people*

Definition for Presentation A process comprises an identified body of work that can be characterized as a set of related activities or a defined sequence of steps and decisions The process name is in action verb - noun form (usually) The name is singular The result must be discrete and countable The result is what the customer of the process wants The process is initiated by a triggering event that could be an action, time, or a condition Workflow Modeling: Tools for Process Improvement and Application Development, Sharp & McDermott

Process Examples Suggested Process What We Call It Why Not a Process Customer Relationship Management Process Area Doesn’t deliver a single, specific result; a set of related business processes meeting an overall objective Acquire New Customer Business Process Delivers a single, specific result, and meets all other criteria. An “end-to-end business process Assess Prospect Financial Status or Setup Customer Sub process Too small - both deliver specific results, but are intermediate results in an E2E process Calculate Customer Credit Limit or Create Customer Account Activity, Step, Task Much too small - a part of a sub process. Possibly described in a procedure or use case. Workflow Modeling: Tools for Process Improvement and Application Development, Sharp & McDermott

What is a Process Model?

BPMN Basic Elements Activities Gateways Events Data

BPMN Pools White Box Pools Black Box Pools Pool with Swimlanes

BPMN Pool Communication

What is Process Mining?

Process Mining Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst

Three types of Process Models

Event Logs A process consists of cases A case consists of events such that each event relates to one case Events within a case are ordered Events can have attributes Set the context / expectation for the presentation by discussing current level of analytics maturity at GFS

Play-Out Set the context / expectation for the presentation by discussing current level of analytics maturity at GFS Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst

Play-In Set the context / expectation for the presentation by discussing current level of analytics maturity at GFS Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst

Play-In Set the context / expectation for the presentation by discussing current level of analytics maturity at GFS Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst

Replay Set the context / expectation for the presentation by discussing current level of analytics maturity at GFS Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst

Replay Set the context / expectation for the presentation by discussing current level of analytics maturity at GFS Process Mining Discovery, Conformance, and Enhancement of Business Processes, Wil M. P. Van Der Aalst

Data Prospecting with Neo4j

General Approach Given that the artifacts from the SLDC are recorded and linked in JIRA (i.e. issue tracker) they can be viewed as a graph Find a promising project in this case Wholesale Order Capture (WOC) and load the issues with their associate links into a labeled graph (i.e. Neo4J) Use Cypher queries to assess the “Prospect” for developing event logs Remediate good prospects (i.e. if possible) Extract the event log from the graph database and load it into Prom

Traceability Network Set the context / expectation for the presentation by discussing current level of analytics maturity at GFS

Graph Model Set the context / expectation for the presentation by discussing current level of analytics maturity at GFS

Challenges Compiling Event Logs Picking events and activities for the model Finding good the timestamps Assessing the relevancy and accuracy of the timestamps Relating and ordering events and activities across systems

Neo4j Demo

ProM Demo

Conclusions While the WOC project was very successful from a project context, it was not as successful in a process improvement context Controls for workflow and traceability between issues needs to be improved Neo4j is a very effective tool for prospecting and processing event logs Process mining of the SDLC is feasible but will take significant effort on the data side (no surprise here !)