Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) Dynamic and generic workflows in.NET Bart De Smet

Agenda Introduction Workflow: what and why? Research Dynamic workflows Methodologies Instrumentation framework Generic composition Generic data-driven workflows Design decisions Performance and conclusion Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 2

Introduction What is workflow? Graphical construction of applications  Business processes, human workflows  State machines versus sequential workflows Typically long-running processes  E.g. order processing with human approval Key advantages Graphical inspection by various stakeholders Availability of runtime services  Raise the level of abstraction  E.g. persistentie, tracking, scheduling, etc Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 3

Introduction Windows Workflow Foundation Workflow in a general purpose framework Part of Microsoft’s.NET Framework 3.0 Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 4

Introduction Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 5 Activity Sequential workflow definition If-Else Parallelism Interactions

Research Dynamic workflows Problem statement  Long-running versus changing business environment  Visual obstruction by aspects Adaptation at run-time  Modification of workflow instances  Injection of aspects, e.g. logging, authorization, etc Generic composition Problem statement  Repetitive tasks, e.g. data querying and retrieval  Ultimate vision: composition by the end-users Generic blocks for composition Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 6

Agenda Introduction Workflow: what and why? Research Dynamic workflows Methodologies Instrumentation framework Generic composition Generic data-driven workflows Design decisions Performance and conclusion Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 7

Dynamic workflows What? Dynamic adaptation of workflow instances Examples  Changing business policy (e.g. extra order approval)  Injection of runtime aspects (e.g. logging, auth’z)  Inspection of workflow instances (e.g. debugging) How? WF Dynamic Updates  Flexible framework-level mechanism  Additional custom tools layered on top Instrumentation framework Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 8

WF Dynamic Updates Internal modification By code inside a running workflow instance  Access to internal state  Fast  Prior knowledge of adaptation kinds required  Requires pre-configured communications to the outside External modification By code at the host level  Access to external state  Adaptation in an ad-hoc manner  Relatively slow  Timing issues concerning the state an instance is in Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 9

Instrumentation Best of both worlds External modification + internal modification Suspension points for exact timing Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 10 Instrument Runtime Event Handlers Inspect, inject, resume

Instrumentation Examples Logging Debugging Time measurement Authorization Tracking Real-time inspection Execution history Workflow Monitor Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 11

Agenda Introduction Workflow: what and why? Research Dynamic workflows Methodologies Instrumentation framework Generic composition Generic data-driven workflows Design decisions Performance and conclusion Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 12

Flowcharts Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 13 Iterative Decision logic Reporting Calculations Error handling

Generic composition What? Generic building blocks  Reusable blocks for workflow composition  Typical tasks such as data retrieval Important design decisions  Easy translation from flowchart to workflow  Dataflow versus workflow  Combination with Service Oriented Architectures  Workflow versus procedural: what’s the cost? How? Custom activities in WF Designer rehosting Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 14

Generic blocks Data retrieval Data gathering block IQueryManager  DBMS independent PropertyBag storage Iteration Foreach block Decision logic IfElse + conditions Reporting of results Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 15

Design decisions To iterate or not to iterate... Stale data during iteration  Loop body long-running?  Persistence is your enemy Granularity of workflow instances Chatty or chunky? Reduction of database communication More logic at the database tier?  Introduces workflow need at a different layer “Dataflow” isn’t trivial Bindings required for explicit dataflow (no pipelining) Generic & weakly typed versus specific & strongly typed Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 16

Service oriented architecture Workflow level Granularity and invocation mechanism? Block level Publish block functionality (e.g. query manager) Windows Communications Foundation (WCF) Keep internal communication efficient Make external communication WS-* compliant Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 17

Result Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 18

Rehosting Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 19

Agenda Introduction Workflow: what and why? Research Dynamic workflows Methodologies Instrumentation framework Generic composition Generic data-driven workflows Design decisions Performance and conclusion Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 20

Procedural versus workflow Iterative workflows Procedural is better WF scheduler overhead Data gathering Parallelism reduces damage Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 21 Let the runtime services do useful work!

Exploiting parallelism Intra-workflowInter-workflow Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 22 Boost scheduler

Instrumentation Internal modification External modification Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 23 Instrumentation framework Reduces # of ad-hoc external modifications Workflow is set to idle by suspension points Instrumentation framework Reduces # of ad-hoc external modifications Workflow is set to idle by suspension points Impact by scheduler

Conclusion Vakgroep Informatietechnologie – Breedbandcommunicatienetwerken (IBCN) 24 WF featuresConsiderations Updates PerformanceGranularitySOA Persistence SchedulingTrackingDesigner