© Donald F. Ferguson, All rights reserved. Topics in Computer Science: Modern Internet Service Oriented Application Development Lecture 5: Composite Applications Dr. Donald F. Ferguson (Admin:

2 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Contents

3 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Contents Components and composite applications –Introduction –Setting the context –Composite applications –Types of composition; Styles of integration –Modeling, visual notation and executable processes –Structural composition: A conceptual model – Service Component Architecture –Data flow composition: Message/event/service bus oriented composition (iPaaS) –Basic building blocks –Message/data flows and processes –Control flow composition: Workflow and orchestration oriented composition –Basic building blocks –Processes and workflows Next assignment Discussion

4 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Components and Composite Applications

5 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Introduction

6 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications APIs Galore

7 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Setting the Context – “ The World is not Enough. ” The world gives you …You ned to make …

8 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Composite Applications – Some Definitions “In computing, a composite application is a software application built by combining multiple existing functions into a new application. … A composite application consists of functionality drawn from several different sources. The components may be individual selected functions from within other applications, or entire systems whose outputs have been packaged as business functions, modules, or web services.” ( “Composition refers to a way of delivering enterprise solutions by assembling them from prebuilt components, instead of building them from scratch. It also includes personalization and customization abilities, so that users can easily and quickly modify specific functionality in the solution.” ( us/library/bb aspx) us/library/bb aspx “A composite application orchestrates independently developed programs, data and devices to deliver a new solution that none of the previously available applications could deliver on its own. Each resource accessed by a composite application uses a different data model. In most cases, the composite application supports user interactions beyond those provided by the leveraged applications. In other cases, a composite application may act as a service. Some composite applications are built by leveraging other composite applications.” (

9 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Composite Applications – Some Pictures

10 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Setting the Context You have learned –How to build a reusable “component” –That surfaces a well-defined, reusable API –And how to monitor, trace, manage and secure the API. Any moderately sized company or organization, there are –Thousands of internal APIs. –100s of published APIs. –100s of consumed, public, external APIs –Tens (or hundreds) of thousands of available, external, public APIs. By and large, –Application development involves –Controlling the ordering, flow and execution (if … then, while … do, etc) using “code” –Of API calls (JDBC, class libraries, etc.) –Modern, Internet SOA Application Development involves –New techniques for controlling the ordering, flow and execution –Of “web” callable APIs.

11 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Composition Diagrams The diagrams are always “nodes” and “arcs.” Structure: node A node B, where x is Requires/depends/calls/synchronizes with/… Control: –Execute A and then execute B, where A and B are tasks. –There is a shared set of “data objects” the tasks manipulate. Data flow: Do A to the message/document/… and then do B to A’s output Control FlowStructureData Flow

12 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Simplistic Taxonomy of Integration Integration UI Information Integration ETL, MapReduce Federated DB Message/ Event CEP/CPE Message Flow RT events Workflow Orchestration Human Centric Automation

13 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Modeling, Visual Notation, Executable Models Development projects tend to be systematic or opportunistic. Opportunistic projects and some systematics project team will tell you that they do not use modeling because –We tried it and it did not work. –Too slow and too much time. We are agile and want to get to the code. –That’s the way old guys do it. I ask the following questions –Do you have meetings and scrums? –Do you use a whiteboard? Do you draw diagrams? –Does someone copy down the information and convert to PPT or diagrams? –Do you take pictures of the whiteboard? –Do you the pictures and PPT? –Do you realize you are doing modeling, just poorly? My view –A model and visual notation is like PPT, but there is precise and consistent interpretation of shapes, colors, symbols, … –Like PPT, there is an underlying file format/schema for the diagram. –An executable model has a file format that is complete enough (with possible editing) that some engine can interpret the file the same way that a JVM interprets bytecodes or a CPU interprets instructions.

14 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Three Core Concepts Editor Visual Notation Diagram “Program” “Engine”

15 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Of Course, maybe that ’ s just me …

16 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Structure Composition Diagrams

17 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Service Component Architecture

18 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Flexible Implementation – Some of which we will Cover

19 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Eclipse SOA SCA Tool

20 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications OSGi Blueprint are Similar

21 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications OSGi Blueprints

22 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Comments Structural composition and models is a recurring concept –OSGi –SCA –J2EE dependency references –Camel –UML Module and Component Diagrams –etc. The concept waxes and wanes in popularity, and seems to be waning. –The explosion of micro-services and ad hoc applications. –Structure integrated into behavior composition –More dynamic, query based binding We will not spend more time on the topic.

23 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Data Flow Composition

24 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Point-to-Point Connections are Bad Doing this … Leads to this … …

25 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Q&A Discussion

26 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Setting the Context

27 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications An “ Old Cranky Guy ” Example SCA

28 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Three Core Composition Models Structure –What are the APIs, and possibly components, that I need? For example, a commerce application needs –Content management system for images, manuals, how-to videos, … –Catalog systems for products, sub-products, prices, … –Inventory management –Credit card processing –Customer account and relationship management –And what gets connected to what, and how. Behavior –Data flow –Updating thing T in component C needs to –Send an update(Q) thin in component B and D –But I need to split the data because B and D only need some of T –And I need to remap the data because B and D have different schema for “T” stuff. –Control flow: Opening a new credit card requires things to happen in order –Validate the application is correct –Perform a credit check –Call a system to create a risk score –Have an analyst approve the application –Send an to the customer for approval –Send a message to the place that is going to make the card –… … People like diagrams. But tend to use similar diagrams types for all composition. And be inconsistent about what symbols means

29 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications You Have Already Seen the Pattern This is the same pattern as API Mgmt But … What happens in the “proxy” is a lot more complicated. The relationship between the proxy and the APIs Is many-to-many And has data flow or sequencing.

30 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Integration is Purple Boomi Example

31 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Enterprise Integration Patterns with WSO2 ESB

32 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications More Examples

33 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications More Examples

34 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications How do you Build the Composite Take off a palette Set properties Put on canvass Set control values Connect flow

35 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Composition Each “clipart widget” has –A unique, documented shape (visual notation). –A little implementation piece of code (think POJO) that does the function. –Rules about what other widgets can connect to it/from it. –A set of properties that tailor this instance’s behavior For example, –When I drag a routing widget onto the canvass to make a routing instance –I specify –The valid, input JSON format. –A the routing rule table –{“Person.iq” <= 50; –{“Person.color” == “Yellow”; –And connect to other things on the palette

36 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Dell Boomi

37 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Test Drive

38 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Workflow/ BPMN

39 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications BPMN BPMN 2.0 is an international standard for business process modeling. The BPMN specification describes how the elements of a process diagram have to look like (notation), how these can be combined with each other (meta model / syntax), what a diagram means (semantics) and how diagrams can be transferred from one tool to another (XML interchange format). Process models describe sequences of business activities from start to finish, e.g. Order-to-Cash, Account Open

40 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications BPMN and Activiti Some terms –BPMN 2.0 is a standard for defining/documenting workflows/orchestrations. –Activiti (and many others) are implementations of the standard –Runtime engine that can interpret and execute a BPMN process –A WYSIWYG editor –With prebuilt, configurable shapes palette that extend base BPMN shapes. Huh? BPMN has the concepts of a “Task,” which is one “step” in a process. But in a process model, this is just “prose” like “Order a Pizza” Implementations come with “common” types of tasks, e.g. Make an SQL call on a database. Call a REST API Put a document on a worklist etc. –A composition canvass for assembling shapes into processes. –Debugger, monitoring, reporting, versioning, etc. This is –The same concept as enterprise integration patterns and Boomi –But, –The clipart is different –And the graph mostly represents “control flow,” not data flow.

41 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Simple BPMN Diagram

42 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications BPMN Symbols

43 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications BPMN Symbols

44 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications BPMN Symbols

45 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications BPMN Symbols

46 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications BPMN Symbols

47 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications BPMN Symbols Lots more go here. There are references, e.g.

48 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications BPMN Symbols

49 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications BPMN Symbols Seriously dude? There are references, e.g.

50 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications BPMN Symbols

51 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications And You can Define Processes

52 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications How do you Implement? Notify a person/role. Have them do something. Have the use an app that calls the engine to tell you they are done. Send an . POST a document etc. Decision table Call Drools etc. Insert into a DB Call a web service Run a Java app. etc. Run some JavaScript or Java or … Right at this point in the workflow/process And manipulate the data/documents/control.

53 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications User Task

54 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Script Task

55 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Service Task

56 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications

57 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications

58 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications

59 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications

60 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications

61 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications

62 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications

63 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications

64 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Comments Three Core Aspects of BPMN and Actviti Human Centric Workflow –Define an organization model –Users, Groups –Properties for users and groups –Define workflow processes that have Human Tasks. For each Human Task –Define the form and data that enables a person to perform the task. –Define “rules” for assigning a task to people or groups. Automation/Orchestration –Implement a set of POJOs implementing the Delegate interface. Each POJO implements an automation action by … –Call a REST service –Sending a message on JMS or SQS –Etc. –Define a workflow with one or more Service Tasks. For each Service Task –Specify/bind to a Delegate. –Map process data variables to inputs and outputs of the Delegate.

65 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications AWS Simple Workflow

66 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Amazon Simple Workflow

67 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Application Structure Role of the Activities Worker: The activities worker performs the various tasks that the workflow must accomplish. It consists of: –The activities implementation, which includes of a set of activity methods that perform particular tasks for the workflow. –An activity worker, which uses HTTP long poll requests to poll Amazon SWF for activity tasks. When a task is available, Amazon SWF responds to the request by sending the information required to perform the task. The activity worker then calls the appropriate activity method, and returns the results to Amazon SWF. Role of the Workflow Worker: The workflow worker orchestrates the execution of the various activities, manages data flow, and handles failed activities. It consists of: –The workflow implementation, which includes the activity orchestration logic, handles failed activities, and so on. –An activities client, which serves as a proxy for the activities worker and enables the workflow worker to schedule activities to be executed asynchronously. –A workflow worker object, which uses HTTP long poll requests to poll Amazon SWF for decision tasks. If there are tasks on the workflow task list, Amazon SWF responds to the request by returning the information that is required to perform the task. The framework then executes the workflow to perform the task and returns the results to Amazon SWF.

68 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Assignment 3

69 © Donald F. Ferguson, All rights reserved.Modern Internet Service Oriented Application Development – Lecture 5: Composite Applications Backup