1. Ashish Jain, Mike Bizub, & Vignesh Sukumar
Microsoft CSE&O
How Microsoft IT does Integration

2. Agenda Recap of who we are and what we do
Composable architecture and how we’ve implemented the Routing Slip Pattern
Hot-Path Telemetry
ALM, Dev-ops, and Governance
Business Continuity and Enterprise Disaster Recovery
Migration Tools update
Appendix

3. Enabling Business Processes
Microsoft Integration Landscape
150M+
Messages Per Month
1,000 +
Partners – Internal and External
3,400+
Integration Streams Centrally Managed
200+
Microsoft Systems
Enabling Business Processes
Supply Chain
Contract Management
Licensing
Sales
Benefits
Legal
Payroll
Finance
Travel
Suppliers
Distributors
… etc
$1.6 B
Daily bank treasury transaction
100+
BizTalk Servers in use today
Message Standards
X12 – 30% EDIFACT-14%
XML-12% SWIFT-12%
Other – 32%
Multi-Platform
BTS 2016, MABS, Logic Apps

4. Recap – The People (giving credit where it’s due)

5. Recap – B2B Approach
Use one Integration Account and set of logic apps across multiple Businesses and Trading Partners (Partner info, Agreements, Document Types, Schemas, Maps, Batch Configurations)
Flow concept static but type can be overridden based on message (e.g. different logic apps that process X12 or EDIFACT messages)
Metadata used to populate runtime properties specific to a TP or a LOB (currently stored in a CosmosDB and accessed via Azure Function with caching)
A ‘Routing Slip’ can override the URL of next logic app to load (can be overridden using APIM Policy)

6. Msg Subscriber Processor
Composable Architecture
Source
Destination
BTS On-P
File Trigger
File
Processor
SFTP Trigger
SFTP Processor
HTTP Request
Trigger
File Processor
HTTP POST
Deleter
APIM
Transform
Business Prop
Promotion
Msg Pub
Adapter
FFEncode
Get
Context
Router
FFDecode
Determines physical endpoint adapter
APIM Policy, FFDecode, Business Prop Promotion, Adapter Router
APIM Policy
Msg Publisher
Pub / Sub
Msg Subscriber
Msg Subscriber Processor
Patterns and Composable Services
Composable Flows for integrations
Leverage existing patterns or compose a new one
Microservices should be autonomous
Refactor to net-new, don’t change an existing pattern or service
Routing Slip Pattern
APIM, API, and Policies to expose and describe pattern
Context to describe the flow specific parameters that allows for multiple scenarios to flow through one pattern
Generic ARM Templates to describe concepts that may need implementation specific properties at deployment time (e.g. FileAdapter, SFTP, Message publishing and subscribing patterns)
AdapterRouter concept for endpoint implementation details

7. Telemetry Hot-path Warm-path
Run Scope for exception handling within logic apps (similar to a try/catch)
All exceptions are not created equal; may need to enumerate through exceptions before posting
Single logic app concept to move exceptions into an Event Hub (not the same used by Logic Apps)
Azure Functions consume exceptions from Event Hub for both App Insights and internal Ticket management system
Warm-path
OOB Logic App infrastructure plus Integration Account
Event Hub and OMS
OMS drives non-flow business exceptions

8. ALM and Dev-ops IST PST Enterprise Commerce / Volume Licensing
Supply Chain / Core Integration Service
CFE
Common Backlog (shared and triaged across regions)
Feature Branches
‘develop’
CI / CD
Pull request into develop
sprint
Sprint teams live here: iterating through tasks and user stories until features are ready or sprint ends. What’s passing tests makes it to release
‘release’
Pre-production (PPE)
Pull request into release
DRI’s work here: Executing PR’s into ‘release’ and managing the deployment across primary/secondary regions.
Issues found here can be dealt with by DRI following Git best practices for this scenario
‘master’
Production
Pull request into master
DRI’s work here: Executing PR’s into ‘master’ and managing the deployment across primary/secondary regions.
Issues found here can be dealt with by DRI following Git best practices for this scenario

9. ALM and Dev-ops Git for source control.
Each code check-in goes thru pull request (code review), at least 2 approvals required and enforced thru branch policies
Code is compiled – gated check in build
Policies for security or governance enforcement (e.g. scan for potential security risks or ensure the person checking in code is supposed to be)
Azure DevOps (formerly known as Visual Studio Online (VSO)) for build and release management.
Two builds
Gated check in – code check in and performs compilation of the code executes unit tests
Continuous Integration build – used to deploy/release, compiles the code, runs the unit tests and creates a build structure to be consumed by release/deployment.
Release
One release pipeline for each environment
Each release pipeline consists of steps to perform various tasks to complete tasks in a given environment and run functional tests
Branching Strategy
Following a basic Git practice of using a ‘develop’, ‘release’, and ‘master’ branch
Deployments into CI, PPE (Pre-Production), and Production
Deploy core services components on a sprint cadence
Unit test / functional test
Unit test what can be unit tested
Functional test for things like Logic Apps, APIM Policies, and any other non-unit testable capability that makes up the integration service
In general
Agile != ‘No Process’
Dev-ops != ‘No Governance’

10. Security and Governance
IP Whitelist to further secure logic apps (Migration to ISE)
Managed Identities for AAD auth between logic apps (HTTP Actions/Triggers). Remove SAS token from URL
Things to consider in a hybrid environment
Access Management – who has access to my system when considering PaaS and on-premises resources (e.g. BizTalk)
Release Management – Separation of duties between developers and release management in production
Privacy – How to handle highly confidential data or personal data (e.g. cosmosdb, table/blob storage, Azure SQL
Cost Optimization – Once you begin to figure this PaaS thing out, how do you corral the cost

11. AIS Migration Story Metadata driven Architecture
Migration Accelerators
Schema Provisioning
Partner – Agreement json creation
Maps to XSLT conversion
Metadata creation
AB Testing
Metadata Creation
Schemas Json
Maps Json
End Point
Flow Context
Partner Flow Context
Milestones Achieved

12. Enterprise Integration Disaster Recovery
Scheduled Sync Solution’s
Runtime Sync
LogicApps scheduled to run every 3 mins to update secondary region Integration account to make them sync and vice versa post DR.
AS2 Synchronization (MIC Algorithm)
X12 Synchronization (Control Numbers Sync)
EDIFACT Synchronization (Control Numbers Sync)
Artifacts Sync
Integration Account Artifacts sync scheduled to run every 1 hour to secondary region and vice versa if needed.
Schemas
Maps
Certificates
Partners
Agreements
Assembly
Batch Configurations
RosettaNet PIP
Pre/Post DR Scripts Execution
Bump Interchange Control Numbers in secondary region and vice versa post DR.
Flip Azure Traffic Manager(ATM) routing from Primary to secondary APIM and vice versa post DR.
Disable/Enable LogicApps.
Lookup Azure Storage Table sync.

13. Appendix
Open-source Accelerator Tools, Demos and scripts available at Github
Write to us for any integration queries:

14. Recap – B2B Approach
Use one Integration Account and set of logic apps across multiple Businesses and Trading Partners (Partner info, Agreements, Document Types, Schemas, Maps, Batch Configurations)
Flow concept static but type can be overridden based on message (e.g. different logic apps that process X12 or EDIFACT messages)
Metadata used to populate runtime properties specific to a TP or a LOB (currently stored in a CosmosDB and accessed via Azure Function with caching)
A ‘Routing Slip’ can override the URL of next logic app to load (can be overridden using APIM Policy)

15. Recap – B2B Approach Trading Partner Gateway Exchange Protocol
Document
Business
Process /
Orchestration
Line of
Integration
On-premises
Connectivity
LOB
Application
MTPB/HTTPS
SFTP
AS2
RNIF
BTF
EDI X12
EDI EDIFACT
OAGIS
RosettaNet
Custom message processing
Transforms
Business Property Promotion
Flat file encoding
LOB specific enhancement (custom processing)
EHC
EIS Adapter
BizTalk
Flat file decoding
APIM/HTTPS

16. Recap – B2B Approach Logic Apps Gateway APIM TP Service Lines
Replication for Busn. Continuity
Primary
APIM
Logic Apps
EHC
Corpnet
BTS2016
Service Lines
(Business Units)
Secondary
Gateway provides an Active/Active store and forward mechanism for B2B type traffic. Trading Partners post messages to endpoint which round robins between the two regions.
There are two deployed installations of B2B Service. However, only one is active and any time.
At deployment time, artifacts are deployed to both regions. This includes all static data including Logic Apps, Functions, Metadata
During runtime, there are properties that are updated in the Enterprise connectors that must be replicated across both regions for business continuity. In the passive region, there is a set of logic apps responsible for listening to events in primary region and updating the appropriate Integration Account Artifacts
EHC provides secure connectivity to on-premises resources. B2B Service uses corpnet BTS 2016 servers
BTS 2016 provides pass-thru connectivity to existing LOB Applications over a wide range of technologies
BTS 2016 provides the necessary capabilities to send and receive messages from LOB Applications

17. Recap – B2B Approach APIM PAExchange ProtocolAS2V2 PATPC MessageInfoV2
PADocument
ProtocolX12V2
PADebatch
GoodMessagesV2
PALobAdapter
BTSV2
PALob
IntegrationV2
PABusiness
Process
Orchestration V2
EHC
Adapter
ProtocolEdifactV2
BadMessages
GeneratedAcks
ReceivedAcks
BTS On-premises
LOB
Application
Azure Function
Integration Account
CosmosDB
APLob
APBusinessProcess
OrchestrationV2
APMessage
SenderX12V2
APBatch
MessagesX12V2
APDocument
EdifactV2
APExchange
APAsyncMDN
ProcessorV2

18. Composable Architecture
Patterns and Composable Services
Composable Flows for integrations
Leverage existing patterns or compose a new one
Microservices should be autonomous
Refactor to net-new, don’t change an existing pattern or service

19. Composable Architecture
Routing Slip Pattern
APIM, API, and Policies to expose and describe pattern
Context to describe the flow specific parameters that allows for multiple scenarios to flow through one pattern
Generic ARM Templates to describe concepts that may need implementation specific properties at deployment time (e.g. FileAdapter, SFTP, Message publishing and subscribing patterns)
AdapterRouter concept for endpoint implementation details