Ashish Jain, Mike Bizub, & Vignesh Sukumar Microsoft CSE&O How Microsoft IT does Integration

Agenda Recap of who we are and what we do Composable architecture and how we’ve implemented the Routing Slip Pattern Hot-Path Telemetry ALM, Dev-ops, and Governance Business Continuity and Enterprise Disaster Recovery Migration Tools update Appendix

Enabling Business Processes Microsoft Integration Landscape 150M+ Messages Per Month 1,000 + Partners – Internal and External 3,400+ Integration Streams Centrally Managed 200+ Microsoft Systems Enabling Business Processes Supply Chain Contract Management Licensing Sales Benefits Legal Payroll Finance Travel Suppliers Distributors … etc $1.6 B Daily bank treasury transaction 100+ BizTalk Servers in use today Message Standards X12 – 30% EDIFACT-14% XML-12% SWIFT-12% Other – 32% Multi-Platform BTS 2016, MABS, Logic Apps

Recap – The People (giving credit where it’s due)

Recap – B2B Approach Use one Integration Account and set of logic apps across multiple Businesses and Trading Partners (Partner info, Agreements, Document Types, Schemas, Maps, Batch Configurations) Flow concept static but type can be overridden based on message (e.g. different logic apps that process X12 or EDIFACT messages) Metadata used to populate runtime properties specific to a TP or a LOB (currently stored in a CosmosDB and accessed via Azure Function with caching) A ‘Routing Slip’ can override the URL of next logic app to load (can be overridden using APIM Policy)

Msg Subscriber Processor Composable Architecture Source Destination BTS On-P File Trigger File Processor SFTP Trigger SFTP Processor HTTP Request Trigger File Processor HTTP POST Deleter APIM Transform Business Prop Promotion Msg Pub Adapter FFEncode Get Context Router FFDecode Determines physical endpoint adapter APIM Policy, FFDecode, Business Prop Promotion, Adapter Router APIM Policy Msg Publisher Pub / Sub Msg Subscriber Msg Subscriber Processor Patterns and Composable Services Composable Flows for integrations Leverage existing patterns or compose a new one Microservices should be autonomous Refactor to net-new, don’t change an existing pattern or service Routing Slip Pattern APIM, API, and Policies to expose and describe pattern Context to describe the flow specific parameters that allows for multiple scenarios to flow through one pattern Generic ARM Templates to describe concepts that may need implementation specific properties at deployment time (e.g. FileAdapter, SFTP, Message publishing and subscribing patterns) AdapterRouter concept for endpoint implementation details

Telemetry Hot-path Warm-path Run Scope for exception handling within logic apps (similar to a try/catch) All exceptions are not created equal; may need to enumerate through exceptions before posting Single logic app concept to move exceptions into an Event Hub (not the same used by Logic Apps) Azure Functions consume exceptions from Event Hub for both App Insights and internal Ticket management system Warm-path OOB Logic App infrastructure plus Integration Account Event Hub and OMS OMS drives non-flow business exceptions

ALM and Dev-ops IST PST Enterprise Commerce / Volume Licensing Supply Chain / Core Integration Service CFE Common Backlog (shared and triaged across regions) Feature Branches ‘develop’ CI / CD Pull request into develop sprint Sprint teams live here: iterating through tasks and user stories until features are ready or sprint ends. What’s passing tests makes it to release ‘release’ Pre-production (PPE) Pull request into release DRI’s work here: Executing PR’s into ‘release’ and managing the deployment across primary/secondary regions. Issues found here can be dealt with by DRI following Git best practices for this scenario ‘master’ Production Pull request into master DRI’s work here: Executing PR’s into ‘master’ and managing the deployment across primary/secondary regions. Issues found here can be dealt with by DRI following Git best practices for this scenario

ALM and Dev-ops Git for source control. Each code check-in goes thru pull request (code review), at least 2 approvals required and enforced thru branch policies Code is compiled – gated check in build Policies for security or governance enforcement (e.g. scan for potential security risks or ensure the person checking in code is supposed to be) Azure DevOps (formerly known as Visual Studio Online (VSO)) for build and release management. Two builds Gated check in – code check in and performs compilation of the code executes unit tests Continuous Integration build – used to deploy/release, compiles the code, runs the unit tests and creates a build structure to be consumed by release/deployment. Release One release pipeline for each environment Each release pipeline consists of steps to perform various tasks to complete tasks in a given environment and run functional tests Branching Strategy Following a basic Git practice of using a ‘develop’, ‘release’, and ‘master’ branch Deployments into CI, PPE (Pre-Production), and Production Deploy core services components on a sprint cadence Unit test / functional test Unit test what can be unit tested Functional test for things like Logic Apps, APIM Policies, and any other non-unit testable capability that makes up the integration service In general Agile != ‘No Process’ Dev-ops != ‘No Governance’

Security and Governance IP Whitelist to further secure logic apps (Migration to ISE) Managed Identities for AAD auth between logic apps (HTTP Actions/Triggers). Remove SAS token from URL Things to consider in a hybrid environment Access Management – who has access to my system when considering PaaS and on-premises resources (e.g. BizTalk) Release Management – Separation of duties between developers and release management in production Privacy – How to handle highly confidential data or personal data (e.g. cosmosdb, table/blob storage, Azure SQL Cost Optimization – Once you begin to figure this PaaS thing out, how do you corral the cost

AIS Migration Story Metadata driven Architecture Migration Accelerators Schema Provisioning Partner – Agreement json creation Maps to XSLT conversion Metadata creation AB Testing Metadata Creation Schemas Json Maps Json End Point Flow Context Partner Flow Context Milestones Achieved

Enterprise Integration Disaster Recovery Scheduled Sync Solution’s Runtime Sync LogicApps scheduled to run every 3 mins to update secondary region Integration account to make them sync and vice versa post DR. AS2 Synchronization (MIC Algorithm) X12 Synchronization (Control Numbers Sync) EDIFACT Synchronization (Control Numbers Sync) Artifacts Sync Integration Account Artifacts sync scheduled to run every 1 hour to secondary region and vice versa if needed. Schemas Maps Certificates Partners Agreements Assembly Batch Configurations RosettaNet PIP Pre/Post DR Scripts Execution Bump Interchange Control Numbers in secondary region and vice versa post DR. Flip Azure Traffic Manager(ATM) routing from Primary to secondary APIM and vice versa post DR. Disable/Enable LogicApps. Lookup Azure Storage Table sync.

Appendix Open-source Accelerator Tools, Demos and scripts available at Github https://aka.ms/Integrate2019 Write to us for any integration queries: btsmigrationtool@microsoft.com

Recap – B2B Approach Use one Integration Account and set of logic apps across multiple Businesses and Trading Partners (Partner info, Agreements, Document Types, Schemas, Maps, Batch Configurations) Flow concept static but type can be overridden based on message (e.g. different logic apps that process X12 or EDIFACT messages) Metadata used to populate runtime properties specific to a TP or a LOB (currently stored in a CosmosDB and accessed via Azure Function with caching) A ‘Routing Slip’ can override the URL of next logic app to load (can be overridden using APIM Policy)

Recap – B2B Approach Trading Partner Gateway Exchange Protocol Document Business Process / Orchestration Line of Integration On-premises Connectivity LOB Application MTPB/HTTPS SFTP AS2 RNIF BTF EDI X12 EDI EDIFACT OAGIS RosettaNet Custom message processing Transforms Business Property Promotion Flat file encoding LOB specific enhancement (custom processing) EHC EIS Adapter BizTalk Flat file decoding APIM/HTTPS

Recap – B2B Approach Logic Apps Gateway APIM TP Service Lines Replication for Busn. Continuity Primary APIM Logic Apps EHC Corpnet BTS2016 Service Lines (Business Units) Secondary Gateway provides an Active/Active store and forward mechanism for B2B type traffic. Trading Partners post messages to endpoint which round robins between the two regions. There are two deployed installations of B2B Service. However, only one is active and any time. At deployment time, artifacts are deployed to both regions. This includes all static data including Logic Apps, Functions, Metadata During runtime, there are properties that are updated in the Enterprise connectors that must be replicated across both regions for business continuity. In the passive region, there is a set of logic apps responsible for listening to events in primary region and updating the appropriate Integration Account Artifacts EHC provides secure connectivity to on-premises resources. B2B Service uses corpnet BTS 2016 servers BTS 2016 provides pass-thru connectivity to existing LOB Applications over a wide range of technologies BTS 2016 provides the necessary capabilities to send and receive messages from LOB Applications

Recap – B2B Approach APIM PAExchange ProtocolAS2V2 PATPC MessageInfoV2 PADocument ProtocolX12V2 PADebatch GoodMessagesV2 PALobAdapter BTSV2 PALob IntegrationV2 PABusiness Process Orchestration V2 EHC Adapter ProtocolEdifactV2 BadMessages GeneratedAcks ReceivedAcks BTS On-premises LOB Application Azure Function Integration Account CosmosDB APLob APBusinessProcess OrchestrationV2 APMessage SenderX12V2 APBatch MessagesX12V2 APDocument EdifactV2 APExchange APAsyncMDN ProcessorV2

Composable Architecture Patterns and Composable Services Composable Flows for integrations Leverage existing patterns or compose a new one Microservices should be autonomous Refactor to net-new, don’t change an existing pattern or service

Composable Architecture Routing Slip Pattern APIM, API, and Policies to expose and describe pattern Context to describe the flow specific parameters that allows for multiple scenarios to flow through one pattern Generic ARM Templates to describe concepts that may need implementation specific properties at deployment time (e.g. FileAdapter, SFTP, Message publishing and subscribing patterns) AdapterRouter concept for endpoint implementation details