1. Wagner Silveira
Microsoft MVP - Azure
Creating a Processing Pipeline with Azure Function and AIS

2. About me Principal Integration Architect Microsoft Azure MVP
MCSE Cloud Platform
About me
wsilveiranz

3. Agenda Case Study Initial Solution Updated Solution Key Components
A year later – what changed?
Lessons Learned

4. Case Study

5. The solution requirements
Onboarding of external EDI messages
Should allow for different transports
Message need to be Validated, translated, enriched, routed
Control on retries and notification over failed messages
Auditing of initial message
Visibility of where a single message was in the process

6. Logical Design

7. Initial Solution

8. Reality Check Big Payloads EDIFACT schema not available
Operational costs
End to end Monitoring

9. Updated Solution

10. How was this different from the initial solution?
Azure Functions
EDIFACT support via .NET Package
Claim check pattern
Dedicated instances
Azure Storage
Payload storage (claim check pattern)
Application Insights
End to End with custom events
Single technology for monitoring and notification

11. Key Components

12. Key Components
Claim check pattern
App Insights Custom Events

13. The claim check pattern
Publishing function separates content from metadata then:
Store content on a safe repository
Publish metadata + storage pointer + Reprocessing count to message engine
Subscribing processes receive metadata and retrieve content when required
The claim check pattern

14. App Insights custom events
End to end traceability using a Correlation ID
Allow for exceptions to be captured the same way
Leverage Functions support for App Insights
Implemented as a reusable component
App Insights

15. Hands on Time?

16. Key Management Leveraged MSI support where available
SQL Server
Storage
Key vault
Kept keys in key vault where MSI where not supported directly
Service Bus

17. Exception Management & Retries on Functions
Catch blocks using a notify and throw pattern
Leverage Function SB binding peek-lock pattern
Adjusted Maximum Delivery Count on SB
Messages that failed where delivered to DLQ

18. DLQ Management Logic Apps polling subscriptions DLQ every 6 hours
Each subscription DLQ could have it’s own logic
Reprocessing count
Resubmission logic
Notification logic
notification
Error blob storage

19. One year later…

20. New technology candidates
Integration Services Engine
Azure Durable Functions
Event Grid

21. What about new features
Azure Functions Premium
Improved support for Key Vault
Improved support for MSI
Virtual Network support + Service Endpoints

22. Lessons learned

23. Review the fine print

24. Operational cost is a design consideration

25. Make the best of each technology

26. Think about the big picture

27. In Summary Weight your technology options
Think about limitations, operational costs
Understand how your solution fit in the big picture
Understand the components roadmap

28. Thank you for attending!