1. Building a Windows Azure Application
Architecture, Patterns, Processes
Andreas Piefke, Architect (MYOB)
28/11/2011

2. Architecture Overview
User Interface
Biz Logic
Biz Logic
Smart Client
Split at existing API
Tweaked the granularity of the existing API where required
Client can operate in local mode or Azure mode
Client may have a synchronised local copy of the Azure database to work offline
Database
Database
Sync

3. Configuration/Management
Architecture cont.
Web Roles
Business
Services
Collaboration
Configuration/
Management
Worker Roles
SQL Azure
Customer
Databases
Configuration/Management
Database
Azure Storage
Customer facing and internal web roles
Multiple worker roles for different purposes
Single tenanted customer database
Configuration and management database for cloud specific services
Azure Storage (Queues, Tables and Blobs) for Management tasks
Communication to internal systems is always initiated by the internal systems
Federated Identity for authentication, the system does not contain and authentication information
Queues
Tables

4. Patterns – Long Running Operations
Sends
Message
Receives
Message
Initiates Operation
Queue
Storage
Client
Web Role
Worker Role
Performs
Operation
Polls for
Completion
Client initiates operation via Web Role
Web role creates job in table storage and sends message with job id
Worker role receives the message …
Worker role performs operation based on the job information and updates job status
Client polls for job completion by calling the web service
Updates
Job Status
Checks
Job Status
Table
Storage

5. Patterns – Data Transfer (Upload)
Requests
Shared Access Signature
Creates
Job
Web Role
Client
Table
Storage
Uploads direct to
BLOB
Example: Upload and existing customer database to Azure
Client requests Shared Access Signature
Shared Access Signature provides time limited delegated access to Azure storage
Client uploads data directly into BLOB storage
Client notifies web role when upload is completed
Web role sends message to worker via queue as per pattern for long running operations
Download works similar, except that the long running operation is initiated first and on completion the client requests the shared access signature for the BLOB storage
BLOB
Storage

6. Patterns – Communication with Internal Systems
Web and Worker roles generate Notifications
Updates
Management
Data
Queue
Storage
BizTalk
Web Role
Receives
Notifications
Communication to internal systems is asynchronous
Communication is always initiated by the internal system
Notifications generated by web and worker roles are put into queues
Possible improvement would be to move a BizTalk Web Service into Azure AppFabric
That would break the policy of communication being initiated by the internal systems
Communication is always initiated by the internal system
Management
Database

7. Processes – Continuous Integration
CI runs automated deployment to Azure
Checks for deployment success
Runs smoke test to verify basic functionality
Automated Azure deployment can be a challenge as some roles may fail to start up for unknown reasons
Rebooting a role usually fixes the problem
Hard to do in an overnight test
As many tests as possible are run in Compute Emulator (or DevFabric for old friends)

8. Processes - QA Requires testing in multiple Environments
Compute Emulator
Multi-machine deployment
Windows Azure
Separate environment
Additional environment for System Integration
Automated Deployment
The aim is to test as much as possible in Compute Emulator
Running multiple clients against a Compute Emulator instance is not supported out of the box and requires some tweaking
rinetd for port request redirection from external network interface to localhost
QA data setup in Windows Azure can be a challenge

9. Processes – Monitoring and Debugging
Audit logging
IntelliTrace
Tracing/Diagnostics
Configuration requires redeployment (check SDK 1.6)
Custom configuration to allow enabling/disabling from service configuration
Tracing is essential to diagnose production problems in Azure.
Some things cannot be tested in Compute Emulator
Database size changes
Creation of new SQL Azure servers

10. Processes – Testing in Azure
Multiple Environments in one Subscription
Hosted Services
Storage Account
Federated Identity
Package once as part of CI
Transform Service Configuration for specific environment
MYOB has 5 teams working in Windows Azure
Significant amount of Windows Azure resources: Hosted Services, Storage Accounts, Cores
Transformation requires configuration to be in service configuration file

11. Challenges Deployment Times Encryption (where to hide a key)
Certificates, Certificates, Certificates
Encryption (where to hide a key)
Scaling and Autoscaling
Latency (where to host)
David Robinson quote: “You guys have crappy pipes”
Deployment times.
Bring your favourite ebook reader or tablet to read a good book
Can vary significantly
Limit the number of deployments that can be done in a day
Lots of things in Azure require certificates
Deployment
SSL (consider test environments, we used a wildcard certificate and CNAMEs)
Authentication
Federated identity
Make sure the certificates are managed properly
Make sure there are processes in place to replace expired certificates
The whole point of cloud is elastic scaling
Patterns and practices has released an Autoscaling application block (WASABi)
Schedule based scaling
Performance based scaling (queues or performance counters)
Latency
Australia is an island
So far there is no local Azure data centre
Geographically closest is Singapore
Connectivity is possibly better to the US
Latency depends on ISP and location within Australia
Test to figure out which location works best for you

12. Challenges cont. Platform continues to evolve Azure SDK Updates
Things can change
Example: New data management views for federation
Azure SDK Updates
Coordination across 6 teams + DevOps
No side-by-side
CI environment with more than one SDK version
Changes to the platform:
Keep an eye on any announcements from Microsoft regarding small changes to the Azure platform to avoid getting caught out
Example:
Query to get a list of tables from information_schema was not narrow enough
SQL Azure introduced 8 data management views in relation to federation (sharding) in the sys schema
The query returned the views which resulted in an exception in the code
New SDK versions are released frequently
Migration usually requires changes to the build/deployment pipeline

13. What would I do differently?
Use AppFabric Queues instead of Storage queues
Use AppFabric Workflow Foundation for long running processes
The platform has evolved since we started development.
AppFabric queues were not available when we started and there was no point changing a working solution
AppFabric hosted workflow foundation is in development but has not yet been released

14. Tools Cerebrata (now Redgate) SQL Azure Backup (Redgate)
Azure Storage Studio
Azure Diagnostics Manager
SQL Azure Backup (Redgate)
Azure Storage Explorer
rinetd

15. Questions