UNC301 - (Microsoft IT) Microsoft Exchange 2007 Architecture and Design at Microsoft Martin Coetzer Technical Consultant Microsoft

Agenda Microsoft IT Exchange environment overview Transport and topology Server roles and hardware planning Mobile messaging scenarios – Client Access Server Internet mail connectivity – Edge Transport Server Mailbox server and storage designs Backup approach (current and future)

Microsoft IT Exchange Environment Overview Multiple Exchange organizations (Corp, Dogfood, Windeploy, WinSE, Extranet, Managed Solutions) Prior to transition to Exchange 2007 –~100 legacy Exchange 2003 servers / 30 mailbox servers After transition to Exchange 2007 –~64 Exchange 2007 mailbox clusters running in the Corp forest at RTM –Goal to reduce it to ~35 in the nearest future Mailboxes – 130,000+ Public Folders – 240,000 folders Distribution Groups – 170,000+ Maximum message size = 10 MB Availability Goal – 99.99% Mailbox size on Exchange 2003: 200 MB Mailbox size on Exchange 2007: 500 MB and 2 GB

Post-Consolidation Environment 4 Sites with Exchange mailbox servers 100 Exchange servers - 30 mailbox server (clustered) x200MB per cluster node 99.99% availability goal 4 Sites with Exchange mailbox servers 100 Exchange servers - 30 mailbox server (clustered) x200MB per cluster node 99.99% availability goal

Microsoft IT Messaging Trends and Needs Category Exchange 2000 timeframe Exchange 2003 timeframe Exchange 2007 timeframe Mailboxes total70,00090,000130,000 Typical mailbox quota100 MB200 MB500MB / 2GB Mobile users per month OWA EAS Outlook RPC/HTTP Limited N/A 50,000 13,000 25,000 60,000 30,000 60,000 Internet message submissions per day (prior to filtering) 6,000,0009,000,00013,500,000 Spam messages filtered per day2,500,0007,500,00011,000,000 Maximum message size5 MB10 MB10+ MB

Microsoft IT Exchange Server 2003 Deployment Limitations of 32-bit platform Scalability limitations –Virtual memory is limited to 4GB Exchange Server 2003 can use up to 3GB max –Kernel Memory limitations Paged Pool 256MB max with /3GB enabled Client connections, Access tokens Non Paged Pool (256MB max without /3GB and 128MB max with /3GB) HTTPs and TCP connections to the Exchange Front End or Gateway Storage Performance demands –Small Database cache size = higher disk I/O and storage demands

Goals of Exchange Server 2007 Deployment at Microsoft Overcome scalability limitations of 32-bit platform Maintain 99.99% availability goal Increase the user mailbox size from 200MB to 500MB and 2GB Reduce hardware, storage, and backup costs Establish mailbox server resilience from storage level failures Establish flexible and scalable middle-tier services (e.g. Client Access, Transport)

Routing Group Connectors RGC RGC RGC Legacy Exchange Server 2003 Environment

Dublin Singapore Sao Paulo Redmond Redmond- Exchange AD Site Link Exchange Server 2007 Environment AD Site with Exchange Servers AD Site without Exchange Servers

Dublin Singapore Sao Paulo Redmond Redmond- Exchange AD Site Link Exchange Server 2007 Environment Exchange Routing AD Site with Exchange Servers AD Site without Exchange Servers

Dublin Singapore Sao Paulo Redmond- Exchange AD Site Link Custom Site Link ExchangeCost=10 ADCost=999 Redmond Exchange Server 2007 Environment set-adsitelink Dublin-to-RedmondExchange -ExchangeCost 10 set-adsitelink SaoPaulo-to-RedmondExchange -ExchangeCost 10 set-adsitelink Singapore-to-RedmondExchange -ExchangeCost 10

Microsoft IT Exchange Server 2007 Server Roles In Exchange Server 2003 timeframe Microsoft IT deployed servers roles dedicated for specific functions Exchange Server 2007 deployment – similar approach –Structured system administration and maintenance –Optimized hardware and software configuration –Targeted load balancing and fault tolerance –Flexible scaling approach –Independent and flexible –server provisioning –Higher Availability

Exchange 2007 Load Balancing and Fault Tolerance at Microsoft IT Enterprise scenario – must build load balancing and fault tolerance provisions –Resilience to server level failures –Tolerance to increased load due to planned server downtime and environmental conditions (spam attacks, “snow day”) ComponentLoad BalancingFault Tolerance Mailbox ServerN/AClustering (CCR) NIC Teaming Hub Transport ServerMBX  HUB: Built in Edge  HUB: Send Connector – multiple bridgeheads Edge Transport ServerInternet  Edge: MX equal cost & DNS Round Robin HUB  Edge: Edge Subscriptions Client Access ServerExternal: Network Load Balancing + ISA 2006 WPLB Internal: Network Load Balancing Unified MessagingUM  IP Gateway: Multiple IP Gateways per dial plan IP Gateway  UM: Built in (Round Robin between UM)

Hardware Configurations RoleConfiguration BHD/Edge 1 4x2.2 GHz CPU*, 8GB RAM CAS 2 4x2.2 GHz CPU*, 4GB RAM UM 2x2.2 GHz CPU*, 4GB RAM MBX 500MB (CCR) 4x2.6 GHz CPU*, 12GB RAM MBX 2GB (CCR) 4x3.0 GHz CPU*, 16GB RAM MBX 2GB (CCR) 8x2.6 GHz CPU*, 24GB RAM MBX 500MB (CCR) 8x2.66 GHz CPU, 16GB RAM 8x2.66 GHz CPU **, 16GB RAM * * 2 way = One Dual Core CPUs; 4 way = Two Dual Core CPUs ; 8 way = Four Dual Core CPUs ** 8 way = Two Quad Core CPUs 1 1 7:1 Mailbox-to-HUB server performance ratio 2 2 4:1 Mailbox-to-CAS server performance ratio SPECint ® 2000 rates were used to compare different processors ( )

Common URL namespace for mobile messaging clients –OWA, Exchange Active Sync (EAS), Outlook Anywhere (RPC/HTTP) (e.g. Integration – all mobile messaging services on a common Exchange Server 2007 CAS platform Mobile Messaging Service Usage –OWA: ~60,000 unique users/month –RPC/HTTP: ~60,000 unique users/month –EAS: ~30,000 unique users/month Highly scalable Exchange Server 2007 CAS infrastructure –Load balancing and fault tolerance Overview

Messaging Topology Dublin: -ExternalURL “ Singapore: -ExternalURL “ Sao Paulo: -ExternalURL “ Redmond: -ExternalURL “ -Autodiscover Service URL “

Design Published Paths /exchange/* /owa/* /public/* /Microsoft-Server- ActiveSync/* /RPC/* /Autodiscover/* /EWS/* /UnifiedMessaging/* Cookie-based load balancing IP-based load balancing The Exchange front-end servers are part of a Web farm pointed to by an ISA Server 2006 array

Microsoft IT Exchange 2007 Edge Server

Deployment Goals: Mailbox size: 2GB; Deleted items retention time: 14 days Daily Transaction Log generation rate per mailbox per day: 25MB (average) Database overhead (fluff) factor: 20% Desired max DB size - 200GB: –200 GB / (2GB + 20%) = 85 mailboxes/DB One Database per Storage Group Databases per LUN: 7 (to facilitate weekly backup schedule) –One Backup Stream Per source LUN DB LUN Capacity requirements for 85mbx * 7DBs= 600 mailboxes/LUN –600 mailboxes * 2GB + 20%[DB overhead] + 5%[Ci] + 10%[Free space] = 1,656GB –1,656GB / 275GB * 2 [RAID1+0] = 12 disks DB LUN Performance requirements –Assuming 1.0 IOPS/mbx at 2GB for very heavy user profile with 5MB of RAM/user –600 mailboxes * 1 IOPS/mbx = 600 IOPS / LUN –1:1 Read:Write mix (300 Reads Writes = 600) 300[R] + 2x300[W] = 900 IOPS 900 / 130 IOPS = 7 disks (10K RPM) Conclusion: Design is capacity bound - 12 disks required for capacity [RAID1+0] Hardware Design: Mailbox Storage Provisioning

Hardware Design: Universal Storage Building Block – 2GB Goal: Design simple and scalable storage building block Goal: Design simple and scalable storage building block Approach: Two disk enclosures Cross-shelf mirroring [RAID 1+0] LUN Capacity: 600 mailboxes = 12 disks [RAID 1+0] 7 Storage Groups/Databases Single Logical Drive Two Database LUNs total 1,200 x 2GB mailboxes One Log LUN 2 logical volumes (L:\ + M:\) 825GB of log space 30 x 300GB drives 24 for databases 6 for transaction logs Data:Log – 4:1 Goal: Design simple and scalable storage building block Goal: Design simple and scalable storage building block

Hardware Design: Universal Storage Building Block – 500MB LUN Capacity: 830 mailboxes = 14 disks [RAID 1+0] 7 Storage Groups/Databases Single Logical Drive Three Database LUNs total 2,500 x 500MB mailboxes One Log LUN 550GB of log space 50 x 146GB SFF SAS drives 42 for databases 8 for transaction logs

Hardware Design: 2,400 x 2GB mailboxes (SAS DAS)

Hardware Design: 3,600 x 2GB mailboxes (FC SAN)

Streaming Backup Backup Window: 4 hours Schedule: Full – weekly, Incremental - daily

System Center Data Protection Manager 2007 VSS based backup from “Active” or “Passive” CCR aware requestor Express Full backup technology (deltas) Incremental backup - every 15 minutes

Q&A

Thank you