11/28/2018 12:08 AM PDC09-FT26 Scale Your Data Tier using Windows Server AppFabric Caching (formerly Project “Velocity”) Muralidhar Krishnaprasad Principal Software Architect Microsoft Corporation © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Grocery Web Site

Typical Architecture Users Load Balancer Sticky Routing Application / Web Tier Application Application Application ASP.Net App Session Cart Hosted in memory Database Data Tier Catalog sits in Database

And then… Flu Hits!

You become a popular Grocer! Web Site’s too slow!! Users Where did my shopping cart go? Web Tier (ASP.Net) … Application Application Application Servers are crashing Database Data Tier Database is hot!!

Windows Server AppFabric Caching can help

Outline The Problem! Caching with the Windows Server AppFabric What is it? Why AppFabric caching? What stage is it in? Overview and Programming Caching Futures Conclusions

What is AppFabric Caching? An explicit, distributed, in-memory application cache for all kinds of data (CLR objects, rows, XML, Binary data etc.) Fuse "memory" across machines into a unified cache Caching clients can be across machines or processes Clients Access the Cache as if it was a large single cache Unified Cache View Cache Layer distributes data across the various cache nodes

… Where does it fit? Users Web Tier (ASP.Net) Data Tier Cache Tier Application Application Application Caching Access Layer Caching Service Cache Tier Database Cloud Data Tier

Outline The Problem! Caching with the Windows Server AppFabric What is it? Why AppFabric caching? What stage is it in? Overview and Programming Caching Futures Conclusions

Why use AppFabric Caching? 1. Share Data Across Applications No more sticky routing 2. Performance Oper Throughput Latency Bulk Fetch Read 28, 000/ sec (2k) 1-2ms 200k/sec (512 bytes) Write 20,000 / sec (2k) 3 ms - 3. Scale out by adding more boxes Operation Servers Throughput Read 2k 1 28,000 ops/ sec 2 52, 500 ops/ sec 3 78, 000 ops/ sec 4. High Availability Protect from Web & Cache Server Failure

Why use AppFabric Caching? 5. Proven Technology Associated Press Custom News live on CTP3 AppFabric caching capabilities caches metadata and news for widgets Serves 16 million hits per day Increased the amount of cached data 6 times.

Why AppFabric Caching? 5. Proven Technology Contd.. Dell.com uses AppFabric caching! Channel Partner Portal uses AppFabric caching Provides targeted content and search results Tracks customer browse history and downloads. Next Generation Sales Platform Solutions to use AppFabric caching (currently under construction) Just two of the many TAP customers in line for production

Why AppFabric Caching? – Best yet 6. Do all this at low cost! . Use inexpensive commodity hardware Ships as part of Windows Server AppFabric 2010 Wave 1 * Different Features of the cache might require higher level SKUs of Windows Server. AppFabric CACHING WORKFLOW HOSTING MONITORING SERVICE BUS SERVICE HOSTING ACCESS CONTROL MANAGEMENT HIGH AVAILABILITY SCALE OUT MULTI-TENANT

Outline The Problem! Caching with the Windows Server AppFabric What is it? Why caching? What stage is it in? Overview and Programming Caching Futures Conclusions

When Can I Get Windows Server AppFabric ? CTP1 June ‘08 Beta PDC Nov ‘09 CTP2 PDC ‘08 CTP3 Mar ‘09 RTM 2010 Wave 1 * CY10 Azure CTP Post RTM (Formerly Called “Velocity”) Microsoft Confidential

Outline The Problem Caching with the Windows Server AppFabric Overview and Programming AppFabric Caching Deployment & Access API How does AppFabric Caching Scale? Features AppFabric Caching Futures Conclusions

Deployment … Users Copy Client DLLs and add app.config Web Tier <hosts> <host name="BL1CDB8083714“ cachePort="22233" cacheHostName="DistributedCacheService"/> ….. </hosts> <localCache isEnabled=“true" ../> <security … /> Application Caching Access Layer Caching Access Layer Caching Access Layer Caching Service Cache Tier Configuration Store (Can be database, File share, etc.) Stores Global Cache Policies Stores Current Partitioning Information Caching Service Enable Application Server Cache Role

Security (new in Beta) Domain Based Security Option 11/28/2018 12:08 AM Security (new in Beta) Domain Based Security Option Domain Account / Local Account based Authentication Only authorized servers can join the cluster Only authorized clients can connect to the cluster Transport Level Security Turn on/off Signing or Encryption Can turn off Cache Security Use Firewalls, IPSec, VLANs to protect cache grant-cacheallowedclientaccount  RedDomain\Machine1$ grant-cacheallowedclientaccount  RedDomain\John © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Administration & Monitoring Centralized administration through powershell Perfmon to monitor the cache (new in Beta) Logging Default ETW, support for file logs

AppFabric Caching API // Create instance of cachefactory (reads appconfig) DataCacheFactory fac = new DataCacheFactory(); // Get a named cache from the factory DataCache catalog = fac.GetCache("catalogcache"); // Simple Get/Put catalog.Put("toy-101", new Toy("Puzzle", .,.)); // From the same or a different client Toy toyObj = (Toy)catalog.Get("toy-101"); // Region based Get/Put catalog.CreateRegion("toyRegion"); // Both toy and toyparts are put in the same region catalog.Put("toy-101", new Toy( .,.), “toyRegion”); Catalog.Put("toypart-100", new ToyParts(…), “toyRegion”); Toy toyObj = (Toy)catalog.Get("toy-101“,"toyRegion");

AppFabric Caching Logical Hierarchy AppFabric Caching Service AppFabric Caching Service AppFabric Caching Service AppFabric Caching Service Named Cache : Product Catalog Named Cache : Electronics Inventory Regions Key Payload Tags Region A 121 xxxx “Toy” “Child” 123 yyyy “Toy” “Chair”.. Machine -> Cache Host -> Named Caches -> Regions -> Cache Items -> Objects Host Physical processes hosting AppFabric Caching instance. Named Caches Can span across machines Defined in the configuration file Cache Item Key, Payload (Object ), Tags, TTL, Timestamps, Version Regions Physically co-located Container of Cache Items May be implicit or explicitly created

Access APIs – Tagging Items Add Tags to Items Tag Search on Default Regions (New in Beta) Tag hotItem = new Tag("hotItem"); catalog.Put("toy-101", new Toy("Puzzle"), new Tag[]{hotItem}, “toyRegion”); catalog.Put("toy-102", new Toy("Bridge"), “toyRegion”); // From the same or a different client List<KeyValuePair<string, object>> toys = catalog.GetAnyMatchingTag("toyRegion", hotItem);

Usage Pattern – Cache Aside (Explicit Caching) // Read from Cache Toy toyObj = (Toy) catalog.Get("toy-101"); Application Caching Access Layer // If Not present in the cache if (toyObj == null) { // Read from backend.. toyObj = ReadFromDatabase(); // Populate Cache catalog.Put("toy-101", toyObj); return toyObj; } Caching Service Database

Outline The Problem Caching with the Windows AppFabric Overview and Programming AppFabric Caching Deployment & Access API How does AppFabric Caching Scale? Features Demo AppFabric Caching Futures Conclusions

Data Distribution - Partitioned Cache Users … Web Tier ASP.Net App ASP.Net App ASP.Net App Caching Client Caching Client Caching Client A C E D B G I H F Cache Service Cache Service Cache Service Cache Tier A C E D B G I H F Scale on Data Size - More machines => More memory to cache Scale on Cache Throughput - More machines => keys distributed across more machines => better throughput

Scale Test Output Load Throughput Latency Server 3 Added Throughput Increases Latency Decreases Load Server 2 Added Throughput Increases Latency Decreases Until Server Saturation Single Server Throughput Increases with Increasing Load Until Server Saturation Throughput Latency

GridDynamics Blog Engine Tests More info at http://blogs.msdn.com/velocity/ or http://www.griddynamics.com/velocity

Outline The Problem AppFabric Caching – The Solution Understanding Cache Data Overview and Programming AppFabric Caching Deployment & Access API How does AppFabric Caching Scale? Features Demo AppFabric Caching Futures Conclusions

Use the right features! Classify your data Examine Requirements Reference or Activity or Resource Data Examine Requirements Performance - Throughput & Latency Consistency – Tolerable staleness Eviction - Is the data evictable? Security - Is there any secure data? Availability – Survive node or cluster failures?

Types of Data Grocery Inventory Grocery Shop Web Tier Reference Activity Resource Primary Read Only Read-Write Not shared Read-Write, Shared Catalog Data Shopping Cart Auction Data/Seat Assignment Web Tier Distributed Cache Shopping Cart Grocery Catalog Grocery Inventory Grocery Shop

Reference Data – Performance Catalog data doesn’t change often Unnecessary network cost to access from different machines Solution – Local Cache Application Application Get(K2) Get(K2) Put(K2, v3) AppFabric Caching Client AppFabric Caching Client Local Cache K2, V2 Routing Table Routing Table Cache1 Cache2 Cache3 Primary for K1,V1 Primary for K2,V2 Primary for K3,V3 K2, V2 K1, V1 K2, V3 K3, V3

Reference Data – Bulk Get Enumerate all items in Region Bulk Fetch from region (New in Beta) 200-300k ops per second Catalog.BulkGet( new List<string>(){“toy-101”, “toy-102”} , “toyRegion”);

Activity Data – Session Cart Integration Load Balance Requests No more sticky routing <sessionState mode="Custom“ customProvider="SessionStoreProvider"> <providers> <add name="SessionStoreProvider" type=“Microsoft.Data.Caching.DataCacheSessionStoreProvider, ClientLibrary“ cacheName="<YourNamedCache>"/> </providers> </sessionState> Drop in AppFabric Caching SessionStoreProvider Application Caching Access Layer … Caching Access Layer Caching Access Layer Session State stored in AppFabric Caching New in Beta – Allow session state to be shared amongst multiple applications Scale your Session Store Dynamically Cache Service Caching Service Caching Service Highly Available

Activity Data - Availability Application (K2, V2) Application PUT Get(K2) AppFabric Caching Client Routing Table AppFabric Caching Client Routing Table Cache1 Cache2 Cache3 Primary for Primary for Primary for K1, V1 Replication Agent K3, V3 K2, V2 K2, V2 Secondary for Secondary for Secondary for K2, V2 K3, V3 K1, V1

Resource Data - Optimistic Locking GetCacheItem returns a version object Every update to an object internally increments it's version Supply the version obtained along with the Put/Remove Put/Remove will succeed only if the passed in version matches the version in the cache Version Based Update Time Client1 Client2 (Different Thread or process) T0 CacheItem item = catalog.GetCacheItem(“PlayerRegion”, ”Zune”); catalog.GetCacheItem(“PlayerRegion”, ”Zune”); T1 ((ZuneObject)item.Object).inventory --; ((ZuneObject)item.Object).inventory--; T2 catalog.Put(“PlayerRegion”, “Zune”, item.Object, item.Version); T3 // Version mismatch // Client must retry again Two clients access the same item Both update the item Second Client gets in first; put succeeds because item version matches; atomically increments the version First client tries put; Fails because the versions don’t match

Resource Data - Pessimistic Locking Client1: GetAndLock ("k1") Client2: GetAndLock ("k1") Client3: Get ("k1") GetAndLock gets lock handle Regular Get succeeds Other GetAndLock on same item fails K1 New in Beta – Take locks on non-existent keys Allows you to co-ordinate creating new object amongst multiple clients

Resource/Activity Data – Tracking Changes 11/28/2018 12:08 AM Resource/Activity Data – Tracking Changes Cache Event notifications Register on any client to notify changes Batched Notifications – (New in Beta) DataCache.RegisterCacheLevelCallback( int filter, DataCacheChangeCallback delegate); DataCache.RegisterRegionLevelCallback( String region, int filter, DataCacheChangeCallback delegate); DataCache.RegisterKeyLevelCallback( String region, String key, int filter, DataCacheChangeCallback delegate); © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Scalable Notifications Register Notification for Key “K3" Call Delegate Store Last LSN Application Map Keys to Partition (say P2) AppFabric Caching Client Partition: P2 Last LSN: 19 Routing Table Poll Required Nodes Nodes Return List of Changes LSN Order Cache1 Cache2 Cache3 Primary for Primary for Primary for Change Log (Partition P2) Del K32 Del K43 Change Log 33 Add K1 34 Del K22 Change Log Partition P1 Add K2 Del K32 K1, V1 K2, V2 K3, V3

Outline The Problem AppFabric Caching – The Solution Understanding Cache Data Overview and Programming AppFabric Caching Deployment & Access API How does AppFabric Caching Scale? Features Demo AppFabric Caching Futures Conclusions

Outline The Problem AppFabric Caching – The Solution Understanding Cache Data Overview and Programming AppFabric Caching AppFabric Caching Futures AppFabric Caching and Azure Platform ASP.Net and Persistance Conclusions

Application Growth You are a popular Grocer!! Lots of Users! Irregular & Elastic Demands Solution - Move to the Services World! Application on-premise with data on SQL Azure Application on Windows Azure with data on SQL Azure Application on Windows Azure with on-premise database connected by AppFabric Service Bus

App on-premise; Data on SQLAzure … ASP.Net Web Tier Application Application Application AppFabric Caching Access Layer AppFabric Caching Access Layer AppFabric Caching Access Layer Application & AppFabric Caching deployed On-premise Caching Service Caching Service Caching Service Caching Worker Role Data on SQL Azure

App on Windows Azure; Data on SQL Azure Web Role … Application Application Application AppFabric Caching Access Layer AppFabric Caching Access Layer AppFabric Caching Access Layer Application & AppFabric Caching on Windows Azure Caching Service Caching Service Caching Service Caching Worker Role AppFabric Caching Worker Role Data on SQL Azure

App on Windows Azure; Data on-premise Web Role … Application Application Application AppFabric Caching Access Layer AppFabric Caching Access Layer AppFabric Caching Access Layer AppFabric Service Bus Application & AppFabric Caching on Windows Azure Caching Service Caching Service Caching Service Caching Worker Role AppFabric Caching Worker Role Data on-premises

Outline The Problem AppFabric Caching – The Solution Understanding Cache Data Overview and Programming AppFabric Caching AppFabric Caching Futures AppFabric Caching and Azure Platform ASP.Net and Persistance Conclusions

Integration with ASP.Net 11/28/2018 12:08 AM Integration with ASP.Net ASP.Net extends cache surface .Net 4.0 Integrate with the Application Cache System.Runtime.Caching namespace AppFabric Caching will be a provider for this namespace Granular Session Updates IPartialSessionState lists keys changed Extensible Output Cache Provider Output Caching not limited to single node © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Future – Cache Through Bulk Access APIs Callback for read-through, write-behind Specified at Named Cache Level Read-Through Called when item not present in cache Callback returns the object/serialized bytes Write-Behind Writes to cache are queued Callback called asynchronously in batches Re-tries upon failure Bulk Access APIs

Central Market Data Store HPC Integration Final Results Store Central Market Data Store (~1 TB Tick Data) AppFabric Caching Market Data Results Final AppFabric Caching Intermediate Store Market Data Scratch Job Input Rollup Operation Split Method Keys Calculation Operation Scratch GridDynamics testing on CTP1 50 times improvement in some cases

Outline The Problem AppFabric Caching – The Solution Understanding Cache Data Overview and Programming AppFabric Caching Deployment and Maintenance AppFabric Caching Futures Conclusions

AppFabric Caching Vision – Cache for all your data Web Scenarios Distributed/Global object cache Low latency access High scale Availability Cache for reference and activity data Scale IIS/ASP.Net applications Enterprise / HPC Scenarios Persistence LINQ enabled cache Integrate with HPC server Heterogeneous client support Co-locate compute and data Software + Services Scenarios Application Cache for Windows Azure More data services BI, Streaming, Reporting OData (REST API)

Additional Resources Related Talks Hands On Labs On the Web PDC09-FT25 Microsoft Application Server Technologies: Present and Future PDC09-FT18 Microsoft ASP.NET Futures Hands On Labs Introduction to Microsoft Project Code Name “Velocity” On the Web Windows Server AppFabric Developer Center http://msdn.microsoft.com/appfabric

YOUR FEEDBACK IS IMPORTANT TO US! Please fill out session evaluation forms online at MicrosoftPDC.com

channel9.msdn.com/learn 11/28/2018 12:08 AM Learn More On Channel 9 Expand your PDC experience through Channel 9 Explore videos, hands-on labs, sample code and demos through the new Channel 9 training courses channel9.msdn.com/learn Built by Developers for Developers…. © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11/28/2018 12:08 AM © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11/28/2018 12:08 AM © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11/28/2018 12:08 AM © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.