Why Assumptions about Cloud Performance Can Be Dangerous to Your Business Ron Wilson, Director Cloud Strategy

Agenda Why Does Performance Matter? Brief Overview of Web and Cloud Performance Challenges Real-World Data: How Are Cloud Providers Performing? Cloud Optimization Opportunities Key Takeaways Q&A

Why Performance Matters: Revenue Source: Steve Velocity Conference millisecond delay 0.59 % fewer searches/users* 50 % more pages/visits than users experiencing the slowest page load times* Noticed that users who experience the fastest page load times view stated that a 2 second slowdown 4.3 reduction in revenue/user* % found that a reduced page load times from ~7 seconds to ~2 seconds, leading to aincrease in revenue and 7–12 % reduction in hardware costs* 50 %

Why Web Performance Matters: Customer Satisfaction Consumer expectations for how quickly a web page should load 5% 12% 30% 36% 17% How long consumers will wait for a page to load before abandoning 1% 2% 10% 27% 60% 40% of Consumers will abandon a site if made to wait Lost revenues Increased costs Reduced customer satisfaction LOB dissatisfaction with IT “eCommerce Web Site Performance Today” white paper August 2009

Why Performance Matters: Cloud Adoption

Typical Web Application Delivery The Web Application Delivery Chain Major ISP Local ISP Mobile Carrier Internet Content Delivery Networks 3rd Party/ Cloud Services Browsers and devices Users Storage Web Servers App Servers DB Servers Mainframe Load Balancers Mobile Components Network Traditional zone of control

Major ISP Local ISP Mobile Carrier Internet Content Delivery Networks 3rd Party/ Cloud Services Browsers and devices Users Storage Web Servers App Servers DB Servers Mainframe Load Balancers Mobile Components Network The Web Application Delivery Chain Network peering problems Outages Network peering problems Outages Inconsistent geo performance Bad performance under load Blocking content delivery Incorrect geo-targeted content Inconsistent geo performance Bad performance under load Blocking content delivery Incorrect geo-targeted content Configuration issues Oversubscribed POP Poor routing optimization Low cache hit rate Configuration issues Oversubscribed POP Poor routing optimization Low cache hit rate Network peering problems Bandwidth throttling Inconsistent connectivity Network peering problems Bandwidth throttling Inconsistent connectivity Configuration errors Application design issues Code defects Insufficient infrastructure Configuration errors Application design issues Code defects Insufficient infrastructure Poorly performing JavaScript Browser/device incompatibility Page size too big Too many objects Low cache hit rate Poorly performing JavaScript Browser/device incompatibility Page size too big Too many objects Low cache hit rate Network resource shortage Faulty content transcoding SMS routing / latency issues Network resource shortage Faulty content transcoding SMS routing / latency issues The Challenge of Ensuring Quality Web Experiences Zone of customer expectation Systems management tools: “OK” …user is NOT happy Zone of customer expectation Traditional zone of control

Major ISP Local ISP Mobile Carrier Internet Content Delivery Networks 3rd Party/ Cloud Services Browsers and devices Users Storage Web Servers App Servers DB Servers Mainframe Load Balancers Mobile Components Network The Web Application Delivery Chain The Business Impact of Poor Web Experiences Over investment in infrastructure Lost revenue Brand damage Dissatisfied customers Increased call center volume Increased costs Lost revenue Brand damage Dissatisfied customers Increased call center volume Increased costs Over spending on CDNs Ineffective SLA’s and partner relationships Systems management tools: “OK” … user is NOT happy Zone of customer expectation Traditional zone of control

Major ISP Local ISP Mobile Carrier Internet Content Delivery Networks 3rd Party/ Cloud Services Browsers and devices Storage Web Servers App Servers DB Servers Mainframe Load Balancers Mobile Components Network Moving Web Applications to the Cloud: Benefits The Web Application Delivery Chain Users Traditional zone of control  Elastic and scalable  Focus on my business while someone manages infrastructure Cloud

Major ISP Local ISP Mobile Carrier Internet Content Delivery Networks 3rd Party/ Cloud Services Browsers and devices Users Storage Web Servers App Servers DB Servers Mainframe Load Balancers Mobile Components Network The Problem: The Cloud Creates Performance Concerns The Web Application Delivery Chain Traditional zone of control Cloud Cloud is opaque Loss of visibility and control Traditional tools don’t apply Cloud is opaque Loss of visibility and control Traditional tools don’t apply

The Answer: Adopt an “Outside-In” User Point of View The Web Application Delivery Chain Users Major ISP Local ISP Mobile Carrier Internet Content Delivery Networks 3rd Party/ Cloud Services Browsers and devices Storage Web Servers App Servers DB Servers Mainframe Load Balancers Mobile Components Network Users Traditional zone of control  Full understanding of performance from user perspective Test/monitor your site the way your customers use it: What they do (key pages and transactions) Where they do it (geographic locations) How they do it (browsers and mobile devices) When they do it (normal and peak usage) Determine the impact on their behavior and your business Gomez Platform Web Performance Management Web Performance Business Analysis Web Performance Business Analysis Web Load and Performance Testing Web Cross- Browser Testing “Outside-in” customer point of view

Major ISP Local ISP Mobile Carrier Internet Content Delivery Networks 3rd Party/ Cloud Services Browsers and devices Storage Web Servers App Servers DB Servers Mainframe Load Balancers Mobile Components Network My users Other users The Problem: The Cloud Creates Performance Concerns The Web Application Delivery Chain Cloud is shared Others can affect my performance Cloud is shared Others can affect my performance My app Other app Other app Other app

Major ISP Local ISP Mobile Carrier Internet Content Delivery Networks 3rd Party/ Cloud Services Browsers and devices Storage Web Servers App Servers DB Servers Mainframe Load Balancers Mobile Components Network My users Other users The Answer: Collective Intelligence The Web Application Delivery Chain Cloud is shared  Multiple contributors help diagnose issues for everyone My app Other app Other app Other app “Outside-in” customer point of view

Cloud Apps: You Must Be Able to Pinpoint Problems The Web Application Delivery Chain Major ISP Local ISP Mobile Carrier Internet Content Delivery Networks 3rd Party/ Cloud Services Browsers and devices Storage Web Servers App Servers DB Servers Mainframe Load Balancers Mobile Components Network My users Other users

A Year In The Cloud An End-user Perspective on Cloud Performance

Introducing… CloudSleuth.net

What We Measured Cloned reference Web application deployed across various IaaS and PaaS providers Added various services such as a Content Delivery Network, when available Structured application to highlight Cloud Performance issues

How We Measured Various locations around the world Backbone and Last Mile locations Every 15 minutes, 24/7, for over 1 year Used a strict definition of availability

Not All Clouds Perform the Same Way Average Response Time of Reference Application, as measured from US backbone locations

Taking the Long View – Response Time © 2010 Gomez – All Rights Reserved

Taking the Long View - Availability

Geographic Latency – GoGrid

Geographic Latency – S3

Going International?

Amazon EC2 Europe West (Dublin) Performance Response time for sample transaction of reference application hosted on Amazon EC2 Europe West (Dublin), as measured from major European cities

It’s Not All About Network Latency

Enough Scary News… … let’s see something good for a change

Geographic Latency – S3

Added Services Help… A Lot!

Unique Opportunities for Optimization Better performance, for little work, at no extra cost?

Best Practice: Define Your Goals and Build a Plan Align goals across your organization Why are we moving to the cloud? Common goals include: 1.Additional Capacity – How much capacity do we need during normal and peak times? 2.Improved End-User Experience – What performance goals are we trying to deliver against? 3.Greater Elasticity – How quickly can the provider we select ramp up to meet our needs? 4.Flexible Bursting – How fast do we need to be able to access additional capacity? If only there was a button to push!

Best Practice: Keep Your End-users in Mind Test your cloud applications the same way your customers use it: What they do?  Customers care about completing tasks NOT whether the homepage is available Where they do it from?  Your customers don’t live in the cloud, test from their perspective across multiple devices and browsers When they do it?  Test at peak and normal traffic levels, to find all the problems What expectations do customers have?  Is 4 seconds fast enough or does it have to be quicker? Geographic disparities 4 sec’s 22 sec’s

Best Practice: Performance Testing Cloud Capabilities Evaluate vendors based on your goals… Capacity Test vendors to 15-20% past estimated capacity goals Elasticity Baseline end-user performance before & after testing Test during pre-deployment and in production Ramp elasticity testing to peak levels Burstability Isolate the cloud elements from other infrastructure to test Test the “failover process”

Best Practice: Set SLAs to Match Your Goals Set SLAs based on your goals… End-user availability and response times Capacity & elasticity objectives Burstability goals Set SLAs based on how you are using the cloud… SaaSPaaSIaaS Transactional Success Rate Fast end-user experience across key geographies Availability and performance of key components, services, and API’s Hardware & OS availability / performance Network connectivity Capacity and elasticity requirements

Putting Together a Cloud Management Strategy Key Steps Vendor Evaluation Deployment Testing Production Monitoring Service Level Management Validate the burstability & elasticity of cloud solutions Identify application performance bottlenecks Measure performance under normal and peak conditions Baseline and monitor service level objectives Measure how end-user performance impacts behavior Monitor performance & availability from the end-users’ perspective

Questions? THANK YOU