1. Revolutionizing enterprise web development Memcache Optimization

2. Agenda What is memcache and what components do you need? Why use it and how do you know it’s working? Limitations and configuration options Basic troubleshooting Common configuration strategies Tips

3. What is memcache? “Memcached is an in-memory key-value store for small chunks of arbitrary data” A system to cache data in RAM rather than in a database. Replaces Drupal’s default database cache.

4. Who Uses It? Facebook YouTube Craigslist Twitter Flicr LiveJournal Wikipedia Bebo Wordpress.com Digg Typepad Yellowbot Many others… Sources: http://memcached.org and http://lineofthought.com/tools/memcached

5. How does memcache.d fit into an overall performance architecture? Only 1 component of the architecture Compliments other tools such as Varnish and APC.

6. Why use memcache? Helps improve the scalability of your site. Reduces request execution time on the web server. Reduces database load. Allows more distribution of resources and processing. In some use cases, can improve page response time* * This is very application specific, for most sites the improvement will be so small that most users will not be able to tell the difference. (e.g. 0.1 second improvement on stock Drupal install)

7. Stats Performance and Scalability Cache read performance: 2x faster Cache write performance: 40x faster Reduces number of database queries by up to 50% Actual statistics* (Drupal core vs. Memcache) Cache Read Operations: 3.5 sec vs. 1.5 sec Cache Write Operations: 100 sec vs 2.5 sec DB Queries: 70 vs. 33. * Cache Read/Write stats are for performing 10,000 operations ** Your results will vary, tests performed on a single dev VM with default D7 install.

8. Components of a memcache system Memcache.d Daemon (v 1.4.x) PHP Extension PECL Memcache (> v 2.2.1) Use version 3.0.5 or later PECL Memcache.d (> 2.x) TIP: Use the setting “memcache_extension” to select which library to use if you have both installed. Memcache Module Drupal 7.9+, Pressflow 6, or Drupal 6 with core patch

9. PECL Memcache vs. Memcached PECL MemcachePECL Memcached Stable, not as frequently updated Newer, more features Easy to installMore complicated install with multiple dependencies Slightly faster in some cases igbinary support

10. What is igbinary? Alternate to built-in php serialize/unserialize functions. Higher performance and lower memory utilization in many cases. Binary serialization rather than string. Requires special php.ini configuration to fully take advantage of. Requires special calls to use. Resources: http://codepoets.co.uk/2011/php-serialization-igbinary/ https://github.com/igbinary/igbinary/ http://drupal.org/node/839444

11. Installing Memcache.d On Debian based systems: Sudo apt-get install memcached Sudo apt-get install php-pear For PECL Memcache Sudo pecl install memcache-beta For PECL Memcached Sudo apt-get install libmemcached libmemcached-dev pcre3 pcre3-dev (optional) Sudo pecl install igbinary Be sure to do the config steps! Sudo pecl install memcached

12. How do you know it’s working? Memcache Statistics (Memcache Admin Module) Enable “Show memcache statistics at bottom of each page” You should see at least 10 – 20 cache operations listed on several bins including: “cache”, “cache_menu”, and “cache_bootstrap”. Refresh the page a couple times, you should see a hit number greater than 0 on almost every line and almost no cache set operations.

13. How do you know it’s working? Devel Module Enable the option “Display query log” sorted by source. Should see no queries hitting the main Drupal cache tables (e.g. “cache”, “cache_bootstrap”, “cache_menu”, etc.)

14. Diagnostic/Troubleshooting Techniques Use Telnet to directly query cache instances Useful commands: Stats – displays general stats Stats slabs – displays per slab stats Stats items – displays per item stats Flush_all – Deletes everything in the cache Get / set – Gets or sets a cache item Use the memcache stats on each page and Memcache Stats Report to see what’s happening Most of the same info as via telnet. TIP: You may get an error from the report if you don’t have at least one memcache bin defined.

15. Important Memcache Stats Cache Hit/Miss Ratio > 99% hits ideal Connections < 1024 (less than configured limit) Evictions == 0 Free/Used Memory > 20% free memory

16. Sizing Memcache.d Look at the cache_* tables in your database. Sum the total size of these tables (e.g. in phpmyadmin) Multiply by at least 2 and round up to the nearest multiple of 8MB. TIP: Try to keep instance size under 512MB. Example: Cache table size=30MB, x2 = 60MB. Round up to 64MB.

17. Settings.php Options: cache_backends cache_default_class cache_class_form (cache_class_ ) Drupal 7 only session_inc (D6 only for now) memcache_servers memcache_bins memcache_key_prefix memcache_options Configuration options

18. Recommended Memcached Options (in settings.php): $conf['memcache_options'] = array( Memcached::OPT_COMPRESSION => FALSE, Memcached::OPT_DISTRIBUTION => Memcached::DISTRIBUTION_CONSISTENT, Memcached::OPT_BINARY_PROTOCOL => TRUE, ); For a single memcache.d instance, this option is slightly faster: Memcached::OPT_DISTRIBUTION => Memcached::DISTRIBUTION_MODULA, NOTE: These settings are only valid if using PECL Memcached

19. Limitations Max key size: 250 bytes Can use larger keys, but pay a hashing penalty Max data size: 1 MB Has no redundancy or locking * Most of these limitations can be overcome by custom compiling the Memcache.d libraries or adding other extensions to Memcache.d.

20. Common configuration strategies 3 Common Strategies: Basic configuration Basic distributed configuration Multiple bins distributed configuration

21. Basic Configuration Good for: Sites that run on 1-2 web servers. Mostly read-only content or infrequently changing content.

22. $conf['cache_backends'][] = 'sites/all/modules/memcache/memcache.inc'; $conf['cache_default_class'] = 'MemCacheDrupal'; $conf['cache_class_form'] = 'DrupalDatabaseCache'; $conf['memcache_servers'] = array(‘mysql.localdomain:11211' => 'default',); $conf['memcache_key_prefix'] = ‘mysitename'; $conf['memcache_options'] = array( Memcached::OPT_COMPRESSION => FALSE, Memcached::OPT_DISTRIBUTION => Memcached::DISTRIBUTION_MODULA, Memcached::OPT_BINARY_PROTOCOL => TRUE, ); Basic Configuration: settings.php

23. Simple distributed configuration Good for: Sites running on 2 or more web servers. Sites that are a mix of read-only and regularly changing content, but mostly read-only.

24. $conf['cache_backends'][] = 'sites/all/modules/memcache/memcache.inc'; $conf['cache_default_class'] = 'MemCacheDrupal'; $conf['cache_class_form'] = 'DrupalDatabaseCache'; $conf['memcache_servers'] = array( ‘web1.localdomain:11211' => 'default', ‘web2.localdomain:11211’ => ‘default’, ); $conf['memcache_key_prefix'] = ‘mysitename'; $conf['memcache_options'] = array( Memcached::OPT_COMPRESSION => FALSE, Memcached::OPT_DISTRIBUTION => Memcached::DISTRIBUTION_CONSISTENT, Memcached::OPT_BINARY_PROTOCOL => TRUE, ); Simple distributed configuration: settings.php

25. Dedicated bins distributed configuration Good for: Site running on 3 or more web servers. Frequently added or changing content. Large amounts of cached content.

26. // Same settings as previous configs $conf['memcache_servers'] = array( ‘memcache1.localdomain:11211' => 'default', ‘memcache2.localdomain:11211’ => ‘default’, ‘memcache3.localdomain:11211' => ‘static', ); $conf[‘memcache_bins’] = array( ‘cache’ => ‘default’, ‘cache_bootstrap’ => ‘static’, ‘cache_menu’ => ‘static’, ‘cache_field’ => ‘static’, ‘cache_filter’ => ‘static’, ‘cache_path’ => ‘static’, ); // Same settings as previous configs Dedicated bins distributed configuration: settings.php

27. Other Tips Do not set “Minimum cache lifetime”. Do not set expiration more than 30 days. You do not need to enable the Memcache module or enable Drupal caching to use Memcache. When using multiple bins, make sure you explicitly define at least one cache table to each bin to avoid errors on the stats report. When splitting out multiple servers/bins, move things out in this order: Mostly read-only content (menus, filters, paths) Page cache High traffic cache tables (session, cache)

28. References Memcache.d: http://memcached.org/ PECL Memcache: http://pecl.php.net/package/memcache PECL Memcached: http://pecl.php.net/package/memcached Drupal Module: http://drupal.org/project/memcache Memcached.php: http://www.phpdeveloper.org/news/10247

29. References PECL Memcache vs. Memcached: http://code.google.com/p/memcached/wiki/P HPClientComparison http://brian.moonspot.net/php-memcached- issues Alternate Memcache.d Init script: http://www.bxtra.net/articles/2011-07- 27/memcached-multiple-instances-startup- script-initd-on-centos-56-64-bits

30. Thank You Shawn Smiley, Lead Engineer d.o: Shawn_Smiley e: shawn.smiley@achieveinternet.com