Optimizing MySQL To Let People Argue Jeremy Bingham, OSCON 2011

Working With What You’re Given, or, “The Legacy Tables”

Lots of things that work on a small scale fall apart with millions of pages a day.

Working With What You’re Given, or, “The Legacy Tables” Lots of things that work on a small scale fall apart with millions of pages a day. Just identify the problems!

Working With What You’re Given, or, “The Legacy Tables” Lots of things that work on a small scale fall apart with millions of pages a day. Just identify the problems! Simple, right?

Working With What You’re Given, or, “The Legacy Tables” Lots of things that work on a small scale fall apart with millions of pages a day. Just identify the problems! Simple, right? Improvements possible, but you do have to look back.

Real Tables, Real Problems

Look for bad indexes. –Stories used a 20 byte VARCHAR as a primary key. –Comments used a 20 byte VARCHAR + an integer as a primary key. –Converted both to use integers. –Had to keep the old indexes around, of course.

Real Tables, Real Problems This will sound really dumb…

Real Tables, Real Problems This will sound really dumb… … check the indexes are actually there. For some reason, no one ever bothered to add an index on nicknames on the user table.

The Three Tools I’ve Used Most

EXPLAIN Probably the single most useful tool at your disposal

Slow Query Log If something’s in there a bunch of times, it’s probably a sign.

SHOW PROCESSLIST; And its brother, SHOW FULL PROCESSLIST; Can be deceptive; sometimes what seem to be problem queries are actually fine, but are binding up because of some other query. EXPLAIN helps here.

Upgrade?

Pros of Upgrading New features Get beyond limitations of previous versions –We were held back by a lot of MySQL 3.23isms

Cons of Upgrading

Something can go horribly wrong.

Cons of Upgrading Something can go horribly wrong. Like your data corrupting itself randomly after the upgrade.

Configuration Let us configure our servers in accordance with the MySQL lifestyle.

Configuration tricks we’ve used Turn off XA transactions innodb_flush_log_at_trx_commit = 0 Per table InnoDB files Large Pages Work those buffer pool settings according to your procs and RAM. Actually read the docs, and don’t just go off the ancient sample configs. They leave out a lot of options, or leave in deprecated ones. They’re in Sumerian anyway.

Flog the Hardware

2008 –Master/slave 16GB RAM 8 core with dedicated /tmp, 6x73GB RAID 10, tuned XFS 2010 –Master/slave 32GB RAM, quad core, SSD

Election Season Or, my Christmas comes every two years, but lasts between six and ten months.

Each cycle builds on the last, but is special and unique

Taking Opportunities Redesigning everything and trying to get it right, but still making sure old stuff works.

Using What I’ve Learned Cracking tables apart Judicious partitioning –Where applicable, anyway –You’ll need lots of experimentation and thought with MySQL’s partitioning Ferret out poorly thought out primary keys

Using What I’ve Learned Trim out truly unneeded columns and tables Move less used columns off to the side Better foreign keys and join tables Stat tables, to avoid certain painful queries Lots of testing with a full copy of the data

The Most Important Thing Cache, cache, cache! Cache pages to disk –Great in 2006, blew up horribly during the Iowa caucuses in 2008 Started storing pages in memcached for anonymous users in 2008

More Caching Cache fragments and collections for logged in folks too. Particularly intense queries can be run from a cron so they’re always available. Slower updates can be kicked to a background daemon.

Caching’s Terrible Downside

Never underestimate how much people will gripe if something’s not exactly up to date. Somehow, they’ll notice.

What in God’s name was I thinking?

Jeremy Bingham Photo Credits: – Siegfried, public Domain – PDP-10, Michael L. Umbricht – Stone axe, Didier Descouens – Hammer, Evan-Amos – Adze, Luigi Zanasi – Teamsters strike in 1934, US Government work

Questions?