0 Optimization Success Story: Match.com’s Cultural Change Jim McDonald, Sr. Online Marketing Manager, Match.com

1 Overview  Match.com’s Culture Change  Told from a Relationship /Dating Angle

2 Why Relationships Grow Stale  Frequency Down  A/B Testing once every two, three weeks – sometimes not for months!  Friends & Colleagues bragging about doing it all the time!

3 Why Relationships Grow Stale  Frequency Down  Staying Home Up  Use internal A/B testing platform, data & analytics  Bored with routine  Results never seem to change  Want more Scientific Approach  Anecdotes and gut feeling are often used  Rank determines test  Don’t want to be seen in public  Never have anything new or worthwhile to share

4 Why Relationships Grow Stale  Frequency Down  Staying Home Up  Playing Darts Up  Throw darts to determine next A/B versions to test

5 Why Relationships Grow Stale  Frequency Down  Staying Home Up  Playing Darts Up  Complacency Up  People do Not like Change!  People know what to expect with the Status Quo

6  Things Getting Old  A/B tests take time to test and iterate  Seems like you’re spinning your wheels  Takes forever to commit Why Think about Change

7  Things Getting Old  New Person Enters Picture  Someone catches your eye  You learn from someone else  New Colleague, Director who as used MV testing  Someone questions the status quo  You catch someone else’s eye  Someone else learns from you  You bring a new paradigm to your company/department  You questions the Status Quo

8  Things Getting Old  New Person Enters Picture  Lost Trust  Data behaving strangely  Not totally confident in results Why Think about Change

9 What are the Benefits of Meeting Someone New?  Brings Back Passion  Tests are Easier  Tests are Quicker  Results Baby!  You want to be seen

10 What are the Benefits of Meeting Someone New?  Brings Back Passion  Brings Back Trust  No more excuses  You don’t question the results  You know what is affecting change  Sector Variable reporting  No more throwing darts  No more design or test by committee

11 Why Break Ties with Your Old Relationship?  You’ve Seen the Light  Like what you see  Everything makes sense  Everything seems certain and conclusive  Sometimes the grass IS greener  Sometimes you need an independent sound-check

12 Why Break Ties with Your Old Relationship?  You’ve Seen the Light  You have More Self Esteem  Know you can do better  Good is not good enough where better is expected

13 Why Break Ties with Your Old Relationship?  You’ve Seen the Light  You have More Self Esteem  The Data Proves Itself  Gives you definition  You know specifically what works and what doesn’t  Gives you direction  no more darts, no more design by committee  No more unanswered questions

14 What Happens in the New Relationship?  You Do it all the Time Now  MV Testing is Iterative  MV Testing is Scalable  Touches every facet of an online product  Not just going through the motions  MV Testing produces results!

15 What Happens in the New Relationship?  You Do it all the Time Now  You Want to Be Seen  You Look REALLY Good!  Results speak for themselves  You ARE Affecting Change!  You Want People to Talk to You  You want people to ask questions  Real knowledge is power

16 What Happens in the New Relationship?  You Do it all the Time Now  You Want to Be Seen  Become an Ambassador  Share your Knowledge  Communicate Benefits with Other Departments  Tell them it’s FREE for them  Insure a Higher Power is Involved  Your Boss, Your Bosses Boss…

17 What Happens in the New Relationship?  Doing it all the time Now  You Want to Be Seen  Become an Ambassador  Share the Benefits  SPEED!  DIRECTION!  Reporting  Data is easily understood  Simplicity  My marketing assistant and I implement all code (three snippets)  Optimost generates code for us and sets up the campaign

18 Testing Methodology – How We Test  Two Streams  Big Idea Stream  Looking for large incremental lifts  Do A/B Testing  Promote if beats Control (current best performer) to significance  M/V Stream  Test derivations of best-performer  Constantly promote best performer at significance

19 Testing Methodology - Testing Example

20 Testing Methodology - Results  Within First Month  10% lift in subscriptions  One minor tweak on landing page  Recently  Another 10% incremental lift in subscriptions  Another minor tweak  Approximately 3% additional lift from other combinations  10% reduction in marketing costs

21 Final Thoughts  MV Testing = Easy  Much easier than going through a development prop cycles  Gives you control over iterative changes  MV Testing = Inexpensive  What does a 1, 2, 5, 10% incremental lift with add to your revenue  What does developer and analyst time cost with traditional internal A/B systems  Monthly fee is relatively nominal – worth testing  MV Testing = FUN!  Optimost makes my job fun  Like playing a game and getting paid for it  Multivariate Testing = Results  Subtle results all add up  Dramatic results will be uncovered  You WILL be surprised with the results

22 Questions?  Jim McDonald  Sr. Online Marketing Mgr.  Match.com   “A website that can’t be found is useless… …A website that can’t be used is pointless”