TELLING YOUR PRODUCT STORY WITH METRICS Govind Ramanathan

- Marketing Vision Customer Impact Innovations Target Markets Competitive Advantage We communicate as a Product Manager Customers New Features Feature Improvements Performance Business Value Executive Team Product Vision Roadmap Revenue Resource Utilization Stakeholders Current Status Roadblocks Timeline Partner Issues Product Team Overall Vision Feature Priority Overall experience User story Roadmap

We run as fast as possible Power Point Slides Spreadsheets Market Requirement Document Feature Specifications Timeline Backlog User Stories Go to Market Strategy Vision Mission Roadmaps Kanban Scrum Waterfall

Think Words to Numbers With Documents Vision Build a platform that creates quality leads to our customers at low cost Status The Feature to screen quality leads is 50% done The APIs needed to measure conversion is defined With Metrics Vision Increase conversions to 30% Increase quality leads to 55% Reduce cost per acquisition to 20% Status Conversions are at 15% Quality of leads increased to 40% Cost per acquisition is 45%

Metrics It is not… Replacement to a Vision statement Metrics are derived from your vision statement. It helps to enforce the vision. Replacement to Scrum or the engineering progress Replacement to Roadmaps It is… Language to communicate progress accurately Goal for the teams to focus on Improve it and feel good about the work Method to filter the vision into numbers Means to force us to think about actual benefit/value we are creating To measure and communicate success pre and post product launch Tool to reduce chatter on details and focus on the bigger picture

Types of Metrics Feature Metrics For large feature measure the progress Example Product: Inventory management system. Own fulfilling orders and replenishing inventory. Metric: Average time to fulfill order is less than or equal to 2 days Orders fulfilled greater than 2 days is less than 10% Buffer inventory <= 30%

Types of Metrics Product Metrics Directly measures product improvements Measures the product in isolation Example Product: Sell Textbooks Online Metric: Decrease online ordering time to less than 2 minutes Reduce shipping average shipping cost to less than $3

Types of Metrics Company Metrics Measures the entire suite Reports on the improvement of the eco-system Forces compromises on individual products for the overall Product Suite Example Company: WeSellLotOfStuffOnline.com Metric: Increase Revenue by 30% to $30M per quarter Decrease Operating Cost by 10% to $20M per quarter Increase customer base by 50% to 5M customers

How to define Metrics? Identify the entity of value Identify the audience for the metric Avoid metrics that may be influenced by externalities Metrics should be impacted only by what it measures Look back at your vision Would improving this metric satisfy your vision If more than one metrics satisfies the vision then pick the most important You can have more than one but make sure they are mutually exclusive Look at the top-level metrics to determine if lower-level metrics are targeted reasonably

Things to watch out… Always monitor over optimizing for a metric Evaluate if counter metrics needs to be created Evaluate if new metrics are needed Define and evangelize how you obtain the data for your metrics Including pre and post processing Data is never perfect…but we can get close Resist the temptation to add revenue as a metric This is only useful at the company level. At the product level you lose focus Never count product bugs or support calls as a way to monitor product improvements

Continue measuring the product pre and post launch

THANK YOU In God we trust; all others must bring data. W. Edwards Deming