Performance Metrics CPS 181s April 10, 2003

Should Performance Be Measured?

MetricsMetrics  Assessing the progress and health of on-line businesses  Financial progress: sales, margins, profit, market share  Strategic metrics require entire examination of the company: perception of value proposition, marketplace offering, effectiveness of implementation  Branding and implementation metrics focus on >supply-chain performance >organizational dynamics >marketing communication effectiveness – including branding  Senior and stock market analysts tend to have a bias toward financial metrics  provide history of the company in the marketplace  do not reflect the strategy of the company  Managers need early warning system – metrics

Should Senior Managers be Concerned About Metrics?  Organization metrics represent performance targets of the company  Percent increase in revenue  Percent increase in unique visitors  Length of time visitors remain on site  Cost of customer acquisition  Metrics valuable as employee reward or taking strategic action  Change processes  Change strategy  Change product strategies

Metrics Drive Behavior in a Number of Ways  Help define the business model  Concrete goals with precise measurements  Focusing on measurement can increase the precision of the value proposition  Help communicate strategy  Document performance targets  Communicates strategy to workforce  Creates buy-in to the metric-setting processes  Help track performance  Instantaneous feedback concerning site performance real-time  Usage, visitors, length of time on site, average sales, page views

Metrics Drive Behavior in a Number of Ways...  Help increase accountability  Metrics must be linked to the performance appraisal system  Individual performance can be appraisals tied to companywide, team-specific, and individual metrics  Site usability can be tied to the interactive design function  Customer service metrics can be tied to the customer service department  Help Align Objectives  Align precise objectives, departmental functional goals, companywide strategic activities as a whole

Current Challenges to Specifying Metrics for Online Businesses  Companies Strategies Change Rapidly – Internet time requires rapid change  Measurement is resource intensive  Setup and maintenance of capital intensive/human resource intensive systems and procedures  Data capture, data mining, and information use require time and commitment of senior executives  Online measurement systems are vulnerable – easy to manipulate final results

Current Challenges to Specifying Metrics for Online Businesses...  Soft metrics are not valued by the investment community  Investment community looks at revenues, margins, number of visitors, length of time on site, customer acquisition costs  But, ease of customer use, best value perception, and other market research measures are frequently early warning indicators of site performance  Meaningful metrics change on Internet time – often as quickly as the time it takes to learn best how to interpret the retained original data

What Should We Measure?

How Can We Assess the Health of eCommerce Firms?...  The Balanced Scorecard  Introduced by Kaplan and Norton  Finance balanced by other domains of the business  Includes internal business processes and customer responses  Four categories >Financial >Customer >Internal business systems >Learning and growth

Kaplan-Norton Balanced Scorecard FINANCIAL To succeed financially, how should we appear to our shareholders? INTERNAL BUSINESS PROCESS To satisfy our shareholders and customers, at which businesses must we excel? CUSTOMER To achieve our vision, how should we appear to our customers? LEARNING AND GROWTH To achieve our vision, how will we sustain our ability to change and improve? Vision and Strategy Source: Kaplan, Robert S. and David P. Norton Using the Balanced Scorecard as a Strategic Management System. Harvard Business Review 74 (January-February): 76.

K-N Financial and Customer Metrics  Financial Metrics  Revenue, revenue growth, gross margins, operating income, net margin, earnings per share, cash flow  Reflect recent planning period, to some degree all previous planning periods  Customer Metrics  Management relationship of customers by the firm  Market share  Customer acquisition  Customer satisfaction  Customer profitability  Customized by the target segment

K-N Internal Business Process Metrics  Innovation includes customer’s perception of innovativeness of the company  Operations – quality of entire supply  Postsale service – quality being offered to customer

K-N Learning and Growth Metrics  Employee  Selection  Training  Retention  Satisfaction  Information Systems  Quality of infrastructure  Timeliness, accuracy, and utility of data  Motivation  Empowerment  Alignment of company and personal goals

Limitations to the K-N Balanced Scorecard  No clear definition of strategy or business models  Unclear location of organizational capabilities or resources in framework  Unclear where partnerships reside in framework

The Rayport-Jaworski Performance Dashboard and Strategy Framework  The Performance Dashboard - intended to reflect the health of the business  The Strategy Framework Drives the Necessary Metrics  Six critical steps >opportunity assessment >business-model metrics >customer-interface design >market communications and branding >implementations >financial

Introducing the R-J Performance Dashboard Financial Metrics Financial Metrics Steps of Strategy: Customer Interface Implementation and Branding Performance Dashboard: Areas Addressed: Size of market opportunity Attractiveness of target segments Competitive landscape Customer acquisition Changes in customer behavior Site experience and usability Branding Technology infrastructure Internal organization Relationship with suppliers and partners Fulfillment capability Revenue Profit Cost Balance sheet Market Opportunity Assessment Market Opportunity Assessment Business Model Implementation & Branding Metrics Implementation & Branding Metrics Customer Interface & Outcome Metrics Customer Interface & Outcome Metrics Business Model Metrics Business Model Metrics Opportunity Metrics Opportunity Metrics Uniqueness of value proposition Attractiveness of offering Firm capabilities relative to competition Sustainability of competitive position Evaluation

Components of the R-J Performance Dashboard  Opportunity Metrics  Ability to discern unfulfilled needs in the market place is critical  Market-opportunity metrics asses the degree to which the firm can accurately gauge the market opportunity  Business-model Metrics  Capture subcomponents of the business model: egg diagram, resource systems, financial metrics  Value Proposition or Cluster Benefits Metrics >target segments, benefits offered, capabiliti4s that drive benefits >focus on customer’s perceptions >performance relative to the competitor’s benefits

R-J Branding and Implementation Metrics  Focus on the supply-chain performance, organizational dynamics, marketing communication effectiveness (including branding)  Fulfillment of the brand process metrics: customer awareness of the brand  Innovation metrics: ability to rapidly innovate, even if in a potentially discontinuous fashion

R-J Customer-Interface Metrics  Customer-interface and customer-outcome metrics  Customer’s perception of the firm’s performance on each level of the 7 C’s of the customer interface  Example: customer’s rating of customization compared to competitors’  Other critical interface metrics  Including value proposition

R-J Customer-Outcome Metrics  Capture process measures that the firm will produce favorable customer responses such as satisfaction and loyalty  Focus on subjective customer-outcome metrics  satisfaction  length of stay at site (stickiness)  overall evaluation of the experience  Focus on objective customer-outcome metrics  customer acquisition costs  average order size  customer profitability  number of visits per month

Financial Metrics  Revenues  Costs  Profits  Balance–sheet metrics Lise Buyer

Life Cycle of a Company STARTUP/ BETA STARTUP/ BETA CUSTOMER ACQUISITION CUSTOMER ACQUISITION MONETIZATION MATURITY Develop a platform for rapid growth by building a strong team and creating a flexible site. Build market share as quickly as possible by aggressively spending on partnerships and promotion. Increase revenues and customer lock-in by developing new revenue streams. Control costs and optimize marketing expenditures to achieve profitable growth. 6 Months- 1 Year1 Year - 2 Years2 Years - 5 years> 5 Years Strategy Business.com TNBT.com mValue.com Paypal.com Evite.com Yahoo.com Amazon.com Schwab AOL Examples* * At time of writing (3/2000)

What are the Steps to Implement the R-J Performance Dashboard?  Step One: articulate business strategy  market opportunity assessment  business model  customer-interface design  branding  implementation  evaluation

Steps Two and Three  Step Two: translate strategy into desired outcomes  not settling target levels but specifying outcomes  Step Three: devise metrics  identifies specific metrics that reflect the desired outcomes  look-to-book ratios

Bill Gurley on the Power of Conversion Rates Source: Gurley, J. William The Most Powerful Metric of All. CNET News.com, 21 February. URL:

Steps Four and Five  Step Four: link metrics to leading and lagging indicators  map entire set of indicators  including leading & lagging indicators  Step Five: calculate current and target performance  current conversion rates  advertising expenditures  degree of positive word of mouth

Blueprint to the R-J Performance Dashboard Step One: Articulate Business Strategy Step Two: Translate Strategy Into Desired Outcomes Step Three: Devise Metrics Step Four: Link Metrics To Leading And Lagging Indicators Step Five: Calculate Current And Target Performance Define goals and value proposition Market Opportunity Opportunity size? Competitive environment? Market Opportunity Opportunity size? Competitive environment? Customer How to acquire customers? How will customers change? The customer experience? Customer How to acquire customers? How will customers change? The customer experience? Implementation and Branding How to develop brand? How to go to market? Implementation and Branding How to develop brand? How to go to market? Financial Financial consequences in terms of revenue, profit, cost and balance sheet? Financial Financial consequences in terms of revenue, profit, cost and balance sheet? Market Opportunity Market size and growth Average age and income Competitor concentration Market Opportunity Market size and growth Average age and income Competitor concentration Customer Market share Purchases / year Success rate Service requests / customer Customer Market share Purchases / year Success rate Service requests / customer Implementation Customer brand awareness System uptime percentage Number of IT staff % inaccurate orders Implementation Customer brand awareness System uptime percentage Number of IT staff % inaccurate orders Financial Revenue Profit Earnings per share Debt to Equity ratio Financial Revenue Profit Earnings per share Debt to Equity ratio For each metric, determine the metrics that it affects and that affect it Map the linked set of metrics, indicating leading and lagging indicators Ensure that there is a balance between leading and lagging indicators For each metric, determine the metrics that it affects and that affect it Map the linked set of metrics, indicating leading and lagging indicators Ensure that there is a balance between leading and lagging indicators Develop resource system required to deliver the strategy For each metric, calculate current level of performance Determine target level required to meet outcomes described in Step 2 Ensure that targets are consistent with each other For each metric, calculate current level of performance Determine target level required to meet outcomes described in Step 2 Ensure that targets are consistent with each other Business Model Unique value proposition? Capabilities vs. competition? Business Model Unique value proposition? Capabilities vs. competition? Business Model Customer perceived benefits Exclusive partnerships $ invested in technology vs competition Business Model Customer perceived benefits Exclusive partnerships $ invested in technology vs competition

What are the Three Sources of Metrics Information That Firms Can Use to Chart Their Progress?  Industry standard metrics  Benchmark values  On-line Information

Online Information  Market research – primary customer data  Analysts reports – blending market data  Financial information – income statement, balance sheet, statement of cash flow  Use of Bizrate – use of dimensions of service  Use of Forrester Reports – e-commerce reports – competitors, consumers, technology evolution in a segment of the industry  Use of Hoover’s Online – financial and market information for both online and offline companies

U.S. Top 25 Web & Digital Media Properties (March 2000) Source: Media Metrix ( March 31, 2000

Market Research Source - Bizrate.com

Analyst Source - Forrester

Financial Information Source - Hoovers Online

Which is Better? Online or Off-line Data Sources

Mapping Internet Research Onto The Performance Dashboard Market Implementation Customer Financial Market ResearchAnalyst Financial Information