Concept evaluation and testing Innovation Management 2012 Stefan Wuyts

Agenda Concept evaluation Concept testing The full screen

Concept evaluation

The overall evaluation system

Element of cost: Cumulative expenditures curve Many high-tech products Many consumer products Time

Element of risk: Risk/Payoff matrix Cells AA and BB are “correct” decisions. Cells BA and AB are errors, but they have different cost and probability dimensions. Usually BA (the “go” error) is much more costly – but don’t forget opportunity costs! Consider how “new-to-the-world” the product is as that has an impact on the risk level

Real options logic as response to risk Source of real options approach: early discussions on exploration versus exploitation (Schumpeter 1934; March 1991) Exploitation: refinement, choice, efficiency, selection, implementation, execution  Clear results, short-term Exploration: search, variation, risk, experimenting, flexibility, discovery  More uncertainty regarding results, long-term Problem exploitation: suboptimal (stable) equilibria; Problems with exploration: insufficiently developed ideas, not reaping the benefits of experimentation (more search than application), no distinctive competence; Problems of both: they are self-reinforcing. Challenge: find the right balance. Real options logic: postpone the decision until uncertainty is reduced

Probability distribution Imagine two investment alternatives (A1 and A2), for which the probability of payoff (in $) is initially unknown. The decision maker can either: Gather extra information, postpone choice  improving future returns; Use current information, choose  improving current returns. Probability distribution A2 After further exploration in A2 (scenario 1) A2 After further exploration in A2 (scenario 2) Before further exploration in A2 A2 A1 Pay-off ($) -9 -6 -3 3 6 9

Real options logic summarized: Bet on different horses: option on >1 strategic alternatives; For now, invest in multiple ideas until there is more clarity as to their market potential; Investing in a real option provides the right, but not the obligation to further invest at a later point in time (or to ‘abandon’ the option at a later point in time). Appropriate if: Uncertainty regarding link between investments and results; Decisions today determine opportunities tomorrow (time dependence, path-dependence) Possibility for abandoning/executing the option. Applied increasingly in high-tech environments where these three conditions are oftentimes met (e.g. Philips)

Planning the evaluation system Keep in mind: Everything is tentative Financial analysis Marketing Avoid potholes Damaging problems FDA approval, customer skills, manufacturing cost You are dealing with people Example: in the NPD literature, it has been shown to be very difficult to pull the plug for unsuccessful projects. versus

Example of Potholes in new product development: Complexity IT IS ROCKET SCIENCE PHILIPS ADMITS WHAT EVERYONE KNOWS: DIGITAL GADGETS ARE WAY TOO COMPLICATED FOR THE AVERAGE CONSUMER (Newsweek 2004) NEWSWEEK: Is it true you ran a test, giving 100 of your top managers one weekend at home to get various Philips gadgets operating? KLEISTERLEE (CEO Philips): Yes, we did. And indeed, a number failed, returned frustrated and some even angry; another group that succeeded returned quite proud. It strengthened our conviction that we must start making things easier for consumers or we will never see the real promise of the digital revolution come to life. And we must do it now.

Example of Dealing with People: Hard to pull the plug. Why? One of the most-cited reasons: “Escalation of Commitment”, the tendency to continue investing in a strategy, despite negative feedback, continuation in a failing course of action. Solutions: How can we reduce escalation of commitment bias? (Boulding, Morgan & Staelin 1997) More information? Problem of information distortion Emphasize uncertainty or external causes of failure? Calculate Net Present Value of continuing to invest versus NPV of not continuing to invest? Establish a rule a priori that determines when to stop Sequential decision decoupling (people who decide on continuation ≠ people who are strongly involved) If at first you don’t succeed, try, try again. Then quit. No use being a damn fool about it. -- W.C. Fields

Quantifying evaluation: the A-T-A-R model Profits = Units Sold x Profit Per Unit Units Sold = Number of buying units x % Aware of product x % who would Try product if they can get it x % to whom product is Available x Repeat measure (average # units bought per person per year x # units repeaters buy in a year) Profit Per Unit = Revenue per unit - cost per unit

Concept testing

Why concept testing? We are not starting from scratch: We already had a Product Innovation Charter (PIC) approved as a guide to select ideas; We already have gained first insight into the market area singled out by the PIC; Concepts always undergo some initial reaction by management (based on heuristics or method like A-T-A-R). How should we proceed? Concept testing. Concept testing takes evaluation one step further, but it precedes technical work (not the same as prototype testing). It is intended to (1) remove poor concepts, (2) get first idea of purchasing likelihood, and (3) make the concept more concrete (attributes)

Concept testing is not feasible if… …prime benefit is personal sense (e.g. taste). …concept involves new art and entertainment. …concept embodies new technology that users cannot visualize or address needs customers can’t articulate yet (e.g. first microwave).

Concept testing requires clear concept statement (e. g Concept testing requires clear concept statement (e.g. new electric razor whose screen is so thin it can cut closer than any other electric razor on the market) Necessary steps include the selection of the concept test format; preliminary indication of price; specification of respondent group and response situation; preparation of interviews

Application Example Nestlé Refrigerated Foods (Contadina Pasta) 24% “definitely buy” + 51% probably “would buy” Adjusted trial, rule of thumb: 80% of the “definitely” + 30% of the “probably” will actually buy, or: (0.8 x 24%) + (0.3 x 51%) = 34.5% Assuming 48% awareness and 70% availability, we get : AW x T x AV = 0.48 x 34.5% x 0.70 = 11.6% Target households x trial rate = 77.4 million x 11.6% = 9 million Repeat for similar products = 39%; average customer repeat = 2.5 times; No. of units bought per repeat purchase occasion = 1.4 Additional sales because of repeats: 39% x 2.5 x 1.4 = 136.5% Hence, the “R” in the A-T-A-R model = 1 + (.39 x 2.5 x 1.4) = 2.365 A-T-A-R Sales prediction: 9 million x 2.365 = 21.285 million. Note that this deviates from the calculations in the book! Good exercise to identify and think about the differences in calculations.

The full screen

Toward more informed allocation decisions Refine and rank-order the concepts (on basis of mathematical models or checklists) Forces feasibility evaluation along technical and commercial dimensions, and summarizes what must be done. Cross-functional: involve major functions (marketing, technical, operations, finance), new products managers, staff specialists (IT, distribution, procurement, PR, HR)  Decide whether or not to allocate further resources to each concept

Example (simple) mathematical model: profile analysis Profile analysis: I = T*C*P/D I = index of attractiveness T = probability of successful technological development C = prob. of commercial success if technological success P = Likely profit D = development cost Keep it simple; Need for qualitative input (accuracy, error); Pay attention to deliberate underestimation of costs; Select several ideas => higher returns.

Example: checklist + scoring model Cooper’s NewProd study identified must-meet and should- meet factors across many product successes and failures Must-Meet Criteria (rated yes/no) Should-Meet Criteria (rated on scales) Strategic alignment Strategic (alignment and importance) Product advantage Product advantage (unique benefits, meets customer needs, provides value for money) Existence of market need Market attractiveness (size, growth rate) Likelihood of technical feasibility Technical feasibility (complexity, uncertainty) Return versus risk Return versus risk (NPV, ROI) Environmental health and safety policies Synergies (marketing, distribution, technical, manufacturing expertise) Show stoppers (“killer” variables)