Marketing & Sales Roundtable Beta Program Best Practices November 2004

©2004 Rosemary Remacle 2 11:30–Introductions/Objectives 11:45 - Beta Program Best Practices: Presentation and Discussion 1:15–Summary and Wrap-up Agenda

©2004 Rosemary Remacle 3 What Do We Mean: “Beta Test”? Testing of a working product performed by real users in a production environment Last test before formal product launch, production shipments begin (GA) A test of all elements and process required for successful GA product The Beta program definition will be driven by product category and type of users – HW, SW, Services

©2004 Rosemary Remacle 4 Objectives for a Beta Program Product Gain deeper understanding of how product works in a ‘real world’ environment Stress products more completely and in different ways than in a lab setting Fine-tune product/service functionality (tweak vs. major overhaul) Test configuration planning, installation and support procedures and documentation (including technology and delivery partners where applicable) Test specifications and metrics to ensure quality levels of the specifications in a range of environments (e.g. usability, reliability, performance, etc.) Marketing Test corporate and product positioning platform, message architecture and plans for customer segment programs Develop a systematic way for capturing customer information and input Build key early customer relationships, generate references and referrals Understand weaknesses in company infrastructure required to support all customers’ requirements (total product)

©2004 Rosemary Remacle 5 Beta Program Development Collect and analyze user feedback Identify, prioritize issues to be resolved Develop integrated functional response plan Refine beta testing process Summary beta report doc Define beta testing process, participants and objectives Produce integrated functional plan - Engineering - Marketing - Sales - Professional Services - Support - Partners Develop criteria for beta test customers Identify and Recruit beta customers and set expectations Create beta program documentation Implement beta program - Product Distributed to Beta Users - Survey Vehicles in place - Reporting Process - Timelines, Milestones

©2004 Rosemary Remacle 6 Sample Criteria for Selecting Beta Customers Criteria(Hi – Med – Lo) Member of identified market entry customer segment(s) Does their application test stress the aspects of the product that need it? Match of company’s solution to perceived customer critical need Motivated to make fast decision within market window/Beta program manager in place Early adopter technology adoption cycle profile: high tolerance for Beta problems Referenceability: Big name, technology leader and/or key influencer in market entry segment, willing to be reference Existing business relationships with key members of Beta company team, users Internal Champion Geographic proximity for ease of support Technology or market leader

©2004 Rosemary Remacle 7 Beta Program Best Practices: The Short List Don’t abuse the generosity of beta customers with non customer-worthy product – develop a “phased beta approach” to address risks of putting beta product in big influential prospective customer Choose and recruit your beta customers, users to match market entry customer segment(s) Drive for confidentiality (NDAs, Beta agreements, etc.) Manage the program and beta users, communicate (bi-directional) regularly (program managers in both your company and customers) Provide a simple, easy method for beta users to give feedback, make them feel like part of the your team Encourage beta users to communicate with each other Respond quickly to all requests and problems Document, document, document

©2004 Rosemary Remacle 8 What Motivates Beta Testers ? Early adopter technology adopter cycle value profile Ability to contribute to product development process, be perceived as expert ‘Community’ participation Exclusive access to new product/Head start on resolution of their business problem ‘Free’ product/Incentives Career advancement Where do You Find Them? Sales Force/Executive Team relationships Partners (Channel, complementary products, etc.) Internet groups/Blogs/Bulletin Boards and Lists “Dead prospects” Customers’ competitors Other? User Groups/SIGs

©2004 Rosemary Remacle 9 Elements of Beta Program Research Plan [Realistic] sampling strategy - # of customers - # of participants - Functional representation Efficient user feedback (technical and marketing) collection tools (on-line, web page) Third party tools (e.g. web surveys - Survey Monkey, Zoomerang) for simple, measurable inputs - In-depth, qualitative (e.g. telephone/in-person interviews) for exploration of issues/requirements and positioning strategy validation - Forum discussions (open and hosted: web forum, conference calls, WebEx) Surveys: design for structured analysis! - Topics to be covered by function - Technical and marketing metrics - Qualitative ‘discussion’ topics

©2004 Rosemary Remacle 10 Beta Test Plan Definition - Project parameters, goals, activities and participants - Internal resources required, responsibilities identified -Foundation for beta report/summary Table of Contents - Product definition (in current state) -Target customer segment(s) – “user spec” (time, HW, etc.) - Parameters (# of participants, timeline, etc.) - Goals (UI, support requirements, testimonials, etc.) - Incentives (as appropriate to situation) - Beta team action items and responsibilities - Etc.

©2004 Rosemary Remacle 11 Post-Beta Test Make customer feedback accessible, institutional customer intelligence (Analyze customer feedback in structured way and document all issues) Prioritize issues (severity and repetition) and develop an integrated cross- functional action plan Incorporate beta user feedback into market-driven product roadmap Ensure all company infrastructure requirements identified have been fully addressed Review beta program process and execution performance and develop improvement plan Send out incentives (where appropriate) and thank individual Beta users personally (phone call, , etc.) in a timely fashion

©2004 Rosemary Remacle 12 Some Tips for Managing a Beta Program Establish well-integrated cross-functional team with representation from all stakeholders (executive, marketing, sales, engineering, support, partners, etc) Assign program owner (generally marketing) and define clear team member roles and responsibilities Agree cross-functionally on criteria for choosing a Beta site Promote importance and value of beta test to all functional organizations and share the beta information with all who need it

©2004 Rosemary Remacle 13 Key Beta Test Mistakes “Wrong” beta testers accepted Recruiting family or friends skews perspective Poorly managed beta testers and/or test data Prohibitive time commitment, resulting in drop-outs Viewing Beta program as revenue source Beta test period too short or too long Too few or too many beta users (difficult to manage) “Choosing the wrong beta testers is the best way I know to provide useless results”

©2004 Rosemary Remacle 14 Summary: A Successful Beta Program Product works as advertised Benefits are documented, validated and measured (overall, feature-specific) Product configuration, installation and customer support processes and documentation meet customer expectations Unanticipated or un-addressed total product requirements are identified and prioritized with level of criticality determined

©2004 Rosemary Remacle 15 Presenters Rosemary Remacle, Consultant Market Focus