1. Optimal Database Marketing Drozdenko & Drake, © 2002 1

2. 2 The Analysis Sample Chapter 6

3. Optimal Database Marketing Drozdenko & Drake, © 2002 3 Chapter Objectives Why We Sample Sampling Methods Creation of the Analysis Sample Methods of Saving Point-in-Time Sample Data Analysis and Validation Samples Application of Analysis Findings

4. Optimal Database Marketing Drozdenko & Drake, © 2002 4 Why We Sample Testing is the foundation upon which direct marketing is built. With correct test planning, a direct marketer can: Evaluate new product offerings. Gauge the reaction to price changes by measuring the associated increase or decrease in response rates. Determine the impact of a new promotional format change on response, payment or conversion rates. Identify the target market for a new product test. Gain insight about specific customer groups or segments.

5. Optimal Database Marketing Drozdenko & Drake, © 2002 5 Sampling Methods For samples to be meaningful and unbiased, they must be selected randomly and representatively from the universe of interest.

6. Optimal Database Marketing Drozdenko & Drake, © 2002 6 Sampling Methods (Cont.) Representative Samples - A representative sample is a sample truly reflecting the population of interest from which the direct marketer draws inferences. For a sample to be representative, no members of the population of interest are purposely excluded from the sample. Some direct marketers overlook this very important concept and assume they can apply test results from one population to another. This may work in some cases but not always. Be careful!

7. Optimal Database Marketing Drozdenko & Drake, © 2002 7 Sampling Methods (Cont.) Typically, the only names that should be eliminated from testing are names eliminated in roll-outs such as: DMA do-not-promotes Frauds Credit risk accounts When testing new promotions, some direct marketers also consider eliminating: Names recently promoted for other marketing tests States or cities such as Washington D. C. known to have strict promotional restrictions.

8. Optimal Database Marketing Drozdenko & Drake, © 2002 8 Sampling Methods (Cont.) Random Samples - A random sample is one in which every member of the sample is equally likely to be chosen, ensuring a composition similar to that of the population. Pulling names one after another from the beginning of a geographically sequenced customer database will result in a geographically biased sample. To ensure random samples, many direct marketers utilize what is called “nth selects.”

9. Optimal Database Marketing Drozdenko & Drake, © 2002 9 Sampling Methods (Cont.) For example, ACME Direct, a direct marketer of books, music, videos and magazines is interested in testing a new book concept. The universe of interest is the segment of most active book buyers. Therefore, a random and representative sample will be created by taking an nth select from the universe of concern with no exclusions.

10. Optimal Database Marketing Drozdenko & Drake, © 2002 10 Creation of the Analysis Sample In order to properly determine the characteristics that define responders vs. non-responders you must base it on a sample in which the customer characteristics were “frozen” at the point-in-time of the promotion. You cannot pull customers from the database today and examine such characteristics for a promotion that occurred, for example,six months prior. If you are going to properly determine what caused a customer to either respond or not respond to your promotion, then the characteristics that you examine must be reflective of what the customer looked like at the time you promoted them. Think of this file as a “snap shot” of each customer’s record prior to sending the promotion. This “snap shot” of the customer’s records is also referred to as a “frozen file.”

11. Optimal Database Marketing Drozdenko & Drake, © 2002 11 Creation of the Analysis Sample (Cont.) To illustrate why “point-in-time” customer data is important in making sound marketing decisions, consider the following example: One year ago you sent an offer to 10,000 names from the database a special Club Med vacation package known to be only of interest to young married couples without children. If you examine the sample today (one year later) to determine the characteristics that uniquely separated responders from non-responders and use the customer characteristics as of today, you may be misled as to what a responder looks like. Some responders to the test promotion may have had a baby since the time of the original test (one year ago). As a result, you will erroneously conclude that people with babies also are interested in this vacation package. This is false since at the time of the promotion when the person made their decision to order the vacation package or not, they were childless.

12. Optimal Database Marketing Drozdenko & Drake, © 2002 12 Creation of the Analysis Sample (Cont.) The process to create a frozen file for analysis purposes is as follows: Select names (including all customer data) to be test promoted On the database, using a unique key code (Chapter 4), mark the names that were selected for this test promotion Create a file of the selected names with their address and customer ID information only and send to the lettershop for promotion Create a file of the selected names with all customer data for later analysis (the frozen file) Once customer responses come in: Update the customer records on the database with response information using the unique key code Update the frozen file with response information You are now ready to conduct the analysis to determine the characteristics that separate responders from non-responders based on the frozen file updated with response information.

13. Optimal Database Marketing Drozdenko & Drake, © 2002 13 Methods of Saving Point-in-Time Data Saving a “snap shot” of all customer data for every test sample promoted is an expensive and costly proposition. Depending on the size of the direct marketer’s testing program, this can quickly create major computer storage capacity issues. To circumvent this problem, some smaller direct marketing companies save an entire copy of their marketing database on regular basis (e.g., quarterly). When analysis of a product promotion is required, an analysis sample is created “on the fly” by identifying the names promoted for the test (and who responded) via a unique key code residing on the database. Once the names are identified on the database, customer data from the frozen customer database closest, but prior, to the actual promotion date is merged and an analysis sample created.

14. Optimal Database Marketing Drozdenko & Drake, © 2002 14 Methods of Saving Point-in-Time Data (Cont.) This process is illustrated below. The danger with this method is the customer data to be appended to the test names may not represent the customer’s status near the time of the promotion. In the above figure we note the appended data is two months prior to the actual promotion date. Ideally, the customer data should represent the customer’s status as near the actual promotion date as possible. As a result, saving a snap shot of the customer data with each test sample is preferred.

15. Optimal Database Marketing Drozdenko & Drake, © 2002 15 Methods of Saving Point-in-Time Data (Cont.) If your company cannot do this due to data storage capacity issues, this is the preferred alternative. Just remember, do not merge a promotion file with a version of the customer database reflecting the customer’s status after the date of the promotion. Why?

16. Optimal Database Marketing Drozdenko & Drake, © 2002 16 Analysis and Validation Samples Before analysis of any sample is performed, the sample is typically spilt into two. The analysis is often performed on two-thirds of the sample and the results validated or calibrated on the remaining one-third of the sample.

17. Optimal Database Marketing Drozdenko & Drake, © 2002 17 Analysis and Validation Samples (Cont.) Why? Remember, a sample is just that, a sample; and samples have a certain level of error associated with them. The validation sample is used to ensure the analyst does not make erroneous conclusions based on the error variance associated with the sample. Creating an analysis and validation sample is a very important step in the analysis process flow. The importance of the validation sample will be addressed fully in Chapter 11.

18. Optimal Database Marketing Drozdenko & Drake, © 2002 18 Application of Analysis Findings Once the target market is defined based on the analysis of a frozen file, you are ready to select names from the entire database meeting your criteria. When preparing to select names from the entire database based on an analysis of a frozen file, ensure the data residing on the customer database is current. Make sure you are selecting the correct names by keeping the information residing on the database fresh and relevant.

19. Optimal Database Marketing Drozdenko & Drake, © 2002 19 Database Modeling Analysis Process Flow Chart Define test segment Names pulled and saved from the database Sample split 2/3 for analysis Application of analysis findings to the database for roll-out Customer Database Validate findings and refine results Responses matched to create the frozen analysis file Names sent test promotion Analysis of responders vs. non-responders Sample split 1/3 for validation “The Frozen Analysis File”