By Gordon W. Grossman
Reader’s Digest created a systematic and disciplined approach to testing new products, primarily because of creation of expensive one-shot books—and testing began long before a product saw the light of day.
This complicated process existed because of the size of the marketplace for new products. When a company knows that millions of copies of a successful one-shot can be sold, everyone wants to make each one as nearly perfect as possible.
The pre-testing procedure started with a questionnaire mailed to a sample of the promotable customer list. The recipient was told that he and some other Reader’s Digest buyers were being asked questions about a number of products which were being considered for future production. This questionnaire typically described about 30 different titles. The purpose was to winnow out losers—to eliminate material that fascinated the editors, but not potential buyers.
The titles were listed with the number of pages, the proposed price and two- or three-line titles. The choices were:
• I would order
• I might order.
• I have little interest.
• I have no interest.
The next stage was a much more targeted questionnaire with just six product concepts described in a single mailing. The descriptions were expanded to several paragraphs, and all major product features were included. Various titles could be tested for the same book (in separate questionnaires, of course), and emphasis on certain aspects of the book also could be measured. In practice, while [particular] titles sometimes made a major difference, emphasis seldom did.
A questionnaire is still not a real sales situation, no matter how much care is taken in its construction. Any questionnaire makes the prospect into an expert rather than a potential buyer. The practical definition of a “successful” questionnaire is one that ranks new products in the same order as a series of simultaneous sales tests to the same market would. This result was achieved regularly by both Reader’s Digest and Time Life Books.
The ultimate questionnaire would include a scoring technique that would accurately predict sales and profits. Although a number of very smart analysts have tried many twists and turns in scoring, this has never been achieved— and I doubt it ever will be.
One drawback to this ideal is that it is quite common for the product to change between the initial probing and final sales presentation.
After the second round of questionnaires, it was time to put actual sales material in the mail with a “dry test.” This is a real direct mail sales test conducted before the final commitment to produce a product has been made. The presentation is as close as possible to the one that is planned for the launch mailing, with important exceptions.
The Federal Trade Commission issued an “advisory opinion” on dry testing. In paraphrase, the FTC said that it had no objection as long as four conditions were met:
• No representation is made that the product definitely will be produced.
• There must be adequate notice of the conditional nature of the offer.
• Those who order are promptly informed if it is not produced.
• There can be no substitution of another product
Naturally, Reader’s Digest and Time Life Books always adhered rigidly to these guidelines. The dry test necessarily replaces current terms with conditional ones, saying things such as this product “is now being planned,” it “will have 620 pages” and there “will be” various other features.
The marketing problem in a dry test is to simulate actual sales conditions as closely as possible.
Dry tests can accurately predict response to an offer. They cannot predict subsequent performance, and this is a significant limitation. For a series book or periodical prod uct, the marketer learns only what the up-front response wifi be. After that, the new product is on its own.
But Reader’s Digest found ways around this problem. For example, a book called “Nonfiction Best Sellers” was being produced and sold each year as a one-shot with consider able success. The company wanted to find out if the product could be made into a nonfiction series as a counterpart to the fiction in Reader’s Digest Condensed Books.
A small group, of customers was shielded from the annual promotion and received an offer some months later describing the book as the first volume of a series. By speeding up production of the next books, the Digest could send books two and three at about their normal series interval. These books subsequently were sold as annuals to the rest of the promotable list in the next two years. The result was a profit on each book, and a clean test reading of performance on the series.
The verdict? The test ,showed that the series would only be moderately successful and it was not produced at the time, but it’s a rare instance of profiting handsomely while testing lavishly.
A much more serious example of the limitations of dry testing occurred at Time Life Books. Its dry tests always described the series, but naturally focused on the “lead book”—the first book in the lineup—just as the subsequent launch promotion did. In testing for a proposed series called “The Human Body,” two volumes were far and away the most popular: “The Healthy Heart” and “Diet and Exercise.” These titles were so close in appeal that Time Life books decided to split the launch mailing.
Half of the launch universe received an offer for “The Healthy Heart” while the other half saw “Diet and Exercise” as the lead book. The second book sent to each group was the one they did not receive as volume one.
Each lead book produced a very satisfactory response, and it looked like Time Life Books had another winner. However when book two was shipped, it turned into a disaster of almost proportions. Payments f it were so bad the series was stopped immediately, and a huge investment had to be written off.
Research showed what had happened. The buyers of “Diet and Exercise” were mostly in their 30s and 40s. Those who purchased “The Healthy Heart” were almost all men, age 50 or older. Regardless of which book went first, the second one was a total turnoff to the very different group of people who had bought book one.
When the product was planned, this hadn’t occurred to anyone (including the marketing consultant on the project, who happened to be me). So the process wasn’t perfect, but every successful series and all major one-shots did come out of it.
On larger projects, if the dry test was successful, the product would move forward to the production stage. One more set of full-scale direct mail tests would then precede the launch. These were conducted with just enough lead time to incorporate the results into the launch mailing. These tests, mailed between the dry tests and the “wet” launch, were, naturally, called “damp” tests.
The damp test was the time to test alternative prices, dif ferent premiums and new copy approaches. After multiple regression analysis was adopted, the sample for regression analysis was mailed.
Everyone who ordered was notified right away that the product wasn’t currently available, given a firm shipment date and the option to cancel. If the customer chose, he could elect to receive the product. Both Reader’s Digest and Time Life Books were able to salvage a healthy percentage of damp test orders, even though these tests took place months before products were available for shipment.
The sequence of multiple questionnaires, then a dry test leading to the decision to develop the product, followed by a damp test for fine tuning, was the route followed by every major new offering from both Reader’s Digest and Time Life Books for many years.
About the Author
Veteran circulation consultant Gordon W. Grossman is a member of the Direct Marketing Association Hall of Fame. This piece is from “Confessions of a Direct Mail Guy,” published by Bottom Line Books, Stamford, CT. Click here to purchase. Source Direct Magazine – February 2007.
http://www.marketingscoop.com/dry-run.htm
