Using Experiments to Launch New Products
It is always a leap of faith to introduce a new product into a market. Justification for the investment of time, effort, and resources lies in the response from consumers, which is never guaranteed. Increasingly, companies are relying on controlled experiments to improve the accuracy of the predicted response, and to tweak their products and/or services accordingly.
Converting Data to Results
Despite the inherent and largely obvious benefits that experimentation offers, its implementation has, until now, too often been inconsistent or unnecessarily restrictive. The tide seems to be changing as major international companies like Uber exploit its methodology to expand the range and diversity of their offerings.
However, the beauty of experiments is that they can be employed by a company of any size. A restaurant chain can use a designed experiment effectively when launching a new dish. The traditional approach would typically involve offering it in select locations and making a decision based on the response.
An experiment offers a much more comprehensive range of results. A large-scale randomized trial will tell decision makers the impact of the new product on the rest of the menu. Are other purchases being crowded out? Does the product bring in new customers? Is it sustainable in the longer term?
“Move away from the mental model of thinking of new products and new features you want to build, in the form of a list of requirements, and instead, think of them as hypotheses”, says Giannis Psaroudakis, Product Director at Optimizely.
Launching an Experiment
There are four critical stages to any experiment. Understand, tweak, and employ them in a way that delivers the data you need to make the decisions that matter.
1. Data and hypotheses
Collate existing data and consult every department for their opinion on the most relevant metrics. Form hypotheses based on this information and design the experiment to address them directly. This is also the time to formulate a plan on how to use the results from the experiment to improve your product and/or service.
2. Select your target market(s)
The selection from within your area of operations must be random. Just take care to include as diverse a range of demographics, outlets, or regions as possible. Avoid duplication as this will distort results. This step requires assumptions – ensure that the ones you make are sound.
3. Have a wider plan
Any new product disrupts the market, including your own share of it. Design your experiment in a way that it has minimal impact on your existing reach… while being as disruptive as possible for your competitors. Pre-plan tweaks to the experiment that will uncover as diverse a range of outcomes as possible.
4. Interpret results correctly
Metrics like sales numbers that may, at other times, be of utmost importance are not necessarily so in an experiment. If sales of the newly-launched product are eating into the revenue from established products without raising overall revenue, is the effort worth it? Decipher the resulting data without the inherent prejudice of wanting the new product to succeed.
Uncategorized