July 13

MVPs — How to Run & Test Effective Experiments

The 7 habits for running highly effective lean startup experiments. Even the most celebrated scientists are not immune to cognitive biases. Ash Maurya quotes Richard P. Feyman who said “the first principle is that you must not fool yourself and you are the easiest person to fool.” Although a product manager need not go to the lengths required by the scientific community to ensure valid results, you still must vigourosly test your own assumptions. Maurya teaches that you can develop habits that will help you do this on a continued basis. These include clearly defining expected outcomes from the start, accepting imperfect information sources, depending on quantitative data, and becoming adept at turning your assumptions into falisifiable hypotheses.

A guide to validating product ideas with quick and simple experiments. An effective experiment must include these four elements — an hypothesis, your riskiest assumption, the method, and a minimum criterion for success. So says Grace Ng, who provides details on how to assemble each of these. First, a customer-problem hypothesis is formed (a problem-solution hypothesis will be formed after it’s been validated), then your riskiest assumption will be identified for testing, then the method of testing chosen, and finally your expected outcomes defined.

Five pitfalls of running lean startup experiments. In another article, Grace outlines some of the pitfalls to look out for when running experiments. She starts with one of the most striking — testing the wrong thing. In the effort to show progress, it’s easy to rush into an experiment without sufficient planning. The result is a poorly defined experiment and an MVP that is not a good representation of the solution you’re trying to test. Similarly, failing to keep the MVP lean, not defining your success criterion and asking the wrong questions will lead you down the wrong road.

Paul McAvinchey

About the author

For over 20 years, Paul has been building and collaborating on digital products with fast-growing startups and global brands, including AOL and WMS Gaming. Currently, he's a co-founder of Product Collective, a worldwide community of product people. Members collaborate on in the exclusive Member Hub, meet at INDUSTRY: The Product Conference, listen to Rocketship.fm, learn at Product Interviews and get a weekly newsletters that includes best practices in product management. In recent years he led business development at DXY, a leading product design firm in the Midwest, and product innovation at MedCity Media, a publishing startup acquired by Breaking Media in 2015.


You may also like