Truly data-driven product development calls for both. Each form of data has strengths and weaknesses in different circumstances. Cliff Gilley asserts that you must know the audience you are analyzing and, considering the vast amount of data that could be collected, avoid analysis paralysis.
When do you use quantitative vs. qualitative research?
To decide which kind of data to work with, Ron Yang suggests that you ask yourself, “would my problem best be solved by feelings or facts?“. Qualitative research helps you to find subjective themes in data collected from interviews and focus groups. Whereas quantitative data helps you find objective data through methods like A/B testing, surveys and analytic tools.
Effective quantitative research requires good experiment design
In-product analytics, error mining, and customer support data are great to understand an existing product. When you’re working on a new product you need a different source of information to guide your decisions. You need to build experiments, and you need to make sure those experiments are designed appropriately. Teresa Torres shares why you should run experiments to test specific assumptions instead of ideas and how to go about doing that.
In-product analytics provide insight on how customers use your product when it’s working. Finding out what happened when things didn’t work can also be enlightening and point to work you need to do on your product, soon. Gojko Adzic calls the process of examining production errors for insights “error mining” and explains why it’s a good idea for you to do a little digging.
Mastering quantitative research is Product Management
If you want to make data-informed decisions about your product, in-product analytics provide a good way to “analyze user behavior and find opportunities to improve the experience.” Suhail Doshi, CEO of Mixpanel, shares his experience with quantitative research and the good and not so good practices teams follow.
What every Product Manager needs to know about product analytics
As a product manager, you want to learn as much as you can about your customers and their needs. Taking a look at in-product analytics and what they tell you about how your customers actually use your product is one way to do that. Sam Tardiff explains what product analytics are and why you should use them, what “empathy debt” is and how to pay it off, and how to guide new feature development with analytics.
You could find the one metric that your team needs. To find this one metric depends on the stage you are at with your product and requires some digging to expose a data-point that is globally illuminating. Early stage products might look at daily active users, but later products may look at something more specific, like how many users completed X number of actions.
Acknowledging first the common adage that data (knowledge) is power, Evgeny Lazarenko goes on to break down some of the key metrics that you should follow as a product manager. These he groups by user engagement, business, and customer service.