Shipt VP of Product Ethan Grob on moving from startup to mature product organization
Brought to you by CustomerIQ, the AI platform to discover and quantify themes across customer feedback channels like calls, surveys, tickets, team messages, and more. Align every team, prioritize work with conviction, and accelerate development with one tool. Learn more here.
Shipt was founded in 2014 by Bill Smith in Birmingham, Alabama. Uniquely, it was acquired by Target only 3 years later and has since grown to become one of America’s largest grocery delivery platforms.
Shipt went from scrappy startup, to still-scrappy-but-operating-within-the-mothership of Target as a wholly-owned but independent subsidiary with over 1,000 team members. The company has the unique challenge of being a three-sided marketplace – retailers, consumers, and gig workers – with over 300k gig workers on the platform, partnering with hundreds of retail and brand partners, and delivering tens of millions of orders each year.
To learn more about the journey from adolescent startup to mature product organization I spoke with Shipt VP of Product, Ethan Grob. Ethan has an impressive and non-traditional product background having gone from consultant, to strategy/ops at Kroger where he helped stand up grocery e-commerce and led the $700M acquisition of Home Chef, to now VP Product at Shipt. He’s also a side hustler building a budding indoor golf empire but we’ll leave that for another day.
Here’s what we learned:
- Making the shift from project-led to product-led culture.
- How Shipt structures a 500-600 member product and tech org.
- Shipt’s focus on outcome and metrics-driven problem solving
- The impact of qualitative vs quantitative data
- How to get more customer feedback
- Building the “Waze for grocery shopping”
- Letting the best ideas win
- Going from good to great in product management
- What he looks for when hiring new PMs
To learn more about Shipt, check out their website, download the app and place your order.
Shipt’s product org has grown ~3X since 2020. How have things changed in that time?
It’s come a long way. We were founded, experienced tremendous growth, and were acquired by Target only a few years later.
So we were still kind of in an adolescent stage of product management when our then-CPO hired me.
When I joined the team, there were only a few product managers and designers involved in the B2B space. It was a very small org.
And so it was very, very much engineering-led, which I think just inherently lends itself to being a little bit more project oriented, being very opportunistic, which comes with the territory for any growing org.
There was a lot of, “We’ve only got a set number of resources or team members that can work on this stuff with certain expertise. So for whatever the biggest opportunity is, let’s just all the people are going to focus on that and get it done.” It’s scrappiness at it’s finest.
A new level of product maturity was something I was really trying to champion when I got here. I was a really big proponent of persistent product teams – having someone who owns the roadmap, knows it deeply, understands the vision, and can speak to a specific capability or set of capabilities and what they’re going to be.
That was a big transition because at that time our model was more flexible – ream members flexed based on the burning needs. We had adopted a squad model where were rotating people in and out depending on the needs of the business at that time. It was great for agility and speed, not so great for supporting and evolving the products that you build.
It was like, “All right, we got this one out, we solved the need for this retailer, now we have to pick up this team and go work on this thing… well that’s on fire now.”
So there was a bit of internal selling to get to that point and understand the importance of continuity.
And it’s important to note you have to be at a certain size or level of maturity as an organization to be able to do something like that.
If you only have one or two PMs, you don’t have that luxury. So trying to get to that level of maturity was pretty important.
And I’d say that we, for the most part, are now operating in that way with more stable persistent product teams, taking the long-view for the tech we support.
How is the product org structured?
We’re mostly structured around a combination of market segments and operational functions, creating a hybrid model. Because we’re a multi-side marketplace we have to focus on each of the users/customers. It includes a Business-to-Consumer (B2C) arm, encompassing our consumer-facing marketplace and a shopper-facing app tailored for gig workers (not consumers in this case, but the needs of the gig economy). Then there’s a Business-to-Business (B2B) side, primarily focused on retailer engagement. There’s also the operational aspect, how do we make our offerings operationally and economically feasible leveraging technology.
The organization is segmented based on which aspect of the marketplace is being supported. For example, the operations team concentrates on balancing gig worker supply with consumer demand, aiming for cost-effectiveness and efficiency. The marketplace team is dedicated to enhancing user experience, delighting our members, and converting new traffic. Meanwhile, the B2B side is retailer-focused, ensuring the growth of their business on our platform, driving incremental volume.
Internally, there has been debate about the best way to organize these teams. Options have included grouping by operations, a growth team (also known as the business team), or a marketplace and marketing team. Alternatively, we’ve considered organizing by tech stack.
Currently, the structure is a mix of these approaches. It features a core internal team and an algorithmic systems team, which works on behind-the-scenes aspects like pay structures, bundling, and order offerings for gig workers. There’s also a marketplace team, primarily focused on member experience within our marketplace.
The ratio of product managers (PMs) to designers is approximately two to one. We have around 75 to 80 people collectively in product management and UX design. We also have a relatively small yet super effective user research team.
On the engineering side, the ratio to product management is roughly four or five to one. This means our engineering and data science team, which we collectively refer to as our tech team, comprises roughly 500 to 600 individuals.
Do you use OKRs?
I certainly see value in OKRs, but to implement them in an effective way is challenging, and I can’t confidently say I’ve seen anybody do it perfectly yet, in any organization. At least the way that “the books would tell you to do it”. Maybe Google or Intel? I don’t know.
For example, when I joined Shipt, our quarterly meetings were very much focused on what each product team had delivered. We were output-driven, and largely by necessity. As a result, no one was really asking “why” any of this mattered and what it did for our business. So, much of my first year at Shift was spent working with business teams to change their approach to be outcome-focused.
I constantly asked questions like, “What problem are we trying to solve here?” and “What’s our hypothesis for what this is going to do for the business?”
Over time, I noticed our non-technical team members becoming more receptive to this and began framing their requests as problems rather than solutions. They started considering a broader perspective during our planning meetings.
While this is still a work in progress and no company does it perfectly, any of these frameworks boils down to being outcome-focused and metrics-driven. It requires having a clear hypothesis of what you aim to accomplish before building something.
To answer your question – we are gradually adopting OKRs at Shipt, and teams are working towards their own key performance indicators. We now have a clear view of the metrics that matter to us at a company level and are rallying our teams around these important objectives.
Do ideas come from top down or bottom up?
It largely depends on the visibility of the matter at hand. Take for example, the algorithms supporting our pay structure, order bundling, and the offerings to our gig workers. Teams handling this kind of stuff tend to have more autonomy largely due to the complexity involved. It gets into some really interesting, intricate math, and honestly, I think it’s intimidating to people unfamiliar. There aren’t as many ideas coming from the top there. It’s not so much a lack of interest, rather a case of wow, that’s complex, but it’s cool. Plus that team has been really effective in removing cost from the model in a sustainable way.
On the other end of the spectrum are elements that are much more tangible, like the marketplace experience. Everyone’s got an opinion on it, as with any consumer app, and there are generally more voices, suggesting things from new components to innovative merchandising tactics. I can’t say if this leaning towards group involvement for tangible matters is right or wrong, but that’s been my observation.
Similarly, elements that are super front and center, like the ones our marketing is built around, or those that are consumer-facing are also subject to a lot of opinions. With real-time feedback, it’s almost inevitable to have various ideas on how such things should be built.
So, the flow of ideas, from top down or bottom up, kind of depends on the level of complexity and visibility involved in the issues at hand.
How do you manage behavioral tracking and analytics and how has that evolved?
Managing behavioral tracking and analytics initially began with prioritizing what was important to track. We already had a team in place, so it was about ensuring that we were logging all the pertinent details that we wanted to monitor.
We utilize several tools including Amplitude, Segment, Snowflake, and Tableau, amongst others to measure our app experiences.
The first hurdle was determining what was happening on certain screens and ensuring we had the necessary tracking set up. This took a considerable amount of time, for example a third of a team’s quarterly capacity, just to set it up appropriately and verify that we could see the necessary data. Then, we had to ensure that this data was flowing properly for our data science team to analyze, build models, or extract it into tools like Tableau for further analysis.
We had a lot of people internally proficient with working in relational databases. Surprisingly, many knew how to access our tables via SQL in Snowflake, which was probably not part of their original job description. I think that was a clear indicator for us that we needed to make behavior tracking and dashboards more available.
This all started as a slight evolution, but now we’re in a great spot. Over the past two years, we’ve reached a point where our data is solid, reliable, it flows appropriately, and teams can access it to make informed business decisions.
One of the things I’m thinking through now with our teams is if/how we establish causal links from all lower-level data points to the top metrics of our vast KPI tree like order volume or membership growth. But we continue to do the best we can, amidst the multiple connections and correlations we know today, we continue our efforts to make this powerful data work for us.
What about the qualitative side? How do you manage customer feedback?
We collect customer feedback from various sources, including real-time input within our Shipt Marketplace and Shopper apps – it’s so critical for us. We collect anything from operational details like inaccurate store information or hours of operation, to suggestions for a smoother shopping process. We strive to use this to enhance not only the customer experience but also to make it easy for our gig workers to be as helpful and effective as possible.
On the marketplace side, users often highlight technical issues they encounter – bugs, support issues, etc. but we also see things around promotion expectations, assortment questions, etc.
Historically we’ve probably leaned more on the qualitative side than quantitative. What we believe to be our key differentiator at Shipt is the shopper and driver community we’ve built. All the gig workers that choose to do work on Shipt’s platform are a step above what you’ll find with competitors so we really lean on that qualitative element; but it shows up in the quant metrics too.
Past decisions were predominantly influenced by potential impacts on these communities. Using scoring systems like NPS or CSAT, we could identify and avoid actions that could potentially harm our shopper sentiment, subsequently affecting the overall customer experience. We maintain open lines of communication with our retailers, members, gig workers, and internal team members to ensure we continually meet the needs of everyone who uses the products we build in this ever-changing landscape.
How do you encourage more people to give feedback?
I remember times in my career when we’d sit and wonder why customers were doing what they were doing. Then this realization hit us – let’s just ask them. So we setup systems to ask them feedback in the moment – right when they were doing something. You may think you need complex machine learning models to infer that behavior but you’d be surprised at how many insights you can glean from simply asking the question.
Whether they’re shopping for groceries or completing a task, ask in the moment. Keep it simple, make it easy for them to respond or give them a space to enter feedback if they feel compelled to do so. For example, they could log an issue they just encountered with the product or service.
What you don’t want is to send them something a day later, asking them to fill out this 15-minute survey. The results could be biased, you’ll get a lot better, higher quality feedback by asking one-off questions in the moment. The balance you’ll have to strike is to ensure it’s not too frequent or intrusive – rather it’s optional and available at the right moment.
Don’t get me wrong; I believe there’s a big place for post-experience feedback. But in my experience, real time feedback has consistently offered really solid insights.
We do a few other things as well. One is the use of the five-star rating at the conclusion of an order. There’s a feedback loop for both shoppers and customers where they rate how the order or the shopping experience was. Did you get everything you wanted? Were there too many substitutions? Was the delivery on time? It’s these simple things we can draw a lot of insight from – some for ourselves, some for the retailer, some for the shoppers themselves, etc.
Additionally, we also use tools like clarifying questions – if a customer selects a specific star rating, we ask, “What about the order made you give it those five stars?” It helps us understand customer satisfaction better, and we can use those insights to build those profiles and flow them through into actionable features. More importantly, it helps us course correct in real time and provide a better shopping experience for both shoppers and customers.
What’s something notable you’re building as a direct result of feedback?
One cool initiative we are currently working on is in the context of our shopper application. Many aren’t aware of how challenging it can be to shop for groceries for someone else. Especially if you’re shopping at a new store, it can take approximately 25 to 50% (or more in some cases) more time to complete an order when browsing for items. This is true even as a consumer.
We’re tackling this challenge by working towards directing our shoppers to the specific aisle and place on that aisle where they can find the stuff they’re looking for. The tricky part is, it’s different at every store. Aisle numbers vary, store formats and sizes are distinct, and the layouts are diverse.
But because of our shopper community, we have hundreds of thousands of shoppers who are in stores every day. So we thought, why not crowdsource this information? So, we’ve built a feedback loop for shoppers specifically on item location and its accuracy, for the instances where we don’t already have this information.
Of course, there are challenges. If we ask for more structured feedback, we might miss important details. If we leave the format more open, we’ve got to handle all the unstructured data – sentences, numbers, typos, incomplete thoughts. So, we’re building a model to parse out the crucial information and attribute that to a specific store.
There’s no sugarcoating it, it’s a bit tricky, the store layouts differ from place to place, and the feedback is complex. But it’s top of mind for any shopper in real-time, and that makes the task worthwhile. This solution, we believe, will truly improve the shopping experience by reducing time and confusion for our users – ultimately leading to our members getting exactly what they want in their order.
You mentioned you coached sales reps and account managers on the B2B side how to better communicate with the product team. Can you tell us more about that?
Yeah, the challenge of getting sales and account managers to ask better questions when customers make requests is a real one, especially in organizations where “product thinking” didn’t really come first.
First, it requires a significant amount of retraining to deal with the intake process efficiently. For instance, if I’m the chief growth officer and my team has a brilliant idea, the question arises, who do we take it to? Who’s the right person that can potentially get this built? When in reality, the first question should be what problem do we think this thing is addressing and how big is that problem?
Now, the first thing to understand here is that just because you’re taking an idea to a product team or an EM doesn’t necessarily mean that your idea will, or should, be built immediately. Rather, what we’re trying to cultivate is a culture for better intake and collaboration. We need to initially identify who to approach, you know, like having a “who to call” list that outlines the responsibilities of all the PMs on the team.
Now if you desire a particular user experience, you need to identify if there’s a problem with our product content. That’s when you get in touch with the catalog domain leader, see if they’re already working on something similar, or get your idea added to their backlog.
Then, to make sure your idea isn’t lost in the backlog – you should come with a hypothesis. Like, “I think this adjustment will increase the conversion by 400 basis points because we’ve test-proven this, or we have some industry research backing this claim.” In other words, if you believe doing something will yield a particular result, quantify it.
What I usually tell my teams (on the product side) is this: yes, as a product manager, you need to defend your roadmap, but if someone presents an idea with clear financial, operational, or strategic impact that you can’t argue against, then it may be time to pivot. Just as a stakeholder needs to make a compelling case to you, you should be expected to do the same. If what they’re suggesting appears to be better than what you’re currently working on, then explore the new avenue.
It’s a fantastic challenge for both teams because it paves the way for stronger product intake backed by solid hypotheses and a product development team that’s sure of the value they’re creating with their backlog.
So the golden rule is: don’t just fall hard for the thing you want to build; it’s the age-old “fall in love with the problem, not the solution”. If there’s something better on the table that’s faster, cheaper, more effective, etc… go for it!
How do you manage and track customer feedback?
Incorporating feedback into our product operations is a continuous process. It’s important to keep this ongoing, bringing new ideas or changing needs of the business to your product contacts continuously. We then work through the sizing, determine the possible impact, and get it stocked in our backlog.
Our PMs keep grooming this backlog as we operate through our (typically) two week sprints. The idea is to do this consistently and as frequently as possible, with more detailed refinement during a dedicated week. That week should be dedicated to reviewing dependencies (especially cross-team or cross-functionaly), figuring out the appropriate metrics, and understanding what can actually fit into a quarter or set of sprints.
Having said that, it’s crucial to strike a balance. At the end of our quarterly planning, I always encourage my teams to have a 95% confidence level about what they can deliver so that we can plan our financials, customer communications, and other important aspects. I’m sure the product purists disagree with this part but it’s just reality for many companies.
Our primary tool for managing the backlog and roadmap is ClickUp. It helps us maintain a pretty robust backlog that comes handy during our sprint planning. The main objective is to identify what’s the highest impact thing we can do right now and also keep solid data on our velocity, what we’ve delivered, metrics we’re tracking, project plans, etc.
For qualitative feedback on the consumer side we leverage Medallia among other tools. And of course we also run focus groups, in-depth interviews, user ethnographic research and in-market observation, and ongoing surveys.
On the B2B side, aggregating feedback from different retailers or CPGs, it’s a challenge. We try to understand not only the benefits of a project to one particular retailer but to the entire group from A to Z so our solutions scale to a wider set of B2B customers, while balancing the needs of the gig workers on our platform, or our members.
What unique perspectives about product management have you developed over your career?
I think anyone can be a great product manager, it doesn’t take a specific background or pedigree. This may sound a little biased because of the path my career has taken, but let’s remember that product management is a relatively new field in the history of careers. A lot of people I come across are like, “I don’t understand what product is or how it works.”
The way I see it, anyone could be a product manager if they tried. Not to self-deprecate, but at its core, product management is just creative problem solving via technology. It’s learning a new language, adopting new processes and routines, but it’s not entirely different from solving a problem in say, an operations role. It’s all about evaluating a business process, understanding feasibility, viability, and desirability, setting goals, determining the time frame, and ensuring scalability.
I suspect that the confusing part for many folks is the technology aspect. Unless you’re in the field, it can feel a tad mystical. But, breaking it down, technology simply involves a set of rules, plenty of creativity, and problem-solving.
For Product Managers it’s enticing because it’s almost like running your own mini-business. You get to work a lot cross-functionally, with your stakeholders and users, and witness the product being built. So that’s what I find interesting.
The key thing is to demystify it. It’s not a scary thing. It’s an approachable thing. We’re just solving problems and using technology to do it.
What separates an average PM from a great PM?
The exact path depends on what the company needs at that time.
Look, there are times when what you really need is an execution machine – someone ruthless and relentless on just getting things done.
And then there are times when what you need is a visionary. Someone who knows the ins and outs of an industry and its users so well that they can evolve a product into something beyond anyone’s expectations.
Now, if you’re a junior level PM with dreams of becoming a CEO, I’d lean more on developing skills on the vision side, while being dependable on the execution of course. Being a great story teller is essential. But your story needs more than rhetoric, it needs to be backed by solid data and a team that loves working with you. You’ve got to be able to bring the design, data science and engineering teams together, no egos allowed. It’s important to not get too attached to a specific idea and you have to be data-driven to determine what those ideas should be.
Operational reality matters too. Can you deliver what you promised? If you estimate that something will be done in a quarter, will the team be able to do it? Have you accounted for potential complications, such as needing another team’s input or development? You need to effectively anticipate challenges, technical and non-technical, in order to ensure smooth execution of the work.
So really, you need skills in both vision and execution. If you desire to be a strong PM as an individual contributor, focus on the execution side and be someone your team loves to work with. If you envisage a future in product leadership, be ready to stretch across disciplines, thinking about the long term and broad implications; not just for the technology side, but the business side as well.
What signals do you look for when hiring?
When hiring, my priority isn’t necessarily looking for someone with product experience, although it is important to some degree. What I really look for is how candidates approach problems. I like when they can provide examples from their past of the challenges they faced, what they chose to do, and how they accomplished it.
I place a big emphasis on translatable skills, because how someone learns new skills, or works cross-functionally, is very important in day-to-day product management. Picking up on how an applicant tackles a problem is key for me.
And of course, there are the basics. I want to see if the candidate understands the subject matter. Can they empathize with the needs of their users and so on? Are they good communicators and professionals? Would I put this person in front of a customer? How well versed are they in running effective product ceremonies? Do they have a sense of urgency? All of this depends on the scope of the role itself.
Contrary to what may be standard, I tend to structure my interviews more like a business role with a technology lens as opposed to a purely product management interview. Again, you could say my approach may be a bit biased because of my background, but that’s how I go about it.
Huge thank you to Ethan, VP of Product at Shipt for sitting down and speaking with us. You can find him on Linkedin here. To learn more about Shipt check out their website or download the app.
