Portfolios and Probabilities: A better way to think about product roadmaps
Last year, I published a short series of newsletters about probabilistic product management. That series was entirely separate from this newsletter. I’m reposting some of it here with minimal edits.
In today’s issue I will share a way of thinking about roadmaps that’s proven most robust in my product management practice.
There’s no shortage of model frameworks to build a roadmap. But as the saying goes: “All models are wrong, some are useful” [George Box].
I’ll want to share with you an approach to thinking about roadmaps that has been useful in my product practice.
Some would say it’s boring because it relies on some [rather basic] math. I would call it “grounded” instead. And that’s exactly what makes it so powerful.
First, three important statements before we begin:
Roadmap is a portfolio of bets
Each bet has an expected value
Most bets will fail, i.e. their EV will be 0 or even negative
It’s best to think about roadmaps in purely economic terms. Your company has a budget which it can invest in a portfolio of projects. The return from some of those projects will exceed the investment greatly. Some will break even, but most will lose money. And that is absolutely okay, as long as the combined return from those bets is positive.
Venture Capital firms operate under the same principle. And if you look at a typical fund, you will notice how VC investments have a power law distribution of returns:
A handful of startups (typically a single-digit percentage) generate 5x - 100x returns
20-30% return 1-2x or break even
70% or more result in a loss
The graph below illustrates this very well. [Source: https://www.linkedin.com/pulse/vc-power-law-james-church/]
Successful VC firms are sophisticated investors who figured out the most important truth about investing in technology: each project has a limited upside and an unlimited downside:
The upside is limited by market size, competition, efficiency of distribution, execution, and other factors.
The downside is unlimited because you can invest resources indefinitely [money, effort, etc].
Therefore, VCs construct their investment portfolios to limit the downside and apply the principle of asymmetric payoff: the likelihood of a positive return from the combined portfolio must be higher than the likelihood of losing money. As long as 1-2% of bets return at least 10x of their investments, the rest of the bets can lose money. Mathematically, it’s still a win.
Start thinking in portfolios
The same logic applies to roadmaps. Individually, most initiatives will never reach their goals. But collectively, they will achieve a good ROI.
The most important product management job when building and managing a roadmap is to limit the downside.
Mathematically, this means:
Making sure that the Expected Value of each roadmap item is above 0 at all times
Stopping the projects whose EV has turned 0 or became negative
Here, Expected Value = Impact x Probability of Impact
Some examples
This is not a new approach. Look at the most popular roadmap prioritization framework: Reach, Impact, Confidence, Effort (RICE). What you’re really trying to estimate here is the Expected Value of a product initiative by using proxy metrics. And this is where it gets complicated.
In the real world, there will be many input variables that influence the EV of a roadmap item. And most of those variables are non-linear, i.e. they are probabilistic functions of some other variables.
Consider the Effort variable in the RICE framework. Effort depends on how skilled the engineering team is at solving a technical problem. But that “skill” is not a constant. If the team of engineers doesn’t have any attrition, their skill improves with time. If a new engineer joins mid-development, the average level of skill at the team decreases because this new team member needs onboarding.
A myriad of other variables fluctuate as well: market dynamics, regulation, consumer behavior patterns, trends, and company strategy — all those things change the EV of a roadmap item.
This means that the Expected Value of a roadmap item can become negative at any time. Even after the development has already started.
So, how do you manage a roadmap then?
Accept that a double-digit % of your roadmap will have a negative or 0 Expected Value. This number will typically be between 50% and 70%.
Define leading indicators which you can track to foresee the decline to 0.
Have a set of stopping criteria for each initiative, i.e. the leading indicator thresholds which will stop the project.
Liked what you read? Get in on future posts here: