Win probability charts like the one above have become the icons of popular predictive data analysis. I love data, but let me whisper a heresy to you. I detest these charts. I Instead of provoking thought, insight and questions, they close minds. They support the ideas of inevitability, of odds too great to overcome.
Congratulations to the Patriots who faced a 99% probability of losing the game, but passed the ball down the field into the endzone twice, and rendered this fallacious prediction worthless. Another example for Nassim Taleb to use if ever he writes a sequel to the Black Swan.
The problem with predictive analysis like this is they never capture all the variables. They don’t account for great players like Tom Brady can increase their level of play, the emotional swing of a team building momentum, those moments when people enter what Csikszentmihalyi called Flow. Predicting the future is damn hard, and no matter how much data we jam onto disk, or how sophisticated our adversarial neural networks become, we still won’t be able predict the future accurately. Just ask the weatherman who does it for a living.
When I started in venture, I took a naive view that with enough data, I could filter and find great startups to invest in. At Google, I learned how powerful data can be. But almost nine years in, I see how fallacious that perspective is. I’ve done the analysis.
You can find variables that correlate to success. You can increase or reduce the number of variables to examine. You can overfit models till you’re blue in the face. But predictive modelling will never capture exceptions. The models are based on the notion that they capture the typical outcome. What startup founder is typical? What startup journey is typical? All of them are outliers. In fact, every startup faces a chart like the one above. The odds are the business will fail.
Two years ago, the two most exciting startups in enterprise might have had charts like the one above. One was a gaming company whose MMO didn’t succeed. But when the business launched its internal messenger, it became perhaps the fastest growing SaaS company ever. The second was a PaaS that was quite close to bankruptcy. The CEO decided to open source the code, and two years later it’s a key piece of every modern infrastructure stack.
The most wonderful part about Startupland is the frequency with which enterpreneurs face predicted success charts and pull off a Tom Brady performance. Damn the win probability, full speed ahead. I’m very grateful to be able to witness some of those black swam performances because they inspire me every day. They remind me that the numbers are just numbers, and they change direction just as easily as the wind.
Published 2017-02-05 in Startups