How can you predict, far in advance, if businesses will succeed or fail?
I’ve spent my career asking this question. As a data scientist and venture capitalist, it’s my white whale. My colleagues and I stare red-eyed at data sets; poking, prodding and begging for secret clues. Most of our breakthroughs give us a euphoric buzz, right up until we dig a little deeper and realize we were dead wrong the whole time. Sometimes we actually do make progress, but more often we’re reminded why science is the Great Humiliator.
This obsession can make me a little opinionated. Okay, super opinionated. For example, sometimes I’m asked to speak on interview panels with other venture capitalists, and it never goes well.
The way these panels tend to work, I’m usually seated next to other VCs who are asked what they look for in a startup. Each panelist says something like “the most important thing is the startup’s founding team. It’s all about the team.”
Everyone nods in approval. The audience is happy. The moderator is happy. The other panelists are happy. It’s like saying “I love puppies.” You can’t go wrong.
Then it’s my turn. I say something like “actually, I don’t pay much attention to the team because, if you look at empirical data, you’ll find the team typically doesn’t explain more than around 12% of the variance in outcomes.”
…somebody gasps…
That’s when the other panelists scowl at me. One says “I don’t know what data yoooooou’re talking about, but in myyyyyy experience the team is the most important thing.”
…and so it begins…
Growing up in a house where debate was the family sport, part of me wants to lock horns. Like Cuba says in Jerry Maguire, “that’s the difference between us. You think we’re fighting, and I think we’re finally talking.”
In those moments, I want to point out how people’s “experience,” even if it spans decades, almost never constitutes a statistically significant sample size when brought to bear on an isolated mental decision. I want to talk about cognitive bias and behavioral economics. I want them to realize they don’t even know their true investment criteria (beyond intangible feelings, CYA instincts, peer pressure and rampant attribution error). They don’t know their own accuracy, because they’ve never bothered to keep a tally. If they’re like around 90% of VCs, they probably have a miserable portfolio – yet this never causes them to question their own investment criteria, even for an instant.
The entire venture capital industry, as measured by the NVCA venture capital index, posted negative returns for the entire decade between 2000 -2010 and, according to a Harvard study, 75% of VC investments don’t even break even. Just because VCs are considered experts doesn’t mean the gross majority have any idea what they’re doing.
If there was ever a good time to pause, take a deep breath and count to ten, this is it.
Then, a few months ago, something changed for me. I was chatting about investment strategies with a couple VCs at a swanky NYC cocktail party when I realized how self-righteous I’d (ashamedly) become. They were especially bright investors, and while they used a totally different approach, they made a lot of sense.
That’s when my heart caught up with something my head already knew. They were probably right, and so were some of the other panelists I’d sat next to over the years. Yes, some VCs are truly ignorant of their own ignorance. But at that moment, what I perceived as “scientific truth” was tempered by a flash of awareness that there are more than one ways to predict the future.
The world of predictions is a world of probabilities. As Daniel Kahneman put it, probability is a waypoint between ignorance and knowledge. None of us know what the actual future will be, but we make our best guesses based on what’s most likely. There’s always a margin of error, and the quality of our predictions is based on the size of that error.
Given the near-infinite things one could consider when predicting a business’s fate, gobs of variables – and even more combinations of them – hold a range of predictive values ranging from around 1% (a tiny bit) to 99% (a huge amount). There can also be multiple approaches with the same score.
In other words, while the future is one destination, there are many paths to get there. Some paths are better than others. There can also be multiple paths that are equally good.
This is one of those cliché, humdrum and otherwise trivial realizations that, at a specific moment in time, hit home for me in a new way. I’m not saying all VCs or predictive strategies are equally good, but I am saying this realization has helped me be more open minded and receptive. Sometimes, in life and investing, there are many paths to one place.