Would you make loved ones fly in an airplane that hadn’t been safety tested? Would you give your child a drug that skipped out on clinical trials? Probably not (at least I hope not). How about your innovation? Would you bet your career, salary and colleagues on advice that had never been tested? No… of course not… well… actually… yes.
Name one innovation guru who has subjected his/her pet theory to predictive testing.
Go on.
I’ll wait.
Think of the last innovation book you read. When was the last time you saw numbered “steps to success” from a guru? Write the examples down, then go back and see if any were tested. Did they meet the minimum requirements of the Scientific Method most of us learned in science class? The purpose of the Scientific Method isn’t to be uptight or formalistic. It’s the very process of separating good ideas from bad. Fact from fiction. Truth from falsehood. Reality from illusion.
With only a few exceptions – who probably aren’t the ones you’re thinking of – innovation gurus don’t subject their theories to predictive testing or validation. It simply isn’t done and, until now, you probably hadn’t noticed.
How can this be? How is it possible that something like 99.991% of the innovation advice you’ve enjoyed has endured zero predictive vetting whatsoever? The answer is: it probably never occurred to the guru either.
It isn’t that your favorite gurus are necessarily dishonest, lazy, sloppy or ignorant. Chances are they’re extremely intelligent, well intentioned and have a lot of insight. Yet once they come up with a theory, for some reason the discipline of “innovation” has never asked them to take the next obvious step – to test the theory going forward. It simply hasn’t been part of the discipline.
Innovation gurus more or less follow this process when building theories:
Look at past data. Whether it’s a couple first-hand examples or a huge pile of historical data, gurus begin by examining history.
Notice patterns. Most of the time historical data is next whittled down to a manageable number of cases (from three to a few dozen examples). These are usually examples of companies that have done extremely well in one way or another. From this smaller number, gurus next try to find any patterns shared by the examples.
Brand the patterns and prescribe them to the world. Write a book. Speak at TED. Teach MBAs. Sit on boards. Sell consulting. Have your picture taken with the President. Dish with Oprah. Do your best to sound scientific and inspirational the whole time.
What’s missing are critical next steps: predictions from the hypothesis, experiments, randomized testing, statistical vetting. It’s one thing to find commonalities between a small number of hand-picked examples, and another to test if those traits have any predictive value in the real world beyond those limited cases.
The problem is simple – it’s relatively easy to design a feel-good brainstorm or find commonalities between just about anything. For example, what do Mozart, Theodore Roosevelt and Thomas Edison have in common? Given a few hours, most of us could come up with a lot of “patterns” shared by these men. Given a year we could also write a book about how the patterns can make any innovation more successful. We could add tons of detail about each person, sprinkling in novel and delightful facts, anecdotes and tidbits to solidify our points. What’s wrong with that?
I’ll tell you what’s wrong. For starters, it’s unsubstantiated hogwash. We just pulled it out of our… ears. As such, our advice is just as likely to do harm as it is to do good.
It isn’t a victimless crime. Whenever we rely on un-tested advice during major business decisions, we’re putting the fate of our innovations at risk. Bad advice can kill good businesses. There’s a word for this in medicine, law and many other advice-giving fields: malpractice.
If you’re sick of living in a world where 70% – 80% of new innovations fail, it’s time to stop condoning malpractice. At the very least, it begs for all of us – consumers of innovation theory – to become tougher critics and to demand more from gurus. Otherwise we’re just feeding, and funding, rampant malpractice in a domain society depends on for its very future.
What about you? Let’s be honest. Has your favorite innovation guru been tested? The answer might surprise you.