There’s a crisis in the scientific academic world. It’s called the Replication Crisis. Scientists have found that they cannot replicate the results published by many scientific studies. The same thing is happening in the world of business.

Over the last 15 years I’ve read several hundred business books, and I’ve written one. Across those 15 years, one of the most interesting is a book called The Management Myth, which traces the history of management science back to its less than solid origins. The book illuminates the history of scientific management and Taylorism, and supplies evidence that much the original data underpinning these theories was fabricated. I was curious if this was a broader trend, and it may be. I found other cases.

More recent examples include Scientific American debunking the theory behind Jim Collin’s work, Good to Great. A team of academics failed to reproduce Kahneman’s work on social priming theory. In 2014, Jill Lepore wrote a polemic arguing Clayton Christensen’s disruption theory wasn’t supported by the examples in his book.

Over the holiday break, I read another 10 business books. One revealed how to interpret body language in others. Another applied sports principles to the business world. A third ported military theory to the business world. Each tries to simplify the world, reduce its complexity, in effort to offer some form of reproducible insight. If you do this, you will find great success.

The three books listed in the paragraph above suffer from survivorship bias, picking winners and using them as the data set to extrapolate a theory of success. In the machine learning world, this is known as overfitting - drawing a conclusion without all the data required. In other words, drawing conclusions from anecdotes or small sample sizes.

As I study and meet and analyze more and more businesses, I find myself drawn to the idea that each is a snowflake. Each startup is unique composition of its team, in the circumstances of its genesis, and the ecosystem in which it blossoms. Business concepts like the ones mentioned above can provide a common language about ideas. They can help us tell stories to motivate and inspire. Perhaps, they can provide some form of analytical framework.

But until the success of these theories can stand the test of reproducibility or time, that’s all they should be. After all, even some of the most successful CEOs in the world cannot reproduce their success the second time or third time. Building a business is really hard, and building an enduring company even more so, because each enduring business is built in a different way.

Each January, I remind myself of this idea. To start the year with a fresh mind, that the world is different, meaningfully different than it was last year, and that means jettisoning the assumptions of the past - until the assumption can be reproduced consistently.

Image credit: Alexey Kljatov