Startup Best Practices 23 - Leveraging The Illusion of Explanatory Depth in Interviews

Draw an image of a bicycle that depicts how the bicycle works. You might draw something like this bike above.

Or this one.

Or this one. But as Gianluca Gimini discovered when he asked 50 people over the course of six years to draw a bicycle, most people cannot - despite their great confidence of the contrary. Gimini rendered these drawings (on the right) to highlight incorrect most people’s understanding of a bicycles anatomy truly is.

This is the illusion of explanatory depth or IOED, which Rozenbilt and Keil described in 2002

People feel they understand complex phenomena with far greater precision, coherence, and depth than they really do; they are subject to an illusion — an illusion of explanatory depth. The illusion is far stronger for explanatory knowledge than many other kinds of knowledge, such as that for facts, procedures or narratives.

The authors argued IOED occurs for four reasons. First, we overestimate how much we remember.

Second, there are several levels of explanation for how things work. You could describe the bike as two wheels, pedals and a chain. A deeper level of explanation might introduce brakes, brake pads. Yet another level might delve into the physics of how bicycles balance (something physicists haven’t fully explained yet).

Third, the explanation varies depending on the sophistication of the questioner. Fourth, we don’t practice explaining things very often.

So how does this illusion apply in the world of startups? The illusion enables a great interviewing technique.

When worked at Google, I interviewed product managers. I was given a set of questions to ask, and one of my favorites was “how would you redesign one of your favorite products?”

Interviewees might pick a toothbrush or a piece of software or camera. Over the next 10-15 minutes, I would poke and prod. Why do you like the product? How does it work? Which are the most important design considerations? And finally, how would you improve it?

Though I didn’t know it then, this line of questioning attempts to expose the illusion of explanatory depth. Can the candidate communicate the basic functions of the product but also plumb the nuances of the product and the key design decisions? Why was the toothbrush handle molded that way? How does the software onboard users? How does the single lens reflex camera work?

This technique applies equally well for marketing technology stacks to PR managers to query planning database engineers.

Using the IOED in interviewing questions enables interviews to identify candidates who deeply understand how to build a bicycle, or a marketing stack, or an optimized query plan.

Published 2017-02-26 in best-practices 


Tomasz Tunguz is partner at Redpoint. I write daily, data-driven blog posts about key questions facing startups. I co-authored the book, Winning with Data. Join more than 20,000 others receiving these blog posts by email.


Read this next: