In WWII, researcher Abraham Wald was assigned the task of figuring out where to place more reinforcing armor on bombers. Since every extra pound meant reduced range and agility, optimizing these decisions was crucial. So he and his team looked at a ton of data from returning bombers, noting the bullet hole placement.
They came up with numerous diagrams that looked like this:
Most of his team members observed “Wow! Look at all those bullet holes in the center of the fuselage and on the wing tips! The armor clearly ought to go there, because those are the areas that are most marked-up in red!”
But Wald realized that they were only looking at those bombers which SURVIVED, and he correctly argued that these areas were instead precisely the damage areas that were already most survivable, while the areas which were NOT marked by bulletholes meant they were fatal. In so doing, he helped us understand “survivorship bias” — that is, if we only sample from the successful outcomes, we avoid seeing the crucial factors that caused failure, which in many cases are the most important factors of all.
Such survivorship bias can lead to conclusions and strategies which are precisely the opposite of optimal, so pay attention to the datapoints that you may have already artificially and incorrectly eliminated.
Steve’s a Seattle-based entrepreneur and software leader, husband and father of three. He’s half Canadian, and east-coast born and raised. Steve has made the Pacific Northwest his home since 1991, when he moved here to work for Microsoft. He’s started and sold multiple Internet companies. Politically independent, he writes on occasion about city politics and national issues, and created Alignvote in the 2019 election cycle. He holds a BS in Applied Math (Computer Science) and Business from Carnegie Mellon University, a Masters in Computer Science from Stanford University, and an MBA from the Harvard Business School, where he graduated a George F. Baker Scholar. Steve volunteers when time allows with Habitat for Humanity, University District Food Bank, Technology Access Foundation (TAF) and other organizations in Seattle.