Are Humans Equipped for a Big Data World?

Evolution does not work quickly. It takes many generations for our genetic code to adapt to changing environments and circumstances. What this means is that our 21st century human genome is still basically the genome of a caveman.

Our genome was well-adapted to the environment of our hunter-gatherer ancestors, because that environment lasted for hundreds of thousands of years. Unfortunately, the 21st century world we live in bears little resemblance to the prehistoric world. Most of the change has occurred in the past few centuries, and ongoing change is only accelerating.

The prehistoric world was a “small-data” world. It was in this world that the human propensity toward confirmation bias first developed. Confirmation bias is the tendency to favor information that confirms our preconceived beliefs — and to ignore data that contradicts those beliefs.

When a caveman walked in the woods and heard a rustling noise behind him, he had very little data on whether it was a leopard or a squirrel making that sound. Cavemen had to infer danger or opportunity from limited data inputs. When they heard a growl or a squeak, it would reinforce a hunch, and they would act accordingly. But our ancestors were mostly inferring things and acting on hunches based on locally accessible information. The small data set they relied on was a product of their five senses and their limited personal experience of the small world in which they wandered.

A caveman’s life was one of actions, not data browsing. Coming to the conclusion that the rustle behind him was a leopard required a very quick decision. The instant he heard a growl, he needed to act. So forming a hunch — and then being extremely open to rapid confirmation or reinforcement of that hunch — was a critical survival tactic.

As a result, confirmation bias was more important to human survival in a small-data world. It became highly ingrained in our genetically controlled brain development.

But ever since the Age of Enlightenment, and certainly since the advent of the Internet, we no longer live in a small-data world. Today we are being overwhelmed by the data around us. We live in a Big Data world.

On every topic of interest, Google puts petabytes of data at our fingertips. Add to that Wikipedia, crowdsourcing and a thousand TV channels. Every minute of every day, email users send about 200 million messages, and Facebook users share nearly 700,000 pieces of content. Our problem is no longer inferring from limited data. It is finding the valuable signal in the vast noise — finding what is important in that huge mountain of raw data.

I’ve come to believe that confirmation bias has become a human liability in our modern Big Data world. Today, if you believe that childhood vaccinations cause autism, there are websites that will substantiate that conclusion. If you believe in ghosts or extrasensory perception, there are plenty of websites that will confirm and reinforce that view. You can find support for almost any crazy belief, even if the vast majority of the information in an unbiased search might contradict your preconceived notions. Humans are good at ignoring stuff they don’t agree with.

This is an unforeseen consequence of the Internet and the sudden accessibility of almost infinite, diverse data that can be freely posted by almost anyone. I think it partly explains the polarization of our politics, and even the rise of extremism around the world. Now one can find external support for almost any preconceived belief. Different groups can exist in alternate realities that are internally consistent and self-supporting, but totally contradictory to the beliefs of other groups.

Confirmation bias has gone from being a useful survival tool to a source of increasing polarization and disharmony. And I submit this is being driven by our move from a small-data world to a Big Data world. Almost every wild thought can now be confirmed on the infinite Internet.

Once we inferred to survive, now we self-reinforce to disagree. In a strange way, we’ve gone from one reality to many realities, each internally reinforced. This is not a good trend.

Ted Driscoll, PhD, is the partner leading the Digital Healthcare team at Claremont Creek Ventures. His investment portfolio includes AlterG (formerly Tibion), Assurex Health, Cureus, Fluxion Biosciences, GeneWeave Biosciences, GigaGen, Natera, NuMedii Inc. and Zipline Medical. Reach him @easydjr.

This article originally appeared on Recode.net.