Fact-based research can be biased

Google+ Pinterest LinkedIn Tumblr

Can news be factual and biased? Yes, absolutely! In the modern era of “fake news” and information sources trying to stand out as reputable, there is a growing trend to highlight fact-based research methods. This is great, and definitely a trend I hope continues, but touting something as factual, to imply it is therefore unbiased, is greatly misleading.

Facts and bias—what are they?

Facts can be verified independently. For example, in the best scientific studies, the results of an experiment, the facts, are shared with the broader scientific community. Others can repeat the process and get the same results. Facts tend to be consistent over time. We consider facts to be the truth of a matter, though this may not be the best view.

Bias is when someone’s opinion influences the results. It often implies unfairly manipulating the results of a test, but bias can be incredibly subtle and accidental. Some types of bias are systemic and may be repeated when a fact-based test is performed by other parties. For example, if one group tries to reproduce the results of a biased study and use the same language used by the original researchers, it is reasonable to expect the same biased results could occur again.

Trump’s election shows bias

When election day came in 2016, the news sources declared Hilary’s impending victory right up until the last few hours before the polls closed. Why? All the surveys had shown she held a clear lead. Only one major source even hinted at an outside chance Trump could win.

Were the polls lying? No. Did the polls misrepresent the data they collected? No. I’m sure there are doctoral research papers on the subject, but there are a couple of simple problems that can creep into any research.

As a researcher, I love learning about, exploring, and talking about bias. I’ve written about it, I’ve presented it, and I’ve taught it.

Common biases that can influence fact-based research

Selection bias is easily the biggest culprit, and it is the one I see most prevalent in modern media. A form of this bias is seen when news sources will ask several experts to share their opinion, but the experts were carefully chosen to represent the views the news source desires.

However, this happens very often on accident. College students are often asked to participate in psychology research because they are in close proximity to the researchers who are located on the university campus, but college students may not represent the general population. Likewise, people tend to surround themselves with like-minded people. Therefore when we do a poll, we find it easier to get data from people like us. We are often quite surprised when we find that a contrary viewpoint is so prevalent.

The Hawthorne effect is another sneaky bias. It’s named from the factory where it was first observed. Researchers monitored the effect of lighting on productivity, but the group being studied was always more productive, regardless of the lighting. It turns out, the test subjects acted differently when they were observed, which is why it’s also called observation bias. This happens all the time—people often answer differently when they’re being polled, or when they know someone is observing them. It is very difficult to avoid and it definitely impacts data in fact-based research.

Social desirability is a third very common bias. Imagine trying to do research on racism. How would you find out if a person held racist views? If you ask them, would you expect many people to say, “oh yeah, I’m super racist!” That is an obvious example, but in fact, this bias can sneak into any type of research. It could be caused by something simple, such as the way the question was asked, or something nuanced, such as the gender or age of the researcher. An older woman may answer a question about marriage differently if the researcher is a young woman vs. an older man, for example.


Many news sources in recent years have become heavily editorialized and report on opinions and perspectives, rather than facts. Fact-based research and reporting is, therefore, a huge improvement. However, don’t be confused: fact-based research can be, and often is, biased.

Web guy, big thinker, loves to talk, teach and write. I make technology easier to use @ John Deere ISG.

Pin It