Information Quality Checks | How To Spot Information Pollution

Identify fake news with information quality checks
Image from Dogtown Media

The internet has allowed anyone with a computer to publish their ideas online, many lacking the expertise, research skills, or objectivity to produce quality information. The result is a deluge of blogs, “news,” social posts, videos, and podcasts, impeccably designed and posted on authoritative-looking websites, tricking us into believing that the ideas are credible.

Information quality has never been so important. The rise of dangerous ideas such as climate change denial, anti-vaxxing, and the authenticity of COVID-19 are a result of people believing misinformation, and as it becomes harder to separate the wheat from the chaff, it also becomes harder to make decisions that save lives. As an anti-vaxxer tries to protect her son against autism, he dies of measles. As people gather to protest against the conspiracy of COVID-19, they indirectly kill. As narcissistic world leaders dismantle fossil fuel regulation,1 the temperature moves closer to the tipping point that drastically alters the earth’s climate,2 putting millions of people at risk.

Good information gives us a more accurate understanding of reality, allowing us to navigate the world effectively. We can make decisions that allow us, the people around us, and the rest of our species to flourish. Quality of information is critical for the wellbeing of humanity, and being able to identify whether a news article, blog, or video is credible can help you to make good decisions. 

We can’t see through the fog of information pollution unless we know how to identify it, and in this article, we’ll explain how.

Why is there so much misinformation?

The web isn’t regulated

Content posted on the web isn’t regulated. This gives people the freedom to post whatever they want, but lacks the principles and rules to ensure that the content is accurate.

While the web itself isn’t regulated, some reputable media companies have validation processes to ensure their information is as accurate as possible, for example The New York Times, the BBC, or the Wall Street Journal.

Greed

The more people who view a piece of content, the more money the creator will be able to make from advertising, paid reviews, paid subscriptions, public speaking, and other business opportunities. It’s in a content creator’s best interests to generate popular content, which isn’t necessarily the most accurate content. When faced with a choice between accuracy and profitability, it can be difficult to do the right thing.

Insufficient research, and no expert review

The world is a complex place, and many ideas are determined and affected by a large number of factors. For a piece of content to be credible, it must be rigorously researched, and if necessary, reviewed by experts. Many content creators don’t know this, and regardless of their good intentions, they end up publishing misinformation that can warp a person’s understanding of reality, leading them to harmful beliefs, and bad decisions.

Disclaimer: this article has been rigorously researched, but hasn’t been reviewed by experts.

Narrow scope

With so much content and such little time, we’ve become skimming experts. We want the specific information that we’re seeking, and we want it ASAP. So when we’re faced with a 5,000 word monster of an article that provides an excellent overview of a topic, we’ll probably close it down and find something shorter, even though the shorter article lacks the depth needed for deep understanding.

Content creators understand this, and the length of their work is shortened to suit. This can narrow the subject’s scope at the expense of explaining it effectively.

To use an example from boxing, a video editor can put together a 10-minute compilation of David Tua’s most powerful left hooks, under the title “David Tua—the best left hook in boxing.” Unless you’ve seen every one of that boxer’s fights, and watched enough boxing to get an understanding of the frequency and power of a typical left hook, you’ll be inclined to believe that David Tua has the best left hook in the sport. This is a trivial example, and it won’t affect your ability to make good decisions, but it’s something that content creators do constantly to grab your attention and get you to click on their content. It doesn’t matter if the video is accurate. What matters is that you click.

Disinformation

The information that we consume shapes our beliefs and behaviours. We can be fed information from nefarious governments, companies, groups, and individuals, who bombard us with disinformation as a way to influence our beliefs and encourage action. One of the most damaging examples of this is Russia’s supposed interference in the 2016 US election, where they bombarded American citizens with emotive social media memes, helping Trump to win the presidency.4

Pressure

Companies and individuals whose income is based on content are under pressure to create. In theory, the less content they create, the less money they make. This creates incentive to produce as much content as possible as quickly as possible, at the expense of thorough research and peer reviews.

Social acceptance

As social animals, being accepted by others is important for our mental health. An effective way to be accepted and respected by others is to create content that seems smart, well-informed, and useful. To the content creator seeking social approval, it doesn’t matter that their information is dumb, shallow, and harmful. They still get the kudos.

Information quality attributes | How to spot information pollution

Information quality is a term usually associated with the quality of information in a system (typically a computer), but we can use some of its metrics to determine if a piece of content is credible. We can also use elements from academia’s CRAAP test5 (currency, relevance, authority, accuracy, purpose). The guidelines below are a mix of these methods, with some additional checks thrown in.

Authority

When a content creator has the expertise and experience to explain a subject accurately, they’re an authority. This is one of the most important indicators for information quality.

Authority can be assessed for a piece of content in two ways: the individual who created it, and the organisation who published it.

Authority of the person

A person’s authority can be determined by their credentials and experience, usually outlined in their profile. Do they match the topic they have covered? An electrical engineer shouldn’t be telling people how to manage their diabetes.

If a content creator hasn’t listed their credentials, or if they’ve listed credentials from an unrecognised educational institution, they may not have the skills or experience needed to explain a topic accurately. People can still educate themselves and draw on their life experiences, and there’s plenty of subjects that the average Joe can elucidate, but if something complex is being explained—medicine, physics, economics, psychology, etc.—you’ll probably get more accurate information from somebody who has studied and practised it.

Of course, it’s easy for a content creator to lie about their credentials. Other information quality attributes should be checked before choosing to believe their content.

Authority of the organisation

Organisations create content for one reason: to attract an audience. Whether a newspaper, blogger, YouTube channel, or business, they’re all creating content as a way to attract or keep “customers,” and make money. Unfortunately, when money is the main reason for doing something, morals are often abandoned, and quality of information neglected. It no longer matters that content is accurate and well-researched; all that matters is that people view and share it.

The organisations with the best standards are established newspapers and media companies, whose purpose is to produce content. They usually adhere to the five principles of ethical journalism—truth and accuracy, independence, fairness and impartiality, humanity, and accountability6—which allow them to create accurate, relevant, and authoritative content. There are plenty of exceptions—most content produced by Rupert Murdoch’s News Corp is trash, including Fox News, Sky News, The Sun, and The New York Post.7 Every media company is also politically biased to some degree,8 which must be recognised when viewing their content. Finally, there’s the troubling theory of every media company acting as a propaganda tool for their corporate overlords, as outlined by Edward S.Herman and Noam Chomksy in their meticulous book Manufacturing Consent: The Political Economy of the Mass Media. But this book doesn’t question the accuracy of the media’s reporting, just the political purpose of the stories they cover, and how they might influence you in ways that benefit the powerful. You can still get facts from a reputable paper like the New York Times, but you should ask yourself why they’re choosing to report those particular facts.

Reputation is the best indicator of an organisation’s authority. Some have built their businesses on accuracy—The New York Times, The Wall Street Journal, BBC, Al Jazeera—but others are more difficult to identify. There’s websites such as Media Bias Fact Check,9 but their analysis has been called unscientific,10 so should be viewed with skepticism. Some organisations such as Nestle, Amazon, Coca Cola, and Disney are infamous for their sins, and their lack of integrity often leads to a lack of standards, extending to any “informative” content they produce. If you’re unsure whether a company has a reputation for producing accurate information, try Googling their reputation and credibility, and reading what others have to say.

Another way to determine authority is by comparing the subject of the content with the organisation’s purpose. If you’re reading a climate change article from Scientific American, it’s clear that the topic matches their area of knowledge, and there’s a greater chance that the article will be accurate. Authority is boosted further if the organisation has been producing this kind of content for a long time (although there are exceptions to this rule).

Purpose

Content is usually created for a purpose, and identifying this purpose can help to determine whether the information is trustworthy. What is the content creator trying to achieve with their content? Are they trying to entertain you, educate you, influence you, or mislead you? Are they trying to sell you a product or service?

Fox News promotes itself as a serious news organisation, and has the trust of roughly one in four Americans.11 But the purpose of its stories are entertainment first, and information second—a more accurate name for the corporation would be  “Fox Infotainment.” There’s nothing wrong with watching Fox for amusement, but watching it for educational purposes is like asking a Nazi to teach you about Jewish history. You’ll end up with a warped sense of reality.

Content that is created primarily to educate is the most trustworthy, especially when created by somebody with authority. Discerning the purpose of a piece of content can help you to decide whether it should be believed.

Referenced evidence 

Data analysis and the scientific method allow us to understand the world more accurately than ever before. If somebody makes an unfamiliar assertion that cannot be known without data, scientific analysis, or another type of real-world evidence, they need to provide a credible reference to back up their claim. This includes climate change denial, proof of conspiracy theories, political scandals, character assassinations, or anything else requiring hard evidence to be correct. 

If a content creator can’t provide you with credible evidence for their claim (which is often the case), their content shouldn’t be taken seriously.

Scope, comprehensiveness, and completeness

“The world is too complex for simple answers, and we don’t want to be the ones giving them.”

Kurzgesagt, Can You Trust Kurzgesagt Videos? 3

Even the most seemingly simple of subjects are complex, with an intricate web of relationships. Few topics can be properly explored in a few thousand words, let alone a few hundred. To explore an idea, a content creator must understand the level of scope needed to portray it effectively. If important details are missed, the viewer may end up with biased or incomplete knowledge.

For example, to understand the dictatorial motives of Israel’s prime minister Benjamin Netanyahu, it’s useful to know about his allegations of political corruption. Even if the scope of the article is small, it’s important to include further reading on the topic to offer the reader supporting background knowledge. The BBC are forerunners for this, providing supporting links throughout most of their news articles, allowing you to better understand the story by broadening your scope of knowledge.

Many content creators are unable to determine scope and comprehensiveness because they lack the knowledge and experience to write about a topic, which is why authority is such an important measure of quality.

Balance

As humans with values and opinions, we naturally introduce bias into content that we create. It can’t be eliminated, but it can be controlled by giving equal voice to different sides of an argument, encouraging the viewer to choose their preferred position. With people becoming more politically polarized over the last few years, content creators are moving further down the political spectrum, and as their ideas become more extreme, so does their content. This makes balance more important than ever. The more polarized we become, the harder it is to empathise with each other, and the less chance we have of cooperating. We become enemies, not friends.

If a piece of content requires both sides to be heard—for example political issues, social commentary, education—and the creator provides a one-sided argument, you may want to find something more balanced. Otherwise, you could come away with a biased opinion.

Objectivity

For content to be objective, the creator must provide you with the facts, and let you interpret them on your own. Much of the web’s content is based on opinion, which is fine when the topic is trivial, but when it’s something important that requires hard facts, the creator should try to be as objective as possible, and allow you to make up your own mind. Objectivity is a core principle for journalists.

Timeliness (also known as currency or relevancy)

Research that was considered credible in its heyday can still be found, despite it being long disproven. Francis Galton’s eugenics, aspects of B.F. Skinner’s radical behaviourism, and many of Sigmund Freud’s ideas gained a great deal of academic support at the time, but have since been obliterated or replaced by better ideas. 

Progress is fast in the modern world, so keeping an eye on the publication date for a piece of content is important, to avoid consuming out of date information. Many older ideas still hold their ground, but it’s usually worth checking for something fresher.

Composition and organisation

As a content creator, presenting an idea in a coherent, logical way is one of the hardest things to do, but it’s critical to getting the message across. It often requires diligent editing and re-editing, ensuring that each sentence, scene, or section follows logically from its predecessor, providing the viewer with the best chance of understanding the argument.

If a piece of content seems scattered and fuzzy, and you’re struggling to follow the argument, the creator may not have the skill to explain it effectively.

Who is the target audience?

Some content creators are trying to achieve a specific goal for a specific audience. Conservative tabloids such as the New York Post write stories with an anti immigration sentiment, which increases sales, and helps to push their conversative agenda. At the opposite end of the political scale, CNN is extremely liberal, and never has a good word to say about Trump (something that is difficult, admittedly).

Identifying the target audience for a piece of content allows you to uncover potential motives, and better understand why certain facts are being reported, and certain language is being used.

Uniqueness

Few ideas are original. The ideas that run through most of the web’s content come from another source, and are being expressed in a new way. Even journalists rely on news agencies for much of their information (Agence France-Presse, Associated Press, Reuters, and Agencia EFE), which they expand on and add their perspective to.

A piece of content doesn’t have to be unique for it to be valuable, but if the idea comes from another source, it can be worthwhile checking out what was originally said.

Reproducibility

In scientific studies, if a method is reproducible, it produces the same result when applied to different data of the same type. The scientific community recently went through a reproducibility crisis (or replication crisis),12 when they found that many scientific studies were difficult or impossible to reproduce, making their claims much less convincing.

If you’re reading a scientific study that hasn’t been reproduced, you can’t be sure that the conclusions of the study are accurate.

Spelling and grammar

If a piece of content is filled with spelling and grammar errors, as harsh as it sounds, the creator may not have the knowledge, skill, or intelligence to explain the topic effectively.

References

  1. Lisa Friedman, 2020, Who Controls Trump’s Environmental Policy?, The New York Times
  2. Timothy M. Lenton, 2019, Climate tipping points — too risky to bet against, Nature
  3. 2019, Can You Trust Kurzgesagt Videos?, Kurzgesagt
  4. Jane Mayer, 2018, How Russia Helped Swing the Election for Trump, The New Yorker
  5. The CRAAP Test: Critically evaluating information sources – transcript, QUT
  6. Five Principles of Ethical Journalism, Ethical Journalism Network
  7. List of assets owned by News Corp, Wikipedia
  8. AllSides Media Bias Chart, All Sides
  9. Media Bias/Fact Check – Search and Learn the Bias of News Media
  10. Media Bias/Fact Check, Wikipedia
  11. John Gramlich, 2020, 5 facts about Fox News, Pew Research
  12. Replication crisis, Wikipedia