1. A Helpful Guide to Telling the Difference
Reliable or Nonsense?
A Helpful Guide to Telling the Difference

2. Confirmation Bias
One obstacle to telling the difference between a reliable piece of information and nonsense is a tendency that exists in ourselves called “confirmation bias.”
Confirmation bias is …
The tendency to ignore data that contradicts an idea or belief we already have.
The tendency to cherry-pick only that data which supports our existing belief.
For example, people who believe the earth is flat may ignore the ample evidence that suggests otherwise.

3. Cherrypicking Data
Another indicator of the reliability or nonsensicality of a source would be the degree to which it cherrypicks the data.
To “cherrypick” data means …
Selecting data out of context.
Selecting only the data that proves their claim.
Ignoring contrary data.
Ignoring the intent of the author.
Violating common sense or a reasonable interpretation of the data.

4. Example of Cherrypicking
McAllister admitted that he found the movie “incredible” and “amazing,” clearly indicating that he considers the movie to be an “outstanding piece of art” from the same studio that produced Citizen Kane (McAllister 4). Actual Data: “I find it incredible that a movie this poorly made, badly acted, and shamefully written could ever have been produced by a major motion picture studio. It’s amazing, especially considering that this studio once produced an outstanding piece of art like Citizen Kane and now has descended to this level of trash.”

5. Ignoring Common Sense
A classic sign of bad research or bad data is when that data fails the “common sense test.”
Sometimes, uncommon events do happen, but how about the explanation for these events?
For instance, a set of buildings might be brought down by an earthquake. Someone might argue that the buildings were destroyed by saboteurs who used bombs to detonate the buildings.
Ask yourself what would have to be true for this event to occur?

6. How Likely Is It?
Several bombs would have to be planted in different locations, and all of them would have to go without being seen or detected by anyone.
All three explosions would have to occur simultaneously.
The explosions would have to be perfectly timed to coincide with an earthquake.

7. Occam’s Razor
Another version of the “common sense” test is an idea called “Occam’s Razor,” named after the philosopher William of Occam.
Occam essentially suggested that the simplest explanation is usually the correct one.
In other words, you “shave away” unnecessary assumptions to find the simplest explanation.

8. The “No Such Thing as Disproven Ideas” Problem
Reliability or nonsense can be tested by asking proponents of a theory what would it take to prove them wrong?
Are those proponents ever willing to accept evidence that proves that they are wrong?
If they are presented with contrary proof, do they accept it and say, “Oh, I see I was wrong,” or do they simply say, “Nope, this is still false?”
(For example, if someone calls your identity into question and you prove it with a birth certificate and an I.D., do they then turn around and accuse you of forging those documents?)

9. “Did I Get This From a Movie?”
Another important test needs to be the source of the theory. Films are especially problematic because their use of visual effects, emotions, sound, and the tools of cinema are powerfully persuasive.
Often, films about real characters or events can present heavily altered versions of reality without making those alterations clear.
For example, consider the scene in Shakespeare in Love in which the Shakespeare character theoretically contemplates writing a play featuring Romeo and…Ethel the Pirate’s Daughter.
Needless to say, there is no factual basis for this idea, but this film still presents the information realistically.

10. “Did I Get This From a Movie?”
Even a movie that is attempting, in all honesty, to present a truthful and accurate rendition of events still makes choices about what scenes to show, what actors to cast – even what music to use.
Unless a film is a documentary from a reliable source, it should be taken with a degree of skepticism.
Never forget: The phrase “BASED on a true story” means “This isn’t the truth.”

11. Benefits and Risks
One test of a conspiracy is to weigh the known benefits against the known risks.
For example, if a judge were to be accused of taking bribes during her administration, the revelation of that information could cost her not only her career, but could subject her to considerable legal and financial penalties.
With that in mind, if the payoff was very small – or not worth the risk – this would call the validity of the accusation into question.

12. X Files
Another way to consider the validity of a conspiracy is the degree to which it requires us to believe in what might be called “alternate reality.”
This is a subset of the “common sense” test.
Are we being asked to believe in aliens, electrical mind beams, etc.?
Do the proponents of a theory accuse reputable scientists or scientific agencies as being merely paid shills?

13. Bottom Line Bottom line, consider if the sources pass the CRAB test:
Credible? Current?
Reliable? Responsible?
Appropriate?
Biased?