Lesson 3A: Confirmation Bias, Fallacy of Confirming Instances, Falsificationism, Slanting by Omission and Distortion

Introduction
In this section we are going to start learn how to detect BS. Let's move beyond the general notion of 'bias' and get more specific about biases and how they affect the strength and validity of arguments.  Recall that one way we can classify biases is according to how much skin the arguer has in the game; that is, the degree to which the arguer stands to gain from his audience accepting his position.  In this respect we can make 3 broad categories of bias: legitimate, illegitimate, and conflict of interest.  By now you should be able to say something about each type.  Moving on...

Cognitive Biases
Another way to classify bias in an argument is according to how the hard-wiring in our brains affect the way the information is presented and interpreted.  A cognitive bias is when our brain's hard-wiring has an unconscious effect on our reasoning.  It is a current area of philosophical debate as to whether cognitive biases are on the whole beneficial or detrimental to our reasoning.  We'll set these concerns aside for this class and operate under the assumption that in many instances cognitive biases do negatively influence our capacity to reason well.  

There are hundreds of cognitive biases but the most common and the one to which we can trace most errors in reasoning is called confirmation bias.  Confirmation bias is when we only report the "hits" and ignore the "misses"; in other words, we only include information/evidence/reasons in our argument that support our position and we ignore information that disconfirms.  Confirmation bias is often (but not always) unintentional and everyone does it to some degree (except me).

What?  You don't think you do?  Oh, I get it.  You're special. Ok, smarty pants.  Here's a test.  Lets see how smart you are.  And don't forget you've already been give fair warning of what's going to happen. The smart money says you will still fall into the trap.

Click on this link and do the test before you continue:
http://hosted.xamai.ca/confbias/index.php
.
.
.
.
.
I said do the test first!
.
.
.
.
.
.
Well?  Vas happened? I'm going to continue with the assumption that you committed the confirmation bias.  Hey, don't feel bad--we're hardwired for it.  Before we move forward and discuss how and why confirmation bias works, let me take you on a philosophical aside.

Aside on Falsificationism
I promised myself I wouldn't do this but it'd be helpful to bring in a little philosophy here.  Please meet my good friend Carl Popper (no relation to the inventor of the popular snack food known as Jalepeno Poppers).

Popper made a very important philosophical observation in regards to how we can test a hypothesis:  he said we cannot test for a hypothesis' truth, rather we can only test for its falsity.  This is called falsificationism.  In other words, there are infinitely many ways to show that a hypothesis is true, but it only requires one to show that it is false.  We should focus on looking to falsify rather than to confirm.

In technical philosophy we refer to an instance of a falsification as a counter example.  A counter example is a case in which all the premises are true but the conclusion is false (more on this later).

For illustrative purposes lets apply this principle to the number-pattern test from the link.  You were given a series of numbers and asked to identify the principle that describes the pattern.  Suppose, (unbeknownst to you) the ordering principle is any 3 numbers in ascending order.  How did you go about trying to discover the ordering principle? You looked at the numbers and like most people thought it was something to do with even numbers evenly spaced.  You looked at the sample pattern and tried to make patterns that conformed to your hypothesis.

For instance, if the initial pattern was 2, 4, 6 you might have thought, "ah ha! the pattern is successive even numbers!"  So, you tested your hypothesis with 8, 10, 12.  The "game" replied, yes, this matches the pattern.  Now you have confirmation of your hypothesis that the pattern is successive even numbers.  Next, you want to further confirm you hypothesis so you guess 20, 22, 24.  Further confirmation again!  Wow! You are definitely right!  Now, you plug you hypothesis (successive even numbers) into the game, but it says you are wrong.  What?  But I just had 2 instances where my hypothesis was confirmed?!

Back to Confirmation Bias
Here's the dealy-yo.  You can confirm your hypothesis until the cows come home.  That is, there are infinitely many ways to confirm the hypothesis.   However, as Popper noted, what you need to do is to ask questions that will falsify possible hypotheses.  So, instead of testing number patterns that confirm what you think the pattern is, you should test number sequences that would prove your hypothesis to be false.  That is, instead of plugging in more instances of successive even numbers you should see how the game responds to different types of sequences like 3, 4, 5 or 12, 4, 78.  If these are accepted too, then you know your (initial) hypothesis is false.

Lets look at this from the point of view of counter examples.  Is it possible that all our number strings {2, 4, 6}, {8, 10, 12}, {20, 22, 24} are true (i.e., conform to the actual principle--ascending order) but our conclusion is false (i.e., the ordering principle is sequential even numbers).  The answer is 'yes', so we have a counter-example.  In other words, it's possible for all the premises to be true (the number strings) yet for our conclusion to be false.  

How do we know our premises can be true and the conclusion false?  Because our selected number stings are also consistent with the actual ordering principle (3 numbers in ascending order).  If this is the case (and it is), all of the premises are true and our conclusion (our hypothesis) is false.  We have a counter-example and should therefore reject (or in some cases further test) our hypothesis.

If you test sequences by trying to find counter-examples you can eventually arrive at the correct ordering principle, but if you only test hypothesis that further confirm your existing hypothesis, you can never encounter the necessary evidence that leads you to reject it.  If you never reject your incorrect hypothesis, you'll never get to the right one! Ah!  It seems sooooooo simple when you have the answer!

Why do we care about all this as critical thinkers?
When most arguments are presented, they are presented with evidence.  However, (usually) the evidence that is presented is only confirming evidence.  But as we know from the number-pattern example, the evidence can support any number of hypothesis.  To identify the best hypothesis we need to try to disconfirm as many hypotheses as possible.  In other words, we need to look for evidence that can make our hypothesis false.  The hypothesis that stands up best to falsification attempts has the highest (provisional) likelihood of being true.

As critical thinkers, when we evaluate evidence, we should look to see not only if the arguer has made an effort to show why the evidence supports their hypothesis and not another, but also what attempt has been made to prove their own argument false.  We should also be aware of this confirmation bias in our own arguments.

Bonus Round:  Where do we often see confirmation bias?
Conspiracy theories and alt-med are rife with confirmation bias.  Evidence is only used that supports the hypothesis.  Alternative accounts of the results are not considered and there is often no attempt to falsify the pet hypothesis.

Confirmation Bias and the Scientific Method:
We'll discuss the scientific method in more detail later in the course but a couple of notes are relevant for now.  The scientific method endeavors to guard against confirmation bias (although, just as in any human enterprise, it sometimes creeps in).  There are specific procedures and protocols to minimize its effect.  Here are a few:

• When a scientist (in a lab coat) publishes an article, it is made available to a community of peers for criticism.  (Peer review)
• Double blinding
• Control Group
• Incentives for proving competing hypotheses and theories wrong (be famous!)
• Use of statistical methods to evaluate correlation vs causation

Confirmation Bias 2:  Slanting by Omission and Distortion
Slanting by omission and distortion are 2 other species of confirmation bias.  Slanting by omission, as you might have guessed, is when important information is left out of an argument to create a favorable bias.

Perhaps a contemporary example can be found in the gun-rights debate.   We often hear something like "my right to bear arms is in the Constitution."  While this is true, the statement omits the first clause of the Second Amendment which qualifies the second, i.e., that the right to bear arms arises out of the historical need for national self-defense. The Constitution is mute on the right to bear arms for personal security.  There also the troublesome word "well-regulated".

Omitting these facts slants the bias in favor of an argument for an unregulated right to bear arms based on personal self-defense.  This may or may not be a desirable right to have, but it is an open question as to whether this right is constitutionally grounded.

Another example of slanting by omission might be the popular portrayal by the media of terrorists in the US of being of foreign origins.  Such an argument omits many contemporary acts of domestic terrorism perpetrated by white American males (for example, Ted Kaczynski aka the unibomber and Timothy McVeigh).

Slanting by distortion is when opposing arguments/reasons/evidence are distorted in such as way as to make them seem weaker or less important than they actually are.  Think of slanting by distortion as something like white lies.

For example, famously, when Bill Clinton said "[he] did not have sexual relations with that woman," he was slanting by distortion in the way he deceptively used the term 'sexual relations'.

Summary

• A common type of bias is confirmation bias in which only confirming evidence and reasons are cited, and falsifying evidence is ignored.  
• A good way to test a hypothesis or argument is to ask whether it's possible for all the premises to true and the conclusion to be false; that is, are there counter examples.  Instead of emphasizing confirming evidence, a good argument also tries to show why counter examples fail.  In other words, it shows why, if all the premises are true we must also accept the particular conclusion rather than another one.  
• As critical thinkers assessing other arguments, we should try to come up with counter examples.
• Slanting by omission is when important information (relative to the conclusion) is left out of an argument.
• Slanting by distortion is when opponents arguments/evidence are unfairly trivialized.