Introduction
In the course of most arguments, factual or empirical claims will be made. A factual or empirical claim is one that can actually or in theory be observed and tested. Two common and closely related types of empirical claims are generalization and is polling. Lets look at each in turn.
Generalizations
A generalization is when an arguer moves from observations about some specific phenomena or objects to a general claim about all phenomena or objects belonging to that group. For example, I go to McDonald's and order a Big Mac and it costs $1.50. Then I go to another McDonald's and order another Big Mac and it also costs $1.50. Based on these observations I generalize to the conclusion that all McDonald's restaurants will change $1.50 for a Big Mac. (I'm such a great scientist)
Another example would be if I ordered 1000 Tshirts that say "I Love Soviet Uzbekistan". Upon receiving the order I might look at 5-10 of the shirts in 2 or 3 of the 10 boxes to make sure they were printed properly. From those specific samples I'd generalize to the conclusion that all the Tshirts were printed correctly. If I'd found printing errors, I'd check more boxes to get a better idea of the proportion of total shirts with printing errors.
There's nothing really fancy going on with generalizations. We all do it a lot in our everyday lives because it's practical and often it wouldn't make sense to do otherwise.
General vs Universal Claims
At this point we should make a distinction between a general claim and a universal claim. A universal claim is that all X's have the property Y. For example, all humans have a heart and a brain. Universal claims are much stronger than general claims.
General claims admit of exceptions but are generally true of a set of objects. For example, generally students like to sleep. We could possibly point out some counter-examples, like Jittery Joe who doesn't like to sleep. But, despite the occasional exception, we can accept the generalization as true.
Lets get technical for a moment and formalize these structures:
A universal claim will generally ;) have the form, "all Xs are Y".
A general claim will generally have the form, "Xs are, in general Y," or "Xs are Y," or "Each X is probably Y".
Sometimes, (surprisingly) in conversation or in an article, the arguer won't spell out for you or use the exact language I've specified here to distinguish between the two types of claims, nevertheless, if you pay attention to context, you should be able to determine which is being made.
The main point to understand is that universal claims don't allow any exceptions whereas general claims do. Also, from the point of view of constructing and evaluating arguments, it is much more difficult to defend an universal claim than a general claim.
One last type of generalization is the proportional claim: As you might expect, this type of claim expreses a proportion. For example, looking through the first 2 boxes of my Tshirts, I notice that 1 out of every 7 is missing the letter "I". So, even though I don't check the remaining boxes I conclude that 1 out 7 of the Tshirts in those boxes is also missing an "I". (i.e., I made a proportional generalization).
The thing is, (as you might expect) there are legitimate and illegitimate generalizations which have much to do with the nature and size of the sample from which the generalization is being made.
Sample Size Me!
For obvious reasons, the larger the sample size, the more accurately it will reflect the properties in the entire group of objects. For instance, if I see one student and she tells me she has student loans, I shouldn't conclude that all students have loans. Maybe I talk to 3 students and they also tell me they have student loans. It could be that I just happened to talk to the 3 students that have student loans, it doesn't mean that all students have them. The sample is still too small for me to legitimately make any inferences about all the students.
Now suppose I talk to 400 students and 100 of them (amazing round numbers!) tell me they have student loans. At this point I might be able to make a reasonable generalization about all students at that particular school or maybe in that particular region or state.
Sample Bias and Representativeness
One worry is that our sample is too small to justify generalizations. The other is that our sample isn't representative enough of the group about which we are making the generalization. For instance, if I wanted to make a generalization about the proportion of US students with student loans, it wouldn't be enough to collect data at only one school.
My sample have to have about the same proportion of sub-groups as does the general population I want to generalize about. Maybe my sample happens to be from a rich school. Maybe not. Either way, this doesn't represent the average school. Maybe that particular state provides excellent funding, maybe not. Again, I want to make sure my sample represents the the proportion of states or schools that do and don't provide excellent funding. I also want to make sure my sample includes the demographics of all US students in about the same proportion.
To have a representative sample of the larger group, what is needed is to take samples from all over the country. That is, the sample from which we will generalize should be broad enough to negate 'clumpy-ness' of certain traits and should be representative of the group we are trying to generalize about.
In terms of evaluating and constructing arguments beware of anecdotal evidence! Why? Remember biases? Biases have a huge influence over what gets reported and what doesn't. If we experience something that runs against our bias we tend to ignore it. While on the other hand, we over emphasize experiences that conform to our biases. When we are using testimony as evidence (i.e. anecdotal evidence), we should be aware of this and how it increases the likelihood that our sample is biased (and therefore not representative of the group of things we are generalizing about).
Here are some common sources/common examples of bias: "it worked for me (or my Aunt Martha), therefore it works for everyone".
The problem with this is that your sample size is exactly 1. If you are going to use a sample size of 1 for a generalization about medicine/treatment or anything that should apply to everyone, then your sample is worth exactly nothing!
Another common bias (and a huge issue/problem in social psychology right now) is generalizing about all human behavior from samples that have a geographical and cultural bias.
In other words, for decades social psychologists and psychologists have been making generalizations about all of human psychology from samples of US college students. As it turns out (from recent cross-cultural studies) US culture is an outlier in terms of what's "normal" psychology throughout the world. Yup. We're the weird ones, not the rest of the world. Oh! Snap! (but of course our way of thinking is the right way!)
In medicine, great lengths are gone to to protect against a biased sample. The gold standard is a large (3 to 5 thousand subjects) double-blind, placebo controlled, long-term replicated study that includes different populations (i.e., ethnic groups) and both sexes. The hallmark of pseudoscience in medicine is that often these standards are not applied or the sample is too small.
Rules for Good Generalizations
We can think of generalizations as following (implicitly or explicitly) this argument scheme:
(S=the sample group, X=the entire group of objects that the generalization will be about, Y=the property we're attributing to Xs)
P1. S is a sample of Xs.
P2. The proportion of Ss (that are part of X) that have property Y is Z.
C. The proportion of Xs that have property Y is Z*.
*see rule 4 below.
Lets use an actual example to get away from the alphabet soup:
P1. The students in this class are a (representative) sample of UNLV students.
P2. The proportion of the students in the class that have student loans is 60%.
C. Therefore, the proportion of UNLV students with loans is around 60%.
Or
P1 10 species of cats is a sample of all the species of all cats.
P2 The proportion of cats in the sample that land on their feet when dropped from over 4 feet is 100%.
C Therefore, all species of cats will land on their feet when dropped from over 4 feet.
To evaluate generalizations we essentially want to scrutinize P1 and P2 and their logical connection to C. To do so we ask if
1) The sample size is reasonable for the scope of the generalization.
2) The sample avoids biases.
3) Objects/Phenomena in the sample (X) do indeed have the property Y.
4) The proportion of X with property Y in the sample is greater or equal to the claim about the proportion of Xs with property Y in the generalization. (In other words, I can't say that 30% of Xs in my sample have property Y, yet in generalize that therefore 40% of Xs have property Y.)
If a generalization violates one of these 4 criteria then it likely isn't a defensible generalization.
No comments:
Post a Comment