November 25, 2024
We want to use correlation to provide evidence of causation:
But the choice of “solution” to confounding — or our research design — always involves a trade off:
Increasing confidence that our correlation yield an unbiased estimate of the causal effect of \(X\) on \(Y\) (internal validity)…
…comes at the price of limiting the kinds of cases we can examine and the kinds of causal variables we can examine (external validity)
Internal Validity
A research design (choice of which cases to compare using correlation) has internal validity when the causal effect of X on Y it finds is not biased (systematically incorrect) / does not suffer from confounding.
External Validity
is the degree to which the causal relationship we find in a study captures/is relevant to the causal relationship in our causal question/claim
Study has external validity if the relationship found is true for the cases we are interested in
Study has external validity if the causal variable in the study maps onto the concept/definition of the cause in the causal claim.
Did Trump rallies cause an increase in hate crime?
“A USA TODAY analysis of the 64 rallies Trump … held [between] 2017 [and 2019] found that, when discussing immigration, the president has said ‘invasion’ at least 19 times. He has used the word ‘animal’ 34 times and the word ‘killer’ nearly three dozen times.”
data from Feinberg, Branton, and Martinez-Ebers
But as we discussed, this correlation might suffer from confounding:
DISCUSS WITH NEIGHBORS: could there be other factors about communities that…
BOARD
One way to solve confounding is to do an experiment:
Kalmoe (2014) examines the effect of “aggressive” and “violent” language on support for political violence.
Kalmoe (2014) finds that “aggressive” and “violent” language increased support for political violence.
(GROUPS)
Solution | How Bias Solved |
Which Bias Removed |
Assumes | Internal Validity |
External Validity |
---|---|---|---|---|---|
Experiment | Randomization Breaks \(W \rightarrow X\) link |
All confounding variables | \(X\) is random; Change only \(X\) |
High | Low |
Before we return to speech and hate crimes
You live in mid-19th century London.
What causes the spread of cholera?
Dominant view was that “miasmas” or “bad air” caused diseases like cholera
John Snow, MD suggested cholera transmitted as “germ” in water.
To provide evidence of his claim, Snow uses correlation: mapped cholera deaths of 1854 outbreak in SoHo.
Leading doctors rejected Snow’s evidence:
Both might produce miasmas.
So… Confounding.
Snow’s solution to confounding: compare people “near pump” w/ different water sources
Brewers | Broad St. Residents | |
---|---|---|
Water Source (X) | Brewery Well/ Beer (Clean) |
Pump (Contam.) |
Location | Near pump | Near pump |
Timing | Aug. 1854 | Aug. 1854 |
Miasmas (W) | Yes | Yes |
Cholera (Y) | No | Yes |
Snow’s solution to confounding: compare people “far from pump” w/ different water sources
Lady and Niece | West End Residents | |
---|---|---|
Water Source (X) | Broad Street Pump (Contam.) |
Another Pump (Clean) |
Location | Mile from Broad St. | Mile from Broad St. |
Timing | Aug. 1854 | Aug. 1854 |
Miasmas (W) | No | No |
Cholera (Y) | Yes | No |
Discuss:
do you find these comparisons convincing (as a way to prevent confounding)?
Why or why not?
This solution to confounding is called…
when we observe \(X\) and \(Y\) for multiple cases, we examine the correlation of \(X\) and \(Y\) within groups of cases that are the same on confounding variables \(W, etc. \ldots\)
How does conditioning solve the problem?
In contrast to experiments, conditioning is possible for any cases and for any possible-cause \(X\):
Conditioning has greater external validity.
Earlier we asked:
data from Feinberg, Branton, and Martinez-Ebers
Correlation between Trump Rallies and Hate Crimes likely suffers from confounding
Compare against confounders imagined by Feinberg, Branton, and Martinez-Ebers
Feinberg, Branton, and Martinez-Ebers compare hate crimes in counties with and without Trump rallies, but condition on (hold constant):
Feinberg, Branton, and Martinez-Ebers find that, even after conditioning, Trump rallies increase the risk of hate crimes by 200%!
Economics PhD Candidates show that conditioning on the same variables…
Any confounding variables that are missing from this diagram?
Conditioning