1. Recap
   • Solutions to confounding
   • Solutions involve trade-offs (internal vs external validity)
   • Experiments
2. Interlude: Cholera
3. Conditioning:
   • what is it?
   • how does it work?

Did Trump rallies cause an increase in hate crime?

“A USA TODAY analysis of the 64 rallies Trump … held [between] 2017 [and 2019] found that, when discussing immigration, the president has said ‘invasion’ at least 19 times. He has used the word ‘animal’ 34 times and the word ‘killer’ nearly three dozen times.”

Rallies caused hate crimes?

data from Feinberg, Branton, and Martinez-Ebers

Must ask…

• What comparisons between cases does it involve?
  • can you recognize this solution when it is described to you?
  • can you describe how to use this solution to test a particular causal claim?
• What assumptions are required for it to work?
• How does it “solve” confounding?
• What trade offs do we make?

Internal Validity: is the extent to which the correlation of \(X\) and \(Y\) found in a research design is the true causal effect of \(X\) on \(Y\) (does not suffer from confounding).

• studies with strong internal validity: we have very good reason to believe that the correlation of \(X\) and \(Y\) actually implies the causal effect of \(X\) on \(Y\) for the cases we observe.
• because we can easily believe/check the assumptions

External Validity: is the extent to which the causal relationship we find in a study is relevant to the causal relationship in our causal question/claim

• Study has external validity if it examines causal relationship for the cases we are interested in

  • if study has sampling bias (sample in study different from population of interest), may lack external validity

• Study has external validity if the causal variable in the study maps onto the concept/definition of the cause in the causal claim.

  • if the \(X\) that is used in the study doesn’t align with the causal claim, may lack external validity

But the choice of “solution” to confounding — or our research design — always involves a trade off:

Increasing confidence that correlation yields an unbiased estimate of the causal effect of \(X\) on \(Y\) (internal validity)…

…comes at the cost of limiting the cases we can examine and causal variables we can examine (external validity)

One way to solve confounding is to do an experiment:

Kalmoe (2014) examines the effect of “aggressive” and “violent” language on support for political violence.

• 512 survey respondents in random sample of US adults were randomly assigned to see two versions of campaign ad
• one used more aggressive words like “fight”, the other less aggressive words.
• people report their support for violence on survey questions

Treatment vs Control

Why do both groups receive campaign ad text?

• Exclusion restriction: only changing “aggressive language” (\(X\))

Outcomes

Kalmoe (2014) finds that “aggressive” and “violent” language increased support for political violence.

• survey respondents were randomly assigned to see two versions of campaign ad
• one used more aggressive words like “fight”, the other less aggressive words.
• people report their support for violence

• do you believe the ‘violent’ ads caused people to support violence?
• can this tell us about possible effects of Trump’s speeches on violence?

(HANDS)

[board to illustrate HOW]

You live in mid-19th century London.

• Every few years, hundreds to thousands of people are killed in cholera outbreaks
• To stop these deaths, you need to answer:

What causes the spread of cholera?

Dominant view was that “miasmas” or “bad air” caused diseases like cholera

John Snow, MD suggested cholera transmitted as “germ” in water.

To provide evidence of his claim, Snow uses correlation: mapped cholera deaths of 1854 outbreak in SoHo.

• Broad Street Pump (source of drinking water) had “fouled” water (X)
• Examined mortality from cholera (Y)
• Proximity to the Broad Street Pump (C) correlated with mortality (Y)
• Proximity to other pumps not related to mortality

• Positive correlation

Leading doctors rejected Snow’s evidence:

• Houses near Broad Street Pump built on 1665 plague burial site.
• Sewers produce foul odors from rotting material/human waste

Both might produce miasmas.

• maybe Plague cemetery/Sewer \(\to\) Miasmas \(\to\) Foul Water
• and Miasmas \(\to\) Cholera

• So… Confounding.

Snow’s solution to confounding: compare people “near pump” w/ different water sources

Snow’s solution to confounding: compare people “far from pump” w/ different water sources

Discuss:

do you find these comparisons more convincing than the simple correlation?

Why or why not?

conditioning

we observe \(X\) and \(Y\) for multiple cases, we examine the correlation of \(X\) and \(Y\) within groups of cases that are the same\(^*\) on confounding variables \(W, etc. \ldots\)

How does conditioning solve the problem?

• Cases compared have same values on confounding variable \(W\) (“holding \(W\) constant”)
• In these groups, \(W\) cannot affect \(X\) or \(Y\) (because \(W\) is not moving, it can’t move \(X\) or \(Y\))
• “Backdoor” path from \(X\) to \(Y\) is “blocked”

In contrast to experiments, conditioning is possible for any cases and for any possible-cause \(X\):

Conditioning has greater external validity.

• can observe any cases that we can measure
• can look at any causal variable that we can measure

Correlation between Trump Rallies and Hate Crimes likely suffers from confounding

data from Feinberg, Branton, and Martinez-Ebers

Possible confounders imagined by Feinberg, Branton, and Martinez-Ebers

Feinberg, Branton, and Martinez-Ebers compare hate crimes in counties with and without Trump rallies, but condition on (hold constant\(^*\)):

• percent Jewish
• number of hate groups
• crime rate
• 2012 Republican vote share
• percent university educated
• region

Logic of Conditioning:

Feinberg, Branton, and Martinez-Ebers find that, even after conditioning, Trump rallies increase the risk of hate crimes by 200%!

• Lots of news headlines like this

• Discuss: are you convinced that this correlation, after conditioning, shows rallies caused hate crimes?

Clinton Rallies and Hate Crimes

Economics PhD Candidates show that conditioning on the same variables…

• Clinton rallies increased hate crimes by nearly 250%!!

• What could be going on here? Do all political rallies cause hate crimes? Or is something else happening?

Any confounding variables on the board that are missing from this causal graph?

• How easily can we measure the presence of Hate groups?

• How easily can we find counties that are identical on six attributes?

Conditioning

• What is it?: Look at correlation between \(X\) and \(Y\), for cases with same value of \(W\)
• How does it solve confounding?: \(W\) held constant, so correlation between \(X\), \(Y\) cannot be due to \(\uparrow \downarrow W\)
• What are the assumptions?: That is a very good question for next week