April 8, 2021

Correlation to Causation

Solutions to Confounding

  1. Recap
  2. Differences in Differences

Recap

Solutions to Confounding

Every way of using correlation as evidence for causality makes assumptions

  • FPCI cannot be solved without assumptions
  • With assumptions, can say confounding/bias is not a problem

Solution How Bias
Solved
Which Bias
Removed
Assumes Internal
Validity
External
Validity
Experiment Randomization
Breaks \(W \rightarrow X\) link
All confounding variables 1. \(X\) is random
2. Change only \(X\)
Highest Lowest
Conditioning Hold confounders
constant
Only variables
conditioned on
1. Condition on all confounders
2. Low measurement error
Lowest Highest
Before and After Hold confounders
constant
variables
unchanging
over time
No confounders
change w/ \(X\)
Lower Higher

Example: Gun Laws

Does easing restrictions on gun laws increase murders committed using guns?

  • Some states in the US require all handgun purchasers to acquire a permit-to-purchase (PTP) license.
  • Only persons with a permit may purchase firearms
  • In late 2007, Missouri eliminated its PTP requirement

Example: Gun Laws

Webster et al (2014) investigate:

  • Did the removal of the PTP law incease firearms homicides in Missouri? >- Conditioning?: Lots of unique features of Missori; no “otherwise similar” state. >- Easy comparison is Before and After

POLL

Example: Gun Laws

Holds all unique, unchanging characteristics of Missouri constant.

But, we have to assume that there is nothing else about Missouri that…

  • changed around the same time as the PTP repeal
  • also affected Firearms Homicides

No long-term trends, changes in measurement, or anticipation either…

Does this plot make it easier/harder to believe PTP repeal caused more murders?

Example: Gun Laws

  • Maybe an upward trend in long term?
  • But maybe 2006-2007 was aberration, 2008 a return to trend?
  • What else happened in 2008?

Could be that other things were changing between 2007-2008 that confound relationship between PTP and Murders

Example: Gun Laws

What can we do to remove confounding from other variables that change over time, like…

  • weather patterns (hot weather \(\xrightarrow{?}\) murders)
  • global financial crises/economic shocks
  • political events
  • One option: compare Missouri to another state with no change in PTP law. Arkansas

Arkansas has a different history that Missouri, so there are differences that are unchanging between them.

But, if Arkansas experiences same regional economic, political, cultural trends as Missouri, they might have the same trends over time.

Missouri/Arkansas different in 2007, but if Missouri had same trend (absent the PTP repeal) as Arkansas, we’d expect…

Missouri would have (counterfactually) had the same trend as Arkansas, if it hadn’t repealed PTP

Design: Difference in Difference

What is it?

  • Compare “treated” cases to “untreated” cases before and after the “treatment” takes place

How does it work?

  • Hold constant unchanging attributes of cases (compare same case before and after “treatment”)
  • Hold constant variables that change together over time in both “treated” and “untreated” cases

Design: Difference in Difference

Consider 2 states \(\mathrm{Missouri}\) and \(\mathrm{Arkansas}\) at two times \(Before\) and \(After\) Missouri implements PTP repeal (“treated”).

  • \(\mathrm{Missouri}\) sees increase in murder rate
  • \(\mathrm{Arkansas}\) sees slight decrease in murder rate

We measure \(Homicide\) (\(Y\)) in both states.

Design: Difference in Difference

So:

  • \(\mathrm{Difference \ 1} = Homicide_{After} - Homicide_{Before}\) gives us change in murders in a \(State\)…
    • holding unchanging attributes of state constant (same case over time)
  • \(\mathrm{Difference \ 2} = \mathrm{Difference \ 1}_{Missouri} - \mathrm{Difference \ 1}_{Arkansas}\) gives us change in murders in \(Treated\) over time, compared to \(Control\)
    • holds shared trends of both states constant (difference in trends)

Design: Difference in Difference

\(Homocide_{Before}\) \(Homicide_{After}\) First Difference
\(\mathrm{Missouri}\) \(4.6\) \(6.2\) \(1.6\)
\(\mathrm{Arkansas}\) \(5.6\) \(5.4\) \(-0.2\)
Second Difference \(1.8\)

Example: Difference in Difference

Confounding Solved

All confounding variables (affect whether a rally occurs; affect hate crimes) that are unchanging over time are held constant

  • change over time in the same case

All confounding variables that change the same in “treated” and “untreated” case are held constant.

  • By comparing change over time in “treated” to change over time in “control”

Design: Difference in Difference

In order to infer \(X\) causes \(Y\) if \(X,Y\) correlated in difference-in-difference comparison…

Must Assume

  • we assume trend in \(Y\) for “untreated” case is the “counterfactual trend” in \(Y\) for what the “treated” case would have done absent “treatment”
  • Equivalently: we assume “treated” and “untreated” have the “parallel trends” in \(Y\).
  • Equivalently: no variables that affect \(Y\) and change over time differently in “treated” and “untreated” cases

Do you believe assumption of parallel trends?

Example: Difference in Difference

Confounding UNSolved

  • Arkansas and Missouri murder rates mostly move together before 2007.
  • But large change in AR in 2001/2002 not in MO
  • But in 2007, before law took effect, murders dipped

Solution How Bias
Solved
Which Bias
Removed
Assumes Internal
Validity
External
Validity
Experiment Randomization
Breaks \(W \rightarrow X\) link
All confounding variables 1. \(X\) is random
2. Change only \(X\)
Highest Lowest
Conditioning Hold confounders
constant
Only variables
conditioned on
1. Condition on all confounders
2. Low measurement error
Lowest Highest
Before and After Hold confounders
constant
variables
unchanging
over time
No confounders
change w/ \(X\)
Lower Higher
Diff in Diff Hold confounders
constant
unchanging and
similarly changing
Parallel trends /
no differently
changing
Higher Lower

Application

Example: Facebook and Hate Crime

Mueller and Schwarz (2018) ask:


Is social-media hate speech related to real-world violence?


  • Are there higher levels of anti-refugee violence in places with more social media access in weeks with more social media anti-refugee hate speech?
  • Address this question in the context of Germany (2015-2017)

Example: Facebook and Hate Crime

variable: Attacks against refugee persons and property


measure: (for each week in each municipality)


Example: Facebook and Hate Crime

variable: Number of anti-refugee posts on Facebook per week


measure:


Example: Before and After

Is it plausible that nothing other than anti-refugee FB posts is changing over time?

Example: Difference in Difference

Could be events that drive anti-refugee posts and hate crimes…

Mueller and Schwarz come up with “treated” and “untreated” municipalities.

  • Internet outages reduce exposure to Facebook.
  • Communities with internet outages in the same week are “untreated” by Facebook hate, but share other trends (e.g. events) in the same week.