April 6, 2021

Correlation to Causation

Solutions to Confounding

  1. Recap
  2. Before/After Comparisons

Recap

Solutions to Confounding

Every way of using correlation as evidence for causality makes assumptions

  • FPCI cannot be solved without assumptions
  • With assumptions, can say confounding/bias is not a problem

Solution How Bias
Solved
Which Bias
Removed
Assumes Internal
Validity
External
Validity
Experiment Randomization
Breaks \(W \rightarrow X\) link
All confounding variables 1. \(X\) is random
2. Change only \(X\)
High Low
Conditioning Hold confounders
constant
Only variables
conditioned on
1. Condition on all confounders
2. Low measurement error
Low High

Conditioning vs. Design

Conditioning removes confounding by:

  • identifying possible confounding variables
  • measuring confounding variables
  • relationship b/t \(X\) and \(Y\) for cases with similar value of confounding variables.

Conditioning vs. Design

Design-based solutions remove confounding by:

  • selecting cases for comparison in order to eliminate many known/unknown measurable/unmeasurable confounding variables.
  • the nature of the comparison holds constant types of confounding variables, not just specific confounding variables.

Conditioning vs. Design

Did Trump rallies increase hate crimes?

Feinberg, Branton, and Martinez-Ebers compare hate crimes in counties with and without Trump rallies, condition on (hold constant):

  • percent Jewish, number of hate groups, crime rate, 2012 Republican vote share, percent university educated, region >- But they left out population, which confounded Trump rallies and Hate Crimes. >- Difficult to find counties without rallies similar in many traits to counties with rallies

After conditioning on population (a confounder): no correlation.

Solutions to Confounding

Did Trump rallies increase hate crimes?

A design-based solution to confounding could be to compare:

  • counties that had a Trump rally to themselves, before and after the rally.

Before and After

Example: Before and After

POLL

  • What kinds of confounding variables are held constant in this before/after comparison?
  • What kinds of confounding might still bias this result?

Example: Before and After

Example: Before and After

Example: Before and After

Example: Before and After

Confounding Solved

All confounding variables (affect whether a rally occurs; affect hate crimes) that are unchanging over time are held constant.

  • because held constant, cannot produce confounding
  • e.g., demographic features, political leaning, location/geography, long-term economic trends, 8chan white nationalists
  • any variable that does not change in the time period of the comparison (in this case, one month) held constant
  • does not matter if we can measure confounders

Example: Before and After

Confounding UNSolved

  • Did Trump rallies take place in places that are already trending toward having more hate crimes?
  • Is it possible that Trump wanted to avoid controversy and pick places that had lower than usual hate crimes? (board)
  • We can address these concerns by looking at longer-term trends…

No unusual trends before the rally.

Example: Before and After

Confounding UNSolved

Because of over-time comparison, we can have confounding from variables that do not cause \(X\) to change, if they also change with \(X\) over time…

  • Does rally change measurement, but not actual number of hate crimes? (Measurement bias)

    • we can examine this concern by measuring hate crimes in other ways.
  • Are there are other changes over the same time-frame (change at the same time as \(X\), rallies)?

    • This is harder to solve

Design: Before and After

What is it?

Compare the same case to itself before and after change in \(X\)

How does it work?

Holds constant all unchanging attributes of the case.

  • any confounding variables that do not change over time cannot produce change in \(Y\) with change in \(X\)

Before and After: Assumptions

In order to infer \(X\) causes \(Y\) if \(X,Y\) correlated in before/after comparison

Must Assume

  1. There are no other confounding variables that affect \(Y\) and change with \(X\)

Before and After: Limitations

This assumption can be violated if…

  • Value of \(Y\) in cases has a long-term trend in one direction
  • \(X\) changes because of extreme changes in \(Y\) (e.g. gun laws respond to uptick in gun crimes)
  • \(X\) changes measurement of \(Y\)
  • other variables that affect \(Y\) change with \(X\) over time.

Solution How Bias
Solved
Which Bias
Removed
Assumes Internal
Validity
External
Validity
Experiment Randomization
Breaks \(W \rightarrow X\) link
All confounding variables 1. \(X\) is random
2. Change only \(X\)
Highest Lowest
Conditioning Hold confounders
constant
Only variables
conditioned on
1. Condition on all confounders
2. Low measurement error
Lowest Highest
Before and After Hold confounders
constant
variables
unchanging
over time
No confounders
change w/ \(X\)
Lower Higher

Alternatives

Trump’s Twitter and Hate Crimes