March 16, 2021

Testing Causal Claims

1. Fundamental Problem of Causal Inference

  • causal hypotheses
  • independent/dependent variables

2. Correlation

  • scatterplots
  • problems with correlation

Example

Trump Rallies and Hate Crimes

In February 2019, Donald Trump held a rally in El Paso, TX. Argued that migrants were dangerous.

  • “A wall is a very good thing, not a bad thing. It’s a moral thing”
  • “[reducing numbers of immigrants in detention would be] cutting loose dangerous criminals into our country.”
  • “The Democrat Party [is] becoming the party of socialism, late-term abortion, open borders and crime.

Trump Rallies and Hate Crimes

In August 2019, an armed man killed 22 people at a Walmart in El Paso, TX. In advance of his attack, he issued a manifesto that stated he was motivated in response to an alleged “Hispanic invasion of Texas.”

Trump Rallies and Hate Crimes

A causal claim:

“Trump’s rally in El Paso increased the likelihood of hate crimes against immigrants.”

In Groups…

  • what is the implied counterfactual claim?
  • If the causal claim were true, what would be the potential outcomes of hate crimes in El Paso in 2019?

Trump Rallies and Hate Crimes

Counterfactual claim:

“If Trump had not held a rally in El Paso (in 2019), then there would have been fewer hate crimes against immigrants.”

Potential Outcomes:

\(\mathrm{Hate \ Crimes}_{El \ Paso}(\mathrm{Rally}) >\) \(\color{red}{\mathrm{Hate \ Crimes}_{El \ Paso}(\mathrm{No \ Rally})}\)

\(\mathrm{Black}\) is factual; \(\color{red}{\mathrm{Red}}\) is counterfactual

Causal Hypotheses

We make causal claims testable by generating…

causal hypotheses (or empirical predictions):

statements about what we should observe if the causal claim is true.

Hypotheses state the potential outcomes implied by the causal claim. Put another way, what is implied relationship between independent (cause) and dependent (outcome) variables.

  • If \(X\) were present(absent), then \(Y\) would be more(less) likely
  • If \(X\) were to increase(decrease), then \(Y\) would increase(decrease)

Independent/Dependent Variables

Because causal hypotheses are observable, stated in terms of variables.

Independent variable:

The variable capturing the alleged cause in a causal claim.

  • often called the “IV” or “X” or “right-hand variable”

Dependent variable:

The variable capturing the alleged outcome (what is affected) in a causal claim.

  • often denoted as “DV” or “Y” or “left-hand variable”

Potential Outcomes are the values of dependent variable a case would take different values of the independent variable

Independent/Dependent Variables

“Trump rallies in a community increase the likelihood of hate crimes against immigrants.”

What could be an independent variable used to test this causal claim?

What could be a dependent variable used to test this causal claim?

An Example:

\(\mathrm{City}_i\) \(\mathrm{Rally}_i\) \(\mathrm{Hate \ Crimes}_i(\mathrm{Rally})\) \(\mathrm{Hate \ Crimes}_i(\mathrm{No \ Rally})\)
El Paso Yes > 1 ?

How would we find the “\(?\)”?

FPCI

Fundamental Problem of Causal Inference

For any case, we can only observe the potential outcome of \(Y\) for the value of \(X\) that the case is actually exposed to.

We can never observe the other, counterfactual potential outcomes of \(Y\) for different possible values of \(X\) that the case did not experience.

Causality is counterfactual, so for any specific case, we can never empirically observe whether \(X\) causes \(Y\).

FPCI

If Trump’s rally cause an increase hate crimes in El Paso, we need to see this:

\(\mathrm{Hate \ Crimes}_{El \ Paso}(\mathrm{Rally}) >\) \(\color{red}{\mathrm{Hate \ Crimes}_{El \ Paso}(\mathrm{No \ Rally})}\)

While the value in \(\mathrm{Black}\) is factual; The value in \(\color{red}{\mathrm{Red}}\) is counterfactual and can never be known.

Solving the FPCI

Solving the FPCI

We cannot observe: \(\color{red}{\mathrm{Hate \ Crimes}_{El \ Paso}(\mathrm{No \ Rally})}\)

But we can observe, e.g.: \(\mathrm{Hate \ Crimes}_{Austin}(\mathrm{No \ Rally})\)

If we assume: \(\mathrm{Hate \ Crimes}_{Austin}(\mathrm{No \ Rally})\) \(=\) \(\color{red}{\mathrm{Hate \ Crimes}_{El \ Paso}(\mathrm{No \ Rally})}\)

Then, we can test our causal claim, to see if:

\(\mathrm{Hate \ Crimes}_{El \ Paso}(\mathrm{Rally}) >\) \(\mathrm{Hate \ Crimes}_{Austin}(\mathrm{No \ Rally})\)

\(\mathrm{City}_i\) \(\mathrm{Rally}_i\) \(\mathrm{Hate \ Crimes}_i(\mathrm{Rally})\) \(\mathrm{Hate \ Crimes}_i(\mathrm{No \ Rally})\)
El Paso Yes \(\mathrm{Hate \ Crimes}_{El \ Paso}(\mathrm{Rally})\) \(\color{red}{\mathrm{Hate \ Crimes}_{El \ Paso}(\mathrm{No \ Rally})}\)
\(\mathbf{\Uparrow}\)
Austin No \(\color{red}{\mathrm{Hate \ Crimes}_{Austin}(\mathrm{Rally})}\) \(\boxed{\mathrm{Hate \ Crimes}_{Austin}(\mathrm{No \ Rally})}\)

\(\mathrm{City}_i\) \(\mathrm{Rally}_i\) \(\mathrm{Hate \ Crimes}_i(\mathrm{Rally})\) \(\mathrm{Hate \ Crimes}_i(\mathrm{No \ Rally})\)
El Paso Yes \(\mathrm{Hate \ Crimes}_{El \ Paso}(\mathrm{Rally})\) \(\boxed{\mathrm{Hate \ Crimes}_{Austin}(\mathrm{No \ Rally})}\)
\(\mathbf{\Uparrow}\)
Austin No \(\color{red}{\mathrm{Hate \ Crimes}_{Austin}(\mathrm{Rally})}\) \(\mathrm{Hate \ Crimes}_{Austin}(\mathrm{No \ Rally})\)

Solving the FPCI

Every solution to the FPCI involves:

  1. Comparing the observed values of outcome \(Y\) in cases that actually have different values of cause \(X\)

  2. Making assumptions that let us treat factual (observed) potential outcomes from some cases as counterfactual (unobserved) potential outcomes of other cases.

Correlation

Correlation is the degree of association/relationship between the observed values of \(X\) (the independent variable) and \(Y\) (the dependent variable)

  • There are formal mathematical definitions.
  • We use the term loosely to describe observed relationship between \(X\) and \(Y\)

Correlation

All empirical evidence for causal claims relies on correlation between the independent and dependent variables.

But, you’ve all heard this:

Correlation

  • Many different ways of doing this (compare group means, regression) are all fundamentally about correlation.
  • correlations have a direction:
    • positive: implies that as \(X\) increases, \(Y\) increases
    • negative: \(X\) increases, \(Y\) decreases
  • correlations have strength (has nothing to do size of effect):
    • strong: \(X\) and \(Y\) almost always move together
    • weak: \(X\) and \(Y\) do not move together very much

Correlation

Formally…

mathematically: correlation is the degree of linear association between \(X\) and \(Y\)

  • Takes values between \(-1\) and \(1\)
  • Values close to \(1\) or \(-1\) suggest high degree of linear association
  • Values close to \(0\) suggest low degree of linear association
  • Value of correlation does not tell us how much \(Y\) changes with \(X\)

Correlation

What is it?

negative correlation: (correlation \(< 0\)) values of \(X\) and \(Y\) move in opposite direction:

  • higher values of \(X\) appear with lower values of \(Y\)
  • lower values of \(X\) appear with higher values of \(Y\)

positive correlation: (correlation \(> 0\)) values of \(X\) and \(Y\) move in same direction:

  • higher values of \(X\) appear with higher values of \(Y\)
  • lower values of \(X\) appear with lower values of \(Y\)

Correlation