\(1\). Conditioning:

• What is it?
• How does it work?
• Assumptions?

\(2\). “Design Based” approach

• what is it?
• how does it work?

Must ask…

• What comparisons between cases does it involve?
  • can you recognize this solution when it is described to you?
  • can you describe how to use this solution to test a particular causal claim?
• What assumptions are required for it to work?
• How does it “solve” confounding?
• What trade offs do we make?

Experiment:

Does exposure to mass-shootings increase support for laws restricting access to guns?

What would an experiment look like?:

What confounding is removed?:

What is assumed?:

Internal vs External validity?

Example:

Does exposure to mass-shootings increase support for laws restricting access to guns? (McGinty et al 2013)

What would an experiment look like?: Recruit people into a survey. Randomly assign respondents to read/not read a news story about a mass shooting

What confounding is removed?: All confounding variables

What is assumed?: Random assignment, changing nothing other than \(X\)

Internal vs External validity?

Did Trump rallies cause hate crimes?

data from Feinberg, Branton, and Martinez-Ebers

Possible confounders imagined by Feinberg, Branton, and Martinez-Ebers

conditioning

when we observe \(X\) and \(Y\) for multiple cases, we examine the correlation of \(X\) and \(Y\) within groups of cases that are the same on confounding variables \(W, etc. \ldots\)

How does conditioning solve the problem?

• Cases compared have same values on confounding variable(s) \(W\)
• In these groups, \(W\) cannot affect \(X\) or \(Y\) (because \(W\) is not moving, it can’t move \(X\) or \(Y\))
• “Backdoor path” from \(X\) to \(Y\) is “blocked”

How does conditioning work?

Just kidding!

Feinberg, Branton, and Martinez-Ebers compare hate crimes in counties with and without Trump rallies, but condition on (hold constant):

• percent Jewish
• number of hate groups
• crime rate
• 2012 Republican vote share
• percent university educated
• region

Feinberg, Branton, and Martinez-Ebers find that, even after conditioning, Trump rallies increase the risk of hate crimes by 200%!

• Lots of news headlines like this

Economics PhD Candidates show that conditioning on the same variables…

• Clinton rallies increased hate crimes by nearly 250%!!

• What is going on here??

We would be wrong to use observed hate crimes in county \(b\) (without a rally) to substitute in for counterfactual hate crimes in county \(a\) (without a rally).

Population differences \(\to\) difference in hate crimes regardless of rally.

After also conditioning on population (a confounder): no correlation.

In order to use conditioning to infer \(X\) causes \(Y\) if \(X,Y\) correlated …

Must Assume

1. There are no other confounding variables (that you have not conditioned on)
   • i.e. you have conditioned on ALL confounding variables (backdoor paths)
   • sometimes called “ignorability assumption”: you assume you can “ignore” other variables without confounding

How can we tell whether this assumption is correct?

DISCUSS WITH YOUR NEIGHBORS

1. Assume there are no other confounding variables (that you have not conditioned on)

• this can never be proven to be true: a “strong” assumption
• we can’t know whether all confounders are blocked because we don’t know true causal graph
• compare this against the assumption of randomization in experiments

In wake of mass shootings, we might ask:

Do mass shooting events cause people to become more supportive of stricter gun control policies?

Newman and Hartman (2017) examine whether exposure to mass shootings cause an increase in support for stricter gun control

• \(Y\) large survey (CCES) asks: “In general, do you feel that laws covering the sale of firearms should be made more strict, less strict, or kept as they are?”
• \(X\) “distance to nearest mass shooting in recent years”

• What might be possible confounding variables?

Which variables do we NEED to condition on? Which variables do we NOT NEED to condition on?

Must Assume

1. There are no other confounding variables (that you have not conditioned on)

• i.e. you have conditioned on ALL confounding variables
• this can never be proven to be true: a “strong” assumption

But…

• don’t need to condition on ALL variables that might affect \(X\) or affect \(Y\): only confounders.

Which variables do we NEED to condition on? Which variables do we NOT NEED to condition on?

Must Assume

1. There are no other confounding variables (that you have not conditioned on)

• i.e. you have conditioned on ALL confounding variables
• this can never be proven to be true: a “strong” assumption

But…

• don’t need to condition on ALL variables that might affect \(X\) or affect \(Y\): only confounders.
• don’t need to condition on ALL variables on a backdoor path. Just one variable per backdoor path is needed.

Newman and Hartman (2017):

Compare proximity to mass shooting and gun control attitudes, conditioning on (holding constant:

• community level factors: income, education, partisanship, racial composition, murder, firearms stores, population density, population
• individual level factors: education, income, age, gender, race, property ownership, having children, in military, military family, partisanship, ideology, religiosity, region of birth

• Are there any potentially really important confounders that might not be included?

Proximity to mass shootings increases support for stricter gun laws… assuming no other confounders

Must Assume

1. There are no other confounding variables (that you have not conditioned on)

But…

• don’t need to condition on ALL variables that might affect \(X\) or affect \(Y\): only confounders.
• don’t need to condition on ALL variables on a backdoor path. Just one variable per backdoor path is needed.
• we can give reasons/arguments for why any remaining confounding is likely small OR biased against the correlation (small bias may not be enough to change our conclusions, or bias is against our conclusions)

In order to infer \(X\) causes \(Y\) if \(X,Y\) correlated after conditioning

Must Assume

2. Variables used to condition relationship between \(X\) and \(Y\) are measured without error.

• even random measurement error in confounding variables leads conditioning to not remove bias.
• Why? You are no longer comparing like-with-like.

Imagine we want to condition on gun ownership, when examining correlation of mass shootings and gun attitudes.

What if we measure gun ownership with random measurement error?

Let’s see what happens… BOARD

In order to infer \(X\) causes \(Y\) if \(X,Y\) correlated after conditioning

Must Assume

3. We can find cases that are the same on confounding variables \(W\) and different in \(X\).

Let’s say we want to examine the effect of laws restricting gun ownership on gun violence across countries:

What are some factors that might affect strictness of gun regulation and gun violence?

Is there, e.g., country that is similar to the US on all confounders?

In order to infer \(X\) causes \(Y\) if \(X,Y\) correlated after conditioning

Must Assume

3. There are cases that are the same on confounding variables \(W\) and different in \(X\).

• as we want to condition on more and more variables…
• less and less likely we find cases that exactly match
• so, cannot condition on all confounding variables

Did Trump rallies increase hate crimes?

1. If we know confounding variables, can we find cases with and without rallies that are the same on many confounding variables?

2. If we don’t know or can’t measure confounding variables, may still be differences between places with and without rallies that produce confounding.

• Is there a better comparison we can make?

We can make Before and After comparisons: What is it?

before and after: examine the change in \(Y\) in a single case (or group of cases) where \(X\) changes over time (from before to after)

Taking the same data from Feinberg, Branton, and Martinez-Ebers…

• we focus only on counties that ever had a Trump (Clinton) rally
• compare the month after the rally \((X = \text{Rally})\) to the month before the rally \((X = \text{No Rally})\)

DISCUSS

If we compare counties to themselves before and after rallies…

1. Which confounding variables are held constant?

2. What are confounding variables that might NOT be addressed in this comparison?

Conditioning removes confounding by:

• identify possible confounding variables
• measure confounding variables
• relationship b/t \(X\) and \(Y\) for cases with similar value of confounding variables \(W\).

Design-based solutions (like Before and After) remove confounding by:

• selecting cases for comparison in order to eliminate many known/unknown as well as measurable/unmeasurable confounding variables.
• the nature of the comparison holds constant classes of confounding variables, not specific confounding variables.
• by a “class” we mean all confounding variables that share certain properties

Which of these possible confounders are held constant in a before-and-after comparison?

How does it work?…

All confounding variables (affect whether a rally occurs; affect hate crimes) that are unchanging over time (before to) are held constant.

• because held constant, cannot produce confounding

Confounding Solved?…

• any variable that does not change in the time period of the comparison held constant

Conditioning:

• What is it?:
  • examine correlation of \(X\) and \(Y\) for cases that are the same on confounders \(W\)
• How does it work?:
  • Holds measured confounding variables constant
• Assumptions:
  • no other confounding variables: can’t check
  • no measurement error: can check
  • similar cases with different \(X\): can check
• While imperfect, we should not dismiss this approach.

Before and After:

• What is it?:
  • examine correlation of \(X\) and \(Y\) within cases where \(X\) changes over time
• How does it work?:
  • Holds all unchanging confounding variables constant
• Assumptions?