February 4, 2019
These are closely-related concepts, but remember:
\(1\). Variables are not causes of or caused by the concept. They observable indicators of belonging to the concept.
These are closely-related concepts, but remember:
\(2\). Variables and measures are both about what is observable, but variables must be general (not refer to values they take for specific cases, nor the procedures for observing them) where as measures must be about procedures for observing specific values
is a difference between the true value of a variable for a case and the observed value from the measurement procedure.
\[Value_{true} - Value_{observed} \neq 0 \xrightarrow{then} measurement \ error\]
bias or systematic measurement error: error produced when our measurement procedure obtains values that are, on average, too high or too low (or, incorrect).
random measurement error: errors that occur due to random features of measurement process or phenomenon and the values that we measure are, on average, correct
Validity/Reliability can fail on variable \(\leftarrow\) measure link
Bias | Random Error | |
---|---|---|
Conceptual Problem | Validity | Reliability |
Pattern | Errors are systematic (deviate from truth, on average) |
Errors are random (correspond to truth, on average) |
When it's OK | If it is UNIFORM across cases | If false negative better than false positive |
When it's Not OK | If it is different across cases/ we want absolute quantities |
If we need precision/ have few cases |
Solved by more data? | No, bias persists. | Yes, random errors "wash out" |
Given the claim: "German communities with more Nutella-followers on Facebook have more anti-refugree violence.", a false negative, incorrectly concluding that the claim is wrong, can be preferable to a false positive, incorrectly concluding that the claim is right.
Often, in social science, we prefer to wrongly conclude that there are no differences between groups than to wrongly conclude that there is a difference.
Random measurement error (e.g., getting the wrong value of Nutella Facebook followers due to randomness in who makes their location visible on Facebook) leads to false negatives, because differences between groups that we compare are harder to detect.
It consistently deviating from the true value of \(0\)
There are many large errors, but, on the whole, \(X2\) is centered on the true value of \(0\)
(\(1\)) Researcher subjectivity/interpretation - Researcher systematically over-weights, under-weights dimension of concept
Expert interviewers assess "political knowledge". Might overweight language skills in measure of political knowledge
\(\xrightarrow{Downward \ Bias}\) political knowledge of people who have less grasp of language of interview
(\(2\)) Obstacles to observation
Researcher subjectivity:
Obstacles to observation:
Instead:
If truly random: errors cancel out with many trials
We would have to observe too many cases.
"Most Americans prefer a ban on semi-automatic firearms."
We can't interview all Americans…
Is \(1500\) people enough?
population: full set of cases (countries, individuals, etc.) we're interested in learning about
sample: subset of the population that we observe and measure
inference: description of the population we make based on a sample
Measuring attitudes on gun control in the US:
The population:
The sample:
The inference:
For sampling to work, we need to
To do get both we need:
random sampling: sampling cases from the population in a manner that gives all cases an equal probability of being chosen
If we wanted to know: what is the average commuting time for students in this course?
population: all students in this class
sample: students in this class who are present during the last 2 minutes of Friday's lecture
The difference between the value of the measure for the sample and the true value of the measure for the population
sampling error can sometimes be measurement error, but it is not always measurement error.
sampling error is measurement error if your variable is the value of some population (e.g. mean attitudes in a province) and you get a sample that does not correctly reflect the population.
It is not measurement error if the value you are interested in the response to some variable on a survey.
Measurement Error:
Sampling Error: