September 25 2024

Evaluating Descriptive Claims

Outline

  • Assignments
  • Evaluating Descriptive Claims
  • Concepts

Assignments:

  • Assignment 1

    • Recognize when people are attempting to motivate you to do something
    • Recognize what kinds of claims they are using
    • Show us you know can distinguish between claims
    • use of Vox articles is optional; just a way to come up with contemporary political issues
    • Don’t quote a claim directly; paraphrase it to make the claim more clear.

Objectives

(1) types of descriptive claims

(2) How to judge: is descriptive claims correct or not?

(3) What are concepts? How do we make them?

Evaluating Claims

Evaluating Claims

We’ve already:

  1. Seen how claims are embedded in reasons/justifications given to us to act in particular ways

  2. Seen how to distinguish between types of claims

  3. Seen how science gives us principles of evidence: severity, falsification

Evaluating Claims

But we saw that falsification is hard:

If we find evidence that appears to falsify a claim, it could be…

  • claim is false
  • theories linking claims to empirical predictions are wrong
  • assumptions of evidence (warrants) are wrong

Evaluating Claims

In the remainder of the course, we will identify:

  • what are the kinds of assumptions that are required for common evidence for descriptive/causal claims?
  • is the evidence we are given capable of showing the claim to be false?
  • what are common ways that these assumptions can be wrong?
  • what are forms of evidence that make stronger (less plausible) vs weaker (more plausible) assumptions?

A toolbox

Think of this as a toolbox:

  • tools to judge the severity of evidence
  • tools to identify where evidence may not be convincing
  • tools to suggest what kinds of evidence may be more convincing

You can use these tools when deciding whether to accept claims, or when making your own claims

Descriptive Claims

Descriptive Claims:

Why do we care about using science to assess descriptive claims?

  • they are often claims about whether there is a problem, what is the nature of the problem
  • claims about relevance of a particular value judgment
  • evaluating causal claims requires us to evaluate descriptive claims

Descriptive Claims:

Descriptive Claims:

Descriptive Claims:

“Countries around the world have become less democratic since 2016.”

Descriptive Claims:

descriptive claims:

claims about what exists (or has existed/will exist) in the world:

  • what phenomena exist (what kinds of things exist?)
  • what is the type of a specific phenomenon (what is this thing?)
  • amount/frequency of phenomena (how much of something is there?)
  • relative amount/frequency of phenomena across different places/times (how much of something is there here vs. there/now vs. then?)
  • what patterns are there in the shared appearance/non-appearance of different phenomena (does this thing usually appear together with that other thing?)

Descriptive Claims: Varieties

  1. “Democracy is a type of political regime.”

  2. “Russia is a democracy.”

  3. “58% of countries worldwide are democracies.”

  4. “Countries in Western Europe are more democratic than those in Eastern Europe.”

  5. “Countries around the world have become less democratic since 2016.”

Descriptive Claims: Evaluation

What would you want to know before you could judge whether this claim was correct or incorrect?

“Countries around the world have become less democratic since 2016.”

Descriptive Claims

When we evaluate evidence for descriptive claims, we want to think about severity

  • can the evidence actually show claims to be wrong?

or conversely…

  • what are the ways in which evidence may erroneously lead us to accept claims that are false?

Descriptive Claims: Evaluation

Concepts:

define our terms in a way that is transparent and can be used systematically. If concepts are opaque or idiosyncratic \(\to\) STOP!

Variables:

translate concepts into something that we can (in principle) observe. If variables do not correspond to the concept / correspond to other concepts \(\to\) STOP!

Measurement:

devise transparent and systematic procedures with known uncertainty to observe those attributes of specific cases. If measurement procedure is opaque, likely to suffer from bias, or has high degree of uncertainty \(\to\) STOP!

Evaluating Descriptive Claims:

A useful definition:

In social science we discuss “cases”, not in the legal sense, but in this sense:

case:

a specific individual, organization, entity, event, or action, existing in a specific time and place.

We are often interested in identifying what general categories this specific case belongs to, what is its “type”. Or measuring attributes of that case. (How much of something it has, e.g.)

Cases

  1. “Democracy is a type of political regime.”

  2. “Russia is a democracy.”

  3. “58% of countries worldwide are democracies.”

  4. “Countries in Western Europe are more democratic than those in Eastern Europe.”

  5. “Countries around the world have become less democratic since 2016.”

Concepts

Concepts

At a general level:

concepts: abstract or general categories that we (humans) apply to particular cases/instances. They abstract away from the highly particular, complex, and often unique features of reality.

Our thought and our language is rooted in concepts!

What is a chair?

What is a chair?

What is a chair?

What is a chair?

What is a chair?

What is a chair?

What is a chair?

Ludwig Wittgenstein, Philosophical Investigations

On Exactitude in Science

“… In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast Map was Useless, and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and Winters. In the Deserts of the West, still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in all the Land there is no other Relic of the Disciplines of Geography.”

  • Jorge Luis Borges

Concepts

Why do we need them?

Without concepts/abstraction:

  • all experiences we have are totally unique
  • we cannot anticipate regularities/similarities in the world
  • we cannot predict what will happen next
  • we cannot function/act

Concepts

But… abstraction comes at a price

Our concepts may abstract too much, with possibly bad consequences.

  • Forests
  • Boundaries of countries: e.g. Syria

Conceptual Limits

Conceptual Limits

Concepts

are imperfect abstractions from reality, but we need them.


How do we develop scientific concepts?

Concepts and Science:

Building on our insights about/criteria for science:

(1) Concepts need to be transparent:

clear and accessible definition. Even if we disagree over the label, we can all assess whether a case meets the definition.

(2) Concepts must be formulated to be used systematically

Science is about systematic evidence. Cannot define concepts that include loopholes for one case. Definitions must be tied to attributes that are observable

(3) Concepts should be tied to prediction (not strictly necessary to evaluate claims)

science is fundamentally about prediction, finding regularities. Concepts that do not bring together cases that have some shared behavior/action do not help us understand.

Concepts and Science:

If concepts are not systematic, not transparent…

very difficult for evidence to show claims are wrong…

  • if person making claims won’t even define terms
  • if definitions appeal to loopholes (cherry picking to get desired answer)
  • if definitions do not appeal to observable traits (how can someone replicate evidence for themselves?)

concepts:

are abstract or general categories that we apply to particular cases using a set of rules/criteria that determine membership in the category.

For concepts to be useful in testing claims scientifically…

  • defining traits must be something we can all observe (empirical). (e.g., democracy cannot be defined by an ethereal ‘Democratic Spirit’)
  • the traits are about what it means to be in this category. “To be a democracy is to have \(x,y,z\) traits.”

For concepts to be useful in building theories (not our focus):

  • traits are relevant to predicting how cases described by the concept affect other things or are affected by other things. Relevant to ordinary use of the word

An example:

erythrocracy:

  • definition: a state in which the flag/color for the ruler is red
  • defining trait is observable
  • traits tell us what it “means to be” erythrocracy
  • but, fairly useless concept when it comes to prediction
  • Nevertheless, could test claims about whether erythrocracies are different in some way

Countries with flags that are \(50\%\) red:

An example: Democracy?

Claim: Countries around the world have become less democratic since 2016.

Answer: No, the number of democratic states has not changed since 2016.

  • Of 193 UN member states, 9 were democratic in 2016 and are all still democratic today

An example: Democracy?

Countries around the world have become less democratic since 2016.

  • country: a sovereign state that is recognized by the United Nations
  • democracy: a state that has “democracy” or “democratic” in its chosen, formal name.

An example: Democracy?

  • People’s Democratic Republic of Algeria
  • Democratic Republic of the Congo
  • Democratic Republic of Timor-Leste
  • Federal Democratic Republic of Ethiopia
  • Democratic People’s Republic of Korea
  • Lao People’s Democratic Republic
  • Federal Democratic Republic of Nepal
  • Democratic Republic of São Tomé and Príncipe
  • Democratic Socialist Republic of Sri Lanka

An Example: Democracy?

a (minimalist) definition of democracy:

A democracy is a government in which political decisions are made by people who acquire power through competitive elections, the results of which are respected (losers leave office).

You decide…

Scientific Concepts: An Example

Which definition of “democracy” is right?

  1. Kim Jong Un’s: “a state with ‘democratic’ in its name.”

  2. Minimal: “government in which political decisions are made by people who acquire power through competitive elections, the results of which are respected.”

  3. Our class definition

Scientific Concepts: An Example

From the perspective of evaluating descriptive claims, scientifically…

Scientific Concepts: An Example

Which definition of “democracy” is right?

Each definition could be used in scientific evidence

  • defining traits are about “what it is to be a democracy”
  • traits are observable
  • though, not all definitions relevant to prediction (elaborate)

Choice of which definition ought to be labeled “democracy” reflects value judgment and common usage

  • e.g., from the perspective of physics, the name “electron” is not relevant to how we define the properties electrons.

Concepts and Science:

Insight:

Depending on the questions we ask, values we espouse, etc., there are many different definitions for the same words.

Even if we disagree with definition for a specific word, good scientific concepts can be understood and used by others regardless of whether they agree with our definitions.

Concepts and Science:

Kim Jong Un may insist that “a state with ‘democratic’ in its name” is a “democracy”.

But the point of contention is the term “democracy”

a definition of regime type 1.a:

regime type 1.a is a government in which political decisions are made by people who acquire power through competitive elections, the results of which are respected (losers leave office).

Concepts and Science:

Kim Jong-un would have to accept that the Democratic People’s Republic of Korea does not meet this definition empirically:


“a government in which political decisions are made by people who acquire power through competitive elections, the results of which are respected (losers leave office).”


even if he rejected that this is the definition of “democracy”

Concepts:

  1. Concepts are abstractions from reality

  2. To answer descriptive claims, we need to define concepts

  3. Scientific concepts are about what relevant and observable traits makes something an “X”.

  4. Scientific concepts are “objective” in the sense they can be used even if we disagree with them

    • “subjective” in that value judgements may make us prefer one definition of a word over another.