• Evaluating Descriptive Claims
• Concepts

(1) types of descriptive claims \(\to\) diagnostic frames

(2) Toolbox: How to judge is descriptive claim correct or not?

(3) What are concepts?

• when are concepts suitable for scientific evidence?
• when do concepts fail weak severity?

1. “Democracy is a type of political regime.”

2. “Russia is a democracy.”

3. “58% of countries worldwide are democracies.”

4. “Countries in Western Europe are more democratic than those in Eastern Europe.”

5. “Countries around the world have become less democratic since 2016.”

6. “Countries experiencing higher inflation have had greater declines in democracy.”

Why do we care about using science to assess descriptive claims?

• they are often claims about whether there is a problem, what is the nature of the problem (diagnostic framing)
• claims about relevance of a particular value judgment
• evaluating causal claims requires us to evaluate descriptive claims

https://www.instagram.com/reel/DU81DvkDe7r/

How can we decide which claim about “threat to democracy” is right?

“Countries around the world have become less democratic since 2016.”

What would you want to know before you could judge whether this claim was correct or incorrect?

“Countries around the world have become less democratic since 2016.”

We need a toolbox for scientific evidence:

• tools to judge the severity of evidence (is this evidence capable of finding the claim to be wrong)
• tools for identifying what can go wrong with evidence (so we can know whether these risks have been addressed)

You can use these tools when deciding whether to accept claims, or when making your own claims

When we evaluate evidence for descriptive claims, we want to think about severity

• can the evidence actually show claims to be wrong?

or conversely…

• what are the ways in which evidence may erroneously lead us to accept claims that are false?

Claim: “House is not on fire” Evidence: smoke detector

Concepts: define our terms in a way that is transparent and can be used systematically. If concepts are opaque or idiosyncratic \(\to\) STOP!

Variables: translate concepts into something that we can (in principle) observe. If variables do not correspond to the concept / correspond to other concepts \(\to\) STOP!

Measurement: devise transparent and systematic procedures with known uncertainty to observe those attributes of specific cases. If measurement procedure is opaque, likely to suffer from bias, or has high degree of uncertainty \(\to\) STOP!

A useful definition:

In social science we discuss “cases”, not in the legal sense, but in this sense:

case:

a specific individual, organization, entity, event, or action, existing in a specific time and place.

We are often interested in identifying what general categories this specific case belongs to, what is its “type”. Or measuring attributes of that case. (How much of something it has, e.g.)

What cases are relevant to these claims?

1. “Democracy is a type of political regime.”

2. “Russia is a democracy.”

3. “58% of countries worldwide are democracies.”

4. “Countries in Western Europe are more democratic than those in Eastern Europe.”

5. “Countries around the world have become less democratic since 2016.”

6. “Countries experiencing higher inflation have had greater declines in democracy.”

At a general level:

concepts: abstract or general categories that we (humans) apply to particular cases/instances. They abstract away from the highly particular, complex, and often unique features of reality.

Our thought and our language is rooted in concepts!

• concepts are labels with definitions

How would you define what a “chair” is?

Ludwig Wittgenstein, Philosophical Investigations

Takeaway: definitions/concepts are…

1. incomplete: they don’t perfectly give us rules to cover all possible situations
2. about family resemblances: there isn’t one unique element of “chairness” that all chairs share, but they are somehow related
3. homogenizing: concepts lump together diverse things into one category

• But can’t we develop fully precise concepts without these problems?

On Exactitude in Science

“… In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast Map was Useless, and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and Winters. In the Deserts of the West, still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in all the Land there is no other Relic of the Disciplines of Geography.”

• Jorge Luis Borges

Why do we need them?

Without concepts/abstraction:

• all experiences we have are unique
• we cannot anticipate regularities/similarities/patterns in the world
• we cannot predict what will happen next

But… abstraction comes at a price

Our concepts may abstract too much, with possibly bad consequences.

• Imposing abstract concepts onto reality (Forests)
• Reality pushes back against abstraction (Boundaries of countries)

… are imperfect abstractions from reality, but we need them.

How do we develop scientific concepts?

If I claim…

“Countries around the world have become less democratic since 2016.”

And I tell you that my definition of “democracy” is”

“I know democracy when I see it”

Can this be “scientific evidence”?

If I claim…

“Countries around the world have become less democratic since 2016.”

And I tell you that my definition of “democracy” is

“A democracy is country where leaders are chosen by free and fair elections and everyone is equal before the law, or the country is the United States.”

Can this be “scientific evidence”?

If I claim…

“Countries around the world have become less democratic since 2016.”

And I tell you that my definition of “democracy” is”

“A democracy is a country where the leader implements the Will of the People.”

Can this be “scientific evidence”?

(1) Concepts need to be transparent

clear and accessible definition. Even if we disagree over the label, we can all assess whether a case meets the definition.

(2) Concepts must be formulated to be used systematically

Science is about systematic evidence. Cannot define concepts that include loopholes for one case. Definitions must be tied to attributes that are observable

(3) Concepts should be useful:

helpful in understanding patterns

If concepts are not systematic, not transparent…

evidence may not be able to show claims are wrong (fails weak severity)…

• if person making claims won’t even define terms
• if definitions appeal to loopholes (cherry picking to get desired answer)
• if definitions do not appeal to observable traits (how can someone replicate evidence for themselves?)

concepts:

are abstract or general categories that we apply to particular cases using a set of rules/criteria that determine membership in the category.

For concepts to be useful in testing claims scientifically…

• defining traits must be something we can all observe (empirical). (e.g., democracy cannot be defined by an ethereal ‘Democratic Spirit’)
• the traits are about what it means to be in this category. “To be a democracy is to have \(x,y,z\) traits.”

For concepts to be useful in building theories (not our focus):

• traits are relevant to predicting

erythrocracy:

• definition: a state in which the flag/color for the ruler is red
• defining trait is observable
• traits tell us what it “means to be” erythrocracy
• but, fairly useless concept when it comes to prediction…

• Nevertheless, could test claims about whether erythrocracies are different in some way

Countries with flags that are \(50\%\) red:

Claim: Countries around the world have become less democratic since 2016.

• How do we define “democracy”?
• What definition is Trump using?
• What definition is the Center for American Progress using?

Claim: Countries around the world have become less democratic since 2016.

• Evidence: No, the number of democratic states has not changed since 2016. Of 193 UN member states, 9 were democratic in 2016 and are all still democratic today

• People’s Democratic Republic of Algeria
• Democratic Republic of the Congo
• Democratic Republic of Timor-Leste
• Federal Democratic Republic of Ethiopia
• Democratic People’s Republic of Korea
• Lao People’s Democratic Republic
• Federal Democratic Republic of Nepal
• Democratic Republic of São Tomé and Príncipe
• Democratic Socialist Republic of Sri Lanka

a (minimalist) definition of democracy:

A democracy is a government in which political decisions are made by people who acquire power through competitive elections, the results of which are respected (losers leave office).

V Dem definition of democracy:

Countries where elections are free, fair, and reoccurring; elected officials wield political power de facto; suffrage is universal; political parties and candidates can form freely and compete; the environment around elections make for a reasonably level playing field with speech, media, and civil society freedom; there are constraints on the executive by the legislature and the judiciary; and a strong rule of law ensuring that equal civil liberties are respected.

data description

“Countries around the world have become less democratic since 2016.”

Different concepts lead to different variables, different measures, different answers.

Even if we disagree that these are definitions of “democracy”, we can still evaluate the claim using these definitions.

• What if we defined democracy as “I know it when I see it”?
• What if we defined democracy as “Countries where the government enacts the general will”?

Which definition of “democracy” is right?

1. Kim Jong Un’s: “a state with ‘democratic’ in its name.”

2. Minimal: “government in which political decisions are made by people who acquire power through competitive elections, the results of which are respected.”

3. Varieties of Democracy:

Countries where elections are free, fair, and reoccurring; elected officials wield political power de facto; suffrage is universal; political parties and candidates can form freely and compete; the environment around elections make for a reasonably level playing field with speech, media, and civil society freedom; there are constraints on the executive by the legislature and the judiciary; and a strong rule of law ensuring that equal civil liberties are respected.

Which definition of “democracy” is right?

Each definition could be used in scientific evidence

• defining traits are about “what it is to be a democracy”
• traits are observable
• though, not all definitions relevant to prediction (elaborate)

Choice of which should be labeled “democracy” reflects value judgment and common usage

• e.g., from the perspective of physics, the name “electron” is not relevant to how we define the properties electrons.

Insight:

Depending on the questions we ask, values we espouse, etc., there are many different definitions for the same words.

Even if we disagree with definition for a specific word, good scientific concepts can be understood and used by others regardless of whether they agree with our definitions.

Kim Jong Un may insist that “a state with ‘democratic’ in its name” is a “democracy”.

But the point of contention is the term “democracy”

a definition of regime type 1.a:

regime type 1.a is a government in which political decisions are made by people who acquire power through competitive elections, the results of which are respected (losers leave office).

Kim Jong-un would have to accept that the Democratic People’s Republic of Korea does not meet this definition empirically:

“a government in which political decisions are made by people who acquire power through competitive elections, the results of which are respected (losers leave office).”

even if he rejected that this is the definition of “democracy”

1. To evaluate descriptive claims, we need to define concepts

2. Scientific concepts are systematic definitions about what relevant and observable traits makes something an “X”.

3. Scientific concepts are “objective” in the sense they can be used even if we disagree with them

   • “subjective” in that value judgements may make us prefer one definition of a word over another.