1) What makes for “unscientific” evidence?

2) What are some examples of “unscientific” evidence?

3) Objective:

• be able to identify unscientific evidence
• explain why it is unscientific

Transparent procedures

Systematic use of evidence

Test claim against alternatives

Acknowledge uncertainty

All [instances of scientific evidence] is alike; each [instance of unscientific evidence] is [unscientific] in its own way.

Despite an infinite number of topics for research:

• Scientific evidence shares similarities in how it adheres to the four attributes.
• Unscientific evidence can fail to meet four criteria in many different ways.

1. What is the guest’s argument (claim)?

2. What is the primary evidence he offers to support it?

3. How does this evidence fail to meet the attributes (of scientific evidence)?

Appeals to personal experience:

A claim based on one’s own personal (nonsystematic) observation or one’s own reaction to an observation

Personal experience can be misleading:

• We generalize too quickly from a small number of cases (acknowledge uncertainty)

• Our exposure to the world may be skewed (systematic use of evidence)

• We observe the world selectively (consider alternatives; systematic use of evidence)

Claim

“Masks don’t help prevent COVID”.

Evidence

“I went into a crowded store with an N95 mask and got COVID”

Claim:

Tanks are a deathtrap now. With neither side having air superiority, you’re left with infantry & artillery – essentially WW1.

— Elon Musk (@elonmusk) January 7, 2023

Implicitly: there is no point in providing tanks to Ukraine

Evidence

Lots of videos on Twitter of tanks being destroyed by drones…

…and this:

Claim

“Taking mass transit is more dangerous than driving.”

Evidence

“Terrorist attacks and mass shootings have targeted mass transit”

This does not mean:

• evidence from personal/lived experience is always wrong
• evidence from personal/lived experience cannot be important/informative

1. What is his main argument (claim)?

2. What is the primary evidence he offers to support it?

3. How does this evidence fail to meet the four attributes (of science)?

Appeal to common sense

• argue that a claim is true because it is something “everyone knows” or “just makes sense”

Problems with common sense

• Often based on analogy emphasizing superficial similarities between very different situations (systematic use of evidence)

• Often common sense reasons against the claim (consider alternatives)

Claim

Counterclaim

1. What is Miller’s main argument (claim)?

2. What is the primary evidence he offers to support it?

3. How does this evidence fail to meet the four attributes (of science)?

Appeal to authority

Arguing that claim is true because a person with authority says it is true.

1. Experts may be wrong/expertise irrelevant (Transparent Procedures)
2. Experts may have an agenda (Transparent Procedures)
3. We can “cherry pick” supportive experts (systematic use of evidence)
4. Experts can disagree (acknowledge uncertainty,consider alternatives)

Claim:

“As far as the cyber … I don’t think anybody knows it was Russia that broke into the DNC. I mean, it could be Russia, but it could also be China. It could also be lots of other people. … So we have to get very, very tough on cyber and cyber warfare. It is — it is a huge problem. … The security aspect of cyber is very, very tough. And maybe it’s hardly doable.” - Donald Trump

Evidence:

I have a son. He’s 10 years old. He has computers. He is so good with these computers, it’s unbelievable.

Claim:

There is a “hidden variable” that explains the apparent randomness in quantum mechanics.

Evidence:

“God does not play dice” - Albert Einstein

Claim:

“You should take hydrocodone or oxycodone to manage your pain.”

Evidence

“I am a doctor.”

• Undisclosed: The maker of those drugs provides me free lunches; pays for my conference travel (see here)

Claim:

Fraud in 2020 US Presidential Election

Evidence

Hot off press, new Lott study estimates 11,350 absentee votes lost to @potus Trump in Georgia. Another 289,000 "excess (fraudulent) votes" across GA, AZ, MI, NV, PA, WI. As promised this a.m. on Bannon's War Room. Postpone GA Cesspool election!! https://t.co/geqEhnKqCR

— Peter Navarro (@RealPNavarro) December 29, 2020

In a recent paper @JohnLottJr purports to show evidence of fraud in GA and PA in the 2020 election. In a new paper, we (@aeggers, Haritz Garro, and me) show that Lott’s claims are based on a fatal flaw. Once corrected, there is no evidence for fraud. https://t.co/yVtTIdBC0Q pic.twitter.com/dYwQazZICj

— justingrimmer (@justingrimmer) January 4, 2021

What do we do when some experts say one thing, and others say another?

• Without features of scientific evidence, no basis for distinguishing between experts.
• We can choose whatever expert/study we want (becomes theology)

1 hand: yes 0 hands: no 2 hands: I don’t know

Evidence for a claim that references a “study” that:

• does not address the procedures of that study
• does not discuss the broader body of research on the topic

uses science as an appeal to authority…

Studies Show That You Should Come Into This Room Right Over Here, And That You Should Come Alone

Recent studies suggest that it’s very important that you come into this room right over here just now. Studies also suggest that it’s equally important you do not waste time asking questions … but just step this way through the door very quickly and by yourself, immediately.

The studies were conducted, and the evidence is conclusive: this room just over here is where you should go next, without anyone coming with you. Scientists did them, the studies. Which ones? Yes.

The studies are all finished now, and you don’t need to see them. The studies suggested that it’s not important you see the studies at all, but it is important that you hurry along, quickly now, because time is of the essence. The studies say now is the best time for you to be alone in that room.

Beware:

• Appeals to the authority of “science”/“researchers”/“studies”
• This changes science into magic

Does not mean distrust scientists/experts, but instead…

• credentials may be useful, but cannot be the only basis of trust
• does their work have transparency, systematic evidence, consideration of alternatives, acknowledgement of uncertainty?
• what are the assumptions they make?

It is reasonable and scientific to interrogate the evidence and assumptions the are used to support a claim:

• ordinary people should ask questions, but need the skills to do so
• scientists should answer those questions, but need the skills to do so

1. Many ways for evidence to be unscientific
2. Comes down to not meeting four attributes of scientific evidence
3. You get better at recognizing when this happens

• In this course, we introduce you to common problems that arise when we present evidence for claims, and the possible solutions