January 18, 2022

What is Science?

Plan for Today

Admin: Reflection 1 due Friday

1) Recap Weber’s insights

2) Attributes of scientific evidence

3) Example

Recap

Scientific Evidence

Two Key definitions

claim:

(For our purposes) a statement about what is true or valid.

  • If you can add “is true” to the end or “it is true that” to the beginning of the statement without changing the meaning, it is a claim.

Don’t overthink this definition. It is as simple as it seems.

Two Key definitions

basis:

(For our purposes) the basis for a claim is the reason we should accept the truth or validity of that claim. It includes

  • the evidence that is used to “prove” the claim is true

  • and the assumptions required for the evidence to be valid “proof” (sometimes called the “warrant”)

colloquially, we refer to both parts as “evidence”

An example:

claim: “It rained last night”

basis: “I can see the street is wet”


What is the evidence?

What are the assumptions behind that evidence?

Using Science to Evaluate Claims

Science is distinct as a form of thought in that:

(1) Only certain claims can be investigated

(next week)

(2) Has special rules for using evidence and making assumptions to evaluate claims

today

What is makes evidence
“scientific”?

Attributes of science?

Attributes of science?

Attributes of Scientific Evidence:

Systematic use of evidence

Transparent procedures

Acknowledge uncertainty

Test claim against alternatives

Transparent procedures

How did you arrive at your conclusion?

  • What data / observations did you use?
  • What comparisons did you make?
  • What choices as a researcher did you make, and why?

Why?

  • Others can know the assumptions required to find result
  • Others can challenge your choices/assumptions
  • Others can replicate your work (objectivity)

Consider alternatives

Test claim against other competing claims

  • Whichever claim “survives” many different tests is best

Why?

  • Openness to being wrong (no assumption above challenge)
  • Evidence consistent with different assumptions (objectivity)
  • One piece of evidence can be consistent with many claims; Best claim generates most useful predictions

Systematic Use of Evidence

Observations we make:

  • Clear rules for what we observe
  • Clear rules for how we observe
  • Clear rules for comparison
  • Avoid “cherry-picking”

Why?

  • Cherry picking assumes a truth, picks data to support it
  • Systematic rules allows others to investigate for themselves
    • No “secret sauce”: (objectivity, enables challenging assumptions)

Acknowledge uncertainty

Limitations of finding:

  • What questions remain unanswered after the study (what didn’t it tell us)?
  • What assumptions were made that might not be true?

How certain or precise are the answers?

  • Acknowledge possibility of results being driven by chance
  • Acknowledge possibility of spurious relationships

Poll

Which of these four do you think is the most important part of scientific evidence?

Example

Consider a claim:

A person’s prejudice against a group of people can be reduced through meaningful, interpersonal contact with a member of that group.

  • This is the “contact hypothesis”, famously articulated by Allport (1954)

A specific question:

Does contact reduce prejudice?

Are efforts to change the minds of opponents same-sex marriage through a short conversation more successful when those conversations are with homosexuals (as compared to heterosexuals)?


Poll

An investigation

2014 paper in Science:

Researchers conducted an experiment to answer this question:

What would you want to know…

about Transparent Procedures?

Treatments:

Who was treated?

How did they get treated?

What were the survey questions?