Hypothesis Testing

STAT 20: Introduction to Probability and Statistics

Agenda

  • Concept Questions
  • Review of Hypothesis Testing
  • Break
  • Worksheet: Hypothesis Tests
  • Lab 3 Work Time

Concept Questions

Which of the following statements below represents claims that correspond to a null hypothesis (as opposed to an alternative hypothesis)?

Hint: try to write them using parameters (statements about means / proportions / etc)

A. King cheetahs on average run the same speed as standard spotted cheetahs.

B. For a particular student, the probability of correctly answering a 5-option multiple choice test is larger than 0.2 (i.e., better than guessing).

C. The mean length of African elephant tusks has changed over the last 100 years.

D. The risk of facial clefts is equal for babies born to mothers who take folic acid supplements compared with those from mothers who do not.

E. Mean birth weight of newborns is dependent on caffeine intake during pregnancy.

F. The probability of getting in a car accident is the same if using a cell phone than if not using a cell phone.

01:00

A pharmaceutical company developed a new treatment for eczema and performed a hypothesis test to see if it worked better than the company’s old treatment. The P-value for the test was \(10\%\). Which one of the following statements are true?


A. The probability that the null hypothesis is false is \(10\%\).

B. The P-value of about \(10\%\) was computed assuming that the null hypothesis was true.

C. The new drug is significantly better than the old.

D. The alternative hypothesis is 10 times more likely the null.

01:00

Hypothesis Tests

  • We have two models of how the world works, a Null Hypothesis and an Alternative Hypothesis
  • Null is our default assumption, usually that “nothing interesting is going on”
  • Alternative is some more exciting option we want to test if it could be true
  • How it works: the data needs to convince me to abandon my null hypothesis with evidence
  • If the data seems incredibly unlikely under the null hypothesis, I reject the null
  • If the data seems somewhat likely under the null hypothesis, I fail to reject the null

Test Statistic

  • I compute some statistic from the sample in front of me
  • Under the assumption that the null is true, I ask what the sampling distribution of that statistic would look like
  • AKA under resampling from the null, what are typical values of the statistic and the probabilities of those values
  • I compare the test statistic I computed to the sampling distribution under the null

P Value

  • P Value: probability under null of seeing a statistic as or more extreme than the one I observed
  • Always between 0 and 1.
  • Small p value (like 0.01) means under null, seeing a value like this has 1% chance, so reject the null
  • Statistic looks like it did not come from null
  • Large p value (like 0.3) means data reasonably could have come from null, so I fail to reject the null

Small vs Large P Value

Permutation Test

  • I have two groups, A and B
  • B has been given some stimulus, A has not
  • Compute a statistic for A and B, look at difference \(\hat{s}_{B} -\hat{s}_{A}\)
  • Null Hypothesis: Stimulus independent from observed statistic, so \(E[\hat{s}_{B}-\hat{s}_{A}]=0\)
  • Alternative Hypothesis: Group B should have higher expected value than A
  • If null was true, if I shuffle who is in which group, expected difference is 0 and variance is from sampling variation
  • Is my observed statistic MORE EXTREME than what I would expect from resampling the null?

Break

05:00

Worksheet: Hypothesis Tests

  • Today’s problem set is all about answering one question:

Is yawning contagious?

What do you think?

Answer at pollev.com.

WS: Hypothesis Tests

What do you think about the of the experiment in the video?

  • Discuss with your group, identifying anything that seems amiss.
01:00

WS: Page 1

20:00

WS: Page 2

20:00

Do you still think yawning is contagious?

Answer at pollev.com.

Break

05:00

Lab 3 Work Time