MaizePrepSTATS 250
Exam 1 · Lec 04

Simulation-based hypothesis tests

The four steps of a test, the null distribution as a what-if picture, simulated p-values, and the course's strength-of-evidence scale.

25 min

Key ideas

Four steps

(1) Ask a question: define the parameter and state H0H_0, HaH_a. (2) Collect data from a random sample. (3) Analyze: build the null distribution, place the statistic, find the p-value. (4) Conclude in context with the strength of evidence.

Hypotheses

H0H_0 always has an equals sign and names a parameter value; HaH_a uses >>, << or ≠\ne with the same number. No hats: hypotheses are about parameters.

Null distribution

A what-if picture of π^\hat\pi values if H0H_0 were true. It is centered at π0\pi_0, and each dot is one simulated repetition of the whole study.

p-value

The probability, assuming H0H_0 is true, of a statistic as extreme as or more extreme than the one observed, in the direction of HaH_a. Simulated p-value = fraction of simulated statistics in that region.

Strength of evidence

p>0.10p > 0.10: not enough evidence; 0.05<p≤0.100.05 < p \le 0.10: some; 0.01<p≤0.050.01 < p \le 0.05: strong; p≤0.01p \le 0.01: very strong evidence against H0H_0 and for HaH_a. STATS 250 does not use a single 0.05 cutoff.

The course evidence scale (printed on the help card)

p-valueWe say we have
p>0.10p > 0.10not enough evidence
0.05<p≤0.100.05 < p \le 0.10some evidence
0.01<p≤0.050.01 < p \le 0.05strong evidence
p≤0.01p \le 0.01very strong evidence
Worked example

A spinner app that seems to like red

A game's spinner has four equal colors, so red should come up 25% of the time. A player suspects red comes up more often. In 40 spins, red appears 16 times. The player simulates 200 sets of 40 spins with π=0.25\pi = 0.25; 9 of the 200 simulated proportions are 0.40 or higher. Carry out the test.

  1. Step 1: π\pi = the true long-run proportion of spins that land on red. H0:π=0.25H_0: \pi = 0.25 versus Ha:π>0.25H_a: \pi > 0.25.

  2. Step 2: π^=16/40=0.40\hat\pi = 16/40 = 0.40.

  3. Step 3: The null distribution is centered at 0.25. The region of extreme follows HaH_a: values at or above 0.40. Simulated p-value =9/200=0.045= 9/200 = 0.045.

  4. Step 4: 0.01<0.045≤0.050.01 < 0.045 \le 0.05, so strong evidence.

Answer

The data give strong evidence that the true long-run proportion of spins landing on red is greater than 0.25. That is strong evidence, not proof.

Check yourself

1
Lec 04

For H0:π=0.60H_0: \pi = 0.60 versus Ha:π<0.60H_a: \pi < 0.60, an observed π^=0.48\hat\pi = 0.48. Among 300 simulated proportions under H0H_0, 21 are at or below 0.48 and 279 are above it. What is the simulated p-value? Give the exact decimal.

Math shortcuts are on for STATS 250. Type ^ then digits for powers, _ then digits for subscripts.
2
Lec 04

For H0:π=0.35H_0: \pi = 0.35 versus Ha:π≠0.35H_a: \pi \ne 0.35 with an observed π^=0.28\hat\pi = 0.28, where should a correct null distribution be centered?

3
Lec 04

A test gives p = 0.035. On the STATS 250 scale this is:

4
Lec 04

Three samples test H0:π=0.5H_0: \pi = 0.5 versus Ha:π>0.5H_a: \pi > 0.5 with equal sample sizes. They give π^=0.46\hat\pi = 0.46, 0.550.55 and 0.630.63. Which gives the smallest p-value?

5
Lec 04

A test of whether a coin is biased gives p = 0.41. Which conclusion is correct?

6
Lec 04

Select every correctly written pair of hypotheses.

MaizePrep is an independent study resource. It is not affiliated with, sponsored by, or endorsed by the University of Michigan. Course names and numbers are used only to describe which class the material covers. Every practice question is original, written in the style of the course.