İnteraktif Dersler

Hypothesis tests

Lecture notes

Definition

A claim and a sample

A hypothesis test asks whether a sample fits a claim about a population. The claim gives a model, for example “70% of the seeds germinate”, so we know which samples to expect. A sample always differs a little from the truth, so the real question is how far is “too far”.
Example
A supplier claims that 70% of its seeds germinate. In a sample of 20, 10 germinate: . But 20 good seeds would not give exactly either. Is 10 too few?
Definition

What the claim predicts: the null distribution

The claim to be tested is the null hypothesis . If holds, the count follows a known law, the null distribution: here for the number of seeds that germinate. It shows what chance alone does when the claim is right.
46810121416182014
Example
Under the mean is and the standard deviation . The most likely count is 14, . A count of 10 is about 2 standard deviations below the mean.
Rule

How unusual is the result?

One result is always unlikely, so we ask for the probability of a result at least as extreme: as far from what expects, or farther, in the direction that doubts the claim. If it is small, the result would be unusual when the claim is right.
468101214161820
Example
For 10 germinated seeds the doubt goes downwards: (calculator: ). Results at least this extreme happen about 4.8% of the time if is true.
Definition

The p-value

The p-value is the probability, assuming is true, of a result at least as extreme as the observed one. It is worked out inside the null distribution and lies between 0 and 1: the smaller it is, the stronger the evidence against .
468101214161820
Example
The seeds: , the total height of the shaded bars to . In words: if is true, results at least as extreme as 10 germinated seeds happen about 4.8% of the time.
Rule

H0 and H1 from a claim in words

A claim about a parameter (a population share or mean , never the sample value or ) becomes two statements. holds the equality: or . is what the test looks for: , or , never . Choose from the question, before looking at the data.
Example
“A supplier says 70% of the seeds germinate; a grower thinks fewer.” , .“Is the machine off the 500 ml target?” , .“Are deliveries slower after the change?” With the mean of after minus before: , .
Rule

One-sided or two-sided

If names a direction ( or ) the test is one-tailed: one end of the null distribution counts. If says “different” () it is two-tailed: both ends count, and for a symmetric curve the p-value is twice the one-sided one. The direction comes from the question, never from the data.
0.03590.03591.8−1.8−3−2−10123
Example
A statistic . One-sided: . Two-sided: . At 5% the one-sided test rejects and the two-sided one does not, so must be chosen before the data are seen.
Definition

The significance level

The significance level is chosen before the test, often 5% or 1%. It is the probability of rejecting when is true: a wrong rejection, the Type I error. With about 1 test in 20 rejects a true . The p-value belongs to the sample; the level is fixed beforehand.
02468101214
Example
The rule “reject if 10 or more of 20 parcels are late” has, when : is true, probability of rejecting. So in 1000 honest checks about would wrongly reject: is the actual level of the rule.
Rule

Tails and the critical value

For a normal statistic the critical value cuts off the tail with area . One-tailed: the whole level in one tail, at 5%, at 1%. Two-tailed: half the level in each tail, at 5%, at 1%. A statistic beyond it lies in the critical region.
5%1.645−3−2−10123
Example
At 5% one-tailed the area above is , so : check . Two-tailed each tail has , so .
Rule

Two ways to decide, one answer

Rule 1 (p-value): reject if the p-value is at most . Rule 2 (critical region): reject if the statistic lies in the critical region. They never disagree: a statistic beyond the critical value has a tail area below .
0.03041.875−3−2−10123
Example
, , , , , : . Rule 2: . Rule 1: . Both: reject .
Definition

The terms on one picture

The test statistic summarises the sample. The critical value is the border; the critical region lies beyond it and has probability under ; the acceptance region (non-rejection region) is the rest. The p-value is the tail area beyond the observed statistic. The decision compares them.
5%95%1.645−3−2−10123
Example
One-tailed at 5%, : critical value ; critical region (5%); acceptance region (95%). A statistic is in the acceptance region: , do not reject .
Rule

The exact binomial test

Count the successes in independent trials and use the law of : . The p-value is a binomial tail: for , for . Reject when it is at most the level.
02468101214
Example
Parcels: late, , 5%; 10 of 20 are late. ( ). Since , reject .
Rule

Writing the conclusion

The decision about plus a sentence in context, in the frame of the course. UK / Cambridge: “Reject : there is sufficient evidence, at the 5% level, to suggest that …” or “Do not reject : there is not enough evidence …”. IB: “… at the 5% significance level there is (not) enough evidence that …”. AP: “Reject : there is convincing evidence that …” or “Fail to reject : there is not convincing evidence that …”.
Example
Late parcels (), AP: “Reject : there is convincing evidence that the true share of late parcels is greater than 0.3.” UK / Cambridge: “Reject : there is sufficient evidence, at the 5% level, to suggest that the share of late parcels is greater than 0.3.”
Rule

The critical region of a binomial test

Find the rule in advance: the critical region is the set of counts that lead to rejecting . For it is with the smallest value such that . Search from the top. The counts below form the acceptance region.
02468101214
Example
, , , 5%. but . Critical region ; acceptance region . A sample with lies in the critical region.
Rule

The actual significance level

A count takes whole values, so tail probabilities jump and rarely land exactly on the level. The critical region is the largest one whose probability is at most the level; its true probability is the actual significance level, often a little below the stated 5%. Report it: it is the real chance of a wrong rejection.
0.050.10.227780.113390.0480100.0171110.005112
Example
, , , 5%: , , . The region has actual level , not 0.05. The next value, , would wrongly reject in about 11.3% of the cases.
Rule

A two-tailed binomial region

For the critical region has two parts, and . The UK / Cambridge rule gives each part probability at most half the level; the actual level is the sum. Find and separately, each against .
912151821242730
Example
, , 5%: but , so and by symmetry . Region or ; actual level ; acceptance .
Rule

Three conventions, one table

UK / Cambridge: reject when the tail probability is at most the level (two-tailed: each tail against half the level). IB: reject only when the p-value is strictly below the level; binomial tests are one-tailed only (two-tailed: not examined at IB). AP: the two-sided p-value is twice the smaller tail. Exactly on the border the verdicts can differ.
1/803/813/821/83
Example
, , , , level 12.5%: , exactly on the level. UK / Cambridge: reject. IB: do not reject. AP: reject. Two-sided , : AP ; UK: 14 is outside or : do not reject.
Tip

Mark schemes and the word “accept”

Under UK / Cambridge, some mark schemes print the decision for a non-significant result as “accept ”. It means only: not enough evidence to reject . It does not say that is true. Write your own answer in the fixed frame, “Do not reject : there is not enough evidence …”. IB and AP never use “accept ”.
Example
Seeds: 80% claimed, 28 of 40 germinate, . Answer: “Do not reject : there is not enough evidence, at the 5% level, to suggest that the germination rate is lower than 0.8.” A mark scheme may print “accept ” beside it: read it as “not enough evidence to reject”.
Rule

The normal test for a proportion

For a large sample is close to normal under , with mean and standard deviation . The statistic is z = \dfrac{\hat p - p_0}{\sqrt{p_0 (1 - p_0) / n}}, with the null value in the standard error. The p-value is a normal tail beyond (AP: 1-PropZTest).
0.03311.837−3−2−10123
Example
, ; 69 of 100: , , . ; two-sided .
Tip

How good is the approximation? The conditions

The exact binomial p-value is the truth; the normal one approximates it, better with the continuity correction ( for ). Trust it when the sample is random, is at most 10% of the population, and and are both at least 10 (AP) or 5. Otherwise use the exact test.
Example
, , (counts and ): exact ; normal (difference ); with the correction (difference ). Failing case: , has .
Rule

The z test for a mean, sigma known

For a normal population with known (or large ), under . The statistic is z = \dfrac{\bar x - \mu_0}{\sigma / \sqrt n}, the distance of from in standard errors. Find the tail beyond (doubled if two-sided) and compare it with the level.
0.030452.550
Example
, , , , : , , (two-sided ). Reject for the one-sided test.
Rule

The critical region for the sample mean

Rule 2 can be written for itself: reject when is beyond (for ), where is the normal critical value. Two-tailed: with for half the level. It gives the same decision as the p-value.
5%52.1950
Example
Same test: . Region ; lies in it: reject . Two-tailed at 5%: , region or .
Rule

When sigma is not known: the t test

Usually is not known and the sample standard deviation takes its place. Then t = \dfrac{\bar x - \mu_0}{s / \sqrt n} follows a t distribution with , with heavier tails than the normal curve. Use , not , for any sample size under IB and AP (Cambridge allows for a large sample).
−3−2−101232.291−2.291
Example
, , , , , 5%: , , . Two-sided (T-Test); : reject .
Watch out!

z instead of t: a p-value that is too small

A common mistake is to put into the z formula and read the normal tail. The t distribution has heavier tails, so the true tail is bigger and the z p-value is too small: it overstates the evidence and can reject a null hypothesis that should stay. The difference is largest for small samples.
−3−2−101232.067−2.067
Example
The sample of the t test above, but : , . The t test gives two-sided : do not reject. The wrong z test gives and would reject.
Rule

Matched pairs: one sample of differences

When each route or subject is measured twice, the columns are not independent. Subtract pair by pair in a stated order and test the differences as one sample: , , . The order of subtraction decides the sign of and the direction of .
−2−1012345
Example
Delivery minutes on 8 routes, before minus after: 2, 1, -1, 4, 2, 2, 2, 3; , . : , , : reject .
Rule

Conditions for a test for a mean

Trust a test for a mean when the sample is random, is at most 10% of the population, and is close to normal: a normal population, or large (UK / Cambridge and AP: at least 30; IB: above 30), or a small sample with no strong skew and no outliers. IB also says the underlying distribution must be normal for the test.
4555657580
Example
Ten fill levels: 48, 49, 50, 50, 51, 51, 52, 53, 54, 75. With look at the data: 75 is far above the upper fence . A strong outlier in a small sample: the test is not justified until the value is checked.
Rule

Which test fits the question?

Ask four questions. Which parameter? A proportion (exact binomial or normal test), a mean or a correlation. One sample or pairs? Pairs become one sample of differences. Is known? Known: ; unknown: with . Which conditions? Randomisation, 10%, counts or normality. For a correlation: , .
Example
(1) “Do 70% of seeds germinate?” A proportion in a sample of 20: exact binomial test. (2) “Does the machine fill 500 ml?” A mean with known: test. (3) “Is the delivery time lower after the change?” Pairs, unknown: test on the differences.
Rule

Simulate the claim

Without a formula, a test can be run on the computer: draw many samples from the model of with a seeded random generator and count the share that are at least as extreme as the observed result. That share estimates the p-value (CCSS S-IC.A.2: would this result make me question the model?).
1/3205/3215/1625/1635/3241/325
Example
A coin model: tails with chance . Would 5 tails in a row make you doubt it? Exactly, 3.1%. Three presses of 100 simulated series in the application give series with 5 tails: about 3 in 100, as expected.
Tip

A simulation next to the exact value

A simulated p-value is a share of runs, so it carries its own error: it typically misses the exact value by about . To make the gap 10 times smaller you need 100 times as many runs. Put the exact value next to the simulated one whenever it is known.
0.000.010.020.030.040.050.060.070.00260.03741000.02850.039510000.02780.031210000
Example
Exact . Typical gap: : ; : ; : . The application’s first presses give of 1000 () and of 10 000 (): within one typical gap.
Rule

What the p-value says

Three sentences are always right. 1. The p-value is computed assuming is true. 2. A small p-value says the result would be unusual if were true. 3. A p-value that is not small is not evidence that is true. Behind them: the p-value is the probability, assuming is true, of a result at least as extreme as the observed one. It is about results, given , never about itself.
Example
Seeds: . If (70% germinate) is true, results at least as extreme as 10 of 20 happen about 4.8% of the time. That is below 5%, so the result would be unusual if were true: reject at the 5% level.
Rule

A large p-value is not evidence for H0

When the p-value is large, the sample is not unusual if is true. The honest conclusion is “do not reject (not enough evidence against it)”. It is not evidence for : other parameter values would fit the sample just as well, and a small sample cannot tell them apart.
24681012141618
Example
A fair coin is flipped 20 times and shows tails 8 times. , : the smaller tail , so the two-sided p-value is far above 0.05: do not reject . But gives exactly tails on average, so the sample fits just as well.
Watch out!

Misreading 1 (WRONG): the probability that H0 is true

WRONG: “The p-value is the probability that is true.” Verdict: the p-value is calculated assuming is true, so it cannot also measure how likely is. It is a probability of results given . A hypothesis about a population is either right or not; the p-value gives it no probability.
Example
Seeds: does not say that the supplier is right with probability . It says that if the supplier is right, results as extreme as 10 of 20 happen about 4.8% of the time.
Watch out!

Misreading 2 (WRONG): the probability of chance alone

WRONG: “The p-value is the probability that the result is due to chance alone.” Verdict: the p-value starts from the assumption that only chance is at work () and asks how often results this extreme then occur. It does not split the result into a part due to chance and a part due to something else.
Example
A delivery check gives for 10 late parcels of 20 against . That is not “a 4.8% chance that the result is a fluke”: it is the share of samples of 20 parcels, from a company whose late share really is , that give 10 or more late parcels.
Watch out!

Misreading 3 (WRONG): 1 − p as the chance of H1

WRONG: “ is the probability that the alternative hypothesis is true.” Verdict: is the probability, assuming is true, of a result that is less extreme than the observed one. It is still a statement under , and says nothing about how likely is.
Example
Seeds: under , the area of the bars right of 10, about . It is not “a 95% chance that the seeds are worse than claimed”.
Watch out!

Misreading 4 (WRONG): a small p-value proves H1

WRONG: “A small p-value proves the alternative hypothesis.” Verdict: a small p-value is evidence against , with a known risk of rejecting a true . Evidence is not proof: at the 5% level about 1 test in 20 of true null hypotheses is rejected. Say “reject : there is evidence that …”.
Example
Among 1000 tests of true null hypotheses at 5%, about reject, each with a small p-value, and none proves its alternative. In the seeds example () the right sentence is that there is evidence that fewer than 70% germinate.
Watch out!

Misreading 5 (WRONG): a large p-value shows H0 is true

WRONG: “A large p-value shows that is true.” Verdict: a large p-value only says that the sample is not unusual if is true. Many other parameter values would also give such a sample. The right words: “do not reject : there is not enough evidence …”.
Example
Bottling: target 500 ml, , 16 bottles with mean : , two-sided . Right: do not reject . It does not show that the mean is 500: a machine 1 ml low would give a similar sample.
Tip

Significant is not the same as important

A test says whether a difference can be told apart from sampling variation, not whether it matters. With a very large sample even a tiny difference gives a small p-value; with a small sample even a large one may not. Report the size of the difference next to the p-value, and look at an interval for it.
0.400.450.500.550.600.41200.60801000.50020.519810000
Example
with . For : , , two-sided : reject at 5%. For : , : nothing. The share is the same 51% both times.
Rule

The test for a correlation

Could a sample of pairs give such an when the population correlation is 0? Test with t = \dfrac{r \sqrt{n - 2}}{\sqrt{1 - r^2}}, \qquad df = n - 2, or compare with the critical value of from a table: the same decision. Conditions: random pairs, a roughly linear pattern.
−3−2−101232.772−2.772
Example
, , two-tailed 5%: , , . Critical , so critical : reject ; there is evidence of a linear association.
Tip

The shuffle test beside it, and correlation is not cause

Shuffle the values at random against the values: this destroys any link, so each shuffle shows an that “no association” gives. The share of shuffles with at least the observed estimates the p-value, close to the test. Even a significant shows an association, not that one variable causes the other.
−1−0.500.510.694
Example
The application’s ten pairs (; ): , , . Of 1000 seeded shuffles have (; another press: ). All orderings give the exact .
Definition

Type I and Type II errors

Type I error: reject when is true; probability . Type II error: do not reject when is false; probability . The power is the probability of rejecting when it is false. Which error costs more depends on the context, so say in words what each one means.
p ≥ αp < αH₀1−ααIH₁βII1−β
Example
Fire alarm, : “no fire”. Type I: it rings with no fire (a false alarm). Type II: there is a fire and it stays silent (a missed fire), the one to keep very small. Spam filter, : “not spam”. Type I: a good message goes to the spam folder. Type II: spam reaches the inbox.
Rule

Alpha, beta and power as areas

Draw the curve of under (centre ) and under a true value ; the critical value cuts both. is the area of the curve beyond . is the area of the curve before . The power is the area of the curve beyond .
52.195053αβ1−β
Example
, , , 5%, : . True mean : , , power : a mean of 53 is found about 73 times in 100.
Rule

The errors of a binomial test

For a binomial test the pictures are bar charts. Fix the critical region from (say ). Then is with , the actual level; is with the true chance , and the power is with . The same region cuts both bar charts.
0510152005101520
Example
, , region : . True chance : , power . With 20 trials the test misses a true in about 4 cases out of 10.
Rule

What changes the power

A bigger sample shrinks the standard error, so the curves overlap less. A truth farther from is easier to see. A larger level gives more power but a larger Type I chance. A smaller works like a bigger sample. For a fixed sample a smaller makes larger; only more data lowers both errors.
00.51505152535455560.050.727
Example
Previous test (power at 53): : , , power . Truth : . Level 1%: , power only . gives , as does.
Tip

Everyday test 1: seed germination

A binomial test in four stages. 1 , and the level. 2 The null distribution with the observed result. 3 The p-value, or the critical region and the actual level. 4 The conclusion in context, in the frame of your course, and the error that could have been made: a Type II error after “do not reject”, a Type I error after “reject”.
22242628303234363840
Example
80% germination claimed; 28 of 40 germinate. , , 5%: . Critical region (actual level ); 28 is not in it. Do not reject . If the seeds really are worse, this is a Type II error.
Tip

Everyday test 2: a bottling machine

A mean with known : the test, two-sided because the question is “is it off target?”. The critical region for is the target . Add the power: how often would the test notice a machine that is a given number of millilitres low? A test that notices only half of the real faults leaves a lot to the Type II error.
0.06680.0668498.5497498499500501502503
Example
Target 500 ml, , 16 bottles, mean : , , two-sided . Region or : 498.5 is inside the acceptance region, do not reject . If the machine is really 2 ml low, the power is only .
Tip

Everyday test 3: delivery times

Before-and-after data on the same routes are matched pairs: test the differences with a test. A confidence interval for the mean difference is a cross-check: if 0 lies outside the 95% interval, a two-sided test at 5% rejects as well; if 0 lies inside, it does not.
−1012340.65633.0937
Example
Eight routes, before minus after 2, 1, -1, 4, 2, 2, 2, 3: , , . One-sided : reject; two-sided . The 95% interval () excludes 0, as the two-sided test does. If the change did nothing, rejecting would be a Type I error.
Tip

Everyday test 4: is a spinner fair?

A spinner has four equal sectors, so a fair one lands on a chosen sector with chance . A binomial test checks one sector. The question decides the tails: “comes up too often” is one-sided, “is the spinner fair?” is two-sided. The same data can pass one question and not the other, so choose the question before spinning.
02468101214
Example
A sector comes up 9 times in 20 spins ( expected). . One-sided: , reject; region , actual level . Two-sided (AP): , do not reject. Both answer different questions.
Tip

Which course asks what

Cambridge 9709: P6⁠5⁠1 to P6⁠5⁠5 (terms, binomial test, mean of a normal population, Type I and II errors). UK A level (DfE, Edexcel 9MA0, AQA 7357): O1 to O3. IB AI: SL4⁠11 (hypotheses, levels, p-values, the t-test, one- and two-tailed) and AHL4⁠18 (critical values and regions, a mean, a binomial proportion, ρ = 0, errors). AP: 3⁠5 to 3⁠8, 4⁠4, 4⁠5. CCSS: S-IC⁠A⁠2. NL wiskunde D vwo: et-08. Not here: Poisson tests, chi-squared tests, two-sample tests; no MEB, GCSE / IGCSE, NL A or C.
Example
An IB AI pupil needs the cards on hypotheses, levels, the p-value, the t test, tails, critical regions, the one-tailed binomial test, the correlation test and the errors. A Cambridge 9709 pupil needs the binomial test with exact probabilities, the actual level, the z test and the errors. An AP pupil needs the conditions, the z test for a proportion, the t test and the AP wording.

Exercises

A cereal box says it holds 500 g on average. A buyer thinks the boxes hold less. State and and say whether the test is one-tailed or two-tailed. Easy
Show solution
The parameter is the population mean of the content of a box.
The equality goes in : .
The buyer thinks “less”, so . One direction only: one-tailed.
, ; one-tailed
Check → the equality is in only, and names the direction of the buyer’s doubt.
Write and say whether the test is one-tailed or two-tailed. (a) “Has the share of late parcels gone up from 0.3?” (b) “Is the spinner fair?” (a sector should come up with chance 0.25). Easy
Show solution
(a) “Gone up” names a direction: , , one-tailed.
(b) “Fair?” names no direction: , , two-tailed.
(a) , one-tailed. (b) , two-tailed.
Check → the direction comes from the words of the question, not from the data.
A test gives the p-value . Decide at the 5% level and at the 1% level whether to reject (rule: reject if ). Easy
Show solution
At 5%: , so reject .
At 1%: , so do not reject .
Reject at 5%; do not reject at 1%
Check → the level is chosen before the test; the same p-value can pass one level and not the other.
, , is known, and . Find and the p-value (4 decimal places). Easy
Show solution
The standard error is .
Then .
The p-value is the upper tail: .
and
Check → , so at the 5% level is not rejected.
A fair coin is a model for 10 flips: . A series gives 9 tails. Find the exact p-value as a fraction and as a decimal (4 decimal places). Easy
Show solution
Every outcome of 10 flips has probability .
and .
.
Check → , so the result would be unusual for a fair coin: the model is doubted at the 5% level.
A two-tailed test at the 5% level gives . Do you reject ? Find the two-sided p-value (4 decimal places). Easy
0.01070.0107−2.31.96−1.96−3−2−10123
Show solution
The critical values of a two-tailed test at 5% are .
, so lies in the critical region: reject .
One tail: . Two-sided: , below 0.05.
Reject ; two-sided
Check → both rules agree: in the critical region and .
, for trials, level 5%. Find the critical region and its actual significance level (4 decimal places). Medium
Show solution
Under , , with mean .
Search from the top: but .
So the critical region is and the actual level is , not 0.05.
; actual level
Check → : the region is the largest one that fits the level.
A poll of 100 pupils finds 62 who cycle to school. Test against with the normal test: find and the two-sided p-value (4 decimal places) and write the AP conclusion at 5%. Medium
Show solution
and .
One tail ; two-sided .
AP: Reject : there is convincing evidence that the true share of pupils who cycle is different from 0.5.
, ; reject
Check → the exact two-sided value is , also below 0.05.
Nine bottles have mean volume cl and . Test against at 5%, unknown: find , , the two-sided p-value (4 decimal places) and decide. Medium
Show solution
and , .
Two-sided ; also . Do not reject .
, , ; do not reject
Check → the wrong test would give the smaller .
Five routes are timed before and after a change (minutes saved): 3, 4, 1, 5, 2. Test against at 5%: find and the one-sided p-value (4 decimal places). Medium
Show solution
; squares of the deviations , so and .
with ; one-sided . Reject .
, ; reject
Check → the critical value for , one-tailed 5%, is .
A student tests with and the rule “reject if ”. What is the probability of a Type I error (4 decimal places)? Is the rule within a 5% level? Medium
Show solution
A Type I error is rejecting a true . Under , .
Its probability is .
, so the rule is not within 5%. The rule has and does fit.
; no, it is above 5%
Check → the probability of the region is the actual significance level: it must be at most the level.
, , , , level 5%. Find the critical value of and the probability of a Type II error when the true mean is 106, and the power (4 decimal places). Medium
104.93100106αβ1−β
Show solution
and .
with mean 106: , so .
The power is .
, , power
Check → : the power and share the true-value curve.
A binomial test of against uses at 5% (UK / Cambridge rule). Find the critical region, the acceptance region and the actual level (4 decimal places). Hard
Show solution
Each tail at most ; . but , so the lower part is .
By symmetry the upper part is ; acceptance region . Actual level .
or ; acceptance ; actual level
Check → IB does not examine this two-tailed binomial test.
, , , , level 6.25%. Find the p-value and decide under UK / Cambridge, IB and AP. Hard
Show solution
, exactly equal to the level 6.25%.
UK / Cambridge and AP: reject when , so reject. IB: reject only when is strictly below the level, so do not reject.
; UK / Cambridge reject, IB do not reject, AP reject
Check → the conventions differ only for a p-value exactly on the level.
, , , . Find the normal p-value, the exact p-value and the p-value with the continuity correction (4 decimal places). Which normal value is closer to the exact one? Hard
Show solution
, , and .
Corrected: and . Exact: . The corrected value is closer.
Normal , corrected , exact ; the corrected value is closer
Check → the counts are and , both at least 10, so the approximation is allowed.
A machine should fill 12 cl with known cl. A sample of 25 has mean cl. Test against at 5%: find , the two-sided p-value (4 decimal places), the critical region for and decide. Hard
Show solution
, and the two-sided .
Critical region : or . Since , reject .
, ; reject
Check → a borderline result: say there is evidence, not proof, that the mean fill differs from 12 cl.
A sample of 16 has mean and . Test against with the test: find , and the p-value (4 decimal places). Decide at 5% and at 10%. Hard
Show solution
, , .
One-sided : above 0.05, do not reject at 5%; below 0.10, reject at 10%.
, , ; do not reject at 5%, reject at 10%
Check → at 5% the critical value is ; the wrong test would give .
The test of against with uses the region . Find and the power when the true chance is and when it is (4 decimal places). Hard
Show solution
The region stays the same; only the distribution changes: and power with .
: , power . : , power . A truth farther from is found much more often.
: , power ; : , power
Check → and the power add up to 1 each time.
A sample of 12 pairs has . Test against at 5%: find , , the two-sided p-value and the critical (4 decimal places), and write the UK / Cambridge conclusion for “study time and a test mark”. Hard
Show solution
with ; two-sided . Critical , so critical .
Do not reject : there is not enough evidence, at the 5% level, to suggest a linear association between study time and test mark.
, , critical ; do not reject
Check → is below the critical and : the two rules agree.
Are you a teacher?

Start live lessons with these applications for your whole class. Free to try for one month.

Try it free

Fits these school years

IB (MYP, Diploma): Diploma, year 1, Diploma, year 2

Cambridge International: A Level

Netherlands: Vwo · Mathematics A, Vwo · Mathematics D

England (GCSE, A Level): Year 12 (A Level), Year 13 (A Level)

USA (Common Core): Algebra 2, Precalculus

What the applications of this topic teach 6 applications

Hypothesis language: H0, H1, tails and levels

  • Reading the p-value with its fixed sentence
  • Writing the null and alternative hypotheses from a claim
  • Telling a one-sided from a two-sided test
  • Naming every part of a test on one picture

Test a proportion: exact, critical region and normal

  • Doing the exact binomial test with a tail sum
  • Finding the critical region and the actual significance level
  • Using the normal approximation with a continuity correction

Test a mean: z, t and matched pairs

  • Doing the z test for a mean with σ known
  • Doing the t test when σ is unknown
  • Treating matched pairs as one sample of differences
  • Checking the conditions for a test of a mean

P-value by simulation and the correlation test

  • Simulating the null model and counting results at least as extreme
  • Telling the right sentences about the p-value from the misreadings
  • Testing whether a correlation goes beyond chance

Errors and power: Type I and Type II

  • Naming Type I and Type II errors in context
  • Finding α, β and the power as areas
  • Saying what changes the power

Tests in everyday life

  • Running a test in four stages with own numbers
  • Choosing the test that fits each story
  • Naming the error that may have been made after the decision