Type I and Type II errors, explained with examples
Every significance test can reach the wrong conclusion. Type I and Type II errors name the two ways that can happen, and AP Statistics asks you to describe them in context almost every year.
On this page
The two errors
In a significance test, you start by assuming the null hypothesis H0 is true and look for convincing evidence against it. There are four possible outcomes:
- H0 is true and you fail to reject it: correct.
- H0 is false and you reject it: correct. The probability of this is the power.
- H0 is true but you reject it: a Type I error, a false positive.
- H0 is false but you fail to reject it: a Type II error, a false negative.
A helpful way to remember: a Type I error is crying wolf when there is no wolf. A Type II error is missing the wolf that is really there.
The probabilities
The probability of a Type I error equals the significance level α, if H0 is true. If you test at α = 0.05, then when H0 is actually true, you will wrongly reject it 5% of the time. The probability of a Type II error is written β, and it depends on how far the truth is from H0. Power = 1 − β: the probability of correctly rejecting a false H0.
A courtroom analogy
In a trial, the null hypothesis is “the defendant is innocent”. Convicting an innocent person is a Type I error. Letting a guilty person go free is a Type II error. The legal standard “beyond reasonable doubt” is like a small α: it makes Type I errors rare, at the cost of more Type II errors. Every decision rule trades one kind of error against the other.
Describing errors in context
On AP free-response questions, a generic definition earns little. You must describe the error using the context of the problem. A reliable template:
- Type I: “We conclude that [Ha in context] when actually [H0 in context] is true.”
- Type II: “We do not conclude that [Ha in context] when actually it is true.”
Worked examples
Example 1: a new medicine
A company tests H0: the new drug is no more effective than the old one, against Ha: the new drug is more effective. Describe both errors and a consequence of each.
- Type I: concluding the new drug is more effective when it is not. Consequence: patients switch to a drug that is no better, possibly at higher cost or with more side effects.
- Type II: failing to conclude the new drug is more effective when it really is. Consequence: patients miss out on a better treatment.
Example 2: a factory process
A machine should fill bottles with a mean of 500 ml. An inspector tests H0: μ = 500 against Ha: μ ≠ 500, and stops the line if H0 is rejected. Describe the errors.
- Type I: stopping the line because the mean seems different from 500 ml when it is actually 500 ml. Wasted time and money.
- Type II: not stopping the line when the mean really has drifted away from 500 ml. Customers get under- or over-filled bottles.
Example 3: which error is worse?
A test for a serious disease has H0: the patient does not have the disease. Which error is more serious, and how should α be chosen?
- A Type II error means a sick patient is told they are healthy and goes untreated, which is usually more serious than a Type I error (a false alarm followed by further tests).
- To reduce Type II errors, use a larger α, such as 0.10, accepting more false alarms.
Example 4: the probability of a Type I error
A test is carried out at α = 0.01. If H0 is true, what is the probability of a Type I error?
- P(Type I error) = α = 0.01.
How to increase power
Power is the probability that a test detects an effect that really exists. Higher power means fewer Type II errors. You can increase power by:
- Increasing the sample size n. This reduces variability in the statistic, so real differences stand out. It is the best option because it does not increase Type I errors.
- Increasing α. This makes rejection easier, but also increases the chance of a Type I error.
- A larger true effect. The further the truth is from H0, the easier it is to detect. You cannot control this, but it explains why small effects need big samples.
- Reducing variability, for example with better measurement or a blocked design.
AP questions often ask what happens to power, or to the probability of each error, when one of these changes. With n fixed, decreasing α decreases Type I errors but increases Type II errors and lowers power.
Errors and P-values
A small P-value does not mean the probability of a Type I error is small in this particular case; the error probability is set by α before collecting data. And failing to reject H0 does not prove it is true; it may be a Type II error caused by a small sample. This is why we say “not convincing evidence” rather than “H0 is true”.
Common mistakes
- Giving a textbook definition without the context of the problem.
- Swapping the two errors; check whether H0 is actually true or false in the scenario.
- Saying increasing α increases power without noting it also increases Type I errors.
- Thinking you can make both error probabilities smaller without a larger sample.
Practice questions
Try each one before opening the answer.
Q1. H0: a coin is fair. You conclude it is biased, but it is fair. Which error?
Show answer
Q2. H0: a new teaching method makes no difference. It really works, but your test fails to show it. Which error?
Show answer
Q3. A test has power 0.85. What is the probability of a Type II error for that alternative?
Show answer
Q4. A researcher changes α from 0.05 to 0.01 with the same sample size. What happens to power?
Show answer
Q5. What is the best way to increase power without increasing Type I errors?
Show answer
Frequently asked questions
Are Type I and Type II errors on the AP Statistics exam?
Yes. They are part of Unit 6 and recur in later inference units. Free-response questions often ask you to describe an error in context and give a consequence.
Can you make a Type I and a Type II error at the same time?
No. A Type I error can only happen when H0 is true, and a Type II error only when H0 is false, so in any one test only one is possible.
Are these errors in A level Maths?
The ideas appear in A level statistics. A level Maths mainly uses significance levels and critical regions; Type I and Type II errors are named explicitly in A level Further Maths.
Key takeaways
- Type I: rejecting a true H0 (false positive).
- Type II: failing to reject a false H0 (false negative).
- P(Type I) = α; P(Type II) = β; power = 1 − β.
- Describe errors in context and give a consequence.
- A larger sample increases power without raising α.