1. The z Test and Decision Errors

A filling machine has a target. A sample comes back a little high. We will decide whether that result calls for investigation, then ask how often the decision rule would make a mistake. After that, we will apply the same z test logic to proportions and to comparisons between two groups.

2. From estimating a mean to checking a target

In the last lesson we used a sample to estimate an unknown mean. Here the target is already set. The question is whether the data give us a reason to doubt it.

Three extra millilitres in the sample average might be routine sampling variation. Before calling it a process change, we need to see what an on-target process would usually produce.

3. Write the two claims before looking at the sample

The null is the starting claim about the process mean. The alternative says which change would count against it. Because both underfilling and overfilling cost money, the team chooses a two-sided question.

The team also decides what false-alarm frequency it can tolerate. Five percent is a rule set in advance. We will use it after seeing how unusual the observed average is under the null.

4. What sample averages look like under the null

Imagine repeating the same sixty-four-bottle audit on a process centered at five hundred. Each audit supplies one average. Those averages form the curve we use to judge the result.

Individual bottles vary by twelve millilitres in standard deviation. Averages vary less, by one point five. This standard error is the relevant scale because our observation is a sample average.

5. Put the observed gap on a common scale

Find five hundred and three on the upper curve. It sits three millilitres above the target. Each standard error is one point five millilitres, so the gap spans two standard errors.

Follow the marker down. The center becomes zero and the observed result becomes z equals two. Every point has moved to a new scale. The set of sample outcomes is the same.

6. How unusual is a gap this large?

Start from the gap we actually observed. Since either direction matters, count an average at least three millilitres below the target or at least three above. Look at the two ends of the null curve.

The total shaded area is called the p-value. It answers a question about what a sample might show if the null held. We can now calculate that area on the standard normal scale.

7. Use the evidence to make the decision

The upper tail beyond two accounts for a little over two percent. The matching lower tail makes the total about four and a half percent. That is below the limit set earlier, so reject the null.

Tell the manager there is evidence of a mean different from the target. A result above the limit would leave that question unresolved. The cost of an adjustment is a separate business judgment.

8. Set the same rule using a cutoff

The significance level can also determine a cutoff before we have data. Put half the false-alarm budget in each tail. That gives a boundary about one point nine six standard errors from the null center.

The observed mean is outside the range centered on the target. Center that same width on the observed mean and the target is outside the confidence interval. The two comparisons express the same test decision.

9. A different business question changes the tail

A positive score supports a higher mean, so the right tail is small. A different-mean question counts both tails. A lower-mean question counts the large area to the left of this positive score.

These are different questions chosen in advance. At five percent, the one-sided critical score has magnitude about one point six four five. A two-sided five percent test splits the budget and needs a larger magnitude.

10. Now evaluate the rule over many decisions

We have made one decision. Now ask how good the rule would be if the factory used it every day. A false alarm means investigating an on-target process. A miss means continuing when the mean has shifted.

Both mistakes have costs. The significance level controls false alarms under the null model. To evaluate misses, we need to choose a shift that matters to the business. A tiny shift is harder to detect.

11. A fixed rule has a false-alarm rate

Suppose the factory fixes an alarm at a three-millilitre gap. Before taking any sample, we can calculate how often that alarm would sound for an on-target process.

The same tail area appeared in our first test because the observed gap was also three millilitres. Here the gap is fixed by policy. The probability describes the rule over repeated samples.

12. Keep one cutoff when the true mean changes

Set the cutoff so that five percent of sample averages exceed it when the true mean is at the boundary, one hundred and seventy seconds. This fixes the alarm rule.

Now suppose the true mean is one hundred and eighty seconds. The cutoff stays in place. The area below it on the new curve is the chance of a miss, beta. The area above it is power, the chance of detecting this particular delay.

13. Two ways to change how the rule performs

A stricter rule lowers false alarms by moving the cutoff to the right. With the same data quality and true delay, that misses more slowdowns.

With four times as many independent observations, the standard error halves. The same delay is then easier to distinguish from the target at the original false-alarm rate. Better sampling and measurement still matter.

14. Choose the z test from the business question

We began with an average fill. If the outcome is whether an order was returned, the quantity of interest is a proportion. If the business compares two groups, take a difference between their sample statistics.

The rows show where the standard error comes from. The next examples work through those differences. Once the gap is on the z scale, choose the tail just as we did for the bottling decision.

15. Set up a one-sample proportion test

The retailer is asking whether the share of orders returned has fallen below ten percent. Count each order once. Under the null boundary we would expect forty returns in four hundred orders.

Forty expected returns and three hundred and sixty expected orders that were kept both exceed five. That supports the normal approximation we use in this course. Use the benchmark proportion for the test because it describes the null model.

16. Test whether the return proportion is lower

Seven percent is three percentage points below the benchmark. Divide the gap by the null standard error of one and a half percentage points. That puts the sample two standard errors below the benchmark.

For the retailer’s lower-return question, use the left tail. Its area is about two point three percent, so reject at five percent. The other rows show what we would calculate for other questions chosen in advance.

17. Set up a two-sample test for means

The company wants to know whether the warehouses differ in average handling cost. Subtract warehouse B from warehouse A. The observed difference is six dirhams, and the equality claim sets the population difference to zero.

There are two independent samples, with sixty-four costs in each. The problem gives both population standard deviations. That is why we can use a normal reference for the standardized difference.

18. Test whether the two mean costs differ

Each sample mean has its own variance. Add those two variances, then take the square root to get two dirhams and fifty fils. The observed gap of six dirhams is two point four standard errors above the null value of zero.

Both directions matter in the question, so count the matching tail on each side. Together the areas are below five percent. We have evidence that mean costs differ. A claim that warehouse A costs less would use the other tail and get a very different result.

19. Set up a two-sample test for proportions

Site A saw ninety purchases out of three hundred visitors, and site B saw sixty out of three hundred. Their sample proportions differ by ten percentage points. The question is whether the populations differ.

If the proportions are equal, there is one common population proportion to estimate. Divide all purchases by all visitors, which gives twenty-five percent. At that benchmark, both expected counts in each sample exceed five.

20. Test whether the two purchase proportions differ

The numerator is the observed difference, ten percentage points, minus the null difference of zero. The denominator uses the pooled proportion from the previous slide. That makes the observed difference about two point eight three standard errors above zero.

The planned question asks about a difference, so use both tails. Their total is below five percent. Report evidence that purchase proportions differ. A directional question would use just the tail named by its alternative.

21. Try the sequence on a new claim

The word less tells you which departure matters. Pause and write the claims, then divide the gap from the benchmark by the standard error.

The sample mean is one point eight standard errors below the benchmark. Its left-tail area is below five percent, giving evidence that mean pay is lower. Next we will replace the known standard deviation with an estimate from the sample.
