This is the third part of a series of posts on sample-to-population inferences, and progressively developing students understandings.
Comparing two groups:
Is what we are seeing when we compare the features of our samples of our two groups (in particular the median/mean) MORE than just sampling variation?
Our way of judging this difference gets more sophisticated as we move up our curriculum. We essentially have a four-stage progression with details found HERE for NZ Curriculum Levels 5 to 7. The fourth stage introduces formal bootstrapping methods for constructing confidence intervals.
STAGE 1: Curriculum Level 5 guide (Year 10 – 11ish): if the median of one group is outside the middle 50% of the other group then you can make the call that “group 1 tends to be bigger than group 2 back in the population” (for samples of size 20 – 40).
I usually convince students that this works in two ways. Firstly, by using the lesson progression developed by Pfankuuch et al (here are the key resources, but make sure you check out the workshop 1 material too. This wiki also has all the resources and a summary together nicely (thanks Pip!). The lesson progression uses a class investigation where students to take multiple samples from Karekare college, firstly comparing boys vs girls heights and secondly comparing times to school for students busing or walking.
Students CAN make the call that students who bus tend to take longer to travel to school than students who walk back in Karekare College – this is seen visually by the consistent shift between the multiple samples.
This idea is reinforced with arm-waving again. It is important this time to make sure you have bling on each arm (in this case watches) to represent the medians. You can see below that we have the situation where the two groups are so far apart that although the middle 50%s move (each arm) with repeated sampling, the medians (watches) are consistently outside the middle 50% (arms) of the other group (for at least one group). Please note that this one is a bit tricky to pull off – our arms aren’t quite designed right for this sort of functionality…
Students CANNOT make the call that boys tend to be taller than girls back in Karekare College – visually the multiple samples don’t show the same consistency. That is, there isn’t enough of a shift between the samples of the two groups to say whether boys tend to be taller than girls or if girls tend to be taller than boys back in the population.
With similar arm-waving ideas as previously – this time with repeated sampling we show that if the two group in the original sample are close together, the middle 50% (arms) and medians (watches) may stay in the same positions or may switch round with repeated sampling due to sampling variation – so we can’t actually tell from this sample what’s happening with the difference between the two groups back in the population.
STAGE 2: Curriculum level 6 guide (Year 11ish): if the distance between the medians is greater than a proportion of the “overall visible spread” (OVS) then we can make the claim that B tends to be bigger than A back in the population. The proportional relationship between the medians and OVS is linked to both sample size and spread.
The idea that sampling variation is influenced both by sample size, and spread of the population is an important idea to start developing with students. We work as a class to create multiple samples of different sizes (as discussed in Part 1) and to explore the link between the difference in medians and the size of the overall visable spread. Resources are available on Census@School here.
STAGE 3: Curriculum level 7 guide (Year 12): if the informal confidence intervals for the medians do not overlap then we can be pretty sure that the median of B is bigger than the median of A back in the population.
Students take to this guide quite naturally and just run with it if the work has been put into developing their understanding of informal confidence intervals, why we need them and how we incorporate sampling variation measures. We need to make sure we emphasise the slight difference in the conclusion we are drawing – this time it is directly about comparing population medians.
Again, I reinforce ideas of sampling variation and confidence interval construction with hand-waving. When our groups are close together, with a sample size of n then the confidence intervals are THIS wide and with repeated sampling they will “jiggle” round quite a bit due to sampling variation. [insert suitable sound effect at *]. If we increase the sample size then the confidence intervals will get narrower*, AND there will be less “jiggle” so we can be pretty sure that back in the population we will see the differences in the medians of the two groups.