PART 3 – COMPARING TWO GROUPS Developing big ideas with sample-to-population inferences …

This is the third part of a series of posts on sample-to-population inferences, and progressively developing students understandings.

Comparing two groups:

Is what we are seeing when we compare the features of our samples of our two groups (in particular the median/mean) MORE than just sampling variation?

Our way of judging this difference gets more sophisticated as we move up our curriculum.  We essentially have a four-stage progression with details found HERE for NZ Curriculum Levels 5 to 7.  The fourth stage introduces formal bootstrapping methods for constructing confidence intervals.

STAGE 1: Curriculum Level 5 guide (Year 10 – 11ish) if the median of one group is outside the middle 50% of the other group then  you can make the call that “group 1 tends to be bigger than group 2 back in the population” (for samples of size 20 – 40).

boxI usually convince students that this works in two ways.  Firstly, by using the lesson progression developed by Pfankuuch et al (here are the key resources, but make sure you check out the workshop 1 material too.  This wiki also has all the resources and a summary together nicely (thanks Pip!).  The lesson progression uses a class investigation where students to take multiple samples from Karekare college, firstly comparing boys vs girls heights and secondly comparing times to school for students busing or walking.
Students CAN make the call that students who bus tend to take longer to travel to school than students who walk back in Karekare College – this is seen visually by the consistent shift between the multiple samples.  

This idea is reinforced with arm-waving again.  It is important this time to make sure you have bling on each arm (in this case watches) to represent the medians.  You can see below that we have the situation where the two groups are so far apart that although the middle 50%s move (each arm) with repeated sampling, the medians (watches) are consistently outside the middle 50% (arms) of the other group (for at least one group).  Please note that this one is a bit tricky to pull off – our arms aren’t quite designed right for this sort of functionality…
giphy (5).gif

Students CANNOT make the call that boys tend to be taller than girls back in Karekare College – visually the multiple samples don’t show the same consistency.  That is, there isn’t enough of a shift between the samples of the two groups to say whether boys tend to be taller than girls or if girls tend to be taller than boys back in the population.

With similar arm-waving ideas as previously – this time with repeated sampling we show that if the two group in the original sample are close together,  the middle 50% (arms) and medians (watches) may stay in the same positions or may switch round with repeated sampling due to sampling variation – so we can’t actually tell from this sample what’s happening with the difference between the two groups back in the population.

giphy (6)

STAGE 2: Curriculum level 6 guide (Year 11ish):  if the distance between the medians is greater than a proportion of the “overall visible spread” (OVS) then we can make the claim that B tends to be bigger than A back in the population.  The proportional relationship between the medians and OVS is linked to both sample size and spread.

L6 box

The idea that sampling variation is influenced both by sample size, and spread of the population is an important idea to start developing with students.  We work as a class to create multiple samples of different sizes (as discussed in Part 1) and to explore the link between the difference in medians and the size of the overall visable spread.  Resources are available on Census@School here.

STAGE 3: Curriculum level 7 guide (Year 12):  if the informal confidence intervals for the medians do not overlap then we can be pretty sure that the median of B is bigger than the median of A back in the population.  


Students take to this guide quite naturally and just run with it if the work has been put into developing their understanding of informal confidence intervals, why we need them and how we incorporate sampling variation measures.  We need to make sure we emphasise the slight difference in the conclusion we are drawing – this time it is directly about comparing population medians.

Again, I reinforce ideas of sampling variation and confidence interval construction with hand-waving.  When our groups are close together, with a sample size of n then the confidence intervals are THIS wide and with repeated sampling they will “jiggle” round quite a bit due to sampling variation.  [insert suitable sound effect at *].  If we increase the sample size then the confidence intervals will get narrower*, AND there will be less “jiggle” so we can be pretty sure that back in the population we will see the differences in the medians of the two groups.

giphy (7)

STAGE 4: Curriculum level 8 (Year 13):  Students complete their sample-to-population inference journey with us in their last year of high school by meeting a formal re-sampling method, namely bootstrapping, to create confidence intervals.

Further details on teaching bootstrapping for understanding can be found here (Session D – clarifying inferences) or here  (original roll-out teacher professional development). 

Part 1 – Sampling variation

Part 2 – Confidence intervals


PART 2 – CONFIDENCE INTERVALS Developing big ideas with sample-to-population inferences …

This is the second part of a series of posts on sample-to-population inferences, and progressively developing students understandings.

The need for a confidence interval (NZ Curriculum Level 7/Year 12-ish onwards)…

The next step in developing students’ appreciation of sampling variation is for them to understand that making a sample-to-population inference where they give a point estimate of the population parameter is setting them up for failure – they will always be wrong.  Instead they need to use an interval of plausible/believable values within which the population parameter is most likely to fall between, based on our best estimate of the size of the sampling variation.

My lesson progression, outlined briefly below in relation to developing the Level 7 informal confidence interval, is adapted from others’ work including Pip Arnold (Pip’s work) and Lindsay Smith (Lindsay’s work).

  1. Introduction to/ recap of sampling methods including how and why we sample.
    I set a silly homework assignment the period before this lesson, then get the students to brainstorms all different ways we could select five students to check their homework.  Generally students come up with our standard sampling methods needed (eg simple random, systematic, stratified, convenience…), occasionally with a little prompting from me. We then “check homework” which always involves students receiving a lolly if they get selected.  This does lead into interesting conversations on RANDOM (see my previous blog on this!) and our perception of RANDOM.  It does pay to save a few lollies for the end for any students that haven’t been selected!
  2. The need for confidence intervals – meet the Kiwis… 
    Students work in pairs each take four samples (n=15) from the kiwi population and use their samples (after creating dot plots, box plots and summary statistics) to finish the following sentence “From my sample data I estimate that the median weight for all New Zealand kiwis is….“.   Usually students who have worked together will give the same/similar answers but we get a variety of answers across the class.  Sometimes we even get a student or two who give an interval for their answer – send them to stand in the back corner of the room immediately!  The big reveal, of course, comes with the teacher-prompt of “but your answers are all different – who is right?“.  Generally your more confident personalities will claim that they’re right, while the quiet students sit quietly looking concerned.  Pad this out as long as you need to to make your point that “they are all wrong“.  How can they know EXACTLY what the population median will be?
    NOTE: Exactly how you deal with this will depend on the relationship you have with your class – it may be better to soften this statement to “you might be right, you might be wrong – in the real world you would never know…”.  I enjoy stirring my class a little and the reactions I get are always lots of fun!
    We now bring back into the room the students sent to the back of the class and praise them for being three steps ahead of the rest and putting an interval around where they thought the population median would be.
    We could use our medians from our repeated sampling (collect the class results on a big graph) and read off where most of the medians lie to give us a good idea of the interval for the population median.  This works fine in the classroom where we have easy access to the population to actually complete repeated samples, but in reality we want methods that we’re happy working with when we only have one sample.
  3. Increased sample sizes = decreased sampling variation. Therefore our interval should get narrower as our sample size increases.  See PART 1 for how I reinforce this with students.  Hand gestures are very useful again here: If your confidence interval is THIS wide for this sample size – what happens when you INCREASE your sample size? – make sure you include a sound effect as your confidence interval gets narrower with the increased sample size!
  4. Increased spread in the population = increased sampling variation.  Therefore our interval should get wider as our spread increases.  I use the simple situation (from Pip) of comparing student heights for new furniture in an intermediate school (Year 7 & 8) or a middle school (Year 7, 8, 9 & 10).  We ask which teacher is likely to get a closer estimate of the students heights?  
    Bring back your hand visuals again:  If your confidence interval is THIS wide for this spread – what happens when you INCREASE your spread? – make sure you include a sound effect as your confidence interval gets wider with the increased spread!   Of course, both here and for increased sample size we can start showing the ICI visually with a (red) line on our box plot, centered about our median.
  5. Formula for informal confidence interval (ICI).  I introduce the suggested formula for students to use, and we check that it meets our requirements.  Yes, the bit we add on or subtract does get smaller when we increase the sample size (as we’re dividing by a bigger number); yes, the bit we add on or subtract does get bigger when the spread (interquartile range IQR) increases (as we’re multiplying by a bigger number)
  6. Checking our informal confidence interval formula works most of the time.  We have class set of 100 different samples from the kiwi population (here if you want it) where students calculate 5 different ICIs themselves, then we collect these to check whether they captured the population median or not (collection sheet is here).  I am very clear to reinforce with students that we are working in TEACHING WORLD so we can do exactly what we’re doing – testing our ideas to convince ourselves they work.
  7. Reinforcing the ICI has been developed and tested by people who know what they’re doing, ready for us to use this year in class.  It is a step along our sample-to-population inference journey which culminates next year with the introduction of formal methods for constructing confidence intervals.
  8. EVERY TIME!!!!! you construct a confidence interval you should be interpreting it (even if it’s just in your head!).  This is almost a mantra in my class.  Students need to continually remind themselves what the point of creating the confidence interval is, and by interpreting it carefully we are doing this.  Remember, every confidence interval interpretation should include the statistics, the population, “pretty sure” (or equivalent indication of uncertainty), the variable, numbers and units.  For example “we’re pretty sure that the population median height of all Year 12 boys in New Zealand is somewhere between 173cm and 182cm“.
    Other formative assessment questions such as “Would it be believable that the median height of Year 12 boys in New Zealand is 185cm? Why? Why not?” are also super-important to check students’ understanding.  And of course slipping in annoying questions such as “Why do we use confidence intervals?”

Part 1 – Sampling variation

Part 3 – Comparing two groups

PART 1 – SAMPLING VARIATION Developing big ideas with sample-to-population inferences …


This is a post that’s been rattling round in my brain for months now, I’m finally getting it down in words.  Firstly, a huge shout-out to Anna Martin (@annamartinnz ) for all the help, suggestions, proofing,… she’s given so kindly to make sure this all makes sense! 

Right this has all got soooo long that I’ve split it into three parts (1) Sampling variation (2) introducing confidence intervals and (3) comparing two groups

Our NZ Stats curriculum is awesome and has developed a clear learning progression for developing (and assessing) students’ sample-to-population inferential understandings.  This progression is based on work from Maxine Pfannkuch, Chris Wild, Pip Arnold and others (for example Pfannkuch, M. J., Wild, C. J., & Parsonage, R. (2012). A conceptual pathway to confidence intervals. ZDM – The International Journal of Mathematics Education, 44 (7), 899-911; Arnold, P., Pfannkuch, M., Wild, C. J., Regan, M., & Budgett, S. (2011). Enhancing students’ inferential reasoning: From hands-on to “movies”. Journal of Statistics Education, 19 (2), 1-32) and is about having clear visual cue to develop and reinforce student understanding.  We are looking at what we can say about what is happening for two groups back in the population, based on samples, generally focusing on the median and mean. 

I’ve popped a summary together here of how I view the learning progressions from Year 10 to Year 13, including new ideas at each level, what needs reinforcing, and “watch-fors” (Year 10, Level 1 Multivariate data 91035, Level 2 Inference 91264, Level 3 Formal Inference 91582).  Please let me know if it’s helpful, ask for clarification, play spot the errors etc.  But… after teaching these progressions again this year, across all three levels (and still going!) there are some common themes and sticking points for students (and teachers).  Here’s my thinking on it.

Sampling variation: The variation in a sample statistic from sample to sample.

This is a BIG IDEA concept that we start developing in Year 10 with our students, repeat and build upon for the next three years.  Initially I look at this in a one sample situation with students.  Key teaching activities and tools are listed below (with links where possible):

  • Students in class each take a random sample from the same population (bag of data cards each) of size n (usually about 30).  After a dot plot, box plot and calculating summary statistics (usually median), they then use their sample to answer the investigative (summary) question – for example “What is the median height of doozers from Fraggle Rock?”.  Students notice that everyone’s samples are similar but different – they SEE sampling variation.  Collect summary statistics (usually lower quartile, median, upper quartile) from each sample into a class graph.  Students can SEE sampling variation in the summary statistics.  Links for these activities can be found here (Yr10 Karakare College population), here (Yr12, Kiwis population), here (Yr13, Pugs-in-Costumes-on-the-internet populations or Doozers).

These are some samples from Karekare College – our next step would be to create box plots above the dot plots. 

  • Chris Wild has animations here which track the median and middle 50% for repeated samples, reinforcing what you have done by hand.boxes_1samp_mem_30_600


  • iNZight had dynamic plots collecting the distribution in means or medians from repeated sampling

giphy (4).gif

  • Finally (and most often!) I do a lot of hand-waving…. firstly the simple situation of one sample where my hand is capturing sample median from repeated sampling (that is – its tracking the equivalent of the blue line in Wild’s animations above)

giphy (1).gif

The impact of sample size (NZ Curriculum Level 6 (Year 11-ish) onwards)…

The activities above can all be adapted to compare sampling variation with different sample sizes.  This year I also had students using the Kiwis population in Excel, combined with a random number to take larger samples.  The excel file is here if you want it (full lesson details are here near the bottom of the page), and the pictures of my class collection of repeated samples of different sizes are pictured below.  In this picture, the middle 50% (box) is shown, with the green section being from the lower quartile to the median, the median is in pink and the blue section is from the median to the upper quartile. This clearly shows that sampling variation gets less when you have larger sample sizes.  Please note that there are 700 kiwis in our population so when you are sampling 400 it is a large proportion of the population.


Hand-waving sampling variation with a change in sample size:

  • Take your left hand, have it track the change in sample medians for repeated sampling of size 30.  Got it?  Okay, now…
  • Take your right hand (don’t stop that left hand!), and have it track the change in sample medians for repeated sampling of size 300…
  • Okay – you’re laughing (or your students are) as it’s quite tricky to have your hands waving differently but… the point is (really easily, visually, laughably) reinforced that with larger samples the sampling variation is reduced.

giphy (3).gif

Part 2 – Confidence intervals

Part 3 – Comparing two groups