It’s not only scores that have scales

The DfE have now published something on the new KS2 tests and how the results will be presented, the catchily titled “Scaled Scores“.

I have a couple of issues with the document.

Let’s start with what it has to say about the scale itself.

We can’t give full information about what the scale will look like yet. We need to wait until pupils have taken the tests and the tests have been marked before we can set the national standard and the rest of the scale. We can’t set the scale in advance; this cohort is the first that has reached the end of key stage 2 having studied sufficient content from the new national curriculum. If we were to set the scale using data from pupils that had studied the old national curriculum, it is likely it would be incorrect.

We do know the scale will have a lower end point below 100 and an upper end point above 100. Once we have set the national standard we will use a statistical technique called ‘scaling’ to transform the raw score into a scaled score. We will publish this after the first tests have been administered.

The interesting part of this is the thought that until the test has been sat the national standard cannot be set. This does kind of indicate that rather than ‘the national standard’ being based on some idea of what a student working at ‘the national standard’ should be capable of, it will instead be based on the score that some predetermined percentage of students achieve in the test.  So it should be possible to publish either the test score that will equate to ‘the national standard’ or the percentage of students that are expected to achieve ‘that national standard’. As it has done neither I can only assume that this number will be fixed for best political advantage after the test scores are known.

Secondly, the document says,

The standards underpinning the scale will be maintained as long as there is no large-scale change to what the tests cover. Once the national standard has been set in summer 2016, we will maintain the standard in subsequent years by using a process known as ‘test equating’. When we trial future tests in schools, we also administer a separate ‘anchor test’. This test remains the same over time. It allows us to link scores from one test to another to ensure standards are maintained.

This is where I have the real problem.

Key Stage 2 comprises Year 3,4,5 and 6. The students that will be taking this test in 2016 will have had two years of study on the new National Curriculum. With the very best of intentions, many schools are still learning exactly what the curriculum requires of them and their students. Everyone is learning how best to work with the new curriculum.

So how can it be sensible (and we’ll leave the idea of ‘fair’ there as well) to set ‘the national standard’ based on such a set of circumstances?

One of two things will happen.

Either future cohorts will have their accomplishments suppressed as their test scores are scaled to fit the 2016 outcomes. ‘The national standard’ will be maintained but the results will be next to useless in assisting with transfer information, one of the problems the new system was supposed to improve.

Or, because cohorts in 2017, 2018 and 2019 will have had the benefit of much greater percentage of their time studying under the new curriculum, with teachers who have much better understanding of its requirements, it is likely that they will perform better than the 2016 cohort and ‘the national standard’ will have been exceeded. This will, of course, allow the DfE to claim this as clear evidence that their reforms are paying dividends.

I know which one my money is on happening.