Assessment is a complex beast that once seemed so simple. When I started teaching we assessed in KS3 by getting pupils to produce a piece of extended writing at the end of the topic and assessed that against national curriculum level descriptors. This created a whole host of problems. The level descriptors were never meant to be used to assess individual pieces of work but were designed to sum up where a pupil might be by the end of the key stage. They were also full of highly subjective statements. Take a look at the KS3 Geography descriptor for level 5.
Pupils describe and begin to explain geographical patterns and physical and human processes. They describe how these processes can lead to similarities and differences in the environments of different places and in the lives of people who live there. They recognise some of the links and relationships that make places dependent on each other. They suggest explanations for the ways in which human activities cause changes to the environment and the different views people hold about them. They recognise how people try to manage environments sustainably. They explain their own views and begin to suggest relevant geographical questions and issues. They select information and sources of evidence, suggest plausible conclusions to their investigations and present their findings both graphically and in writing.
How do you actually know if pupils recognise “how people try to manage environments sustainably”? Which people? Which actions? Should they recognise that some people try to drive their car less or that there are carbon off-set schemes? Should they recognise that rainforests can be conserved through national parks and bio-reserves or the complexities of managed exploitation? You can do the same thing with any statement in any prose based descriptor. The purpose of a summative assessment is to create a shared understanding. A “5” needs to mean the same thing whether that is “meets the following criteria” or “is in the top X% of the population”.
At GCSE it was once easier to make a summative assessment of where pupils were at as we had a bank of old exam papers and their grade boundaries. We knew that on a certain paper you needed X% of the marks to achieve Y. Those days are gone. It has been depressing to see the desperation with which teachers have scrabbled around to try and create their own grade boundaries or to use the boundaries created by others. The subject groups on Facebook are full of people sharing the grade boundaries they are using; and every one is different. The grade boundaries will also shift each year to prevent grade inflation and the grade boundaries will vary anyway according to the difficulty of the paper. There is no shared understanding of what each grade means.
This has been in my mind a lot of the last few months as I have tried to create a new assessment model for the department that allows us to make a summative judgement about whether a pupil is on track or not. This is all I need a summative judgement to do. I am lucky to work in a wonderful school that doesn’t insist on teachers doing the impossible and pretend to know which grade pupils would currently be on; and as a Head of Department and as a teacher all I really want to know is, are they where they should be or does there need to be something done differently.
The first thing I did was to separate out formative and summative assessment. I want to use formative assessment well to constantly see what pupils know, understand and can do and provide them with this feedback. I want it to lead to responsive teaching.
One way we have done this to start each topic with a checklist of what we expect pupils to have learnt. We can use this as teachers to help us then plan the learning for that topic and pupils can use it to self assess where they are at and identify any gaps in their knowledge. We can test these through low stakes quizzes and in reviewing their work. We have tried to make the statements in these documents as clear and unambiguous as possible.
The strands link to our pathway statements. These tell us what we expect a pupil to be able to do in different aspects of Geography and how these will change depending on their targets. These are very helpful at identifying next steps that pupils need to make and providing feedback but not for making summative assessments.
In order to ascertain whether a pupil is On Track we are using raw scores from tests. These tests check the learning across the topic as presented in the learning checklist. A test for the year 7 topic shown above looks like this.
As you can see, this test isn’t just a test of specific knowledge but of their ability to apply this knowledge to new situations. We create a mark scheme for these tests to ensure consistency. At KS4 we are largely using the specimen papers and mark schemes produced by the exam board (but of course with no grade boundaries provided).
Once we have those raw scores from this test we compare how they did with how we would expect them to do. This tells us if they are on track. To do this, we rank pupils against their KS2 data (which is used to determine their eventual GCSE target grade) and then compare this to the test score. The result looks something like this.
This shows that we would expect Ben to receive the highest score in the test, and sure enough he did. Ed should have come in at number 2 but actually his score of 57 puts him in 9th place. His progress requires improvement and I can see he might need some intervention. By looking down the Pathway column on the right I can identify those pupils whose test score suggests the are on track, are exceeding their target (look at Tom up there with all those level 7 pupils!) or are causing concern (should Andrew be near the bottom with a target of a 7?).
We can then expand this up to include every class in the cohort and can get a clearer picture as most assessments are added. We can start to look for patterns over time and see who is climbing the ranks and who is therefore falling.
This model does have its weaknesses. It means we are only comparing results within the school cohort. We wouldn’t know if ALL pupils were under performing. I would though suggest that this is the case anyway. We don’t know that through using internal exams and by applying different grade boundaries we are just kidding ourselves. We are probably being too cautious with saying who is on track and who isn’t. For the last couple of years 90%+ of our pupils hit or exceeded their target and 50%+ achieved A/A*. Our progress score came out at around +1.58. If those results are replicated again then far fewer of those pupils would actual need an RI.
However, I want all our pupils to perform at their very best and by comparing them against each other I can clearly identify who may not be and so offer them more help. That, at the end of the day, is all I want a summative assessment to tell me.