Running through the Minefield

One of the phrases I’ve seen used over the past few years has been ‘the evidence says’. Well, to be more precise, ‘the research says’. It is a phrase which can either provoke people into action, grabbing onto the newest book or document that comes out ready to change the world with it, or send them into spasms of eyerolling, dismissing it with a well-placed ‘research can prove anything’ or ‘it could never happen here.’ These reactions are understandable from both positions. On the one hand teachers and leaders would love there to be a silver bullet which will ‘fix’ all of the issues we face day to day. On the other there has been a lot of guff which floated around either under the label of research or having mutated into something which bears no relation to what was originally said. With all that in mind then, is there a way we can use the evidence and research to our benefit?

The benefits of research

I personally am a big fan of research and evidence (there is a distinction, and I will come back to that later) and I have found it has enhanced my work with my students and my colleagues in many ways over the years. Examining the research has also made me more reflective about my own practices and has given me more in control of what I am doing in my classroom. I don’t just mean in terms of push back to those who may be enthusiastically waving the latest book in my direction, although that certainly has been a factor, but as it has provided me with a greater understanding of why I am doing what I am doing. Having a clearer understanding of not only how something works, but why it works has really helped me to refine what I am doing. It has sharpened my focus. Knowing what the evidence or research says has empowered me to challenge certain things which seem to refuse to get in the bin (VAK and onerous marking policies anyone?) and allow me to be more certain that when I am pouring my energies into something, a limited resource, it is being poured into something which is more likely to have a positive impact on my students.

I am also very aware that phrases like ‘the research says’, ‘just look at the evidence’ and ‘we are following the research’ aren’t especially useful blanket terms and they can be used to obfuscate when people either aren’t happy to explore the processes and policies they are asking of others or are uncertain of what it means themselves. Its previous equivalent is probably ‘Ofsted want’ as a way to get people to comply without question, but that is something that has become a bit more problematic now that Ofsted have shared more about their research base.

It is no wonder then that people swing between confused and cynical with dizzying speed. This is further exacerbated by the fact you can easily come across evidence which seems to contradict itself before you have even had time to digest what the first part might mean for you.

So, what can we do to help to navigate our way through this maze of information?

Complexities in the research

Let’s first tackle the idea of research. What do we mean by this? In the world of science, it is often quite easy to see what it looks like. It is clinical, sterile and controlled. When we think about how this applies to the variable and highly unpredictable wonders that are human children, it can be hard to picture how that might work. They are far from sterile, even in times of deep cleaning and hand sanitising, and just the word control associated with them can produce more than a wry smile from most teachers.

However, research which aims to isolate different factors, use control groups and consider variables and efficacy are indeed part of the world of education. But how these are achieved are also points we need to consider when we think about research.

What were those control groups? How did the researchers gather the information about the impact, or lack of impact, of what they were doing? Did they publish all of what happened or part of it? Could their finding be replicated? How scalable is the research?

One of the interesting areas we looked at when writing Generative Learning in Action was with the section on ‘Teaching’. It seemed that there was a significant research base, encouraging Fiorella and Mayer to include it in the eight strongest generative strategies, which suggested students teaching other teachers is a positive activity which improves their understanding and retention of key information. However, in the studies themselves it was quite difficult to isolate the teaching from other elements involved in the process or control for them. For example, it seemed that just telling the students they would be teaching their peers had an impact, even if they didn’t then have to actually do the final activity. However, if they did do the final teaching stage, the impact on their learning did seem to be greater. But was that because they had gone through the process of explaining it, as opposed to just thinking they would and preparing for that, or was it the impact of the follow up questions which were also part of the process? The research seems to point in the direction that it was a combination of knowing there would be an audience, going through the learning process with that outcomes as a focus and then having your ideas challenged in some way that had the impact. Trying to decide which one had the greater impact or finding out exactly what happened if those conditions changed proved hard. That may mean, if students don’t have their understanding questioned, they won’t gain as much from the process. Or, if they are told they will present but don’t, does it mean the learning won’t be as well embedded? Even when all of the elements are accounted for there are still lots of caveats.

However, these are also good questions to explore, especially when you are considering taking something forward into your classroom.

Other questions too such as how many students were used in the trial or study? Which age groups? What subjects? Was it research that took place in this country or elsewhere? What did the control group suggest? How was the impact actually measured? The question list goes on and on, but if you want to be able to make the most of the research in your own setting, it is worth considering these points as you explore.

There are also some important questions worth considering in relation to the direction of the research too. Who conducted it? How was it funded? How was that information subsequently used? There is plenty of research into the use of different technology in education but when you dig down you discover it was largely funded by the tech company themselves. Whilst we probably shouldn’t dismiss it out of hand, if recent events in the world tell us anything, it is following the money can be an interesting, and illuminating, trail to follow.

It is also worth considering what previous research has been conducted by that particular group or researcher? Do they seem to have a particular area of interest and could that possibly have an influence on their outcomes? That doesn’t mean it is necessarily a bad thing, as it may just reveal a particular series of ideas they have explored, but it might help us to consider why research is happening and some of the conclusions drawn. To return again to the example of Generative Learning, Mayer is particularly interested in how technology can enhance independent learners. Whilst I by no means would want to suggest he is anything other than completely open to evidence which might be contrary to what he has found, and very aware of his own biases and concerns, he is a researcher and it is his bread and butter, I can’t help but wonder if it has shaped some of his journey and the evidence that he has collated. It is just another question to consider when I think about how it will apply to my context.

Context is key

My context is perhaps one of the most important things to consider. As Dylan Wiliam, almost infamously now, said “Everything works somewhere; nothing works everywhere.” (Wiliam, 2018). In education there are best bets as opposed to absolutes when it comes to research and it is worth considering all of those questions regarding boundary conditions (age groups, subject etc) when I think about what I want to use with my students. I might come across research which tells me that back-to-back fast, teacher led reading has a really positive impact for students (there is some research which does indeed seem to point to that), especially for those who are finding reading difficult, but if I work in a school where there is only one tutor time, or English lessons are so chaotic, I will very likely have issues trying to use that at the moment. I might have to consider how I could create the right conditions for this and work towards using it. Whilst I’m doing that, I could then be considering what books might be best for this as well as how and when these texts would be used and what training and support might be needed. As you can see it is not as simple as finding some research and running with it.

I mentioned earlier about the distinction I make with research and evidence too. Research is one type of evidence and I like that there is a broader way of looking at it. By evidence then I mean other types of information such as case studies and lived experiences. Simply by writing a blog about how I see and use research doesn’t make me a researcher, but it does add to the wealth of evidence that is available. We need to be just as careful not to dismiss what has happened in schools, including both what has and hasn’t worked, just as we need to be careful about rolling our eyes when someone mentions research. Knowing how that school was embedding the back-to-back reading could be really useful in me exploring how I could adapt things for my context and have the potential to make it more successful, especially if they were battling some of the same issues as my school might be. It means I have something else to consider which could help me to understand the pitfalls and issues which can come with the research. One of the things Mark and I were really keen to include in both Generative Learning in Action and our upcoming book on CPD, was the case studies of the people who had put the work into practice. Building up the evidence base and seeing a bridge between the number of studies and measurements and actual classroom practice is something we see as powerful and hope others do too. Certainly, it has been helpful for me to move my thinking from some quite complex abstract ideas into the actual world of students and classrooms.

On my last check there were 177,000 articles about metacognition on google scholar. This is the other issue with ‘research says’. I am pretty certain some of that research, even if I had the time to explore a fraction of it, might not all say the same thing. This is possibly one of the biggest issues in terms of the minefield for me. That is why I always start by asking people to go back to what they want to achieve with their students. Once they have a clear idea of this, then we have a direction of travel through the maze of research, case studies, articles, literature reviews and blogs. Keep what you want your students to achieve though at the heart of your search, with plenty of questions to create a trail of breadcrumbs on your way, then hopefully you won’t get too lost.

Zoe Enser is the lead specialist English advisor for Kent schools with The Education People and an ELE working with the EEF. Her second book, The CPD Curriculum: Creating Conditions for Growth, is available for pre-order now.

3 thoughts on “Running through the Minefield

  1. Ashman looks into meta-cognition and shows it is not really clear what it is and the authors of studies are not consistent in their definitions –

    Fletcher-Wood delves into Feedback and picks up on an aspect you mentioned – what are they measuring? In this case it was not achievement, as claimed by the EEF & Hattie, but behaviour -

    Others show the problem Ashman highlights – the definitions of feedback by researchers is totally different-

    The conflicting results in Educational research is a real issue. Holmes et al. (2006) warned – “Unmasking the hidden politics of evidence-based discourse is paramount…”


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s