On research and banning things

Regular readers will know that I am broadly (ok, more than broadly, but not blindly) in favour of the use of technology by students, which experience and evidence suggests can be beneficial. I am also not blinkered to the issues that can prevent this happening.

So the publication of a piece of research that suggest the banning of mobile phones in schools improves outcomes is of obvious interest to me. The working paper can be viewed here, and I would suggest you give it a try. It is very readable.

The importance of any one piece of research can only really be tested by time. The results in the paper stand for themselves and if you agree with them then my listing other papers that suggest the opposite are unlikely to change your mind, in the same way that providing more evidence of the same is unlikely to get those who are positive about technology to pull all the plugs. But good research should make us all stop and think and consider what it means.

We do that by being critical of research. Now some people take that to mean “He doesn’t agree with it so he is going to pick holes in it and pretend it has no value.” Well it doesn’t mean that. It means we have to look at where the weaknesses are, what’s missing, where are there things that need to be explained, where are the unsupported jumps in logic.

To this end I have read the paper and as I went through have just jotted down my thoughts, the questions and clarifications I would like to be able to ask the authors if they were sat in front of me. Some of these notes are fully formed questions, others are just notes of what is in the paper.

• No data on who owned mobile phones. Uses phone ownership among over 13s as a proxy for 13-16yr olds.
• Definition of ‘widely complied with’?
• Context of the schools that imposed the bans
• Wholesale ban vs visibility ban – Is this tested in the report?
• Only schools in cities were targeted
• Survey sought from 450 schools with 91 respondents – is this all the secondary schools in the four cities?
• Survey data is from 91 schools with 90 having complete bans – this seems ‘rather’ high as a proportion – self-selection? Am I reading this right?
• Sample schools contain significantly more minority, SEN and FSM -eligible pupils – answers the context question above.
• Compliance on a 7 point scale, 1 not at all, 7 Completely. Intermediate points not defined.
• Most bans introduced in 2005-10 – coincides with London Challenge time?
• Paper suggests no upswing in results in these schools in previous years results – given London Challenge this mightt suggest that these schools are not representative of the whole.
• Provides no evidence of pre-ban in-school use of phones (though one could take the need to introduce the ban as evidence it was required).
• “it needs not be the case for an individual to use a phone to be distracted by them, their use by others in the classroom may cause disruptions. ” – evidence for this?
• “There may be a concern that only high-achieving schools introduce mobile phone bans, which could lead to overestimating the effects of a mobile phone ban.” – I would have assumed the opposite.
• How the paper handles the issue of other policy changes is “interesting” – ‘The variable OtherPolicy­_st takes a value of 1 for the years after a change at a school occurs. We combine information coming from our survey of headteachers and information from school’s website. We do not observe multiple change of policies/leader in addition to the phone policy change, hence a binary variable can be used.‘ – So there are either other policy changes or there aren’t, no consideration of what they are.
• The statement – “We add to this by illustrating that a highly multipurpose technology, such as mobile phones, can have a negative impact on productivity through distraction. “ is unevidenced – the results in the paper show that there is an effect. They do not show why the effect occurs.

The above necessarily focuses on issues the paper doesn’t for me answer.

My experience is this. Schools introduce phone bans in response to an existing issue within the school. Such a ban is rarely introduced as the sole change and consequently any changes need to be seen in that light.

I have seen so many instances of great schools who tell visitors “We do A, B, C, D, E, F and a little bit of G.” After which the visitors go back to their own school and vigorously implement lots of D and expect their school to become great as a result. They choose D because it is do-able and it suits their biases. Change in complex environments is complex. Picking one element rarely works.

Let’s continue the research. Let’s answer my questions and any others. Let’s see a paper that does the same for schools that have rigorously implemented and enforced cross-school use of mobile technology so we can see the comparison.

And let’s continue to debate the issue.

But right now, I have to go and take the iPad away from my 11yr old so he gets out of bed!

Advertisement