‘Monsters Inc’ holds the key to assessing research impact #REF2021 #OxfordImpact

(Warning several spoiler alerts!)

If the sight and sound of research institutions and organisations bellowing about their impact on life and the universe has a familiar ring to it, you’d be right.

The reference point you are looking for – but have understandably mislaid – is ‘Monsters Inc.’ The brilliant 2001 Pixar animation in which a motley crew of loveable and not so loveable monsters seek to outdo each other in making small children scream and cry in the middle of the night. As you’ll remember, their shrieks of terror are captured and turned into electricity to power Monster City.

Yes, the research community is doing its VERY best to tell us VERY loudly about the HUGE/MASSIVE/INCREDIBLE difference it is making to you, me and the world. And this cacophony of noise is likely to reach ‘11’ on the volume button as we head towards the Research Excellence Framework (REF) in 2021. The REF is the name given to the exercise by which the UK decides which institutions are doing the best research. Since 2014 it has included a component which assesses institutions for the impact their research has had on society.

That institutions should care about the difference they are making to society is a good thing. That they should be assessed as to whether this impact is real (or imagined by the Vice Chancellor) is also a good thing. After all, if you roll back the years to before the 2014 REF when ‘impact’ was first assessed, much of the research community was kicking and screaming about the mere notion. So loudly in fact that we could have solved the world’s energy problems for at least a month. I really do wish it was 50% of the REF instead of the current 25% just to see what would happen.

Now that ‘impact’ is part of the laboratory furniture so to speak, thought is turning to how the devil we should define and measure it? What does it look like? Would we know it if we walked into it? How can we reduce the fear?

Last week I attended a very good event about impact hosted by the NIHR Oxford Biomedical Research Centre and the Cochrane Collaboration (#OxfordImpact). Then Chaired the NIHR working group on public involvement impact. Each meeting in its own ways was trying to get to grips with the issues.

The debate about impact has given me more highs and lows in the last 12 months than watching the England football team over the last 50 years. One minute I’m running round my kitchen shouting ‘I’ve solved it.’ The next minute I’m sobbing at my desk, inconsolable amidst the piles of screwed up bits of paper pleading ‘why, why, why?’ The point being – and here’s my lightbulb moment of the last week (spoiler moment, I am actually quite dim) – this is an issue which has no solution. Rather it’s a question that can only be answered by asking lots of other questions. Which lead to further questions etc. And so slowly we will form a picture. But it will never be perfect I suspect. Where would the fun be in that?

I have distant memories of a similar ‘impact’ frenzy gripping the charity sector many moons ago. It sparked very similar behaviour. Screaming. Hand-wringing. People with ‘Impact’ in their job title (some of my best friends are impact experts). Overpaid consultants. An industry of…well, an industry. Frameworks you wouldn’t hang your washing on. Etc etc. And eventually it has all settled somewhere sensible where people do their best to say what difference they have made. A place where there is more art than science in all truth.

I suppose where I am for the moment is believing passionately that research impact is in the eye of the beholder. And that, in respect of health research, no conversation about impact can get very far without understanding how our publics see it. What difference do they think health research should be making to them and their family and neighbours? Our public, our citizens, can provide the ‘moral narrative’ as it was called in Oxford that can get this debate off on the right foot. Hence the NIHR impact working group will try to tackle this first. With the help of many others.

If you don’t believe me then just watch Monsters Inc. As you’ll know the story ends with our heroic monsters Sullivan and Mike Wazowski exposing the corrupt factory managers for what they are and discovering that its children’s laughter that creates the greatest energy. And they all live happily ever after.

There’s really no need to make us scream.

Have a great evening.

4 thoughts on “‘Monsters Inc’ holds the key to assessing research impact #REF2021 #OxfordImpact

  1. Ha! Nice – I’d settle for a role in Monsters Inc (although not in Monsters University the terrible prequel). Thanks for clarifying a really important point Simon. For all the hand wringing and the desire to get to an objective set of metrics of impact, impact is inherently subjective and driven by values. I look forward to the work your NIHR PPI Impact group is doing because its a part of the conversation that hasn’t been heavily involved in many previous iterations of the work. Good luck!


  2. Thought provoking. I’ve recently spent some time wrestling with the question of how you would assess the impact of NIHR itself. I’m not vastly impressed by the overly celebratory report that DH commissioned from RAND Europe for the 10th anniversary.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s