Two weeks ago I spoke at the ‘International Perspectives on the Evaluation of PPI in Research’ conference which is taking place in Newcastle right this moment.

It was great to see so many people there – old and new friends. An engaged and lively audience. I am only sorry I could only stay the day.

Anyway, here are some of my original notes that I made for my talk. I sort of said all of this, maybe in a different way and in a different (better?) order because I was on my feet. Bit rough and ready in places but I hope all the more authentic for this reason. I think it’s also on video somewhere.

Still not sure about that conference hashtag everyone (#intppieval18) – sounds like someone from Yorkshire saying PPI is evil.

Public involvement in health research: its academic isn’t it?

Very pleased that this conference is happening and I hope that it can be the start of developing a consensus on what is and isn’t important in evaluating public involvement in health research.

The debate about the monitoring, evaluation and reporting of public involvement – or ‘impact’ if I can call it that – currently feels like one of those knots that gets ever tighter the more we pull at the loose ends.

My sense is that academia – who I think are well-represented in this room – are pulling hardest [knot slide] at the strands. I completely understand why that is so. And those of us who have spent a lifetime advancing and promoting public movement at every opportunity and against all the odds should probably be flattered by the attention.

Whether it helps to shed light on, or makes it more difficult to see, the way forward remains to be seen. I think it can do if it works collaboratively with patients, carers and the public as well as many other partners across health research. But that is by no means certain.

I suspect that untying the knot will, in the end, require something sharp to prise it apart. I am not sure what this sharp object looks like. I very much doubt it looks like a framework. But I am fairly convinced our public contributors will be behind it and I am sure they will bring sharpness to the debate in this hall over the next two days in the form of common sense.

Portrayals of public involvement: Where is this monster we have created?

I wanted to begin with some reflections on the debate thus far and in particular how this has manifested itself in the academic literature.

I am very interested in this notion that public involvement has become some sort of a monster – a Frankenstein – which is spiralling out of control to the endangerment of health research.  Interesting = and of course I am biased – because it doesn’t resonate with what I see day-to-day. I mean, where is the monster in this?

There’s no doubt public involvement is a messy enterprise but then you, me, we, are messy, people are messy, to quote the ‘Theatre of Debate’ play about public involvement. However, when I go round the country I meet thousands of patients, carers and the public who are dedicated to making health research better and working constructively with their partners.  I think we should celebrating this commitment to work with us.

Describing  it as ‘busywork – a time-consuming technocratic distraction’ as in this article or in similar terms feels deeply unfair and disrespectful. It reminds me of when I joined the voluntary sector and people used to describe me as a ‘do-gooder.’ Thankfully for science all that good work results in almost £2 billion of funding per year.

The idea also that if we don’t somehow make public involvement conform to some conceptual rules and regulations that it will bring research to its knees feels pretty far-fetched to say the least. It reminds me of a story when i lived abroad.

When I lived in the United States. I had a large and beautiful cabinet hand-made to go in my dining room of my house. This was a traditional mid-West clapper-board house which felt frail to someone used to a terraced, bricks and mortar home. When it arrived I asked the carpenter if it was too heavy and whether it would go through my floor. He said quite simply ‘if it disappears through the floor you’ve got bigger things to worry house.’ (wardrobe). I feel the same way when people speak about monsters. Is it public involvement that needs to change or attitudes within research?

Keeping it real

The other criticism and associated behaviour which interests me is the tendency to view public involvement as an ‘industry.’

This, while all the time creating more and more evaluation frameworks with the aim of putting it in its place. Now there’s a self-fulfilling prophecy if ever I saw one.  We must be in the 10s at least and it won’t be long before we reach a 100.

The issue I have with all the frameworks I have seen so far is that they feel very cumbersome for people to use on the ground. The risk is a growing gap between work that is conceptually strong but is actually divorced from reality and has no utility on the ground for those people who are facilitating the work on a day-to-day basis and the public contributors they are partnering with. (interestingly Antoine Boivin from the University of Montreal showed some data at the meeting which illustrated just this point).

Public involvement is a sprawling, growing, vibrant movement – a cottage industry at best, perhaps – as I have indicated. I wonder whether it would be so if we had evaluated and monitored and reported it in close proximity from the beginning. Or whether it would have been stifled at birth. Such is the stuff of relationships.

Seems to me that the way forward is to develop a suite of tools including frameworks that can be used and adapted by the community in a way which is relevant to their own context and purpose, proportionate and flexible to helping people to learn and develop the best way forward.

NIHR – Why and whether we do public involvement in research is academic?

There may be thirty years between the story of HIV patients trying to get into the Medical Research Council (MRC) to get their views heard about what matters in research and Charlotte Caldwell fighting for her son, Billy, to have their evidence listened to and gain access to medicinal cannabis. But the essence of their stories – of not being listened too, of their evidence by experience being dismissed, of being excluded from decisions that will affect their lives – is the same.

On a system level as Nesta’s excellent report ‘Biomedical Bubble’ pointed out this summer, without plurality of voice in how strategic decisions are made in the life sciences we are likely to see diminishing returns on investment in research and, more importantly, it fail to address the questions that matter to patients, carers and the society.

You may worry about why we should do public involvement but the NIHR is very clear about the underlying rationale and goals. Not only do we think it’s the morally right thing to do. But we also believe it leads to better research and all our evidence not only points in that direction but makes us want to take it further. Many other funders are moving quite rapidly in that direction as well.

We see public involvement as investing in a relationship not some intervention.

I don’t believe that patients and carers are always right but I do believe they always have a right to be at the table helping us make the best possible decisions and my job as National Director is not to represent the patient voice but to try and create the right environment in which their voices can be heard.

For us ‘the why’ is a somewhat academic question (although you always have to operate on the basis that someone is hearing it for the first time) and we are keen to focus less on proving and more on improving what we do. And we believe all partners and certainly the ones represented in this room have a very important part to play in this.

Keeping it real

Let me finish with a few words about how NIHR is approaching the issue of monitoring, evaluation and reporting impact.

Given what I’ve just said the irony is not lost on me when I start by saying that our work in this area is part of an over-arching NIHR-wide programme to develop a values framework loosely inspired by the Wellcome Trust success framework. The governance for this includes an external advisory group that will be co-chaired by a public contributor.

With regards to public involvement our work is being steered by a working group which brings together public contributors, the NIHR Co-ordinating centres, public involvement researchers and partners and colleagues from medical research charities and industry. As you can imagine the discussions are lively. I think we are making some sense of what needs to be done.

One of our guiding principles is that we produce or sponsor work that’s going to be helpful to people in the field. If it appears in the BMJ all well and good but that’s not our primary motivation.  This slide shows our workplan including key projects and timeline.

To put it in some strategic context I think for me the next six months we will go through quite an intense period of information gathering and benchmarking and trying to understand where things are.

The Cabinet Office is currently undertaking an audit of progress with implementing ‘Going the Extra Mile.’ Very shortly we will be inviting all our public contributors to tell us how they feel they’ve added value, whether they feel valued by us and how we can improve what we do in partnership with us. We are also looking at the data we currently collect and how we use it if at all in terms of measurement and reporting.

This will provide a foundation for the next phase which is about building a system of monitoring, evaluation and reporting around the  UK PPI Standards Test Beds which we see as process measures and our 4Rs of reach, refinement, relevance and relationships which, while in highly rudimentary form at the moment, we potentially see as success measures against which we will be asking people to report progress. We have already been doing this within the NIHR Senior Leadership team and it has been helpful.

UK Patient and Public Involvement Standards

On many levels I agree with the analysis in such papers as the recent BMJ/Journal for Medical Ethics piece of the UK PPI Standards but they do tend to fall victim to their own commentary. The most common one being a deep lack of appreciation of the context within which the public involvement movement has to work.

Very few things in public involvement have been achieved easily. Unlike other fields in research where institutions can put their hand up and attract £ millions with consummate ease, public involvement has been in the research equivalent of a street fight, going from institution to institution and lab to lab – lab fighting for thirty years if you like. We have rarely been in the position of being able to afford perfection even if that was possible. A lot of what we do is borne out of local anarchy.

An important part of the context for projects such as the UK PPI Standards has also been the level of involvement of public contributors in its development. That it doesn’t conform to people’s assumptions about standards is probably the whole point. We want to challenge the consensus. We recognise the need and are partly forced to do things differently. But our door is always open to others to join us and help us improve what we do.

Conclusion

It’s great that we are having a more open and (perhaps) civilised debate about this. Long may it continue. The answer can never solely be academic and although the ‘why’ may well be.