Monday, July 1, 2013

The Number Games

Joseph Pultizer once said.. “There is not a crime, there is not a dodge, there is not a trick, there is not a swindle, there is not a vice which does not live by secrecy”…and probably has been the mantra which has been at the core of the NHS latest effort at transparency. At the heart of what Sir Bruce Keogh has stood for..the effort to be transparent, the effort to be open, the effort to shake off the damning Francis report, the desperation perhaps to rise from the plethora of bad news..Mid Staffs, Morecombe Bay…they keep rolling on, don’t they? Harrowing accounts of how the NHS failed those it was set to serve, scapegoats being served out, hung out to dry….

Now I know Sir Bruce- I have met him- and I think his passion to improve the NHS is indeed something to be inspired by. There is no questioning his desire to create transparency but one think I have learnt in management is this. If you want to use data to do performance management and drive up standards, then you have to battle past a treacle of cynicism, a basic mistrust of NHS data, an ingrained belief that the “data isn’t right”. Because if you show data which is then peeled apart and shown to be wrong, you have not only lost face, but also lost the troops. You have lost their trust that the data you produce is meaningful, that it actually is about patient care and not just about ticking some boxes. So when the vascular data came out, I did indeed feel emboldened to believe that perhaps, just perhaps, the powers that be had indeed got it right.

And then I opened the Telegraph. Sensationalised was the fact that a certain surgeon from our local Trust had the highest mortality rate in Aortic Aneurysm repair. Nearly a third of his patients had died..shock and horror.…how do we allow such poor surgeons to operate? The Daily Mail screamed the same thing. Problem? This person hadn't done such surgeries since 2010/2011- and is a specialist in carotid endarterectomy..his mortality rate there? A mere 0%. The Vascular Society made amendments to their document later in the day- but by then the damage was done. Collateral damage, said some. Part of the learning process said others. Its always easy to be philosophical when your own name isn't being dragged through the mud, isn’t it?

Moving away from the individuals, lets look at it in a scientific way. Here is a quote from the document concerned: “Often information about the severity of disease is added to a risk adjustment model but a minority of patient records in the dataset were missing this information for one or two variables and, to avoid dropping these patients from our analysis, we did not use these variables for risk adjustment”
Seriously??…no account of severity of illness? No account of quality of pre and post op care? No account of coexistent morbidity? You now account a death directly to a Consultant..the other members of the team don’t matter any more? Those surgeons who have performed “well”..where’s the recognition for the team around him? The anaesthetist, the nurse, the physiotherapist, the GP..all of whom factor in the 30 day post operative recovery? What use such data then?

Let me give you a personal example. How do you measure me as a doctor? Our local amputation rate was poor in does that mean I am to blame as the departmental lead? No responsibility of the nurse specialist, practice nurse, GP..all of whom are involved in the patients diabetes care? No responsibility of public health- who still cant get a grip on smoking which is known to make amputations worse? Anyhow, lets assume that makes me a bad doctor. Now how about patient feedback? A patient kindly nominated me and I won Hospital doctor of the now I am a good doctor?

The reality is that medicine is complex and you cannot just take a marker out of thin air and fix it to a Consultant saying all of the process is a cross for him to bear. You want to measure and have transparency? Then do so for the team. Measure the outcome of a vascular team. Measure how they perform together. It is a crying shame that the evangelists are quick to quote Formula 1 racing and airlines as models to emulate- but forget to stay away from individuals rather than team while estimating performance. Jenson Button maybe a fabulous driver..without the right team and car? Struggles to qualify. I was there at one called him a poor driver...they said he needed a better team, a better car.

My worry? It switches off people from learning and using data to improve patient care. Cynicism sets in. Gaming begins. To get your mortality rates down, don’t touch any patients with a higher risk. Here’s some simple but scary maths. 10 patients have complicated problems…if a skilled surgeon operates on him/her,, 5 may survive. His mortality rate? 50% ergo BAD surgeon. Walk away from them all? Mortality rate 0%  ergo GOOD surgeon. Who loses? Those 5 patients and their family who could have survived due to the daring and risk taken by the surgeon concerned. 

Be very careful what you wish for in the name of transparency. A lot of things are at stake here. Reputations; morale; the ethos of teams and above all, patient care. I am a big believer in transparency- heck, I perhaps am the only Consultant in my Trust who has no issues with his patient feedback being on-line- but do so properly, not to satisfy any diktats or tick a box. 
One final request to the data evangelists. Don't take any opposition to such poor data as obstruction to transparency. This isn't the invasion of Iraq...where you are "either with us or against us". This is some advice about using datasets appropriately. We all fight the same battle of improving patient care.

In a line from the Book of Pages…”Telling the difference between transparent and invisible is an acquired skill. Until you've practised, you can't make the choice between looking through and looking at”. 


No comments:

Post a Comment