Chris Potter has a thing or two to say about the use of statistics and, in doing so, he takes us to a Friday night steak house that is prone to airway disasters and on a short tour of his love life over the last 20 years.
On appointment to my first consultant post (which, much like my first marriage, is now to be found very much in the past tense) I was immediately packed off on a ‘team-building’ exercise with my new colleagues and subjected to a personality test. The moderator was a big proponent of neuro-linguistic programming and proved himself capable of spouting an astonishing variety of gullible pseudoscientific psychobabble for what seemed indefinite periods.
I discovered a number of fascinating things that afternoon (albeit mostly pertaining to my personal boredom threshold) but was astonished to find that, according to the questionnaire feedback, I was uniquely deficient in the quality of humility. Being a clinician of consummate skill, an aesthete sans pareil and widely regarded by my peers as a paragon of beauty and virtue, this came as somewhat of a surprise. However, as the ravages of time have taken their toll, I have gradually realised that a degree of therapeutic humility (if not absolute nihilism) is not altogether unwarranted in the clinical field.
It is very easy to oversell our skills to patients, and indeed it has been known for a long time that putting a positive spin to consultations may have dramatically positive therapeutic effects [1]. But there is a fine line between boosting patient morale and making misleadingly optimistic claims to cloud the true clinical picture. A variety of statistical legerdemain has been used over the years to facilitate this process, and I fear I may need to get a bit technical here. Now, I for one have absolutely no time for those chin-stroking eggheads in the medical statistics department, with their tedious pedantry and hair-splitting dogmatism but, just occasionally, they come into their own.
"There is a fine line between boosting patient morale and making misleadingly optimistic claims to cloud the true clinical picture"
Let us look at an example from the recent COVID brouhaha. The Moderna vaccine trial [2] showed only a 0.07% infection rate in the treatment arm, with 1.2% of the placebo arm infected during the trial period. Thus, there is a relative risk (RR) of about 5% of contracting COVID after the vaccine, giving a relative risk reduction (RRR) around 95%, leading to frequent statements in the news media of a ‘95% efficacy’ for the vaccine. All well and good, but the absolute risk reduction (ARR - 1.2%-0.07%) was only just over 1%, meaning the number needed to treat (NNT) was nearly 100! Now, a vaccine that needs to be given to 100 patients to benefit just one is a very different prospect from one with a 95% efficacy in the mind of the public, but both viewpoints may be defended.
Before you write me off as a tinfoil-hatted anti-vaxxer lizard-king proponent, do bear in mind that the duration of this trial was only around three months for most participants and, although the absolute risk was tiny during this period, in an exponentially growing epidemic the relative risk reduction becomes more important, as the prevalence of exposure to infection inexorably grows. Hence, as a portly diabetic in the prime of life carrying out multiple daily aerosol-generating procedures, I made quite sure I was near the front of the queue for the vaccine and following boosters.
Thus, for many years the cognoscenti have looked down upon the RRR as a misleadingly overoptimistic statistic, ideal fodder for tabloid advertisements and press releases, and highly sensitive to the prevalence of the condition in the general public.
"Suppose the mortality rate of choking under these circumstances is around 60% but, if I swing into action with my steak-knife before the Malbec kicks in, I reduce this to 20%"
For a highly theoretical example, let us suppose my personal mortality rate for awake crash tracheostomy is exactly 5%, and every night I eat at the same steak house that serves extremely cheap cuts of meat to edentulous nonagenarians with pseudobulbar palsies. Suppose the mortality rate of choking under these circumstances is around 60% but, if I swing into action with my steak-knife before the Malbec kicks in, I reduce this to 20%. This represents a RRR of death of 66%, and an ARR of 40% benefitting from my actions, with 5% suffering adverse effects. This represents a NNT of 2.5 and a NNK (number needed to kill) of 20; a decent risk:benefit payoff, I hope you would agree. However, word gets around the town of the body bags piling up outside the dodgy restaurant and Tripadvisor reviews start to become alarming. The clientele subsequently changes to healthy young gym bunnies with perfect teeth and upper airway reflexes. Mortality from a choking episode now drops to just 3% but, if I get busy with the blade, I can further reduce this to 1% with my surgical heroics. Once again, I have a RRR of death 66%, but ARR of only 2%, with a NNT of 50 and still a NNK of 20 from my cavalier knife work. The risk:benefit ratio now very much favours me ignoring the chaotic airway emergencies and concentrating on knocking back the Malbec and perusing the dessert menu.
Thus, the same highly- effective competent, and partly sober clinician carrying out the same procedure with the same efficacy in terms of RRR can have wildly different clinical outcomes depending on the population factors. Only ARR and NNT can take these factors into account and, thus, are understandably beloved of health statistic wonks.Relative risk increases (RRI) can be equally misleading and alarming, as was demonstrated by the UK Committee on Safety of Medicines in 1995. A study had revealed taking a third-generation oral contraceptive led to a doubling of thromboembolic disease in young women (an RRI of 100%!), causing a series of terrifying tabloid headlines, and a cohort of ladies immediately ceasing contraception, and an estimated 13,000 additional terminations of pregnancy over the next 12 months. The absolute risk of thromboembolic disease in this age group was about one in 7000, giving an absolute risk increase of 0.0014% and a Number Needed to Thrombose of 7000. Sounds a bit less terrifying when put that way doesn’t it?
"Now, in the distant past as a gilet-wearing, MX-5 driving, libidinous young registrar, 0.3 of a sexually satisfying encounter was barely worth crossing the payday disco floor"
The NNT is a very simple statistical measure of a treatment’s impact; an intuitive, honest and invaluable aid to decision-making and reinforcing realistic patient expectations [3]. Hence, it is hardly ever mentioned in press releases, trial outcome discussions or pharmacological company literature. Certain interventions, such as parachutes for gravitational challenge (NNT 1), the Milwaukee Protocol for Rabies (NNT 1.2), and defibrillation for cardiac arrest (NNT 2.5) are justifiably promoted for their efficacy. Similarly, none of my radical septoplasty patients have ever shown enthusiasm to endure a revision procedure at my hands, of which I remain enormously proud. Very few ENT interventions have undergone the relevant scrutiny, but steroids for acute pharyngitis (NNT 3) and the Epley manoeuvre for benign positional vertigo (NNT 3) are much more effective than antibiotics for acute sinusitis (NNT 17 but number needed to harm, only eight) or glue ear in children (NNT 5 and number needed to harm of 20).
Those of you of a squeamish disposition should probably avert your eyes from the figures of number needed to screen (570 for mammography, 1250 for colonoscopy, 1410 for prostate-specific antigen). The more broad-minded of our readership (you know who you are) may already be aware of the miracle drug, Flibanserin, approved in the USA for the treatment of female sexual dysfunction [4]. Close scrutiny of the literature reveals that regular treatment leads to an increase of 0.3 “satisfying sexual events” per woman per month. Now, in the distant past as a gilet-wearing, MX-5 driving, libidinous young registrar, 0.3 of a sexually satisfying encounter was barely worth crossing the payday disco floor. Nowadays, given my current difficult domestic circumstances, it would cheerfully see me through to the New Year. Merry Christmas, readers!
References
1. Thomas KB. General practice consultations: is there any point in being positive? Br Med J (Clin Res Ed) 1987;294(6581):1200‐2.
2. Polack FP, Thomas SJ, Kitchin N, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N Engl J Med 2020;383(27):2603-15.
3. Joffe HV, Chang C, Sewell C, et al. FDA Approval of Flibanserin--Treating Hypoactive Sexual Desire Disorder. N Engl J Med 2016;374(2):101-4.
