How do we assess evidence, and how should ENT surgeons use EBM?
Evidence-based medicine (EBM) is the practice of medicine based upon high quality scientific research. There are several formal definitions of EBM, the most widely quoted being that of Professor Sackett: “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.” [1].
Initially, it was used to describe a process by which clinicians made decisions about individual patients and, also, the teaching of this approach. Rapidly it developed to include decisions made upon populations of patients and has expanded further with the idea of evidence-based practice.
Before the introduction and development of EBM, medicine was taught and practised on a theoretical basis: it still mostly is. Many clinicians rely on their experience or reasoning to manage patients. Clinical meetings are often dominated by he or she who talks with authority, quoting papers that the rest of us have neither heard of nor had chance to read.
The use of ‘in my experience’ may be less commonly heard, but its sentiment masquerades in an uncritical approach to medical practice. EBM aims to downplay opinion, which is often limited and biased, and supplement it with evidence from scientific literature in order to establish best practice. It is widely accepted as the gold standard of medical practice.
Although statistical methods were used in medicine in the 19th century, the desire for a rigorous scientific approach to the clinical practice has only recently been implemented. The idea of ‘the art of medicine’ and an unquestioning acceptance of the subjective opinion of the expert held sway for many centuries. It was only towards the end of the 1960s that voices began to challenge traditional opinions as to how medicine should be practised. The idea that subjective expert opinion might be biased was highlighted, along with the variation in clinical practice.
The lack of controlled clinical trials supporting medicine was given attention by Archie Cochrane in Effectiveness and Efficiency, published in 1972. Closer correlation of epidemiological studies with clinical practice in the 1980s further laid the groundwork for a movement away from subjective, expert guidance to an approach based upon high quality, available evidence.
Evidence-based medicine was named when Gordon Guyatt, Professor of Epidemiology and Biostatistics, took over as director of the Internal Medicine Residency Programme at McMaster University, Hamilton, Ontario, Canada in 1990. He wanted to change the programme so that physicians managed patients based not on what authorities told them to do, but on what the evidence showed worked. At first, this approach was called ‘scientific medicine’, but this name was met with resistance from some members of Guyatt’s medical faculty, who resented the implication that medicine, as practised at the time, was not scientific. The approach was renamed ‘evidence-based medicine’.
This stuck and is now widely accepted. The sense and advantages of this approach were quickly identified, and what began as a decision making approach on an individual level, rapidly expanded into a way of formulating decisions made on populations. Guidelines, which had previously been drawn up by ‘experts’ were, in part, replaced by ones in which the criterion for the selection of research and its analysis could be identified. Levels of research were described and ranked, allowing its importance and influence to be determined.
The Scottish Intercollegiate Guidelines Network (SIGN) was established in 1993, its aim being ‘to improve the effectiveness of clinical care for patients in Scotland by developing, publishing and disseminating evidence-based guidelines that identify and promote good clinical practice.’ At present, it has published 50 evidence-based guidelines, covering areas ranging from schizophrenia to breast cancer, and including head and neck cancer. The National Institute for Clinical Evidence (NICE) was set up in 1999 with similar aims.
The guidelines for the ‘Diagnosis and management of head and neck cancer’ were published in 2006 and marked a step forward in our ability as ENT / head and neck surgeons to practise EBM. The guidelines document the levels of evidence, the grades of recommendation and, perhaps most importantly, the search criteria used to amass the data (http://www.sign.ac.uk). Refreshing our memory of these three areas is helpful, because they lie at the heart of EBM. Note that ‘expert opinion’ is considered the least important amongst the levels of evidence.
(1) Key to evidence statements and grades of recommendations
Levels of evidence
- 1++ High quality meta-analyses, systematic reviews of randomised controlled trial (RCTs), or RCTs with a very low risk of bias
- 1+ Well conducted meta-analyses, systematic reviews of RCTs, or RCTs with a low risk of bias
- 1 - Meta-analyses, systematic reviews of RCTs, or RCTs with a high risk of bias
- 2++ High quality systematic reviews of case control or cohort studies / High quality case control or cohort studies with a very low risk of confounding or bias and a high probability that the relationship is causal
- 2+ Well conducted case control or cohort studies with a low risk of confounding or bias and a moderate probability that the relationship is causal
- 2 - Case control or cohort studies with a high risk of confounding or bias and a significant risk that the relationship is not causal
- 3 Non-analytic studies, e.g. case reports, case series
- 4 Expert opinion
(2) Grades of recommendation
Note: The grade of recommendation relates to the strength of the evidence on which the recommendation is based. It does not reflect the clinical importance of the recommendation.
- A At least one meta-analysis, systematic review of RCTs, or RCT rated as 1++ and directly applicable to the target population; orA body of evidence consisting principally of studies rated as 1+, directly applicable to the target population, and demonstrating overall consistency of results.
- B A body of evidence including studies rated as 2++, directly applicable to the target population, and demonstrating overall consistency of results; or Extrapolated evidence from studies rated as 1++ or 1+
- C A body of evidence including studies rated as 2+, directly applicable to the target population and demonstrating overall consistency of results; or Extrapolated evidence from studies rated as 2++
- D Evidence level 3 or 4; or Extrapolated evidence from studies rated as 2+
(3) Systematic literature review
“The evidence base for this guideline was synthesised in accordance with SIGN methodology. A systematic review of the literature was carried out using an explicit search strategy devised by a SIGN Information Officer. Databases searched include Medline, Embase, Cinahl, and the Cochrane Library. The year range covered was 1998-2004, although searches for certain questions went back to 1990. Internet searches were carried out on various websites including the New Zealand Guidelines Programme, the Canadian Medical Association, NELH Guidelines Finder, and the US National Guidelines Clearinghouse. The Medline version of the main search strategies can be found on the SIGN website, in the section covering supplementary guideline material.
The main searches were supplemented by material identified by individual members of the development group. Each of the selected papers was evaluated by two members of the group using standard SIGN methodological checklists before conclusions were considered as evidence.” (http://www.sign.ac.uk)
This precise explanation as to how the information was collected is a particular strength of the SIGN guidelines. It allows the reader to determine how comprehensive the guidelines are and how much credence should be given to them. In the case of the SIGN guidelines, the search criteria are placed on page 66, shortly before the bibliography. An earlier position is desirable; similar to the prominence placed on the materials and methods section of a research paper. In most papers, the methodology of the study indicates how likely the research is to yield useful data. Guidelines do not differ in this respect.
A further strength of the SIGN guidelines is their accessibility. They are available to anyone who cares to look at the SIGN website. Clearly defined search criteria and universality are, in the author’s opinion, two fundamental requirements for any guidelines, and SIGN’s adherence to these features place it, at least on a par, with similar guidelines. We may not agree with the recommendations, but at least we are able to read them and know what methods were used to formulate them. This is not the case with some guidelines, which require membership for access and do not state clearly their derivation.
The SIGN guidelines are becoming out of date. A third requirement for guidelines to be meaningful is that the information within them remains relevant.
Criticisms of EBM
EBM is not without its critics such as described by Cohen, Stavri and Hersh [2] in a five point characterisation of what they believe are the flaws of EBM:
- EBM is a poor philosophic basis for medicine
- The EBM definition of evidence is narrow and excludes important information
- EBM is not evidence-based
- The usefulness of applying EBM to individual patients is limited
- EBM reduces the autonomy of the doctor / patient relationship.
Any clinician who has attempted to find the evidence behind a certain management plan will soon face practical problems and discover that evidence is often lacking. Negotiating databases can befuddle the reader and the paucity of high quality systematic reviews is a common criticism. But we must be careful not to dismiss EBM because of these problems. Sackett et al in their paper ‘Evidence based medicine: what it is and what it isn’t’ begin with the line: “It’s about integrating individual clinical expertise and the best external evidence.” [1]
The emphasis is upon integration of the individual and the general; the best that both the clinician and the scientific literature have to offer. In both cases there may be deficiencies, and pragmatism is required.
There is an attempt to introduce guidelines and algorithms into many various aspects of medical practice. Criticism of this focuses upon a lack of clarity in the criteria used for determining the guidelines / algorithms. Also, a universality of approach can both antagonise well-informed clinicians and prevent innovation within medical practice. A treatment plan which goes against guidelines might lay a clinician open to medico-legal scrutiny. This is particularly irksome if the basis for guidelines / algorithm is not clear and we should resist allowing decision-making by GOBSAT (good old boys sat around a table) [3] to creep in through the back door.
Why ENT is well placed to be of benefit to and benefit from EBM
There is a history of scientific thought backing our speciality, as every author attempting to publish in clinical otolaryngology knows. Clinical databases, such as the Liverpool Head and Neck Database, and more recently the Glasgow Head and Neck Database, collect large amounts of information about patients with head and neck cancer. This data collection concurs with the era of ‘big data’ that is now upon us and will prove invaluable in establishing evidence on which to base our practice. Most patients want to know the facts behind treatment options: by amassing large amounts of data and making it widely available, researchers provide a great service to clinicians and patients.
As the speciality subdivides, clinicians are asked to see a narrower range of patients. Eighty percent of our work is elective, which again reduces the need for a broad based practice. This increases our experience in fewer conditions. ENT specialists may not have more time, but they are not required to be an ‘expert’ in so many areas. Patients expect to receive detailed information and advice from their specialist. No longer is the range of diseases treated, nor anatomical sites covered, an excuse not to remain up to date. Noted deficiencies in the literature will be a spur for research, whether through well designed studies, or, literature reviews using clearly defined search criteria. This increasing specialisation within ENT will be a continued driving force behind the proliferation of an evidence-based approach.
There are already commendable examples of ENT surgeons incorporating EBM into their practice and propagating this by writing and publishing books on the subject [4]. Any way in which clinicians are able to spread EBM throughout the speciality is to be welcomed. Practical ways include converting journal meetings into EBM sessions and approaching the plethora of clinical meetings, such as multidisciplinary meetings (MDM), from an EBM perspective. This latter requires the chairperson firstly, to be aware of the guidelines and also take a firm hand when opinion is overtaken by subjectivity. The author, as a previous chair of a head and neck MDM, can vouch that this does not make for increased popularity.
The benefit of striving towards practising the gold standard of medicine applies to ENT, as with all specialities. Patients now reject paternalistic medical practice and demand the options be presented clearly and fully explained. The law has evolved in line with this, with the Montgomery Test [5] replacing the Bolam Test [6] for proof of medical negligence. A case can no longer be defended upon the basis of what a group of reasonable clinicians would do, but rests on evidence of full discussion of options and informed consent. Well-constructed guidelines summarise the options into a form easily digested by both patient and doctor. Documented discussion of these with evidence of informed consent provides a powerful backing to doctors in an increasingly litigious environment.
EBM is here to stay and not only because it is embraced by policy makers. Big data, computerisation, online medical libraries, databases are all rapidly developing resources widely available to clinicians. Patients’ increased desire for explanations and options, along with changes in medical law, will drive clinicians to use these resources. Rather than be affronted by what some may see as an attack on their autonomy, specialists are advised to direct their intellectual energy into critically appraising their own practice as well as that advocated by multidisciplinary meetings and guidelines.
References
1. Sackett DL, Rosenberg WM, Gray J, et al. Evidence based medicine: what it is and what it isn’t. BMJ: British Medical Journal 1996;312(7023):71.
2. Cohen AM, Stavri PZ, Hersh WR. A categorization and analysis of the criticisms of Evidence-Based Medicine. International Journal of Medical Informatics 2004;73(1):35-43.
3. Greenhalgh, T. How to read a paper. The basics of evidence-based medicine. Fifth Edition. Wiley Blackwell; 2014.
4. Lau A, Jacques T, Tandon S, Lesser T. Evidence-Based Emergency Care. 2nd Edition. CreateSpace Independent Publishing Platform; 2015.
5. Montgomery v Lanarkshire Health Board. UKSC11, (2015) All ER (D) 113 (Mar).
6. Bolam v Friern Hospital Management Committee. High Court (1957) 1WLR 582.
Declaration of Competing Interests: None declared
ABOUT THE AUTHOR
