Surgeon Outcomes – Explanatory Notes

Introduction

This report describing outcomes of individual consultant surgeons has been prepared by the Association of Coloproctology of Great Britain and Ireland (ACPGBI) and The National Bowel Cancer Audit (NBOCA). It is a response to an initiative of NHS England (Everyone Counts: Planning for Patients 2013/4) to create greater transparency and more choice for patients and commissioners. Ten clinical specialties were asked to report on outcomes for every consultant in that specialty.

This is the fourth time such an exercise has been undertaken for colorectal cancer surgery. The National Bowel Cancer Audit has reported surgical outcomes, but these have been reported for NHS Trusts and for regional Cancer Networks and never for individual surgeons. The reported outcomes relate only to surgeons practising in England.

The outcome measure assessed for this publication is the 90-day mortality rate following planned removal of a bowel cancer – that is the proportion of patients undergoing surgery who die from whatever cause within 90 days of their operation.

This year there are two new outcome measures of major resection rate and case ascertainment. Major resection rate is the percentage of the colorectal cancer patients discussed by the multidisciplinary clinical committee who actually go on have their bowel cancer removed by surgery. Case ascertainment is the proportion of patients with bowel cancer entered into the Audit by a trust compared to the numbers predicted by other national data sources. Case ascertainment is a reflection of the ability of a unit to identify all the patients with bowel cancer in a particular area.

It is based on data submitted to the National Bowel Cancer Audit for patients whose bowel cancer was diagnosed between April 2010 and March 2015. The national average 90-day mortality after planned bowel cancer surgery is approximately 3% in this period. This means that about 1 out of every 33 patients undergoing planned surgery for bowel cancer did not survive beyond 90 days after the operation. This report should be seen as the forerunner of what will become a powerful resource for patients, surgeons as well as for commissioners of bowel cancer services. Whilst the ACPGBI is committed to the project, it urges great caution when interpreting early releases of data for the reasons set out below.

The Audit

The data were extracted from the National Bowel Cancer Audit. This audit has been in place for a number of years and has collected data on patients with bowel cancer admitted to NHS hospitals in the UK. The audit collects information about the characteristics of the patients and their tumour, the treatments they receive and their follow-up. The number and overall proportion of patients who could be included in the audit and the completeness and quality of their data has increased year on year.

The audit was designed to look at cancer management and outcomes within NHS Trusts and regional Cancer Networks, not at the performance of individual surgeons, mainly because the treatment of colorectal cancer involves a large team of health care professionals working together, all of whom can influence the outcome for patients. For this reason, there can be difficulties in allocating each patient to an individual surgeon and to ensure that all of a particular surgeon’s cases are included in the analysis. The completeness and accuracy of the reports on the performance of individual surgeons has improved year on year following individual outcome publication.

The nature of the data

90-day mortality is a readily available but very crude measure of the performance of a surgeon. It does not give any information on important aspects of that surgeon’s care, such as success at completely removing the tumour, how well the surgeon interacts with the patient and the frequency of complications after surgery. The outcomes reported here are for a specific 5-year time period (patients diagnosed with bowel cancer between April 2010 and March 2015). This represents only a snap-shot of a surgeon’s overall activity. The surgeon’s 90-day mortality will vary over time as a result of the play of chance. During the reporting period, some surgeons may have had clusters of poor results which may then be followed by clusters of better results depending when the data are extracted.  In other words, caution should be exercised when using these data to predict future surgical performance.

The Data

Most colorectal surgeons will perform 20 or more planned operations to remove a colorectal cancer per year. They will also operate on a number of patients who present for the first time as an emergency with the complications of bowel cancer. In addition, they operate on many other patients with non-cancerous conditions of their bowel. Thus, the patients included in this analysis are only a proportion of the bowel operations that an individual colorectal surgeon has performed.

It has been calculated that a colorectal surgeon would need outcome data on over 150 patients before there is an 8 out of 10 chance of spotting genuine poor performance[1]. For this reason, although there is a wide range of mortality between different surgeons, for the vast majority, their outcome falls within an accepted range. With low numbers, a surgeon could have a high death rate simply by chance and be wrongly labelled as performing poorly. Similarly, low numbers could mask a poorly performing surgeon and engender complacency.

The Surgeon

Most elective colorectal cancer surgery in the UK is performed by surgeons who are members of a multidisciplinary team treating colorectal cancer and are also members of the ACPGBI (a professional organisation dedicated to improving outcomes for patients with diseases of the colon, rectum and anus). A number of surgeons who performed bowel resections may not be included in the data presented in this publication. There are a number of possible reasons for this:

  • They do not work in England. This report is confined to surgeons working in England. Surgeons from Scotland, Wales and Northern Ireland have been excluded.
  • The surgeon does not treat colorectal cancer. Some colorectal surgeons deal with other conditions of the lower intestine.
  • The surgeon may not have performed sufficient elective colorectal cancer resections to allow calculation of their outcome. Surgeons were included if they had performed at least ten planned operations to remove a bowel cancer.
  • The data have been submitted incorrectly by the surgeon’s hospital Trust.

Colorectal Cancer Treatment

Treatment of colorectal cancer involves complex pathways because patients suffering with this condition present in a variety of ways and will require different treatments depending on the site of the cancer within their bowel and the stage of the tumour at presentation.

Treatment of colorectal cancer involves a large team of medical, nursing and other health care professionals. Whilst the surgeon is a key member of this team, the risk of the patient dying in the first 90 days after the operations is also influenced by other factors, including the quality of anaesthetic care and the availability of high dependency and intensive care facilities.

In addition, consultant surgeons increasingly operate in pairs, particularly on more complex cases. Thus, focusing on an individual surgeon’s outcome in colorectal cancer treatment is analogous to awarding the FA cup to the striker, rather than the whole team. A patient seeking information on outcome in colorectal cancer should be advised to look at the outcome for the whole unit, rather than an individual surgeon.

The patient and their cancer

Patients suffering with colorectal cancer tend to be elderly and as well as having the cancer to contend with, they often suffer a number of other medical conditions such as heart and lung disease that put them at a high risk of developing non-surgical complications following major abdominal surgery. Indeed it has been estimated that two thirds of patients who die in the postoperative period following bowel cancer resection succumb from complications of their associated medical conditions (such as heart and lung problems), rather than as a result of surgical complications.

Bowel cancer can affect different parts of the large intestine; tumours in the lower part (rectum) are more difficult to remove and surgery takes longer because of the relative inaccessibility of this part of the bowel. Complications are more common for rectal cancer surgery than for surgery to remove tumours in the colon. Some surgeons may have developed a special interest and skill in dealing with particular types of bowel cancer and therefore treat higher risk patients than other colleagues. Thus the case-mix is likely to vary from surgeon to surgeon and with it the outcome of the surgery.

Data Collection and analysis

All NHS Trusts are required to send data on all patients they treat for bowel cancer to a central database. NBOCA uses the data to report on the performance of NHS Trusts and Strategic Clinical Networks. The same data have now been used to report on how many patients die within 90 days of a planned removal of a bowel cancer for individual surgeons. The mortality rate is calculated by identifying all patients who die after bowel cancer surgery within that period divided by all patients having a planned operation by that particular surgeon. We had to exclude some patients because not all of the data relating to their operation was sent to the audit or because they could not be allocated to an individual surgeon.

The results reported here have been adjusted for case-mix to ensure that surgeons who take on more difficult cases are not unfairly penalised. Some surgeons may have treated patients who had more advanced cancer or patients who had other medical conditions. This adjustment was carried out by using a statistical technique called risk adjustment. Depending on the risk profile of their patients, the results of some surgeons were adjusted upwards (these surgeons had operated on patients for whom a statistical risk model had predicted a lower than average risk of dying after the operation) whilst the results for other surgeons were adjusted downwards (these surgeons had operated on patients for whom a higher than average risk had been predicted).

Annual Report of the Audit (792kB)

The methods used to carry out the risk adjustment

Funnel plots

We have used “funnel plots” to investigate whether there are systematic differences between the risk-adjusted 90-day mortality rates of individual surgeons. In these funnel plots, each dot represents the result for an individual surgeon or NHS Trust. The vertical axis represents the 90-day mortality expressed as a percentage and the horizontal axis represents the number of operations that were included for each individual surgeon.

The “funnel limit” is the curved line. A 90-day mortality result above this line (pink zone) is identified as an “outlier”. As explained earlier, the mortality results vary partly as a result of the play of chance. The impact of this chance variation is likely to be greater if a surgeon has carried out a smaller number of procedures, which explains the curved shape of the funnel limit. Surgeons with mortality rates within the green zone are deemed to have mortality rates within an accepted range for this type of surgery.

The funnel limit is chosen in such a way that if a surgeon’s “true” mortality is the same as the national average, there is a 0.1% (1 in 1000) chance that this surgeon will have a mortality result that is above the funnel limit. If an individual surgeon’s mortality rate is found to be above the funnel limit, before any further steps are taken, it is important to check that the data submitted to the audit were complete and accurate. The Department of Health has published guidance on how outlying results should be investigated and managed.

Summary

The outcomes presented here for individual surgeons working in English NHS Trusts is the percentage of patients who die within the first 90 days after planned removal of a bowel cancer. The mortality results are taken from the National Bowel Cancer Audit, a national project that reports every year on the treatment and outcomes of patients with bowel cancer for NHS Trusts and regional Cancer Networks.

[1] Walker K, Neuburger J, Groene O, Cromwell DA, van der Meulen J. Public reporting of surgeon outcomes: low numbers of procedures lead to false complacency. Lancet. 2013 Jul 4. doi:pii: S0140-6736(13)61491-9.

Charles Maxwell-Armstrong
ACPGBI Executive Lead for Clinical Outcome Publication
November 2016