Technical comment on: Güller et al. Lower hospital volume is associated with higher mortality after oesophageal, gastric, pancreatic and rectal cancer resection

DOI: https://doi.org/10.4414/smw.2018.14582

Güller et al. performed a population-based study based on an inpatient database from 1999–2012 of the Swiss Federal Statistical Office, regarding postoperative mortality after oesophageal, gastric, pancreatic and rectal cancer resection as the independent variable and hospital volume [1]. For multivariate analysis, only sex, age, insurance status and nationality were available. However, the paper’s conclusions do not appear to be supported by the measured variables.

First, the absolute number of annual deaths in the high-volume centres (>10 operations per year) were 6 for oesophageal cancer, 38 for gastric cancer, 4 for pancreatic cancer and 52 for rectal cancer. Therefore chance may play a role in the association found, because of these small numbers. As correctly stated by the authors, “The limitations of the present investigation arise from information, which cannot be ascertained from the database of the Federal Statistical Office, such as pathological stages, grading, chemotherapeutic treatments, comorbidities and performance status. Hence, it cannot be determined to what extent these factors might have influenced our analysis”. But authors then argue that “… the population-based nature of the database, with a large number of patients, is associated with a high degree of generalisability and mirrors the real-world situation in Switzerland”. This is not valid, since, as acknowledged by the authors, their database suffers from an “omitted variable bias”.

Because of such methodological limitations and the small numbers of deaths, the risk of a mathematical fallacy in the authors’ conclusion is substantial. The reason for an increased mortality in low-volume hospitals has not been properly addressed by the authors. Nevertheless, they conclude, that “high-risk operations should preferably be performed in high-volume hospitals to ensure optimal patient outcomes in Switzerland”. This conclusion cannot be derived from the data presented. For example, if the principal cause of increased mortality is a patient’s “do not resuscitate” order (DNR), then the authors would have to conclude that high-volume hospitals should be preferred, because DNR is more frequent in low-volume hospitals.

Authors should address the many limitations when using a low quality study design [2], especially with respect to the known inter-method variability in Hospital Standardised Mortality Ratios (HSMRs) [3], the problem of the Simpsons paradox in fully adjusted HSMRs [4] and the role of DNR as a major confounder of HSMRs [5, 6]. Mislabelling of aggregated low-volume institutions due to fallacy [4], risk avoidance [7–9] and improper statistical material [10] raises further formal, ethical and legal issues. Use of mortality rates may have serious consequences for patient safety because of the risk of avoidance creep, where institutions avoid risky operations in order to improve the mortality rate, or improve mortality rates by lowering the threshold of sickness prompting a surgical intervention [7]. Chart and peer reviews should be the preferred methods to assess performance and to help politicians in decisions that are supported by the best possible evidence [10, 11], as was exemplified by simple layperson reviews of videos during bariatric surgery [12].

References

1 Güller U , Warschkow R , Ackermann CJ , Schmied B , Cerny T , Ess S . Lower hospital volume is associated with higher mortality after oesophageal, gastric, pancreatic and rectal cancer resection. Swiss Med Wkly. 2017;147:w14473. doi:.https://doi.org/10.4414/smw.2017.14473  

2 Hoffmann W , Latza U , Terschüren C ; Deutsche Arbeitsgemeinschaft für Epidemiologie (DAE), Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie (GMDS), Deutsche Gesellschaft für Sozialmedizin und Prävention (DGSMP), Deutsche Region der Internationalen Biometrischen Gesellschaft (DR-IBS). Leitlinien und Empfehlungen zur Sicherung von Guter Epidemiologischer Praxis (GEP) - überarbeitete Fassung nach Evaluation [Guidelines and recommendations for ensuring Good Epidemiological Practice (GEP) -- revised version after evaluation]. Gesundheitswesen. 2005;67(3):217–25. Article in German. doi:.https://doi.org/10.1055/s-2004-813850  

3 Shahian DM , Wolf RE , Iezzoni LI , Kirle L , Normand S-LT . Variability in the measurement of hospital-wide mortality rates. N Engl J Med. 2010;363(26):2530–9. doi:.https://doi.org/10.1056/NEJMsa1006396  

4 Marang-van de Mheen PJ , Shojania KG . Simpson’s paradox: how performance measurement can fail even with perfect risk adjustment. BMJ Qual Saf. 2014;23(9):701–5. Available at: http://qualitysafety.bmj.com/lookup/doi/10.1136/bmjqs-2014-003358. doi:.https://doi.org/10.1136/bmjqs-2014-003358  

5 Walkey AJ , Weinberg J , Wiener RS , Cooke CR , Lindenauer PK . Association of Do-Not-Resuscitate Orders and Hospital Mortality Rate Among Patients With Pneumonia. JAMA Intern Med. 2016;176(1):97–104. doi:.https://doi.org/10.1001/jamainternmed.2015.6324  

6 Bruckel J , Mehta A , Bradley SM , Thomas S , Lowenstein CJ , Nallamothu BK , et al. Variation in Do-Not-Resuscitate Orders and Implications for Heart Failure Risk-Adjusted Hospital Mortality Metrics. JACC Heart Fail. 2017;5(10):743–52. doi:.https://doi.org/10.1016/j.jchf.2017.07.010  

7 Resnic FS , Welt FGP . The public health hazards of risk avoidance associated with public reporting of risk-adjusted outcomes in coronary intervention. J Am Coll Cardiol. 2009;53(10):825–30. doi:.https://doi.org/10.1016/j.jacc.2008.11.034  

8Koka A. The High Cost of Public Reporting | THCB [Internet]. Blog. 2017 [cited 2017 Oct 15] Available from: http://thehealthcareblog.com/blog/2017/09/18/the-cost-of-public-reporting/  

9 Rosenbaum L . Scoring No Goal--Further Adventures in Transparency. N Engl J Med. 2015;373(15):1385–8. doi:.https://doi.org/10.1056/NEJMp1510094  

10 Lüscher TF , Pierre C , Alexandre C . « Lies, damn lies, and statistics »: Bemerkungen zu den Outcome-Zahlen des Bundesamtes für Gesundheit. Cardiovasc Med. 2009;12:229–33. https://cardiovascmed.ch/en/article/doi/cvm.2009.01444/.  

11 Romanens M , Hofmeier B , Ackermann F . Begleitforschung SwissDRG : Aufruf zu einem nationalen Konsens. Schweiz Arzteztg. 2010;91:270–3. https://saez.ch/de/article/doi/saez.2010.14975/. 

12 Birkmeyer JD , Finks JF , O’Reilly A , Oerline M , Carlin AM , Nunn AR , et al.; Michigan Bariatric Surgery Collaborative. Surgical skill and complication rates after bariatric surgery. N Engl J Med. 2013;369(15):1434–42. doi:.https://doi.org/10.1056/NEJMsa1300625