Value of the TTM risk score for early prognostication of comatose patients after out-of-hospital cardiac arrest in a Swiss university hospital

DOI: https://doi.org/10.4414/smw.2020.20344

Introduction

Despite advancements in resuscitation practice, the outcome of patients suffering from cardiac arrest remains poor to this day. The overall rate of survival to hospital discharge after out-of-hospital resuscitation varies depending on the population studied (3.1% to 20.4% [1–3]). The rate of patients with a favourable outcome (defined as return to baseline or moderate cerebral disability with sufficient function for independent activities of daily life, and ability to work in a sheltered environment [4]) is even lower.

Neurological damage results from the ischaemic/hypoxic injury during the cardiac arrest and the reperfusion injury after successful resuscitation. Early assessment of the severity of the oxygen deficiency and the corresponding neurological impact remains challenging. Prognostication of unfavourable outcome in comatose patients based on current guidelines can be made no sooner than 96 hours after return of spontaneous circulation and, according to our experience, but also to the literature, only a minority of the patient outcomes can be predicted with a high degree of certainty [5, 6]. To keep the time of ambiguity for the patient’s family as short as possible and to avoid unnecessary treatment and thus reduce costs, it is crucial to achieve an accurate prognostication as soon as possible. Despite different approaches using a variety of parameters ranging from basic laboratory test results and clinical signs to the use of functional magnetic resonance imaging (fMRI) and electroencephalography (EEG), a validated accurate early scoring system is still lacking.

In 2017, Martinell et al. [7] developed a scoring system (the TTM risk score) for early outcome prediction with 10 independent parameters that are usually already available at the time of patient admission. The parameters comprise age, place of cardiac arrest (at home), first monitored rhythm, no flow time, low flow time, treatment with adrenaline, presence of pupillary or corneal reflex, pH, Glasgow Coma Scale (GCS) motor score, and partial pressure of carbon dioxide (PaCO₂) in the arterial blood gas analysis. The score ranges from −2 to 35, with higher numbers indicating higher risk of unfavourable outcome. The database used for this score was the patient cohort of the Target Temperature Management (TTM) Trial [8], which compared two different temperatures for temperature management (33 vs 36°C) in 939 patients in 36 ICUs mainly in Europe (including Geneva, St Gallen, and La Chaux-de-Fonds, Switzerland), but also in Australia (four ICUs). Since the TTM followed strict inclusion and exclusion criteria, with a standardised withdrawal of life support protocol after prognostication, the validity in a general ICU population of cardiac arrest patients is unknown. With our study, we aimed to confirm and assess the usefulness of Martinell and colleagues’ results in a Swiss population suffering from out-of-hospital cardiac arrest outside a clinical trial. To do this we applied the parameters defined by Martinell et al. in 100 patients treated after successful resuscitation following an out-of-hospital cardiac arrest at the Inselspital, University Hospital Bern, Switzerland.

Methods

We performed a retrospective cohort study in the first 100 patients (January to mid-September 2016) with a complete data set of information prospectively collected in a registry including all patients treated for out-of-hospital cardiac arrest and admitted to the ICU of the Inselspital Bern. The cantonal ethics committee (Kantonale Ethikkommission Bern) approved the registry (KEK BE No. 116/15); since no intervention was performed the need for informed consent was waived for those patients who died. Survivors or their next of kin were informed and consented to the use of the data collected. The study is registered in ClinicalTrials.gov (Identifier: NCT027224609).

Data were manually entered into a RedCap database hosted by the Clinical Trial Unit of the University of Bern. We included all patients with cardiopulmonary arrest with return of spontaneous circulation and a GCS score below 9 at admission. The main exclusion criteria were neurological aetiology of cardiopulmonary arrest such as stroke, intracerebral haemorrhage, subarachnoid haemorrhage, SUDEP (sudden unexpected death in epilepsy) and unwitnessed cardiac arrest with asystole as the initial rhythm. We systematically collected data that were available at admission to the hospital and to the ICU and that were necessary to calculate the TTM risk score: age, place of cardiac arrest, initial rhythm at the scene, no-flow time, low-flow time, administration of adrenaline, bilateral absence of corneal and pupillary reflexes, GCS motor response, and pH and partial pressure of carbon dioxide in arterial blood gas analysis. Missing data (3.3% of all data, predominately the exact no-flow and low-flow times) were replaced with the most optimistic value “0”. With these 10 parameters, we calculated the TTM risk score for each patient (see table S1 in appendix 1).

For outcome, we chose the Cerebral Performance Category (CPC), a scale ranging from 1 to 5, with higher scores meaning worse outcomes (CPC 1 = conscious, alert, able to work and lead a normal life; 2 = conscious, sufficient cerebral function for part-time work in a sheltered environment or independent activities of daily life; 3 = conscious, dependent on others for daily support, has at least limited cognition; 4 = unconscious, no cognition; 5 = brain dead). We used the best CPC score achieved within the first 6 months after cardiopulmonary arrest, and not the 6-month CPC, to avoid classifying patients with good clinical outcomes who died a few weeks later for reasons not associated with the initial cardiac arrest [9].

Outcome was dichotomised into favourable (CPC 1 and 2) and unfavourable (CPC 3–5), and the receiver operating characteristic (ROC) curve was constructed. The area under the curve (AUC) was compared with the AUC measured in the original population described by Martinell [7]. For the ROC curve and the AUC we used Graph Pad Prism Version 8.2; the comparisons of the AUCs were made according to the method proposed by Hanley and McNeil [10] using the Web application developed by Professor emeritus Richard Lowry at Vassar College, Poughkeepsie, N.Y. (http://vassarstats.net/roc_comp.html). Calculation of sensitivity and specificity were done for two cut-offs of the TTM risk score, that is, 10 and 16.

Results

We had complete data for 100 of the first 106 patients in the registry. The AUC of the TTM risk score in our population was 0.810 (95% confidence interval [CI] 0.719–0.901, fig. 1), which is not significantly different from the AUC of the TTM Trial cohort (0.844, 95% CI 0.842–0.846) (p = 0.47).

Figure 1 The receiver operating characteristic curves of the Target Temperature Management (TTM) risk score in the Bern cardiac arrest population admitted to the Inselspital, University Hospital Bern intensive care unit (left). On the right, the ROC curves for a poor outcome at 6 months for other predication scores: the TTM [7] (red), OHCA [11] (blue) and CAHP [12] (green) risk scores. The area under the curve (AUC) of the TTM score in the Bern population (0.81) is not significantly different from the AUC of the original TTM population [7] (AUC 0.842). The AUC for the OHCA score is 0.746, and for the CAHP score, it is 0.746. The figure on the right is from Martinell et al, Crit Care 2017;21:96

Using the TTM risk score cut-offs established by Martinell [7], we calculated for our patients with a risk score cut-off at 10 a negative predictive value (NPV) of 0.65, translating into a likelihood ratio for an unfavourable outcome of 0.22. With a TTM risk score above 16, the positive predictive value (PPV) is 0.93 (tables 1 and 2 ). Test accuracy is higher if the lower cut-off of 10 is used (0.77; cut-off of 16: 0.65) (table 3).

Discussion

A comparison of the risk score’s AUC between the original TTM Trial cohort and our population showed no significant difference, with a nondirectional p-value of 0.45, meaning a similar applicability of the TTM risk score within both cohorts. The relatively large 95% CI of our AUC (0.72–0.90) is due to the smaller cohort used in this study.

Current guidelines, based on available evidence, recommend waiting 72 hours before prognostication of comatose survivors after cardiac arrest, if pupillary and corneal reflexes are absent. If these reflexes are present, an assessment should be made no sooner than 96 hours after cardiac arrest, by using multiple modalities (clinical evaluation, EEG, computed tomography / MRI, neuron-specific enolase as a biomarker) [13]. Throughout Europe and the world, different ethical considerations regarding treatment and limitations of treatment in patients with cardiac arrest exist [14]. Many clinicians feel uncomfortable offering a 4- to 5-day course of highly invasive treatment to a patient whose prognosis is likely to be grim, especially if the patient has relevant comorbidities or advanced age. Using clinical experience, the treating physician has to decide whether a patient with an advance care directive or previously expressed wishes limiting treatment should be offered guideline-directed therapy. The published TTM risk score offers some guidance but is limited by the fact that study’s inclusion and exclusion criteria created a selection bias of patients admitted to the ICU, hampering generalisability. In particular, patients were included in the study only if full and guideline-directed treatment was offered. Although not explicitly written, elderly or frail patients are usually not screened for study purposes because of the high risk of death in these groups. Patients with a high certainty of death regardless of the group to which they are randomised do not add any evidence from the study’s perspective and thus are frequently not included. This effect was seen in our study, where we included all patients and where we can find a difference in favourable outcomes between the TTM Trial cohort (52%) and our cohort (29%). This effect cannot be attributed to worse treatment or premature withdrawal of sustained life support alone; for example, the Bern cohort in the subsequent TTM 2/TAME Trial had an overall rate of favourable outcome of >40% (preliminary data, outcome measured with Glasgow Outcome Scale-Extended).

Our unselected patients admitted to the ICU with a risk score below 10 had a negative likelihood ratio for an unfavourable outcome of 0.22, meaning that a favourable outcome was much more likely than an unfavourable one. In contrast, in Bern, only 3 of 42 patients (7%) with a TTM risk score above 16 at admission survived with a favourable outcome. Keeping in mind that test accuracy is far from perfect, a patient with a TTM risk score above 16 has a low probability of surviving without major neurological disability. Thus the cut-offs of 10 and 16 make a reasonable starting point for discussing prognosis with the family much earlier than guidelines recommend if there are doubts as to whether the patient should be offered full or limited treatment.

There are other scoring systems for outcome prediction based on early available patient and circumstantial factors, such as the CaRdiac Arrest Survival Score (CRASS) [15], the CAHP Cardiac Arrest Hospital Prognosis [12] or the OHCA risk score [11]. As expected, the variables found to have a significant impact on outcome in all scoring systems are similar, with age, duration of the no-flow and low-flow intervals and initial rhythm as main determinants. The AUCs for outcome prediction are between 0.75 (CAHP and OHCA) and 0.88 (CRASS), which are in the range of the TTM score. We chose the TTM score because of its relatively easy applicability compared with the complex calculation of the OHCA score, or the use of nomograms for the CAHP. The CRASS score was presented only recently and calculation of the score is complicated, but we expect online calculators or smartphones apps soon. All the mentioned scores have in common that the individual patient’s outcome cannot be calculation with certainty, only risk categories.

There are several limitations in both studies. A major limitation in cardiac arrest and prognostication studies is that the patient’s outcome is at risk of a self-fulfilling prophecy: if treatment is withdrawn because of a perceived unfavourable prognosis, the patient will inevitably die and the perceived bad prognosis will be fulfilled. To overcome this difficulty, the TTM Trial used a strict protocol-based withdrawal approach, to reduce errors that could be attributed to early withdrawal of therapy. In our population, where patients were not subjected to the strict withdrawal protocol, a more liberal and family-centred approach was used. If the patient’s presumed desire was a limited approach, and the family supported the notion that further treatment was not in the best interest of the patient, we did not insist on the 96-hour timeframe supported by the guidelines. For the NPV at the ≤10 cut-off, this leads to a possible underestimation of the proportion of patients with a favourable outcome but, in contrast, an overestimation of PPV in the above-16 group. Compared to the TTM cohort, ours had fewer patients, which lowers the accuracy of the test in our population and represents a further limitation.

Patients admitted to the ICU after cardiac arrest who die after having their life support withdrawn usually do so because of their unfavourable neurological prognosis [16]. However, there are some patients who die from cardiac causes before awakening or sedation stop, so consciousness cannot be tested. These patients are classified as having an unfavourable outcome, although their neurological outcome might have been favourable. This inherent misclassification will affect any prognostication tool and a certain degree of uncertainty will always remain.

Another limitation of our results is that almost all patients arrived at the emergency room already intubated and sedated, resulting in a GCS motor score of 1 in approximately 90% of this population. Since a GCS motor score of 1 adds 2 points in the TTM risk score, this may shift patients into a higher risk group. Since the original TTM cohort included patients from Europe and Australia, we assume most of those patients also arrived at the hospital sedated and intubated, but this was not reported.

A limitation in our comparison is that the absence of corneal and pupillary reflexes was frequently not documented upon arrival at the emergency room and the ICU, and was recorded only after the first 24 hours in the ICU. Thus, the validity of this parameter is limited.

From a practical viewpoint, a further limitation is that more than a third of the patients in our population had a risk score between 11 and 16, which represents the “grey zone” between the given cut-off values. A reasonable approach for these patients is to use the TTM risk score as a continuous, ordinal scale, with higher scores representing a higher mortality risk, as represented by the ROC curve.

In conclusion, the results of the TTM risk score test in our cohort showed a close similarity to the original results obtained by Martinell et al. [7], especially for risk scores above 16, despite certain limitations, such as a falsely documented low GCS motor score or undocumented corneal and pupillary reflexes upon arrival. With an AUC in the ROC of 0.81, the TTM risk score cannot predict outcome at the time of ICU admission with certainty. The proposed cut-offs at 10 and 16 might guide early patient-centred decisions in patients where a limited treatment approach is discussed with the family.

Appendix 1 Supplementary table

1. Points assigned to categories of the ten independent risk factors for a poor outcome in the calculation of the Target Temperature Management risk score. Poor outcome was defined as Cerebral Performance Category 3–5 at 6 months after out-of-hospital cardiac arrest. Total score ranged from −2 to 35.

2. The AUC was 0.84 in the original cohort and 0.81 in the Bern cohort. This does not allow a definite prognostication for an individual patient. A score <10 is associated with a favourable outcome, a score >16 with an unfavourable outcome.

From:

Kägi E, Weck A, Iten M, Levis A, Haenggi M. Value of the TTM Risk Score for Early Prognostication of Comatose Patients after Out-of-Hospital Cardiac Arrest in a Swiss University Hospital. Swiss Med Wkly 2020;150:w20344. doi:https://doi.org/10.4414/smw.2020.20344. (For the Bern cohort.)

and

Martinell L, Nielsen N, Herlitz J, Karlsson T, Horn J, Wise MP, et al. Early predictors of poor outcome after out-of-hospital cardiac arrest. Crit Care. 2017;21(1):96. doi:https://doi.org/10.1186/s13054-017-1677-2. (For the original cohort.)

References

1 Coute RA , Nathanson BH , Panchal AR , Kurz MC , Haas NL , McNally B , et al. Disability-Adjusted Life Years Following Adult Out-of-Hospital Cardiac Arrest in the United States. Circ Cardiovasc Qual Outcomes. 2019;12(3):e004677. doi:.https://doi.org/10.1161/CIRCOUTCOMES.118.004677  

2 Bougouin W , Dumas F , Lamhaut L , et al. Extracorporeal cardiopulmonary resuscitation in out-of-hospital cardiac arrest: a registry study. Eur Heart J. 2020;41(21):1961–71. doi:.https://doi.org/10.1093/eurheartj/ehz753  

3 Kiguchi T , Okubo M , Nishiyama C , Maconochie I , Ong MEH , Kern KB , et al. Out-of-hospital cardiac arrest across the World: First report from the International Liaison Committee on Resuscitation (ILCOR). Resuscitation. 2020;152:39–49. doi:.https://doi.org/10.1016/j.resuscitation.2020.02.044  

4 Jennett B , Bond M . Assessment of outcome after severe brain damage: Apractical scale. Lancet. 1975;1(7905):480–4. doi:.https://doi.org/10.1016/S0140-6736(75)92830-5  

5 Bongiovanni F , Romagnosi F , Barbella G , Di Rocco A , Rossetti AO , Taccone FS , et al. Standardized EEG analysis to reduce the uncertainty of outcome prognostication after cardiac arrest. Intensive Care Med. 2020;46(5):963–72. doi:.https://doi.org/10.1007/s00134-019-05921-6  

6 Cronberg T . Assessing brain injury after cardiac arrest, towards a quantitative approach. Curr Opin Crit Care. 2019;25(3):211–7. doi:.https://doi.org/10.1097/MCC.0000000000000611  

7 Martinell L , Nielsen N , Herlitz J , Karlsson T , Horn J , Wise MP , et al. Early predictors of poor outcome after out-of-hospital cardiac arrest. Crit Care. 2017;21(1):96. doi:.https://doi.org/10.1186/s13054-017-1677-2  

8 Nielsen N , Wetterslev J , Cronberg T , Erlinge D , Gasche Y , Hassager C , et al.; TTM Trial Investigators. Targeted temperature management at 33°C versus 36°C after cardiac arrest. N Engl J Med. 2013;369(23):2197–206. doi:.https://doi.org/10.1056/NEJMoa1310519  

9 Taccone FS , Horn J , Storm C , Cariou A , Sandroni C , Friberg H , et al. Death after awakening from post-anoxic coma: the “Best CPC” project. Crit Care. 2019;23(1):107. doi:.https://doi.org/10.1186/s13054-019-2405-x  

10 Hanley JA , McNeil BJ . The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36. doi:.https://doi.org/10.1148/radiology.143.1.7063747  

11 Adrie C , Cariou A , Mourvillier B , Laurent I , Dabbane H , Hantala F , et al. Predicting survival with good neurological recovery at hospital admission after successful resuscitation of out-of-hospital cardiac arrest: the OHCA score. Eur Heart J. 2006;27(23):2840–5. doi:.https://doi.org/10.1093/eurheartj/ehl335  

12 Maupain C , Bougouin W , Lamhaut L , Deye N , Diehl JL , Geri G , et al. The CAHP (Cardiac Arrest Hospital Prognosis) score: a tool for risk stratification after out-of-hospital cardiac arrest. Eur Heart J. 2016;37(42):3222–8. doi:.https://doi.org/10.1093/eurheartj/ehv556  

13 Nolan JP , Hazinski MF , Aickin R , Bhanji F , Billi JE , Callaway CW , et al. Part 1: Executive summary: 2015 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science with Treatment Recommendations. Resuscitation. 2015;95:e1–31. doi:.https://doi.org/10.1016/j.resuscitation.2015.07.039  

14 Bossaert LL , Perkins GD , Askitopoulou H , Raffay VI , Greif R , Haywood KL , et al.; ethics of resuscitation and end-of-life decisions section Collaborators. European Resuscitation Council Guidelines for Resuscitation 2015: Section 11. The ethics of resuscitation and end-of-life decisions. Resuscitation. 2015;95:302–11. doi:.https://doi.org/10.1016/j.resuscitation.2015.07.033  

15 Seewald S , Wnent J , Lefering R , Fischer M , Bohn A , Jantzen T , et al. CaRdiac Arrest Survival Score (CRASS) - A tool to predict good neurological outcome after out-of-hospital cardiac arrest. Resuscitation. 2020;146:66–73. doi:.https://doi.org/10.1016/j.resuscitation.2019.10.036  

16 Moulaert VR , Verbunt JA , van Heugten CM , Wade DT . Cognitive impairments in survivors of out-of-hospital cardiac arrest: a systematic review. Resuscitation. 2009;80(3):297–305. doi:.https://doi.org/10.1016/j.resuscitation.2008.10.034