Comparison of different versions of the quick sequential organ failure assessment for predicting inhospital mortality of sepsis patients: A retrospective observational study

Hai Hu , Jing-yuan Jiang , Ni Yao ,5

1 Emergency Office of West China Hospital, Sichuan University, Chengdu 610041, China

2 China International Emergency Medical Team, Chengdu 610041, China

3 Emergency Department, West China Hospital, Sichuan University, Chengdu 610041, China

4 West China School of Nursing, Sichuan University, Chengdu 610041, China

5 Department of Critical Care Medicine, West China Hospital, Sichuan University, Chengdu 610041, China

KEYWORDS: Quick sequential organ failure assessment; In-hospital mortality; Sepsis; Lactateenhanced qSOFA; Modified qSOFA

INTRODUCTION

Sepsis is one of the leading causes of mortality worldwide, characterized by infection-induced physiological, pathological, and biochemical abnormalities resulting in significant healthcare and social implications.In the Third International Consensus Definitions for Sepsis and Septic Shock(Sepsis-3), experts recommend the use of quick sequential organ failure assessment (qSOFA) for the diagnosis of sepsis and screening of patients with severe sepsis who are at higher risk of mortality.

The criteria of the qSOFA to predict poor outcomes in adult patients with sepsis are a respiratory rate of≥22 breaths per minute, systolic blood pressure of ≤100 mmHg (1 mmHg=0.133 kPa), and altered mentation. The qSOFA is fast and straightforward, making it appropriate for emergency department (ED) use. However, recent studies have shown the poor performance of qSOFA in predicting sepsis mortality.To enhance its effectiveness, researchers have developed various revised versions of the qSOFA by including additional parameters, including the lactate-enhanced qSOFA(LqSOFA),the procalcitonin (PCT)-enhanced qSOFA(PqSOFA),and the modified qSOFA (MqSOFA).

LqSOFA assigned a lactate level of ≥2 mmol/L one point based on the qSOFA score assignment method.Similarly, the PqSOFA assigned an additional point if PCT was at a threshold of ＞0.5 ng/mL. Likewise, the MqSOFA assigned an additional point calculated from the ratio of peripheral oxygen saturation and a fraction of inspired oxygen (SpO/FiOratio); a score of 0 was assigned if the SpO/FiOratio was ≥315, 1 point if 314-236, and 2 points if ≤235. For the details of the revised versions of the qSOFA, see the Supplementary Table 1.

However, there are few studies verifying and comparing the effectiveness of these revised versions.Therefore, we aimed to evaluate these revised versions of the qSOFA as an in-hospital mortality indicator and for the prognosis of patients with sepsis.

METHODS

Study design

This retrospective study reviewed data obtained from an electronic register system of patients with sepsis who visited West China Hospital. The electronic register system was a part of Hospital Information System (HIS) in which emergency physicians recorded the information of patients.

The local Institutional Review Committee approved the study and waived the requirement for informed consent due to the retrospective nature of the study. The study complied with the international ethical guidelines for human research, such as the.The accessed data were anonymized.

Settings and subjects

This study included 974 adult patients diagnosed with sepsis in the ED of West China Hospital between January 1 and December 31, 2019. Patients with the following characteristics were included: (1) age ≥18 years and; (2) diagnosis of sepsis (based on the Sepsis-3 criteria,sepsis was defined as life-threatening organ dysfunction caused by the infection. Organ dysfunction was identified as a sequential organ failure assessment(SOFA) score ≥2. The exclusion criteria were: (1) lack of information which hindered the calculation of scores; (2)lack of outcome information; and (3) patients who were intubated when they arrived at the ED.

For all included cases, the following data were retrieved from the electronic register system: demographic characteristics; inital vital signs (body temperature, heart rate, systolic and diastolic blood pressure, and respiratory rate); inital mental status (alert, verbal, pain, unresponsive[AVPU] system); inital oxygen saturation level, initial bedside lactate; initial PCT; other parameters used to calculate the scoring system, and final diagnosis. The outcome was death or survival at discharge. The observation point for survival calculations was set as the discharge time of the last admitted patient.

Statistical analysis

Data analysis was conducted using the SPSS 20.0(IBM, USA) and MedCalc 18.2.1 (MedCalc Software Ltd., Belgium). Continuous variables are presented as means with standard deviations or median with interquartile range and were compared using the Student’s-test or Mann-Whitney-test. Categorical variables are described as number and composition ratios and were compared using the Chi-square test or Fisher’s exact test (when the expected value was ＜5 in one cell).

Receiver operating characteristic (ROC) curve analyses were conducted to evaluate the effectiveness of different score systems as predictors of mortality. The performance of these scoring systems to discriminate between survivors and non-survivors was evaluated by calculating the area under the curve (AUC) of ROC with a 95% confidence interval.The AUCs were compared using Delong’s approach.The cut-off value for each scoring scale was determined as the maximum value of Youden’s index.

Finally, the corresponding accuracy, sensitivity,specificity, positive predictive value (PPV), and negative predictive value (NPV) related to the optimal cut-off point for each score were calculated. A-value＜0.05 was considered statistically significant.

RESULTS

Among the 974 patients, 38 were excluded due to missing data. In total, 936 patients were included in the final analysis of the study (Figure 1).

Among the enrolled cases, there were 835 survivors and 101 deaths. The age of survivors (50.13±18.32 years) was significantly younger than that of the nonsurvivors (58.91±20.92 years) (＜0.05). Moreover, the mental status, oxygen saturation, serum lactate, qSOFA,MqSOFA, LqSOFA, and PqSOFA were significantly different between the two groups (＜0.05). However, no significant difference was observed on sex, temperature,respiratory rate, systolic blood pressure, and PCT between the two groups (Table 1).

Among the four score systems, ROC curve analysis found that LqSOFA was the best predictor of in-hospital mortality of sepsis patients, with an AUC of 0.740, followed by MqSOFA, PqSOFA, and qSOFA with AUC of 0.731,0.712, and 0.705, respectively (Table 2, Figure 2).

Figure 1. Flow chart of case exclusions.

Table 1. Comparison of the baseline characteristics of survivors and non-survivors

Based on the maximum Youden’s index, an optimum cut-off value of 2 was used to predict in-hospital mortality using the LqSOFA, with a sensitivity, specificity, PPV,and NPV of 64.36%, 70.78%, 21.04%, and 94.26%,respectively. The optimum cut-off value of MqSOFA was 2 with sensitivity, specificity, PPV, and NPV of 51.49%,80.96%, 24.65%, and 93.24%, respectively. Likewise, an optimum cut-off value of 2 was used in the PqSOFA, with sensitivity, specificity, PPV, and NPV of 71.29%, 61.68%,18.37%, and 94.67%, respectively. The optimum cut-off value of qSOFA was 2 points with sensitivity, specificity,PPV, and NPV of 39.60%, 91.62%, 36.36%, and 92.62%,respectively (Table 2).

There were significant differences of AUCs between the LqSOFA and qSOFA and between the MqSOFA and qSOFA (both＜0.05). However, no other significant difference was observed (Table 3).

DISCUSSION

The recent increase in sepsis mortality has raised the awareness of the importance of early risk stratification among emergency physicians. Since 2016, early identification and in-hospital mortality prediction of patients with sepsis in the ED with qSOFA has been recommended.The simplicity and availability of the qSOFA has warranted its wide use in the ED.

Figure 2. ROC curves of different versions of the qSOFA in predicting in-hospital mortality. ROC: receiver operating characteristic; qSOFA:quick sequential organ failure assessment; LqSOFA: lactate-enhanced qSOFA; MqSOFA: modified qSOFA; PqSOFA: procalcitoninenhanced qSOFA.

However, our study showed that the sensitivity and specificity of the qSOFA in predicting in-hospital mortality in patients with sepsis were 39.60% and 91.62%, respectively, indicating a weak sensitivity. Our result is consistent with recent studies that emphasized the poor performance of qSOFA in the prediction of sepsis mortality.A retrospective study by Moskowitz et alfound that the qSOFA had a sensitivity and specificity of 39% and 87%, respectively, when used in predicting in-hospital mortality at the time of ED presentation among 24,164 patients with sepsis.Similarly, a prospective study by Askim et alon 1,535 adult patients found that the qSOFA had a sensitivity and specificity of 16% and 96%, respectively, in predicting mortality at the time of ED presentation. Other studies found that the sensitivity of the qSOFA in predicting mortality ranged from 13% to 90%, while the specificity ranged from 86% to 96%.

These studies indicated acceptable qSOFA specificity but unsatisfactory sensitivity, which was consistent with our results. However, high sensitivity is more important than high specificity in predicting sepsis mortality.This is due to the higher risk and cost of false negativity compared to false positivity in patients with high mortality risk.

Based on previous studies, researchers have recently added simplely available indices to the qSOFA to improve sensitivity. In a retrospective study of 836 patients, Liu et alintroduced bedside serum lactate and put forward the LqSOFA, which was shown to be superior to the qSOFA.Our findings were consistent with their study, indicating the superiority of the LqSOFA compared to the qSOFA(＜0.05). Additionally, previous studies have shown the effectiveness of lactate as a predictor of sepsis mortality;therefore, lactate will expectedly increase the effectiveness of the qSOFA in predicting sepsis mortality.This addition to the qSOFA for comprehensive scoring could help emergency physicians to make clinical decisions by combining two indicators instead of using one in isolation.

The SpO/FiOratio is associated with respiratory status; some studies have included it as a key parameter for sepsis mortality.In a retrospective study of 1,137 cases, Guarino et alintroduced the SpO/FiOratio into the qSOFA and developed a modified qSOFA (MqSOFA),which showed better accuracy than the qSOFA. Our study also suggested that the performance of MqSOFA was better than qSOFA. Therefore, the MqSOFA may be a viable tool in predicting sepsis mortality.

Xia et aladded PCT to the qSOFA and developed the PqSOFA, which was indicated to have a higher sensitivity but lower specificity in predicting sepsis mortality compared to qSOFA in a retrospective study of 821 patients. However,no significant difference was observed between the AUCsof the PqSOFA and the qSOFA in our research. Similarly,previous studies have inconsistent results regarding the effectiveness of PCT as a predictor of sepsis mortality.Therefore, further studies are required to determine the utility of the PqSOFA as a revised scoring system for the early prediction of sepsis.

Table 2. The performance of scores with different cut-offs in predicting in-hospital mortality of sepsis patients

Table 3. Pairwise comparison of AUC of scores for predicting in-hospital mortality of sepsis patients

According to our results, the best among the four score systems was the LqSOFA, followed by the MqSOFA,PqSOFA, and qSOFA. Generally, the revised qSOFAs obtained better sensitivity and accuracy in predicting the mortality of sepsis patients than the qSOFA; however, the results remain unsatisfactory. Among the revised versions of the qSOFA,LqSOFA and MqSOFA had similar performance in predicting sepsis mortality (＞0.05). The additional parameters from these measures can be obtained rapidly and conveniently; therefore,the LqSOFA and MqSOFA can be used in the early assessment of mortality in the ED. SpO/FiOratio is more convenient than lactate; therefore, we suggest that the MqSOFA can be used as a candidate for the revised qSOFA to increase the performance of early prediction of sepsis mortality. Furthermore, both the LqSOFA and PqSOFA require a bedside test to be calculated.The MqSOFA was the only altered version of the qSOFA,which contains completely clinical information that makes this score helpful in pre-hospital settings.

This study had several limitations. Firstly, the study population was small. Further studies employing a large sample will provide a higher level of evidence for assessing these scores. Additionally, pre-hospital care was neglected because of unavailable information.Secondly, because of the nature of the retrospective study, we identified altered mentation using the AVPU system. Indeed, altered mentation in the qSOFA should be explained as a different mental status of the patient.Even so, since the altered mentation of each score uses the AVPU system, we believe that this has little effect on the comparison of each score. However, this may affect the test characteristics of each score. Therefore prospective research should be carried out in the future.Finally, our single-center study may cause selection bias.Further multi-center studies are required to validate the performance of these revised qSOFAs prospectively.

CONCLUSIONS

Among the three revised qSOFAs, the PqSOFA had a higher sensitivity and lower specificity than the qSOFA;however, their performances were similar. Moreover, the values of the LqSOFA and MqSOFA in predicting inhospital mortality were greater compared to the qSOFA. As the added parameter of the MqSOFA was more convenient compared than that of the LqSOFA, the MqSOFA is a potential candidate for the revised qSOFA to increase the performance of the early prediction of sepsis mortality.

ACKNOWLEDGEMENTS

We would like to thank all the physicians and nurses who worked in the hospital for helping us collect the data.

None.

The local Institutional Review Committee approved the study and waived the requirement for informed consent due to the retrospective nature of the study. The study complied with the international ethical guidelines for human research, such as the. The accessed data were anonymized.

All authors declared no conflicts of interests.

HH conceived the study, designed the trial, and obtained research funding. HH and JYJ supervised the conduct of the data collection from the database, and managed the data,including quality control. HH and NY provided statistical advice on study design and analyzed the data. All the authors drafted the manuscript, and contributed substantially to its revision.

All the supplementary files in this paper are available at http://wjem.com.cn.