Development and evaluation of a predictive nomogram for survival in heat stroke patients: a retrospective cohort study

Fei Shao ,Xian Shi ,Shu-hua Huo ,Qing-yu Liu ,Ji-xue Shi ,Jian Kang ,Ping Gong,Sheng-tao Yan,Guo-xing Wang,Li-jie Qin,Fei Wang,Ke Feng,Feng-ying Chen,Yong-jie Yin,Tao Ma,Yan Li,Yang Wu,Hao Cui,Chang-xiao Yu,Song Yang,Wei Gan,Sai Wang,Liu-ye-zi Du,Ming-chen Zhao,Zi-ren Tang,,Shen Zhao

1 Department of Emergency Medicine,Beijing Chaoyang Hospital,Capital Medical University,Beijing 100020,China

2 Department of Emergency Medicine,Hebei Yanda Hospital,Langfang 065201,China

3 Department of Emergency Medicine,Beijing Huairou Hospital,Beijing 101400,China

4 Department of Emergency Medicine,the Second Hospital of Hebei Medical University,Shijiazhuang 050000,China

5 Department of Emergency Medicine,the Second Affiliated Hospital of Shandong First Medical University,Tai’an 271000,China

6 Department of Emergency Medicine,the First Affiliated Hospital of Dalian Medical University,Dalian 116011,China

7 Department of Emergency Medicine,China-Japan Friendship Hospital,Beijing 100029,China

8 Department of Emergency Medicine,Beijing Friendship Hospital,Beijing 100050,China

9 Department of Emergency Medicine,Henan Provincial People’s Hospital,Zhengzhou 450003,China

10 Department of Emergency Medicine,Beijing Tsinghua Changung Hospital,Beijing 102218,China

11 Department of Emergency Medicine,General Hospital of Ningxia Medical University,Yinchuan 750004,China

12 Department of Emergency Medicine,the Affiliated Hospital of Inner Mongolia Medical University,Hohhot 010050,China

13 Department of Emergency Medicine,the Second Hospital of Jilin University,Changchun 130021,China

14 Department of Emergency Medicine,the First Hospital of China Medical University,Shenyang 110001,China

15 Department of Emergency Medicine,the Second Hospital of Shanxi Medical University,Taiyuan 030001,China

16 Department of Big Data Research,Goodwill Hessian Health Technology Co.,Ltd.,Beijing 100085,China

17 School of Public Health,Peking University,Beijing 100083,China

18 Beijing Key Laboratory of Cardiopulmonary Cerebral Resuscitation,Beijing 100020,China

19 Department of Critical Care Medicine,Beijing Friendship Hospital,Capital Medical University,Beijing 100050,China

BACKGROUND: This study aimed to establish an effective nomogram to predict the survival of heat stroke (HS)based on risk factors.METHODS: This was a retrospective,observational multicenter cohort study.We analyzed patients diagnosed with HS,who were treated between May 1 and September 30,2018 at 15 tertiary hospitals from 11 cities in Northern China.RESULTS: Among the 175 patients,32 patients (18.29%) died before hospital discharge.After the univariate analysis,mechanical ventilation,initial mean arterial pressure ＜70 mmHg,maximum heart rate,lab results on day 1 (white blood cell count,alanine aminotransferase,creatinine),and Glasgow admission prediction score were included in multivariate analysis.Multivariate Cox regression showed that invasive ventilation,initial mean arterial pressure ＜70 mmHg (1 mmHg=0.133 kPa),and Glasgow admission prediction score were independent risk factors for HS.The nomogram was established for predicting 7-d and 14-d survival in the training cohort.The nomogram exhibited a concordance index (C-index) of 0.880 (95% confidence interval [95% CI] 0.831-0.930) by bootstrapping validation (B=1,000).Furthermore,the nomogram performed better when predicting 14-d survival,compared to 7-d survival.The prognostic index cut-off value was set at 2.085,according to the operating characteristic curve for overall survival prediction.The model showed good calibration ability in the internal and external validation datasets.CONCLUSION: A novel nomogram,integrated with prognostic factors,was proposed;it was highly predictive of the survival in HS patients.

KEYWORDS: Heat stroke;Nomogram;Survival;Prognosis

INTRODUCTION

Heat stroke (HS) is a common critical disease characterized by elevated core body temperature (over 40.5 °C) and neurological disorders.Environmental heat exposure causes more deaths than all natural disasters taken together.It is estimated that between 1979 and 2002,more Americans died from HS than from hurricanes,lightning,earthquakes,floods,and tornadoes combined.In Australia,at least 1,700 people died of HS between 2006 and 2017.On average,heat-related mortality in Chinese metropolises was 32.1 deaths per million people,according to multiple global climate models of 1986-2005.Additionally,critical cases of HS increased significantly in the summer of 2018 in Northern China due to unprecedented high temperatures.

There have been several studies on the risk factors,pathogenesis,treatment,and prevention of HS,such as evidence within sports medicine to support good prognosis with proper management and care,but little is known about how to determine the prognosis of HS early.Moreover,the lack of early,noninvasive,predictable biomarkers of organ damage hinders the identification of timely and effective treatments to mitigate it.The purpose of this study was to screen the factors that can aid in the evaluation of HS prognosis,as well as to develop and validate a nomogram to predict survival in HS patients.

METHODS

Study design

This was a retrospective,observational multi-center study of patients diagnosed with HS and treated between May 1 and September 30,2018.We collected data on HS-related hospital admissions from 15 tertiary hospitals in 11 cities of 9 provinces in Northern China during a 5-month period.

The eligibility criteria were as follows: patients older than 18 years;meeting the diagnostic criteria of severe HS: classic or exertional HS with a history of exposure to hot and humid weather or strenuous activity,concurrent hyperthermia (core temperature of ＞40.5 °C),and related neurological dysfunction including confusion,convulsions,or coma on admission.

We excluded patients who had irreversible underlying diseases affecting mortality and pregnancy.

Data collection

For each patient,we collected the following information:demographic data (age,sex,location where HS occurred),condition of hospitalization (initial mean arterial pressure,invasive ventilation,anal temperature,maximum heart rate,Glasgow admission prediction score,hemofiltration,intravascular hypothermia,and mild hypothermia),and laboratory tests on days 1 and 3,including white blood cell count (WBC),red blood cell count (RBC),hemoglobin count(HBG),platelet count (PLT),alanine aminotransferase (ALT),creatine (CRE),potassium,calcium,myoglobin (MYO),and brain natriuretic peptide (BNP).All the data were collected from the electronic medical records or the hospital database at the time of initial hospital admission.

Outcome analysis

The primary outcome was survival until hospital discharge after HS.The secondary outcomes included 7-d and 14-d survival.

Statistical analysis

We divided the whole dataset into training and internal validation datasets at a ratio of 65:35 using stratified resampling to ensure that both datasets had the approximately same percentage of survivors and non-survivors.An external validation dataset from Beijing Daxing Hospital,including data from 53 patients with HS,was collected concurrently.

The categorical variables of survivors and nonsurvivors were compared using either a Chi-squared test or Fisher’s exact test,as appropriate.Continuous variables are presented as the mean±standard deviation or the median with interquartile range.Survival curves were generated using the Kaplan-Meier method and compared using the log-rank test.Univariate Cox regression analyses were performed to identify potentially relevant prognostic factors.Significant factors with-values less than 0.2 in univariate analysis,combined with the factors from clinicians’ treatment experiences,were included in a forward stepwise multivariable Cox proportional hazards model.

The model discrimination was measured by the area under the receiver operating characteristic (ROC) curve,also known as the concordance index (C-index).A C-index of 1.00 indicates perfect discrimination,whereas a C-index of 0.5 indicates no discrimination.A nomogram based on the results of the Cox proportional hazard regression was established to estimate overall survival using the rms package of R software,whereas the C-index was used to estimate the accuracy of the nomogram.A prognostic index (PI) was established based on the Cox regression model,and patients were categorized into high-risk and low-risk groups according to the optimal cutoff value on the ROC curve.Survival curves were compared between these two groups.The model was also evaluated by a calibration curve that compared the predicted and observed survival probabilities,which facilitated an additional comparison of prognostic accuracy.We evaluated the predictive criterion validity by assessing model discrimination and calibration in the internal and external validation datasets.All statistical analyses were conducted with R software,version 4.0.1.

RESULTS

A total of 175 patients (143 survivors,32 non-survivors)with HS were documented in the 15 hospitals during the 5-month study period (Figure 1).The demographic data and significant variables are presented in Table 1.

Figure 1. Flowchart diagram illustrating patients included in each group throughout the study.HS: heat stroke.

The patients’ average age was 64 years (51-75 years),nearly two-thirds (65.14%) were male,and 32 (18.29%) died before hospital discharge (Table 1).Sex,age,intravascular hypothermia,mild hypothermia,8 laboratory tests on day 1 (PLT,BNP,WBC,HBG,calcium,MYO,potassium,and RBC),and 7 laboratory tests on day 3 (HBG,AST,international normalized ratio [INR],CRE,potassium,PLT,and BNP) did not differ significantly between survivors and non-survivors.Survivors’ initial mean arterial pressure was significantly higher than that of the non-survivors.The median temperature of non-survivors was significantly higher than that of survivors (40.8 °C vs.39.7 °C,=0.002).The mean of the Glasgow admission prediction score was 3 points higher in the survivor group than in the non-survivor group (＜0.001).A significantly higher proportion of non-survivors had invasive ventilation (87.50% vs.25.17%,＜0.001),maximum heart rate ＞100 beats/min (90.62% vs.65.73%,=0.010),hemofiltration (43.75% vs.9.79%,＜0.001),and MYO values ＞107 μg/L on day 3 (100% vs.86.71%,=0.026).

After the univariate analysis,invasive ventilation,initial mean arterial pressure ＜70 mmHg,core temperature,maximum heart rate,hemofiltration,and Glasgow admission prediction score were included (＜0.2).In addition,WBC＞9.5×10/L,ALT ＞40 U/L,CRE ＞81.0 U/L,HBG ＞ 150 g/L,and PLT ＜125×10/L on day 1,as well as WBC ＜3.5×10/L or ＞9.5×10/L,PLT ＜ 125×10/L and AST ＞35 U/L on day 3,were found to be significant.Based on our univariate analysis and the clinical experience of our clinicians,we included 8 predictors in our multivariate analysis: invasive ventilation,initial mean arterial pressure (＜70 mmHg [1 mmHg=0.133 kPa] or ＞105 mmHg),maximum heart rate,lab results on day 1 (WBC,ALT,and CRE),and Glasgow admission prediction score (supplementary Table 1).

Multivariate Cox regression indicated that invasive ventilation,initial mean arterial pressure ＜70 mmHg,and Glasgow admission prediction score were independent risk factors for the prognosis of HS patients (supplementary Table 1).To further interpret our Cox regression model and obtain individualized estimates of outcomes,a nomogram for predicting 7-d and 14-d survival in the training dataset was established (Figure 2).The nomogram had a C-index of 0.880 (95% confidence interval [95%] 0.831-0.930) by bootstrapping validation (B=1000).As indicated in Figure 2,higher values for lab results on day 1 (WBC,CRE,ALT)or maximum heart rate resulted in a higher total score and lower chance of survival on day 7 and day 14.Meanwhile,a higher value of the initial mean arterial pressure or Glasgow admission prediction score contributed to a lower total score and a higher chance of survival on day 7 and day 14.In addition,patients with no invasive ventilation are more likely to survive until day 7 and day 14.

Table 1. Demographic and clinical characteristics of the study population (n=175)

The PI cut-off value was set at 2.085 according to the ROC curve for overall survival prediction.Patients were divided into high-risk (PI ≥2.085) and low-risk groups (PI＜2.085).Supplementary Figure 1 shows significant survival heterogeneity between the two groups (＜0.001),with a larger gap for 14-d survival than for 7-d survival.

Supplementary Figure 2 shows good model discrimination in both the internal and external validation datasets.The C-indexes in the training datasets were lower than those in the external validation datasets but were higher than those in the internal validation datasets.Moreover,in the comparison of the calibration curves for predicting 7-d and 14-d survival in the internal and external validation datasets,the model showed good calibration ability in both datasets.Overall,the calibration ability for 14-d survival was superior to that for 7-d survival.With regard to the difference between the validation datasets,the calibration ability in the external validation dataset was much better than that in the internal validation dataset for 7-d survival but slightly worse for 14-d survival.

Figure 2. Nomograms for predicting 7-d survival and 14-d survival in the training set.

DISCUSSION

Shock,acute renal failure,rhabdomyolysis,acute respiratory distress syndrome,disseminated intravascular coagulation,and acid-base or electrolyte disorders are common potential or direct complications of severe HS.In this study,we observed an elevation of cardiac troponins(cTn) I in nearly half of the patients;however,there was no significant difference when compared with the mortality rate.Finally,we included the maximum heart rate to establish the nomogram.Routinely,people are exposed to high temperatures and cool down through an increase in heart rate,which in turn increases blood flow to the skin.However,this protective mechanism increases myocardial oxygen consumption and may lead to myocardial infarction.Patients with HS combined with acute myocardial infarction often have poor prognosis and high treatment costs.Furthermore,numerical changes in cTn I can be used to predict the prognosis of HS.

Rhabdomyolysis,which usually occurs in the acute phase of HS,was observed in over half of this study’s participants.Heat stress can induce kidney disease through acute tubular injury from heat-and exercise-induced rhabdomyolysis,acute elevation in serum and urine uric acid with crystalluria,extracellular volume depletion,and resultant renal hypoperfusion.Furthermore,simple elevation of core body temperature leads to severe kidney injury,which is associated with tubular death.Increased plasma creatinine concentration,reduced urine volume and urine flow rate,reduced fractional excretion of sodium,and mildly increased urinary specific gravity occur after impaired renal function in patients with HS.In this study,we included CRE in the development of the nomogram.

Liver injury has been reported among patients secondary to the IV infusions of amiodaroneor among patients with HS because of an inflammatory response rather than as an acute response to hyperthermia.Some studies have reported liver injury with an early increase in AST and lactate dehydrogenase(LDH),which peak after 3 to 4 d,and an increase in bilirubin by the second or third day.

In our study,the mortality rate was 18.29%,which was lower than that in Western countries.The increase in HS cases (especially exertional HS) in the summer of 2018 in Northern China has been associated with high temperatures and humidity conditions lasting over two months,which is rare for Northern China.After several outbreaks,media reports on HS gained public attention,which in turn caused related regulations to be introduced.Hospitals and prehospital rescue institutions implemented an emergency plan for treating patients with HS in most cities in Northern China.The prevention of HS is as crucial as proper and timely treatment.

This study had certain limitations.First,this study was a retrospective study with a small sample size;thus,a prospective study with a larger population is required to further validate our findings.Second,our data lacked the time from illness onset to hospitalization and coagulation values that were primary indicators of the prognosis of patients with HS,which may have influenced the model we developed.Finally,we did not consider the influence of climatic factors,which may affect HS and the related outcomes of patients.

CONCLUSIONS

A nomogram for predicting survival in HS patients is developed and internally and externally valid.The inclusive nomogram might represent an efficient and valuable tool for prediction and treatment decision-making of HS patients.

ACKNOWLEDGMENTS

We thank all colleagues who collected the medical information of the patients included in this study: Tao Wang,Yan-fen Chai,Qing-bian Ma,Hong-hong Pei,Feng Xu,Chun-ling Zhao,Tian-peng Zhang,Min Zhang,Zhao Tao,and Zhi Liu.We thank the patients and their families for making this study possible.We would like to express our very great appreciation to other colleagues: Qiang Wang and Zheping Yuan from Goodwill Hessian Health Technology Co.,Ltd.,who contributed greatly throughout the process of data cleaning and analyses.

None

The Ethics Committee Board of Beijing Chaoyang Hospital,Capital Medical University,approved this study and waived the requirement for written informed consent.

The authors state that there are no conflicts of interest related to this manuscript.

FS and XS contributed equally to this article.ZRT,SZ,and FS contributed conception and design of the study;SHH,QYL,JXS,JK,PG,GXW,LJQ,FW,KF,FYC,YJY,TM,and YL collected the data;YW,HC,and CXY organized the database;SY,WG,SW,LD,and MCZ performed the statistical analysis;FS and XS wrote the first draft of the manuscript.All authors contributed to manuscript revision and read and approved the submitted version.

All the supplementary files in this paper are available at http://wjem.com.cn.