Original Article••

Derivation and validation of a two-biomarker panel for diagnosis of ARDS in patients with severe traumatic injuries

Abstract

Background Acute respiratory distress syndrome (ARDS) is common after severe traumatic injuries but is underdiagnosed and undertreated. We hypothesized that a panel of plasma biomarkers could be used to diagnose ARDS in severe trauma. To test this hypothesis, we derived and validated a biomarker panel in three independent cohorts and compared the diagnostic performance to clinician recognition of ARDS.

Methods Eleven plasma biomarkers of inflammation, lung epithelial and endothelial injury were measured in a derivation cohort of 439 severe trauma patients. ARDS status was analyzed by two-investigator consensus, and cases were required to meet Berlin criteria on intensive care unit (ICU) day 1. Controls were subjects without ARDS during the first 4 days of study enrollment. A multivariable logistic regression model was used to generate probabilities for ARDS. A reduced model with the top two performing markers was then tested in two independent validation cohorts. To assess clinical diagnosis of ARDS, medical records in the derivation cohort were systematically searched for documentation of ARDS diagnosis made by a clinical provider.

Results Among 11 biomarkers, the combination of the endothelial injury marker angiopoietin-2 (Ang-2) and the lung epithelial injury marker receptor for advanced glycation endproducts (RAGE) provided good discrimination for ARDS in the derivation cohort (area under the curve (AUC)=0.74 (95% CI 0.67 to 0.80). In the validation cohorts, the AUCs for this model were 0.70 (0.61 to 0.77) and 0.78 (0.71 to 0.84). In contrast, provider assessment demonstrated poor diagnostic accuracy for ARDS, with AUC of 0.55 (0.51 to 0.60).

Discussion A two-biomarker panel consisting of Ang-2 and RAGE performed well across multiple patient cohorts and outperformed clinical providers for diagnosing ARDS in severe trauma. Clinical application of this model could improve both diagnosis and treatment of ARDS in patients with severe trauma.

Level of evidence Diagnostic study, level II.

Introduction

Acute respiratory distress syndrome (ARDS) is common in patients with severe traumatic injuries.1 Although effective therapies for ARDS are limited,2 timely recognition of ARDS and treatment with a lung protective mechanical ventilation strategy improves mortality.3 4 In subgroups of patients with more severe ARDS or those with absence or resolution of shock, therapies such as prone positioning5 or conservative fluid therapy6 may also be beneficial. However, the clinical diagnosis of ARDS is frequently delayed or missed,7 8 contributing to belated, inadequate or inappropriate treatment.

Plasma biomarkers are commonly used in conditions such as myocardial infarction and congestive heart failure to aid in clinical diagnosis and facilitate rapid application of appropriate therapies. Although plasma biomarkers of inflammation, lung epithelial and endothelial injury may facilitate early identification of patients with ARDS,9 10 the utility of plasma biomarkers for diagnosis of ARDS in patients with traumatic injuries has not been systematically investigated in large cohorts of patients. Only one prior study has examined plasma biomarkers for ARDS diagnosis among trauma patients, a small, single-center case–control study limited by discordant timing of plasma sampling and ARDS diagnosis.11

To rigorously examine the diagnostic performance of plasma biomarkers for ARDS among adult trauma patients, we evaluated 11 previously described biomarkers of inflammation and epithelial or endothelial injury in three prospective observational cohorts of patients with severe traumatic injury. We hypothesized that a panel of plasma biomarkers could accurately identify trauma patients with ARDS and that a panel of plasma biomarkers would improve ARDS diagnosis compared with clinician recognition alone.

Methods

Patient cohorts and clinical data collection

The derivation cohort consisted of 439 consecutive patients with severe traumatic injuries requiring admission to the Vanderbilt trauma intensive care unit (ICU) who were enrolled prospectively in the Validating Acute Lung Injury biomarkers for Diagnosis (VALID) cohort at Vanderbilt University Medical Center between 2006 and 2009. The VALID cohort is a prospective observational cohort study of critically ill patients at risk for ARDS. Inclusion criteria, enrollment and consent procedures have been previously described,12 and the VALID study was approved by the Vanderbilt Institutional Review Board (IRB) with a waiver of informed consent if the patient or surrogate were unable to provide consent. Patients were enrolled on the morning of ICU day 2 and phenotyped for ARDS daily for the first four ICU days. Within this cohort, cases (n=78) were defined as ARDS if they met Berlin ARDS criteria13 on ICU day 1 by review of clinical data and chest radiographs by consensus of two physician investigators who were blinded to biomarker levels. Controls were all other subjects (n=315) who did not meet Berlin ARDS criteria during the first 4 days of study enrollment. Patients who only met ARDS criteria on a single day or who first met criteria on ICU day 2, 3 or 4 (n=46) were excluded from the primary analysis but included in two sensitivity analyses first as cases, then as controls.

The primary validation cohort consisted of 200 patients with severe blunt traumatic injuries enrolled in the University of California, San Francisco (UCSF) Activation of Coagulation and Inflammation in Trauma (ACIT) cohort at San Francisco General Hospital between 2005 and 2013.14 Detailed inclusion and exclusion criteria have been previously described.14 The study was approved by the UCSF Committee on Human Research with deferred informed consent, as previously described.14 In the ACIT cohort, patients were enrolled on arrival to the emergency department. ARDS was defined on the basis of Berlin criteria and ascertained by a two-physician review of all patient data for the first 7 days of hospitalization. For this analysis, cases were defined as ARDS if they met Berlin ARDS criteria13 within 48 hours of ICU admission by review of clinical data and chest radiographs by a two-physician investigator consensus. Controls were all other subjects without ARDS during the first 7 days of study enrollment. Patients who developed ARDS beyond 48 hours of ICU admission (n=29) were excluded from the primary analysis.Six patients with indeterminate ARDS status due to missing data were excluded from all analyses.

As an additional validation study, the final two-biomarker model for diagnosis of ARDS was subsequently tested using existing clinical data and biomarker levels11 from 190 patients with severe traumatic injuries. These patients were enrolled in a separate Vanderbilt IRB-approved study examining the impact of gender on ICU infection and mortality in patients with severe trauma or surgical critical illness, referred to here as the gender study.15 This group of patients included 74 control patients with clear chest radiographs during the first 7 days of hospitalization, 11 control patients with clinical evidence of hydrostatic pulmonary edema and 105 case patients with ARDS (defined by Berlin criteria) within 72 hours of ICU admission.11

Plasma biomarker measurements

For the derivation study, biomarkers were selected based on published studies of biomarkers for diagnosis and prognosis in ARDS and were chosen to reflect pathways that are commonly implicated in the pathogenesis of acute lung injury including inflammation, coagulation, and lung epithelial and endothelial injury. All biomarkers were measured in duplicate in plasma that was collected at enrollment on the morning of ICU day 2, using commercially available enzyme immunoassay kits or radioimmunoassay: plasminogen activator inhibitor-1, American Diagnostica (Stamford, CT); von Willebrand factor antigen (VWF), Diagnostica Stago (Parsippany, NJ); procollagen peptide-III, Cis Bioscience International (Bedford, MA); club cell-16 protein (CC16), BioVendor (Chandler, NC); B-type natriuretic peptide, Bachem Bioscience (King of Prussia, PA); surfactant protein-D (SP-D), Yamasa Corporation (Tokyo, Japan); and angiopoietin-2 (Ang-2), interleukin-8 (IL-8), soluble receptor for advanced glycation endproducts (RAGE), R&D Systems (Minneapolis, MN). For the ACIT validation cohort, Ang-2 and RAGE were measured in plasma that was collected 24 hours after presentation to the emergency department using the assay kits from the same manufacturers. For the gender study cohort, Ang-2 and RAGE levels that had been previously measured using kits from the same manufacturers11 were used.

Clinician recognition of ARDS

To compare the performance of the biomarker panel for diagnosis of ARDS to clinician recognition of ARDS, the electronic medical record of each patient in the derivation cohort was systematically searched for documentation of a diagnosis of ARDS during the hospitalization by a clinical provider by scanning for key words including ‘acute lung injury’, ‘acute respiratory distress syndrome’, ‘ALI’ and ‘ARDS’. Prior to implementation, the accuracy of the electronic search algorithm was confirmed by manual review of all history, physical and progress notes for 100 patients in the derivation cohort. In addition, all positive results for key words from the electronic search were corroborated by investigator review of the relevant documents to confirm that a clinical diagnosis of ARDS was present or suspected.

Statistical analysis

Demographic and clinical variables were analyzed within each study cohort using Wilcoxon rank-sum test for continuous variables and Pearson’s χ2 test for categorical variables. Biomarker values underwent logarithmic transformation to reduce right skewness. When measured biomarker levels were below the assay detection limit, a value was imputed at half the lower limit of detection for each biomarker. In the derivation cohort, we used a backward elimination model-building strategy on 1000 bootstrapped datasets to select the biomarkers for further consideration in a logistic regression model. For each bootstrap sample, a full model with all 11 biomarkers was fit. Then, the biomarker with the largest p value (Wald test) was dropped, and a new model was fit with one fewer biomarker. This backward elimination process was repeated until only one biomarker was retained. The biomarkers were then ranked from most significant (the last one to remain) to least significant (the first one eliminated) and the average rank from 1000 bootstrap repetitions was used to select the variables for further consideration. The full 11-biomarker logistic model, and the best two-biomarker (Ang-2 and RAGE) logistic model were then fit. The optimism of the model was evaluated by a 300-iteration bootstrap validation. The performance of the model was measured using receiver operating characteristics curves and the area under the curve (AUC) and compared with the clinician diagnosis of ARDS. The two-biomarker model was used to compute the ORs and 95%CI for ARDS.

To externally validate the two-biomarker model, we fixed the coefficients of the model and computed the predicted probabilities of ARDS in the two cohorts. The discrimination ability of the model was evaluated in each validation cohort, using the Harrell C-statistic. The 95% CIs for the C-statistic were generated with 300 bootstrap samples.

Two sensitivity analyses were conducted in the VALID cohort that included 46 patients who were initially excluded from analysis, those patients who had delayed onset of ARDS (met ARDS criteria on day 2 or later). These patients were included first as ARDS cases then as controls in sensitivity analyses. All analyses were performed with R V.3.3 (R Foundation for Statistical Computing, Vienna, Austria).

Results

Patients

Patient characteristics are summarized in table 1. Across the three patient cohorts, patients with and without ARDS were similar in terms of age, gender and race/ethnicity. Compared with subjects without ARDS, subjects with ARDS had significantly higher injury severity scores and Acute Physiology and Chronic Health Evaluation (APACHE) II scores, along with higher hospital mortality and longer duration of mechanical ventilation.

Table 1
|
Patient characteristics by patient cohort

Derivation of a biomarker model for diagnosis of ARDS

Levels of the 11 plasma biomarkers in the derivation cohort are shown in table 2. Of the 11 biomarkers, five were significantly different between cases and controls in univariate analysis: Ang-2, RAGE, IL-8, VWF and CC16. A logistic regression model was fit for diagnosis of ARDS using all 11 biomarkers, and the model performance was assessed by the area under the receiver operator characteristic curve (AUC). As shown in figure 1, the AUC for the 11-biomarker panel was 0.78 (95% CI 0.73 to 0.85). To minimize overfitting and maximize the potential for rapid bedside measurement, a second model was tested that used only the two top-performing biomarkers (Ang-2 and RAGE). This model is summarized in table 3. The AUC for the two-biomarker panel was 0.74 (95% CI 0.67 to 0.80) (figure 1). To illustrate how the two-biomarker panel might be applied at the bedside for ARDS diagnosis, a nomogram for the model is shown in figure 2.

Table 2
|
Comparison of 11 plasma biomarkers between ARDS cases and controls in the derivation cohort
Figure 1
Request permissionrequest permission icon
Figure 1

Receiver operator characteristic (ROC) curve analysis of the plasma biomarker panels for differentiating ARDS (cases) from controls in patients with severe traumatic injuries in the VALID cohort. The solid line shows predicted probability of occurrence of ARDS for each subject computed from a logistic regression model that included 11 biomarkers (RAGE, PCPIII, BNP, Ang-2, IL-8, TNF-α, IL-10, VWF, SP-D, PAI-1 and CC16). Specificity and sensitivity were computed at each possible cut-off of the predicted probability. The area under the curve (AUC) is 0.78. The dotted line shows the ROC analysis in the same patients using only the two most discriminatory biomarkers (RAGE and Ang-2). The AUC for this model is 0.74. For comparison, the dashed line shows the ROC analysis for clinician recognition of ARDS with an AUC of 0.55. Ang-2, angiopoietin-2; BNP, brain natriuretic peptide; CC16, club cell-16 protein; IL-8, interleukin 8; IL-10; interleukin 10; PAI-1, plasminogen activator inhibitor-1; PCPIII, procollagen peptide-III; RAGE, receptor for advanced glycation endproducts; SP-D, surfactant protein-D; TNF-α, tumor necrosis factor-α; VALID, Validating Acute Lung Injury biomarkers for Diagnosis; VWF, von Willebrand factor antigen.

Table 3
|
Summary of the two-biomarker model for diagnosis of ARDS in the derivation cohort
Figure 2
Request permissionrequest permission icon
Figure 2

The two biomarker (RAGE and Ang-2) multivariable logistic regression model was used to create a prediction model for the probability of ARDS. The value for each predictor variable (RAGE, Ang-2, both in pg/mL) is used to determine a number of points using the point scale at the top. The sum of the individual predictor variable points for the measured RAGE and Ang-2 levels corresponds to the total points and the probability of ARDS shown at the bottom. Ang-2, angiopoietin-2; ARDS, acute respiratory distress syndrome; RAGE, receptor for advanced glycation endproducts.

Sensitivity analyses

Because patients who had either delayed development of ARDS (after ICU day 1) were excluded from the initial model derivation, two sensitivity analyses were conducted. In the first sensitivity analysis, the 46 excluded patients were included as ARDS cases and 11-biomarker and two-biomarker logistic regression models were constructed; model performance was reduced with AUCs of 0.72 (95% CI 0.69 to 0.79) for the 11-biomarker model and 0.67 (95% CI 0.63 to 0.73) for the two-biomarker model (likelihood ratio test, p=0.017). By contrast, in the second sensitivity analysis when the 46 excluded patients were included as non-ARDS controls, model performance was very close to the original derivation with AUC of 0.77 (95% CI 0.73 to 0.84) for the 11-biomarker model and 0.74 (95% CI 0.67 to 0.80) for the two-biomarker model (likelihood ratio test, p=0.071).

Validation of the two-biomarker model in the ACIT cohort

Performance of the two-biomarker model using Ang-2 and RAGE was validated in a separate cohort of 165 patients with severe traumatic injuries who were enrolled in the ACIT study. Performance of the two-biomarker model for diagnosis of ARDS was slightly lower in the ACIT cohort (AUC 0.70, 95% CI 0.61 to 0.77) compared with the derivation cohort.

Validation of the two-biomarker model in the gender study cohort

Performance of the two-biomarker model was also validated in a separate cohort of 190 patients with severe traumatic injuries who were enrolled in a study of gender and ICU-acquired infections. Performance of the two-biomarker model for diagnosis of ARDS was excellent in the gender study cohort (AUC 0.78, 95% CI 0.71 to 0.84). Of note, this cohort included patients with both clear chest radiographs and evidence of hydrostatic pulmonary edema in the control group.

Comparison of the biomarker model to clinician diagnosis in the VALID cohort

To analyze whether the two-biomarker panel outperformed clinician recognition of ARDS, the electronic medical record of each patient in the derivation cohort was systematically scanned for documentation of a diagnosis of ARDS during the hospitalization by a clinical provider. The two-biomarker panel significantly outperformed (Akaike Information Criterion reduced by 25) clinician recognition which had an AUC of 0.55 (95% CI 0.51 to 0.60, figure 1), and a sensitivity of 16% and specificity of 93% when compared with the gold standard diagnosis by two investigator consensus.

Discussion

Although there have been many studies of the prognostic value of biomarkers in patients with established ARDS,16 17 there are relatively few studies of the diagnostic value of plasma biomarkers in ARDS. The primary goal of this study was to derive and validate a plasma biomarker panel for diagnosis of ARDS in adult patients with severe traumatic injuries. We tested 11 biomarkers of various aspects of the pathophysiology of ARDS including biomarkers of inflammation, lung epithelial injury, endothelial injury and coagulation. To minimize overfitting and maximize the potential for rapid bedside application in the future, this panel was narrowed to the top two performing biomarkers and then validated in two independent patient cohorts. The two-biomarker panel of Ang-2 and RAGE performed similarly across the three cohorts and significantly outperformed clinical diagnosis.

Although it has been reported that ARDS is underdiagnosed and undertreated,3 7 8 18 the current study underscores the potential magnitude of this problem. In a detailed review of all history and progress notes for each patient in the derivation cohort, we found that ARDS was largely undiagnosed; clinician recognition had a sensitivity of only 31% for the presence of ARDS, with an AUC of 0.55. Thus, although performance of the two-biomarker panel is only in the moderate range for a diagnostic test, when the performance of the two-biomarker panel is compared with clinician recognition of ARDS, there is clear value. Since timely diagnosis and adherence to low tidal volume ventilation decreases ICU mortality,3 utilization of this simple biomarker panel could expedite diagnosis and treatment of ARDS in severe trauma, leading to improved clinical outcomes. In addition, application of the two-biomarker panel could facilitate identification of patients for inclusion in clinical trials of new therapies for trauma-associated ARDS.

Although we tested biomarkers of many aspects of the pathophysiology of ARDS including inflammation, activation of fibrotic pathways, dysregulated coagulation and a biomarker of increased intravascular volume, the top two performing biomarkers that make up the final two-marker panel are biomarkers of lung epithelial injury (RAGE) and endothelial injury (Ang-2). Both lung epithelial injury and endothelial injury are key features of ARDS that have been well documented in both experimental and clinical acute lung injury, lending biologic plausibility to the findings. Lung epithelial injury is a pathophysiologic hallmark of ARDS19 that is evident both pathologically20 and in functional assays of alveolar epithelial transport function.21 RAGE is highly expressed on the type I alveolar epithelium22; release of RAGE into the airspace and circulation is a biomarker of lung epithelial injury in rodent,22 human experimental23 and clinical studies.24 In patients at risk for ARDS from sepsis, biomarkers of lung epithelial injury including RAGE, SP-D and CC16 are strongly associated with the diagnosis of ARDS,9 suggesting that biomarkers of lung epithelial injury may be useful for diagnosis of ARDS in patients with a variety of underlying risk factors beyond just patients with severe traumatic injuries.

Injury to the microvascular endothelium is also a pathophysiologic hallmark of ARDS.2 Several biomarkers of endothelial injury have previously been associated with development of ARDS including VWF25 and Ang-2. Ang-2, an endothelial growth factor and potent mediator of vascular permeability and endothelial injury, was recently found to be highly predictive of ARDS among patients presenting to the emergency department who were at risk for, but did not yet have, ARDS.10 This study did not include patients with traumatic injuries. The current study extends prior findings to patients with traumatic injuries and suggests that endothelial injury is a key pathophysiologic feature of trauma-induced ARDS that can be used to aid clinical diagnosis.

This study has several strengths. First, because of the large size of the derivation cohort, we were able to test a large number of potential biomarkers in the derivation cohort, and thus the study included candidate biomarkers of diverse aspects of the pathophysiology of ARDS. Second, all three of the cohorts that were used had dedicated ARDS phenotyping done by the study authors, including two physician review of all chest radiographs and clinical data, insuring the validity of the gold standard ARDS diagnosis for this study. Finally, validation of the biomarker panel in two independent cohorts enhances the external validity of the findings.

There are also some limitations. Due to differences in how the three cohorts were phenotyped for ARDS in the three parent studies, each cohort used slightly different definitions for cases and controls. This might, in part, explain why the biomarker panel performance varied somewhat between cohorts. In addition, the gender cohort was a case–control cohort, and the performance of the biomarker panel in this group may be an overestimate of panel performance due to patient selection. As noted above, the overall performance of the two-biomarker panel was only in the moderate range, although biomarkers significantly outperformed clinical diagnosis. Finally, since the studies were limited to patients with severe traumatic injuries, the findings cannot be generalized to other patient populations.

In summary, a two-biomarker panel consisting of Ang-2 and RAGE performed well across multiple patient cohorts and outperformed clinical providers for diagnosing ARDS in patients with severe traumatic injuries. If validated prospectively, clinical application of this model could improve both diagnosis and timely treatment of ARDS.

Overview

Article metrics
Altmetricopen-url
Dimensionsopen-url