You are here

Article Text

Article menu

Download PDFPDF

XML
Original research
Hyper-realistic and immersive surgical simulation training environment will improve team performance
  1. Tuan N Hoang1,
  2. Anthony J LaPorta2,
  3. John D Malone3,
  4. Roland Champagne4,
  5. Kit Lavell5,
  6. Gabriel M De La Rosa6,
  7. Lawrence Gaul7,
  8. Mitchell Dukovich4
  1. 1Naval Surface Forces, US Pacific Fleet, San Diego, California, USA
  2. 2Rocky Vista University College of Osteopathic Medicine, Parker, Colorado, USA
  3. 3Medicine, Uniformed Services University, Bethesda, Maryland, USA
  4. 4Naval Medical Center San Diego, San Diego, California, USA
  5. 5Strategic Operations, Inc, San Diego, California, USA
  6. 6Naval Center for Combat and Operational Stress Control, San Diego, California, USA
  7. 7Chinle Comprehensive Health Care Facility, Chinle, Arizona, USA
  1. Correspondence to Dr John D Malone; jdmalone123{at}gmail.com

Abstract

Background Surgical trauma care requires excellent multidisciplinary team skills and communication to ensure the highest patient survival rate. This study investigated the effects of Hyper-realistic immersive surgical team training to improve individual and team performance. A Hyper-realistic surgical training environment is defined as having a high degree of fidelity in the replication of battlefield conditions in a training environment, so participants willingly suspend disbelief that they become totally immersed and eventually stress inoculated in a way that can be measured physiologically.

Methods Six multispecialty member US Navy Fleet Surgical/US Army Forward Surgical Teams (total n=99 evaluations) underwent a 6-day surgical training simulation using movie industry special effects and role players wearing the Human Worn Surgical Simulator (Cut Suit). The teams were immersed in trauma care scenarios requiring multiple complex interventions and decision making in a realistic, fast-paced, intensive combat trauma environment.

Results Hyper-realistic immersive simulation training enhanced performance between multidisciplinary healthcare team members. Key efficacy quantitative measurements for the same simulation presented on day 1 compared with day 6 showed a reduction in resuscitation time from 24 minutes to 14 minutes and critical error decrease from 5 to 1. Written test scores improved an average of 21% (Medical Doctors 11%, Registered Nurses 25%, and Corpsman/Medics 26%). Longitudinal psychometric survey results showed statistically significant increases in unit readiness (17%), combat readiness (12%), leadership quality (7%), vertical cohesion (7%), unit cohesion (5%), and team communication (3%). An analysis of salivary cortisol and amylase physiologic biomarkers indicated an adaptive response to the realistic environment and a reduction in overall team stress during performance evaluations.

Conclusions Hyper-realistic immersive simulation training scenarios can be a basis for improved military and civilian trauma training.

Level of evidence Level III.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/tsaco-2019-000393

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Background

    Excellent trauma care to obtain the best possible patient outcomes with cost-effectiveness requires trained trauma teams with the highest level of surgical skills and intercommunication. Obtaining appropriate trauma experience with standardized, consistent same member teams is a challenge for both military and civilian institutions, especially with the need to maintain immediate readiness in response to mass casualty shooting and bombing events. Hyper-realistic medical training simulations that use state-of-the-art movie industry special effects, role players, and construction techniques to create realistic operative educational experiences may be an option to improve trauma team performance. Hyper-realistic is defined by Strategic Operations Inc. as “such a high degree of fidelity in the replication of battlefield conditions in a training environment that participants so willingly suspend disbelief that they become totally immersed and eventually stress inoculated.”1 In this study, the Human Worn Surgical Simulator known as the “Cut Suit” was placed on human actors allowing for patient-provider interaction. The anatomically realistic “Cut Suit” mannequin with life-like hemorrhaging blood vessels enables surgical teams to practice performing emergency resuscitation procedures and abdominal operations on realistically simulated traumatic, life-threatening injuries (figure 1). This research study on surgical team training assessed the skill and decision process of the surgeon and team members’ communication (anesthesia providers, nurses, corpsmen/medic technicians) to provide immediate life-saving “damage control” resuscitative surgery to control hemorrhage and prevent or correct hypothermia, acidosis, and coagulopathy to ensure maximal survival rates of military trauma victims.

    Figure 1
    Figure 1

    Human Worn Surgical Simulator-Cut Suit was placed on human actors allowing for patient-provider interaction. The anatomically correct “Cut Suit” mannequin with life-like hemorrhaging blood vessels enables surgical teams to practice operations on realistically simulated traumatic, life threating injuries.

    Methods

    This study was designed to evaluate if US Navy Fleet Surgical/US Army Forward Surgical Teams performance to provide life-saving “damage control” surgery in military Role 2 Echelon of Care facilities where a patient first encounters a surgical team, would improve significantly using hyper-realistic immersive training and a standardized curriculum during 6 days of training. The surgical teams were a population of opportunity for study as they underwent unit training to prepare for military overseas missions. The highly experienced team of clinical proctors including trauma surgeons, emergency physicians, critical care nurses, and senior corpsman/medics observed team performances. Evaluation measures were determined by reductions in (1) patient resuscitation time measured in minutes, (2) number of critical errors, and (3) cortisol stress biomarker response of participants. Other evaluation measures included (4) salivary α amylase levels, (5) knowledge of trauma care key principles, and (6) psychometric surveys assessing team communication, cohesion, readiness, and leadership.

    Research subject population and recruitment

    A total of six research populations of opportunity were recruited, consisting of five US Navy Fleet Surgical Teams (78%) and one US Army Forward Surgical Team (22%). Each team is composed of essentially 18 individuals. The US Army Forward Surgical Team, a consistently stable Reserve Unit of older and experienced personnel, was evaluated on 2-week long time periods separated by at least 6 months. A total of 99 evaluations were conducted on 79 individuals (72% male, 28% female) out of 108 potential research subjects consented to participate in this study which was approved by the Naval Medical Center San Diego Institutional Review Board (table 1). Officer ranks of healthcare professionals (40% with graduate degrees, 20% bachelor degrees) noted from junior to senior rank/experience of Officer 03 to Officer O6 comprised 50% of the study subjects. Significant educational experience (associates degrees 14%, some college 23%) was found in the non-commissioned officer (US Navy Hospital Corpsman, US Army “Medic” Healthcare Specialist) technician enlisted ranks noted from junior to senior rank/experience as Enlisted 2 to Enlisted 7. The first team trained from February 29 through March 5, 2016; the sixth and final team from June 5 to 11, 2017. The 6-day Shipboard Surgical Trauma Training (S2T2) Curriculum, a course developed by US Navy Captain Tuan Hoang, MD, FACS, included both classroom lectures and didactic team training sessions with six different hyper-realistic simulation exercises in an immersive training environment concluding with a multiple-victim mass casualty event. The first simulation scenario was repeated on the last training day six to evaluate performance changes. The authors have previously reported smaller studies on hyper-realistic simulation surgical training of US Military providers.2–4 This study was conducted using an on-site hyper-realistic training simulation environment provided by Strategic Operations (STOPS), San Diego.

    View this table:
    Table 1

    Study population demographics

    Measures of efficacy

    1. Resuscitation time in minutes was defined by the start of the exercise until the simulated patient was stabilized and taken into the operating room.

    2. Critical error evaluation (scale 1–10 higher number representing more errors) on standardized scenarios included the following: team huddle meeting before start of the exercise, communication to members on specific assignments and responsibilities, injury pattern recognition, hemorrhage control and blood bank management, airway and chest tube management, intravenous/intraosseous access, X-ray utilization, Focused Assessment with Sonography in Trauma survey, operative management, patient transfer packaging, and administrative documentation including reassessment prior to transport.

    3. Written knowledge assessment-identical precourse and postcourse 25 question multiple-choice S2T2 examination taken by all team members on critical principles of trauma care. Example question: “Which is not a part of the lethal triad of trauma?”

    4. Psychometric scale surveys-administered on days 0, 3, and 6. Survey details are noted in online supplementary table 2. Resiliency and team cohesion assessments included individual perceptions of perceived stress, unit support, and interdisciplinary teamwork.

    5. Salivary samples were collected from participants at various time points during the training scenarios using oral swabs. Salivary biomarker metrics were cortisol and α-amylase. Assays were performed by Salimetrics, LLC (Carlsbad, CA). Salivary cortisol was assayed in duplicate using a highly sensitive enzyme immunoassay (Salimetrics). The test uses 25 µL of saliva per determination, has a lower limit of sensitivity of 0.003 µg/dL, standard curve range from 0.012 µg/dL to 3.0 µg/dL, an average intra-assay coefficient of variation of 3.5%, and an average inter-assay coefficient of variation of 5.1%. Method accuracy determined by spike recovery averaged 100.8%, and linearity determined by serial dilution averaged 91.7%. Values used in statistical analysis were the average of duplicate assays for each sample.

    Supplemental material

    Statistical analysis

    Statistical analysis was performed on 99 consented subject evaluations from 79 individuals, a significant sample size. The research protocol received highly experienced Ph.D. biostatistician review for adequate sample size based on multiple data points longitudinally obtained from nearly 100 evaluations. S2T2 course written examination results were analyzed with paired samples t-test to evaluate if medical knowledge significantly increased over time. T-tests were conducted on the entire sample (n=99) and for each 18-member surgical team using SPSS V.21.

    Psychometric survey measures included 3×6 (three-time points by six groups) repeated analysis of variance (ANOVA) measures comparing groups of means to evaluate for significant changes over times and interactions between time and group. A significant interaction between time and group was interpreted that the effect of time on the measure of interest varies between groups. Given a significant interaction between time and group, separate ANOVAs were conducted for each group to test for significant changes in the variables of interest for each group separately.

    Salivary data were analyzed using IBM SPSS Statistics, V.23.0 (Armonk, New York, USA). Raw salivary cortisol and α-amylase were reduced to a percent change from the “pre” time point for each individual and then averaged for each group at each time point. Differences between simulation scenarios across simulation time points were evaluated in a 2(SIM)×3(TIME) repeated measures ANOVA. All hypothesis tests were two-sided and the probability of committing a Type I error was set at 0.05.

    Results

    Resuscitation time/disposition

    The mean time to disposition during the trauma scenario decreased from an initial 24.39 minutes (95% CI 18 to 30) to 13.91 (CI 11 to 19) minutes postcourse in an average of all six teams (figure 2).

    Figure 2
    Figure 2

    Time to disposition—same scenario. The mean time to disposition during the trauma scenario decreased from an initial 24.39 minutes (95% CI 1 to 30) to 13.91 (CI 11 to 19) minutes postcourse in an average of all six teams.

    Critical errors evaluation

    The mean number of critical errors decreased from 5.00 (95%CI 3.5 to 6.8) to 1.00 (CI, 0 to 1.8) for the same simulation presented on day 1 compared with day 6 of training (figure 3).

    Figure 3
    Figure 3

    Critical events missed (mean). The mean number of critical errors decreased from 5.00 (95% CI 3.5 to 6.8) to 1.00 (CI 0 to 1.8) for the same simulation presented on day 1 compared with day 6 of training.

    Written knowledge

    The same 25 question multiple-choice examination on key trauma concepts administered on the first and last day of the course showed an average improvement of 21% (table 2). Greatest increases in trauma knowledge were among Corpsmen/Medic enlisted technical staff (26%) and nurses (25%) compared with physicians (11%).

    View this table:
    Table 2

    Written knowledge assessment

    Psychometric surveys

    Statistically significant increases were seen in the following survey results: Unit Readiness 17%; Combat Readiness 12%; Leadership Quality 7%; Vertical Cohesion 7%; Unit Cohesion 5%; Team Communication 3% (figure 4). Statistically, non-significant changes were apparent regarding perceived stress, anxiety, resilience, and job satisfaction.

    Figure 4
    Figure 4

    Psychometric survey data. Clinically significant increases with hyper-realistic training were seen in Unit Readiness 17%; Combat Readiness 12%; Leadership Quality 7%; and Vertical Cohesion 7%.

    Table 3 represents the results of ANOVA analyses conducted for the entire sample. Significant interactions were found between time and group for unit readiness F(2,10) = 2.21, p=0.020 and vertical cohesion F(2,10) = 3.36, p=0.001; this was interpreted that the effect of time on these variables was not the same across all groups. As such, separate analyses were performed for unit readiness and vertical cohesion by group. With regard to unit readiness, two groups did not experience statistically significant changes as manifested by all other groups. As for vertical cohesion, two groups experienced significant increases.

    View this table:
    Table 3

    Psychometric survey—main effects of time

    Salivary cortisol and α-amylase biomarkers

    Salivary cortisol and α-amylase biomarker data are represented as deltas, which have been calculated as the percent change from the “pre” time point in figure 5. In a 2(SIM)×3(TIME) repeated measures ANOVA, there was a main effect of SIM (p=0.008), a main effect of TIME (p=0.000), and an interaction effect (p=0.000) for salivary cortisol. For salivary α-amylase (sAA), there was a main effect of SIM (p<0.05) and a marginal main effect of TIME (p=0.07) in a 2(SIM)×3(TIME) repeated measures of ANOVA.

    Figure 5
    Figure 5

    Salivary cortisol and α-amylase biomarker data are represented as deltas, which have been calculated as the per cent change from the “pre” time point (same scenario day 1 versus day 6). In 2(SIM)×3(TIME) repeated measures of ANOVA, for salivary cortisol, there was a main effect of SIM (p=0.008), a main effect of TIME (p=0.000), and an interaction effect (p=0.000). For sAA, there was a main effect of SIM (p<0.05) and a marginal effect of TIME (p=0.07). ANOVA, analysis of variance; sAA, salivary α-amylase.

    Discussion

    Stress is frequently defined as a state of unacceptable divergence between perceived demands and capabilities to adapt.5 Stressful simulation training can affect the performance of healthcare professionals in different settings.6 High-risk environments of aviation and nuclear power have embraced high fidelity simulation training for decades. With aviation simulators, the effect of stress inoculation training on anxiety and performance was reviewed in 1996 with a meta-analysis of nearly 2000 participants. Results indicated that stress inoculation training, a cognitive behavioral approach to stress management, was an effective means for reducing performance anxiety, reducing state anxiety, and enhancing performance under stress.7 More recently, the Defense Health Board, Office of the Assistant Secretary of Defense Health Affairs reported that “Simulation training is also important for improving team effectiveness, specifically with low-frequency, high acuity emergency situations as the success of these complex procedures depends upon the entire surgical team, not only the surgeon.”8 Our data with a 6-day hyper-realistic scenario to improve surgical trauma team performance provides additional information in support of high fidelity medical training. The training curriculum and environment were truly realistic and stress provoking as indicated by increases in salivary cortisol and changes in sAA obtained with multiple longitudinal samples from a large number of subjects (n=99).

    Cortisol steroid hormone is released in response to stress (fear/avoidant-related activation) and salivary cortisol concentrations have shown to be synchronous with serum cortisol levels.9 In the current study, a substantial increase in cortisol during the initial simulation was apparent, followed by a decline (figure 5). Specifically, there was a 41% increase in salivary cortisol from pre-post, followed by a decline at 20 minutes (19%) and 40 minutes near recovery (8%). In posthoc comparisons, a difference pre-post vs. pre-20 minutes (p<0.05) as well as pre-post vs. pre-40 minutes (p<0.001) was detected. By contrast, on the last day 6 simulation (same simulation as day 1), there was no statistically significant decline from baseline in cortisol levels. The results of this study demonstrate that the hyper-realistic immersive training environment elicited various levels of physiological stress to the participants. These findings also raise the possibility for habituation of the cortisol stress response with repeated exposure, implying possible stress adaptation, as a result of exposure to hyper-realistic simulation.

    A notable decrease in sAA occurred during initial stressful learning situations and continued through early recovery in simulation 1 (figure 5). Specifically, there was a 9% decline in sAA from pre-post, which continued in a downward trajectory at 20 minutes (−21%) and 40 minutes recovery (−20%). In posthoc comparisons, there was a difference pre-post vs. pre-20 minutes (p<0.05) and a marginal difference in pre-post vs. pre-40 minutes (p=0.06). By contrast, no differences in sAA were observed between time points during SIM 6 which was the same scenario as SIM 1. In comparing sAA responses between SIMs, SIM 1 versus SIM 6 were different at the pre-20 minutes and pre-40 minutes times (p<0.05).

    sAA is primarily reflective of the sympathetic stress response (approach-related activation). There is an association between changes in sAA during exposure to stressful stimulus and changes in the anxiety state. sAA is usually thought to increase quickly (ie, less than 5 minutes) after exposure to an acute stressful stimulus and return rapidly to baseline (ie, less than 15 minutes) during calming conditions. More specifically, sAA was shown to increase 220% with a corresponding increase of 17% systolic blood pressure in 15 patients undergoing planned C sections.10 Psychosocial stress has been shown to increase salivary α amylase independently from plasma norepinephrine levels.11 As the parotid gland is innervated by both sympathetic and parasympathetic systems, decreases in sAA have also been reported in response to stress.12 Multiple confounding factors may be influencing our data such as time of day (sAA levels highest in the morning) and smoking (decreases sAA levels) with caffeine, exercise and food increasing SAA levels. Salivary flow rate, age, and sex are reported to have little influence. As with salivary cortisol, the authors think that data imply potential habituation of the sAA stress response with repeated exposure, implying possible stress adaptation. Laporta and Hoang have shown somewhat similar results for salivary cortisol and α-amylase changes on medical student trainees undergoing hyper-realistic training.2–4

    Our stress hormone findings are similar to another high-fidelity simulation study of 14 emergency medicine physicians and 27 paramedics. Cortisol values showed a gradual and statistically significant reduction over time, whereas sAA showed significant increases with each scenario; however, reductions followed each increase, indicating no clear trend.13

    Along with habituation to stress, team performance was documented by specific metric improvements. Resuscitation time decreased by 10 minutes to an average of 13.91 minutes—approaching the mean evaluation time of 10.33 minutes for a highly experienced trauma team in a level 1 DoD facility.14 This is a significant clinical finding as rapid, effective trauma resuscitations decrease patient morbidity and mortality. Critical errors in the standardized scenarios also significantly decreased from 5 to 1, which has an immediate impact with improved injury pattern recognition, hemorrhage control, and airway/chest tube management. Due to the high-fidelity Human Worn Surgical Simulator “Cut Suit,” improved surgical procedure accuracy and intraoperative team problem solving was evaluated by subject matter expert observers. Immediate constructive critiques, coupled with video-feedback allowed teams to remediate and improve.

    As for written knowledge assessment with the same 25 question multiple-choice examination taken by all participants pre and post-course, the largest improvement in test scores were noted in US Navy Hospital Corpsman/ US Army “Medic” enlisted medical technicians (26%) and Registered Nurses (25%) compared with physicians (11%) (table 2). The US Army Forward Surgical Team, a more senior in age unit with individuals engaged in trauma care in their civilian roles, showed a nearly 16% overall improvement indicating that trauma experienced individuals will benefit from a hyper-realistic experience.

    The psychometric surveys were administered before the course, on day 3 and on conclusion on day 6 to assess changes in attitudes, resilience, cohesion, and stress-related psychological variables. The survey questions (online supplementary table 2) were based on published normative scales.15–21 Analyses conducted on the entire sample showed statistically significant increases in combat and unit readiness, unit and vertical cohesion, and team communication and leadership quality. Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) processes based on principles of team structure (communication, leadership, situation monitoring, and mutual support) were emphasized.22 Effects of training on perceived stress were not reported by the participants to vary significantly with the 6 days of training which contrasts to the cortisol and α-amylase stress biomarker data. The authors think that the ingrained military “macho” fearless mentality may be contributing to the denial of self-reporting honest stressful feelings. Although no established criteria exist for clinical significance on these measures, in the experience of the Naval Center for Combat and Operational Stress Control, increases of 5% on similar measures (eg, engagement, leadership satisfaction, cohesion) typically correspond to observable differences in group functioning. This project has generated information on hyper-realistic training simulations effects on important psychological constructs that are crucial to group performance.

    Strengths of this study include the relatively large sample size (n=99) for medical simulation research coupled with 6 days of multifaceted longitudinal data on individual subjects. Although the subjects were US Military members with formal rank structures, interactions follow a standard healthcare team hierarchy of physician, nurse, and medical technician. Challenges to the study involve lack of a control group that would evaluate 6 days of similar curriculum training not in a hyper-realistic environment. Certainly, any training experience can improve test scores, but this study focused on the complexities of human interactions in stressful life-threatening surgical trauma environments. Additional studies are required to evaluate the most time-efficient and cost-effective strategy. Lessons learned from previous and ongoing conflicts require realistic team-oriented training to manage complex trauma patients in both civilian and military mass casualty situations.23 24 In conjunction with the American College of Surgeons, Knowledge, Skills, and Abilities criteria have been developed at the Uniformed Services University of the Health Sciences to address the perishable surgical skills required in a combat zone and provide core metrics for trauma care providers.8 Integration and closer cooperation of military and civilian trauma systems will improve patient outcomes.25 More generalized applications of hyper-realistic immersive simulation training for healthcare providers may improve team communication, reduce iatrogenic medical errors, and increase patient safety.

    Conclusion

    Trauma care is team care. A S2T2 curriculum using a hyper-realistic and immersive simulation environment enhanced US Navy Fleet Surgical Team and US Army Forward Surgical Team performance. The data indicate that hyper-realistic immersive simulation training enhances multidisciplinary healthcare team members’ interactions and improves surgical procedures and processes. High fidelity surgical simulation equipment such as the Human Worn Surgical Simulator “Cut Suit” combined with highly realistic replicated settings will allow surgical trauma teams to improve their life-saving skills and teamwork communication to maximize successful patient outcomes. High fidelity, highly realistic, immersive and stress-provoking surgical trauma training is now an option to improve the readiness and capabilities of trauma teams, especially for mass casualty events in hospitals that are not busy level 1 trauma centers.

    Acknowledgments

    The authors would like to thank Steve Markham, Strategic Operations for technical support of the project.

    References

    1. ↵
      1. Strategic Operations Inc
      . Hyper-realistic tactical training….the next generation. Available at. http://www.strategic-operations.com (Accessed 17 Dec 2018).
    2. ↵
      1. LaPorta AJ,
      2. McKee J,
      3. Hoang T,
      4. Horst A,
      5. McBeth P,
      6. Gillman LM,
      7. Kirkpatrick AW
      . Stress inoculation: preparing outside the box in surgical resuscitation and education. Curr Trauma Rep2017;3:135–43.doi:10.1007/s40719-017-0090-2
      OpenUrl
    3. ↵
      1. Hoang TN,
      2. Kang J,
      3. Siriratsivawong K,
      4. LaPorta A,
      5. Heck A,
      6. Ferraro J,
      7. Robinson D,
      8. Walsh J
      . Hyper-realistic team centered fleet surgical team training provides sustained improvements in performance. J Surg Educ2016;73:668–74.doi:10.1016/j.jsurg.2016.03.004
      OpenUrl
    4. ↵
      1. Siriratsivawong K,
      2. Kang J,
      3. Riffenburgh R,
      4. Hoang TN
      . Immersion team training in a realistic environment improves team performance in trauma resuscitation. Surgery2016;160:586–90.doi:10.1016/j.surg.2016.03.029
      OpenUrl
    5. ↵
      1. Sanders AF
      . Towards a model of stress and human performance. Acta Psychologica1983;53:61–97.doi:10.1016/0001-6918(83)90016-1
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Quilici AP,
      2. Pogetti RS,
      3. Fontes B,
      4. Zantut LFC,
      5. Chaves ET,
      6. Birolini D
      . Is the advanced trauma life support simulation exam more stressful for the surgeon than emergency department trauma care?Clinics2005;60:287–92.doi:10.1590/S1807-59322005000400005
      OpenUrl
    7. ↵
      1. Saunders T,
      2. Driskell JE,
      3. Johnston JH,
      4. Salas E
      . The effect of stress inoculation training on anxiety and performance. J Occup Health Psychol1996;1:170–86.doi:10.1037/1076-8998.1.2.170
      OpenUrlCrossRefPubMed
    8. ↵
      1. Office of the Assistant Secretary of Defense Health Affairs
      . Defense Health Board, Subcommittee on Trauma and Injury. 2018. Available at. https://health.mil/reference-center/reports?page=10#paginganchor (Accessed 6 Mar 2019).
    9. ↵
      1. Dorn LD,
      2. Lucke JF,
      3. Loucks TL,
      4. Berga SL
      . Salivary cortisol reflects serum cortisol: analysis of circadian profiles. Ann Clin Biochem2007;44:281–4.doi:10.1258/000456307780480954
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. Guglielminotti J,
      2. Dehoux M,
      3. Mentré F,
      4. Bedairia E,
      5. Montravers P,
      6. Desmonts J-M,
      7. Longrois D
      . Assessment of salivary amylase as a stress biomarker in pregnant patients. Int J Obstet Anesth2012;21:35–9.doi:10.1016/j.ijoa.2011.09.008
      OpenUrlCrossRefPubMed
    11. ↵
      1. Petrakova L,
      2. Doering BK,
      3. Vits S,
      4. Engler H,
      5. Rief W,
      6. Schedlowski M,
      7. Grigoleit J-S
      . Psychosocial stress increases salivary alpha-amylase activity independently from plasma noradrenaline levels. PLoS One2015;10:e0134561.doi:10.1371/journal.pone.0134561
    12. ↵
      1. VanjaVucicevic B
      . Is salivary alpha amylase useful as a biomarker of stress in oral diseases?J Autocoids and Hormones2012;1:e106.
    13. ↵
      1. Valentin B,
      2. Grottke O,
      3. Skorning M,
      4. Bergrath S,
      5. Fischermann H,
      6. Rörtgen D,
      7. Mennig M-T,
      8. Fitzner C,
      9. Müller MP,
      10. Kirschbaum C, et al
      . Cortisol and alpha-amylase as stress response indicators during pre-hospital emergency medicine training with repetitive high-fidelity simulation and scenarios with standardized patients. Scand J Trauma Resusc Emerg Med2015;23:1–8.doi:10.1186/s13049-015-0110-6
      OpenUrl
    14. ↵
      1. Bonjour TJ,
      2. Charny G,
      3. Thaxton RE
      . Trauma resuscitation evaluation times and correlating human patient simulation training differences – What is the standard?Mil Med2016;181:e1630–6.doi:10.7205/MILMED-D-14-00486
      OpenUrl
    15. ↵
      1. Cohen S,
      2. Kamarck T,
      3. Mermelstein R
      . A global measure of perceived stress. J Health Soc Behav1983;24:385–96.doi:10.2307/2136404
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Vaitkus MA
      . Unit manning system: human dimensions field evaluation of the COHORT company replacement model. Medical Research and Material Command. 1994. Available at. www.Dtic.mil (Accessed 6 Mar 2019).
    17. ↵
      1. Vaitkus M,
      2. Griffith J
      . An evaluation of unit replacement on unit cohesion and individual morale in the U. S. army All-Volunteer force. Mil Psychol1990;2:221–39.doi:10.1207/s15327876mp0204_3
      OpenUrlCrossRef
    18. ↵
      1. Podsakoff PM,
      2. MacKenzie SB
      . An examination of the psychometric properties and nomological validity of some revised and reduced substitutes for leadership scales. J Appl Psychol1994;79:702–13.doi:10.1037/0021-9010.79.5.702
      OpenUrlCrossRef
    19. ↵
      1. Hackman JR,
      2. Oldham GR
      . The job diagnostic survey: an instrument of for the diagnosis of jobs and the evaluation of job redesign projects. Yale University, New Haven, Conn. Dept of Administrative Sciences. Office of Naval Research, Washington DC, Organizational effectiveness research program. 1974. Available at. https://www.dtic.mil>get-tr-doc>pdf (Accessed 17 Dec 2018).
    20. ↵
      1. De La Rosa GM,
      2. Webb-Murphy JA,
      3. Johnston SL
      . Development and validation of a brief measure of psychological resilience: an adaptation of the response to stressful experiences scale. Mil Med2016;181:202–8.doi:10.7205/MILMED-D-15-00037
      OpenUrl
    21. ↵
      1. Kroenke K,
      2. Spitzer RL,
      3. Williams JBW,
      4. Monahan PO,
      5. Löwe B
      . Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection. Ann Intern Med2007;146:317–25.doi:10.7326/0003-4819-146-5-200703060-00004
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Agency for Healthcare Research and Quality
      . TeamStepps team strategies and tools to enhance performance and patient safety. Available at. https://www.ahrq.gov/teamstepps/index.html (Accessed 6 Mar 2019).
    23. ↵
      1. Bradley M,
      2. Nealeigh M,
      3. Oh JS,
      4. Rothberg P,
      5. Elster EA,
      6. Rich NM
      . Combat casualty care and lessons learned from the past 100 years of war. Curr Probl Surg2017;54:315–51.doi:10.1067/j.cpsurg.2017.02.004
      OpenUrl
    24. ↵
      1. Schwab CW
      . Winds of war: enhancing civilian and military partnerships to assure readiness: white paper. J Am Coll Surg2015;221:235–54.doi:10.1016/j.jamcollsurg.2015.04.014
      OpenUrl
    25. ↵
      1. National Academy of Sciences, Engineering and Medicine
      . A national trauma care system: integrating military and civilian trauma systems to achieve zero preventable deaths after injury. 2016. Available at. https://www.nap.edu/catalog/23511/a-national-trauma-care-system-integrating-military-and-civilian-trauma (Accessed 6 Mar 2019).

    Footnotes

    • Contributors LG: data collection and critical revision. RC: data collection/data interpretation. GMDLR: study design/data analysis and interpretation. KL: study design/data analysis and interpretation, writing, and critical revision. TNH, AJL, JDM, MD: literature search, study design, data analysis and interpretation, writing, and critical revision.

    • Funding Data presented are the results of a Congressionally Directed Medical Research Program (CDMRP) Grant Award Research Protocol of the Medical Simulation and Information Sciences Research Program/Joint Program Committee-1 (Team Performance Training Research Initiative). This work is supported by Funded Award Number DM140571, under funding opportunity W81XWH-14-DMRDP-JPC1-TPT. The study protocol NMCSD.2015.0094 was approved by the Naval Medical Center San Diego Institutional Review board in compliance with all applicable Federal regulations governing the protection of human subjects. The data are presented in conjunction with NCRADA-NMCSD 18-490 engaging Strategic Operations, Inc.

    • Disclaimer The views expressed in this article reflect the results of research conducted by the author(s) and do not necessarily reflect the official policy or position of the Department of the Navy, Department of Defense, nor the US Government.

    • Competing interests KL is Executive Vice President, Strategic Operations, San Diego, California.

    • Patient consent for publication Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.