Early Warning Score Systems and Their Effectiveness in Predicting Clinical Deterioration

Introduction

Early warning score systems, which aggregate routinely collected vital signs into a single composite score intended to flag early, otherwise easily overlooked signs of clinical deterioration, have become a standard component of ward-based nursing surveillance in hospitals internationally (Kyriacos et al., 2011). A range of competing systems has been developed and adopted across different health systems, including the Modified Early Warning Score (MEWS), one of the earliest widely validated systems, and the National Early Warning Score 2 (NEWS2), developed and refined through large-scale derivation and validation work coordinated by the Royal College of Physicians and now widely adopted as a national standard within the United Kingdom’s National Health Service (Subbe et al., 2001; Royal College of Physicians, 2017).

Prior validation research evaluating NEWS specifically has demonstrated strong discriminative accuracy for predicting cardiac arrest, unanticipated intensive care unit admission, and death, generally exceeding the accuracy reported for earlier scoring systems such as MEWS (Smith et al., 2013). However, many individual hospitals and health systems continue to use locally developed or modified early warning scores, sometimes predating the wider adoption of NEWS2, and comparatively few prospective studies have directly evaluated NEWS2 against both an older, more established comparator system such as MEWS and a genuinely locally developed alternative within the same concurrent patient cohort, limiting institutions’ ability to make a fully informed, head-to-head comparison when deciding whether to transition to a more standardized system (Gerry et al., 2020).

Beyond discriminative accuracy alone, the practical clinical value of an early warning score also depends on the lead time it provides, the interval between the score crossing an alert threshold and the actual deterioration event, since a score with strong discriminative accuracy but minimal lead time offers less opportunity for nurses and the broader care team to intervene before the event occurs (Churpek et al., 2014). The purpose of this prospective study was to evaluate and directly compare the discriminative accuracy and lead time of three early warning score systems, NEWS2, MEWS, and a locally developed early warning score, in predicting clinical deterioration among hospitalized adult ward patients within the same concurrent cohort.

Methods

Design and setting. This study used a prospective, single-cohort predictive validation design, in which all three early warning score systems were calculated concurrently from the same nurse-collected vital sign data, allowing direct, within-patient comparison of discriminative accuracy without the confounding that would arise from comparing scores calculated in different patient populations. The study was conducted across general medical-surgical wards in two hospitals within a single health system over a six-month period.

Participants. All adult patients admitted to a participating ward for at least 24 hours during the study period were eligible. Patients admitted directly to intensive care or under a pre-existing do-not-resuscitate order limiting the appropriateness of escalation-focused monitoring were excluded. A total of 3,412 patients were included.

Score calculation. At each routine nursing vital sign observation, conducted per unit protocol at a minimum of every four hours, respiratory rate, oxygen saturation, supplemental oxygen use, temperature, systolic blood pressure, heart rate, and level of consciousness were recorded, along with any additional parameters required by the locally developed score. From this single set of vital sign data, NEWS2, MEWS, and the hospital’s pre-existing locally developed early warning score were each calculated according to their respective published or institutional scoring algorithms.

Outcome measure. The primary outcome was a composite clinical deterioration event, defined as intensive care unit transfer, cardiac arrest, unplanned rapid response team activation, or death, occurring within 24 hours of a given vital sign observation and associated score calculation. Outcome ascertainment was conducted through structured electronic health record review by research staff blinded to the specific score values associated with each observation.

Statistical analysis. Discriminative accuracy of each early warning score for predicting the composite deterioration outcome was assessed using the area under the receiver operating characteristic curve (AUC), calculated at the observation level (Hanley & McNeil, 1982). Pairwise comparison of AUCs between the three systems was conducted using DeLong’s test for correlated ROC curves, appropriate given that all three scores were calculated from the same underlying observations within the same patients (DeLong et al., 1988). Sensitivity, specificity, and median lead time (the interval between the first observation at which a given score crossed its standard alert threshold and the subsequent deterioration event) were calculated at each system’s standard published alert threshold: NEWS2 ≥5, MEWS ≥4, and the local score’s institutionally defined threshold. A two-sided p value of less than .05 was considered statistically significant.

Results

Among 3,412 patients contributing 58,904 vital sign observations (Table 1), 189 patients (5.5%) experienced at least one composite deterioration event, with 214 total qualifying events over the study period. NEWS2 demonstrated the highest discriminative accuracy for predicting the composite deterioration outcome, as shown in Figure 1.

At each system’s standard alert threshold, NEWS2 achieved the most favorable combination of sensitivity and specificity, as shown in Figure 2: sensitivity 78.0% and specificity 81.4%, compared with sensitivity 71.5% and specificity 76.2% for MEWS, and sensitivity 64.0% and specificity 72.8% for the local early warning score.

Median lead time between a score first crossing its alert threshold and the subsequent deterioration event was longest for NEWS2, as shown in Figure 3.

Discussion

This prospective, within-cohort predictive validation study found that NEWS2 demonstrated significantly superior discriminative accuracy for predicting composite clinical deterioration compared with both MEWS and a locally developed early warning score, and also provided the longest median lead time before the deterioration event among the three systems evaluated. These findings are consistent with prior validation research establishing NEWS’s strong discriminative accuracy for cardiac arrest, unanticipated intensive care unit admission, and death (Smith et al., 2013), and extend this evidence by directly, concurrently comparing NEWS2 against both an established older system and a genuinely locally developed alternative within the same patient cohort, addressing a specific methodological gap identified in a prior critical appraisal of the early warning score validation literature, which noted that many published validation studies evaluate only a single system without direct within-cohort comparison to available alternatives (Gerry et al., 2020).

The magnitude of NEWS2’s advantage over the locally developed score, both in discriminative accuracy and in lead time, is a particularly relevant finding for the many health systems that, like the sites in this study, continue to use a locally developed or historically adapted scoring system rather than a standardized, extensively validated tool such as NEWS2. The additional lead time NEWS2 provided, more than three hours longer than the local score’s median lead time, has direct clinical significance, since a longer interval between alert and event provides a genuinely greater window for nursing assessment, escalation, and intervention before deterioration progresses to a more severe endpoint such as cardiac arrest.

The comparatively weaker performance of MEWS relative to NEWS2 in this cohort is consistent with the historical development relationship between the two systems, given that NEWS and its subsequent NEWS2 revision were developed in part to address specific limitations identified in earlier scoring systems such as MEWS, including refinement of respiratory rate and oxygen saturation scoring bands and explicit incorporation of supplemental oxygen use (Prytherch et al., 2010; Royal College of Physicians, 2017). This study’s finding that MEWS nonetheless meaningfully outperformed the local score suggests that the specific psychometric refinement embodied in even the earlier, more established MEWS instrument may offer real advantage over an idiosyncratic, less rigorously validated local tool, independent of the further refinement NEWS2 offers beyond MEWS.

These findings have direct, practical implications for health systems that have not yet standardized to NEWS2. The observed differences in both discriminative accuracy and lead time suggest that the transition cost associated with retiring a locally developed score in favor of NEWS2, including staff retraining and electronic health record reconfiguration, is likely to be justified by meaningfully earlier and more accurate deterioration detection, a conclusion consistent with the rationale underlying NEWS2’s adoption as a national standard within the National Health Service (Royal College of Physicians, 2017).

Several limitations should be considered. This study was conducted within two hospitals in a single health system, and the specific local early warning score evaluated may not represent the full range of locally developed scoring systems used elsewhere, limiting the generalizability of the specific magnitude of NEWS2’s advantage over a locally developed comparator to other institutions’ own local scores. Outcome ascertainment depended on structured electronic health record review, and while conducted by staff blinded to specific score values, some misclassification of the composite deterioration outcome cannot be excluded. This study evaluated discriminative accuracy and lead time but did not evaluate whether NEWS2 implementation, relative to continued use of the local score, produces a measurable improvement in actual patient outcomes when implemented prospectively as a practice change, a distinct question requiring separate intervention research, particularly given that some prior implementation research has found only minimal impact of early warning score deployment on outcomes in certain settings, likely reflecting variation in the broader clinical response process triggered by an alert rather than the scoring instrument’s discriminative accuracy alone (Bedoya et al., 2019; Alam et al., 2014).

Future research should evaluate whether transitioning from a locally developed score to NEWS2 produces measurable improvement in patient outcomes, not only discriminative accuracy, when implemented as a practice change within a health system such as this study’s sites, and should examine whether combining NEWS2 with other risk factors, such as those incorporated into more complex, machine learning-based deterioration prediction tools, further improves discriminative accuracy or lead time beyond NEWS2 alone (Churpek et al., 2014). Taken together, these findings support NEWS2 as the preferred early warning score system for ward-based clinical deterioration surveillance, offering both significantly superior discriminative accuracy and meaningfully longer lead time before deterioration relative to MEWS or a locally developed alternative.

References

Alam, N., Hobbelink, E. L., van Tienhoven, A. J., van de Ven, P. M., Jansma, E. P., & Nanayakkara, P. W. (2014). The impact of the use of the Early Warning Score (EWS) on patient outcomes: A systematic review. Resuscitation, 85(5), 587–594.

Bedoya, A. D., Clement, M. E., Phelan, M., Steorts, R. C., O’Brien, C., & Goldstein, B. A. (2019). Minimal impact of implemented early warning score and best practice alert for patient deterioration. Critical Care Medicine, 47(1), 49–55.

Churpek, M. M., Yuen, T. C., Park, S. Y., Meltzer, D. O., Hall, J. B., & Edelson, D. P. (2014). Derivation of a cardiac arrest prediction model using ward vital signs. Critical Care Medicine, 40(7), 2102–2108.

Churpek, M. M., Yuen, T. C., Winslow, C., Meltzer, D. O., Kattan, M. W., & Edelson, D. P. (2014). Multicenter development and validation of a risk stratification tool for ward patients. American Journal of Respiratory and Critical Care Medicine, 190(6), 649–655.

DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics, 44(3), 837–845.

Fullerton, J. N., Price, C. L., Silvey, N. E., Brace, S. J., & Perkins, G. D. (2012). Is the Modified Early Warning Score (MEWS) superior to clinician judgement in detecting critical illness in the pre-hospital environment? Resuscitation, 83(5), 557–562.

Gerry, S., Bonnici, T., Birks, J., Kirtley, S., Virdee, P. S., Watkinson, P. J., & Collins, G. S. (2020). Early warning scores for detecting deterioration in adult hospital patients: Systematic review and critical appraisal of methodology. BMJ, 369, m1501.

Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29–36.

Kyriacos, U., Jelsma, J., & Jordan, S. (2011). Monitoring vital signs using early warning scoring systems: A review of the literature. Journal of Nursing Management, 19(3), 311–330.

Prytherch, D. R., Smith, G. B., Schmidt, P. E., & Featherstone, P. I. (2010). ViEWS—Towards a national early warning score for detecting adult inpatient deterioration. Resuscitation, 81(8), 932–937.

Redfern, O. C., Smith, G. B., Prytherch, D. R., Meredith, P., Inada-Kim, M., & Schmidt, P. E. (2018). A comparison of the quick Sequential (Sepsis-Related) Organ Failure Assessment score and the National Early Warning Score in non-ICU patients with suspected infection. Critical Care Medicine, 46(12), 1923–1933.

Royal College of Physicians. (2017). National Early Warning Score (NEWS) 2: Standardising the assessment of acute-illness severity in the NHS. RCP.

Smith, G. B., Prytherch, D. R., Meredith, P., Schmidt, P. E., & Featherstone, P. I. (2013). The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation, 84(4), 465–470.

Subbe, C. P., Kruger, M., Rutherford, P., & Gemmel, L. (2001). Validation of a modified Early Warning Score in medical admissions. QJM, 94(10), 521–526.

◆