Validation of Central Line-associated Bloodstream Infection Intensive Care Unit Surveillance Data from VICNISS

Study Summary | Results | Interpretation | References

Study Summary

To measure inter-rater agreement, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of data submitted to VICNISS for identifying central line-associated bloodstream infection (CLABSI) in ICU patients, McBryde ES et al performed a retrospective review of hospital medical records comparing reported data with gold standard according to accepted definitions of CLABSI.¹

Six Victorian public hospitals participating in the VICNISS program and with >100 beds were studied. According to usual practice, surveillance outcomes were reported by infection control consultants at each site. Retrospective evaluation of the surveillance process was carried out by an independent VICNISS infection control consultant. Samples of patients reported to have a CLABSI were assessed to determine if they met the case-definition for CLABSI. This definition was based upon the US National Nosocomial Infection Surveillance System (NNIS) criteria. As negative controls, samples of patients reported to have a bacteraemia in the ICU during the assessment period who were not reported as having CLABSI were identified to determine if they met the definition of CLABSI.

Results

Three hundred and ninety eight bacteraemias were observed in patients with central venous catheters in the participating ICUs over the study period (1 January to 31 December 2006). Of these, 81 were reported as CLABSIs and 317 were not reported as CLABSIs.

A random sample of 108 medical records was reviewed retrospectively – 46 were reported as CLABSI and 62 were not reported as CLABSI (all during 2006). Agreement between surveillance reports and VICNISS assessment was 67.6% (kappa value 0.31). Using a gold standard of chart assessment by the VICNISS infection control consultant, 27/46 reported CLABSIs were confirmed as CLABSI, resulting in a PPV of 59%. Of the 62 bacteraemias not reported as CLABSIs by local infection control consultants, 45 were deemed not to be CLABSI by gold standard assessment (NPV 74%). The overall estimated sensitivity was 35% and specificity was 87%.

Discordant results were analysed further - of the 17 false negatives, 9 (53%) were found to be CLABSIs using NNIS criterion 2b, 1 (6%) using NNIS criterion 2a, and 7 (41%) using NNIS criterion 1. Removing the CLABSIs that met criterion 2b from the pool of false negatives led to a hypothetical sensitivity of 50%. Of the 19 false positive, 11 had another source of sepsis. Two did not fulfill clinical criteria for sepsis and two had less than two blood cultures with a possible skin contaminant with no antibiotics instituted. The remainder did not meet criteria on grounds of timing eg blood taken prior to ICU admission.

Interpretation

A large proportion of the false negative cases in this study (9/17, 53%) were CLABSIs defined according to NNIS criterion 2b – a common skin contaminant is cultured from at least one blood culture and the physician institutes appropriate antimicrobial therapy. Removing these cases from the analysis led to an improved sensitivity.

In 2008, the recommended CDC CLABSI case-definition was revised according to National Healthcare Safety Network (NHSN) recommendations. This revision involved removal of criterion 2b from the original NNIS case-definition. Therefore, McBryde et al report not only the validity of the (unrevised) NNIS case-definition, but also the hypothetical validity when using the revised CLABSI case-definition. While large discrepancies between gold standard and surveillance classifications were identified in the current study, the validity of case-definition for CLABSI in other Australian units is not known – it is possible that the current study results are reflective of a broader problem of under-reporting by other units. As with published reports,^2,3 ongoing education and training should now be considered to enhance the reliability of the case-definition for CLABSIs.⁴

In concert with CDC recommendations, VICNISS updated the case-definition for CLABSI in July 2008. Findings of the current study suggest that this is one important step towards improved validity (sensitivity) of the CLABSI module. It may be pertinent for a re-evaluation of validity to be performed for the participating Victorian centres following the revision of this case-definition.

References

1. McBryde ES, Brett J, Russo PL, Worth LJ, Bull AL, Richards MJ. Validation of central line associated bloodstream infection intensive care unit surveillance data from VICNISS. Infect Control Hosp Epidemiol 2009; 30:1045-9.
2. Posa PJ, Harrison D, Vollman KM. Elimination of central line-associated bloodstream infections: application of the evidence. AACN Adv Crit Care 2006; 17:446-54.
3. Rosenthal VD, Guzman S, Pezzotto SM, Crnich CJ. Effect of an infection control program using education and performance feedback on rates of intravascular device-associated bloodstream infections in intensive care units in Argentina. Am J Infect Control 2003; 31:405-9.
4. Emori TG, Edwards JR, Culver DH, Sartor C, Stroud LA, Gaunt EE, Horan TC, Gaynes RP. Accuracy of reporting nosocomial infections in intensive-care-unit patients to the National Nosocomial Infections Surveillance System: a pilot study. Infect Control Hosp Epidemiol 1998;19:308-16.