Healthcare-Associated Infections (HAIs) are considered adverse events. In addition to increasing hospitalization time, morbidity, and mortality in hospitals, HAIs also impose various economic impacts on healthcare systems.1 Ventilator-Associated Pneumonia (VAP) is a serious complication of mechanical ventilation, affecting 8 % to 28 % of ventilated patients globally.2-5 The definition of infection diagnostic criteria for the epidemiological surveillance of HAIs in healthcare services allows for the necessary standardization to identify cases by healthcare professionals from different institutions systematically. However, concerning VAP, the absence of a gold standard in diagnosis complicates the proper evaluation of different case definitions. The sensitivity and specificity variables of available clinical criteria make diagnostic evaluation complex.6,7

The CDC (Centers for Disease Control and Prevention) criteria were revised in 2013, adopting concepts of Ventilator-Associated Events (VAEs): Ventilator-Associated Condition (VAC); Infection-Related Ventilator-Associated Complication (IVAC); Possible Ventilator-Associated Pneumonia (PVAP).8 In Brazil, as in other countries, the regulatory surveillance agency (ANVISA) opted not to follow the changes proposed by the CDC, considering the need to evaluate whether these changes would be feasible for Brazilian hospitals, maintaining the traditional criteria. However, given the high incidence and mortality rate of VAP, there is an urgent need for reliable surveillance methods to accurately identify and track cases, as well as for the global standardization of diagnostic criteria.

In this study, we evaluated the incidence of VAP using both VAE and VAP criteria, comparing the agreement between both criteria.

We followed the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement for reporting this study.9

Data collection and source or information

Data collection spanned 6 months at each site. Given the study's accrual duration (> 1 year) and the coverage across different Brazilian regions with varying initiation dates, seasonal sampling was deemed unnecessary.

The research team from the coordinating center trained all professionals from the HICCs of each participating site to collect data. Training content comprises contextualization of the theme, justification of the project, objectives, methodology (eligibility criteria, design, data collector profile, data to be collected), presentation of the data collection system, step-by-step demonstration of data collection, data analysis, support and materials for participating centers, team documents. Only professionals who completed the training were authorized to collect data, guaranteeing consistency and reliability. All training sessions were recorded in minutes. The participating centers and the training log are available in the Supplementary Material. The research team from each center monitored patients throughout their mechanical ventilation period in the participating ICU. We captured demographic and clinical profile data through the Epimed Monitor System database, an administrative database utilized for quality improvement in Brazil upon the patients' admission to the study ICU.13

The trained local researchers entered mechanical ventilation data daily into the institutional database system (“VAP System”) from the initiation of mechanical ventilation until the day following its cessation. VAPs reported to ANVISA, according to its criteria, were recorded in the database system for adjudication which was carried out independently by two professionals (a physician and a nurse) with experience in HAIs surveillance. They underwent thorough training via videoconferencing and received a manual with instructions. In case of disagreement between the first and the second adjudicators, a third (physician) was consulted. We applied the CDC criteria14 to define VAPs, while a specific algorithm automatically provided the VAE definition in the database system. A nurse experienced in HAI surveillance manually validated the algorithm's diagnosis (Fig. 1; Supplementary Table 1).

Fig. 1.

IMPACTO MR-PAV study flowchart. ICU, Intensive Care Unit; VAC, Ventilator-Associated Condition; IVAC, Infection-Related Ventilator-Associated Complication; PVAP, Possible Ventilator-Associated Pneumonia; VAE, Ventilator-Associated Events; VAP, Ventilator-Associated Pneumonia.

(0.12MB).

The demographic profile of population was described, including age, sex, admission diagnosis, pre-admission origin, Simplified Acute Physiology Score-3 (SAPS3) and Charlson index.15,16

In the beginning, 14 centers achieved the proper feasibility to participate in the study and were invited to the site initiation visit. However, two of then dropped out before the study starts. The exclusion reasons for both centers were related to contractual and technical aspects (Difficulties in the agreement signing, change of PI, lack of response). Therefore, the first 12 hospitals that met the inclusion criteria and expressed interest in the study were selected. The list of participating centers and the reasons for exclusion are available in the Supplementary Material. Table 1 and Supplementary Figure 1 characterizes the sample of 987 patients (1115 ventilation episodes) included in the study. Until the end of the study, 57 ventilation episodes were excluded from the main analysis: 15 could not have data quality guaranteed by the participating site even despite various training and guidelines, and 42, were incorrect registries in the VAP-System, after the deadline, in duplicate or with essential missing data.

The VAP System identified 66 VAEs according to CDC criteria. The incidence density was of 6.7 VAEs per 1000 ventilator days. The identified events are as follows 41 VACs (4.1 VACs per 1000 ventilator days); 19 IVACs (1.9 IVACs per 1000 ventilator days); 6 PVAPs (0.6 PVAPs per 1000 ventilators day). The sites reported 84 VAPs according to ANVISA criteria. The incidence density was of 8.5 VAPs per 1000 ventilator days.

The agreement between adjudicators and VAPs reported by the participating sites according to ANVISA criteria was evaluated. Therefore, 85.7 % of VAPs reported by the centers according to ANVISA criteria were not confirmed by the adjudicators (Table 2) (Supplementary Fig. 2). Furthermore, the reasons for non-confirmation by the adjudicators were: criteria outside the infection window period (5 reported VAPs); absence of mechanical ventilation criteria for definition (9 reported VAPs); absence of radiological worsening (23 reported VAPs); and absence of clinical/laboratory criteria (14 reported VAPs) (Supplementary Fig. 2). A kappa of 0.32 (95 % CI 0.0; 0.6) was found among the adjudicators (Table 2).

Finally, an analysis of agreement between ANVISA (VAP) and CDC (VAE) criteria was performed. There was disagreement in 123 cases. For the remaining 992 cases, the criteria agreed that there was no event, resulting in a discordance rate of 11 %. Manual adjudication (by a nurse with expertise in infection control) was also performed for the VAEs automatically generated by the VAP System. From these data, it was possible to calculate the agreement between manual and automated reporting of VAEs (CDC criteria). A kappa of 0.69 (95 % CI 0.6; 0.8) was found. Prevalence-adjusted Kappa showed higher values for agreement than the not adjusted Kappa, independently of the comparison (Table 2).

Outcomes of mechanically ventilated patients according to the occurrence of events, including the duration of ventilation, ICU stay, duration of antibiotic use, and mortality, are detailed in Table 3. Of the 1115 records analyzed, the overall mean ventilation duration was 8.9 ± 9.9 days, mean ICU stay was 17.3 ± 20.3 days, and mean antibiotic use was 7.0 ± 6.7 days. The death rate during intubation was 37.0 %. For patients with no events recorded according to the CDC criteria, the mean ventilation duration was 8.3 ± 9.7 days, ICU stay was 17.1 ± 20.5 days, and antibiotic use was 6.5 ± 6.3 days, with a death rate of 35.4 %. Similar trends were observed for patients with no events recorded according to ANVISA criteria.

Patients with events recorded based on the intersection of both criteria had significantly higher mean ventilation duration (23.2 ± 11.0 days), ICU stay (28.0 ± 14.7 days), and antibiotic use (20.4 ± 10 days), with a death rate of 46.2 %. This indicates that patients with overlapping criteria for events experienced more severe outcomes. Among the ANVISA event records, patients had a mean ventilation duration of 17.7 ± 12.1 days, ICU stay of 29.5 ± 25.4 days, and antibiotic use of 13.2 ± 7.9 days, with a death rate of 42.2 %. Similarly, CDC event records showed a mean ventilation duration of 17.5 ± 10.2 days, ICU stay of 22.0 ± 16.4 days, and antibiotic use of 14.9 ± 8.9 days, with a higher death rate of 63.6 %.

Specific events categorized under CDC criteria, such as VAC, IVAC, and PVAP, also showed worse outcomes. Patients with VAC had a mean ventilation duration of 16.1 ± 9.7 days, ICU stay of 20.5 ± 17.0 days, and antibiotic use of 13.7 ± 7.7 days, with a death rate of 68.3 %. IVAC patients had higher ventilation duration (19.2 ± 12.1 days), ICU stay (22.4 ± 14.4 days), and antibiotic use (16.5 ± 11.9 days), with a death rate of 63.1 %. PVAP patients experienced the highest mean ventilation duration (21.0 ± 4.0 days), ICU stay (32.0 ± 18.3 days), and antibiotic use (17.3 ± 4.1 days), but a lower death rate of 33.3 % (Table 3).

It was observed that the incidence of VAP increased from 8 % (95 % CI: 6 %‒11 %) at 10 days to 17 % (95 % CI: 13 %‒23 %) at 30 days and 20 % (95 % CI: 14 %‒28 %) at 60 days (Supplementary Fig. 3). The main clinical outcomes and their frequency are reported on Table 3.

The Sankey diagram (Fig. 2) visually represents the flow of mechanically ventilated patients through different stages of Ventilator-Associated Events (VAEs) and Ventilator-Associated Pneumonia (VAP), as assessed by both the ANVISA and CDC criteria. This diagram highlights the progression and overlap between various event categories such as VAC, IVAC and PVAP. The varying thickness of the lines reflects the proportion of patients moving from one event status to another. Notably, the diagram reveals significant transitions from “No Event” to other VAE categories.

Fig. 2.

Sankey diagram comparing ANVISA criteria, automated CDC criteria and adjudication.

(0.18MB).

The findings of this study reveal significant challenges in identifying and classifying VAPs. The discrepancy between the ANVISA criteria and the CDC's VAEs criteria, as well as between the adjudicators themselves, highlights the subjectivity and complexity of the current criteria. The analysis demonstrated that the agreement rate was substantially higher for automatically generated VAEs, suggesting greater objectivity in the CDC criteria.

The incidence density was 6.7 VAEs per 1000 ventilator days in this study. Regarding VAP according to ANVISA criteria, the incidence density was 8.5 VAPs per 1000 ventilator days. Several previous studies have reported the incidence of VAEs. However, the data are very heterogeneous.21-25 The incidence rates reported to the CDC for the first full year of VAE surveillance (2014) ranged from 2.59 to 11.79 per 1000 ventilator days, with higher rates found in university hospitals.26 Lower rates were reported in a study of 7 urban hospitals in Japan (6.4 VAEs per 1000 ventilator days),27 and higher rates were observed in a multinational cohort in Europe (40.8 VAEs per 1000 ventilator days).28 A study with more than 6000 ventilated patients in five medical and surgical specialty ICUs at an academic medical center in China reported VAC, IVAC, and PVAP rates of 13.7, 6.3, and 2.2 per 1000 ventilator days, respectively.29

Regarding traditional VAP criteria (ANVISA), the incidence density was 8.5 VAPs per 1000 ventilator days. Data published by the Epidemiological Surveillance Center of the State of São Paulo indicated 7.08 VAPs per 1000 ventilator days (p50) in adult ICUs of Public Hospitals in 2022.30 The 2022 national report from ANVISA indicates 9 VAPs per 1000 ventilator days (p50) in adult ICUs in Brazil.31. Thus, the data from the study in question is quite consistent with recent reports.

When we analyze the agreement between the two criteria, the result of 11 % discordance is justified because they are fundamentally different criteria – although both are used for VAP. VAE was intentionally designed to be different from the traditional VAP criterion. The VAE requirement for a sustained increase in ventilator settings creates a threshold effect that selects patients with severe illness.5

The new VAE surveillance detected only one-third of conventional VAP cases (among the 165 reported) in a previous study.32 A meta-analysis and systematic review evaluated 18 studies with 61,489 patients receiving mechanical ventilation in ICUs across eight countries and identified that the combined sensitivity and positive predictive value of each type of VAE for detecting VAP did not exceed 50 %, while the combined specificity and negative predictive value exceeded 80 %. It was concluded that VAE surveillance performs well in the absence of a case but does not accurately detect traditional VAP cases in ICUs.33

The high rates of disagreement found in the ANVISA criteria are supported by studies indicating significant interobserver variability in VAP surveillance, reflecting the challenges in consistently applying subjective criteria.7,30 One study sought to evaluate interobserver variability in VAP surveillance. Three infection control professionals and one physician independently evaluated 50 patients with respiratory deterioration through retrospective chart reviews. The three reviewers agreed on only 7 VAPs (kappa = 0.40). It was concluded that interobserver variability in assessing ventilator-associated pneumonia is high.34 In another study, investigators distributed 6 case vignettes to 43 infection control specialists in the United States. An almost equal number of respondents considered that 1, 2, 3, 4, or 5 of the 6 patients met the VAP criteria.35.

Other reasons can also explain this variation. Firstly, we must consider the limitation of the method used for adjudication. The data were entered into the VAP System by the notifier. The adjudicator did not have access to the full patient's medical records, unlike those who reported the events. And, even for reported events, the adjudicators only had access to patient’s medical records the site selected to attach to the VAP System. Another challenge was about sites where the medical records were physical. This made it difficult to access some information on time. This could be considered a potential information bias and a restriction in the comprehensive evaluation of each case. Third, another possible reason for the discrepancies is the heterogeneity of Hospital Infection Control Committees (HICCs) in Brazil evidenced through a National Assessment of Health Service Infection Prevention and Control Programs conducted by ANVISA in 2023. In the evaluation of 1977 services, although 91.40 % reported having doctors and nurses on the HICC team, it was also reported that 23.58 % of these professionals did not have specific training in infection prevention and control.36 During the conduction of our study, it was possible to observe the discrepancy between different public and private participating sites regarding location, team size, professional expertise, accessibility and quality of medical record keeping (physical or electronic), and scientific maturity. This disparity may have directly interfered with data collection, making it difficult at times. Additionally, despite the sample size adjustment, due to the low event rate, the subjectivity observed in interrater agreement, and the presence of imperfect standards, we decided not to conduct a formal diagnostic accuracy evaluation. Therefore, we aborted the Bayesian analysis and focused in a more descriptive analysis.

Lastly, the agreement between the system and nurse (CDC) was substantial, with a Kappa value of 0.69 (95 % CI: 0.6‒0.8) and a low discordance rate of 4.0 %. This indicates a high level of consistency between the system-generated assessments and those conducted by nurse, suggesting reliability in the CDC method. This reflects the objectivity of the criteria, allowing for standardization of surveillance, which is not dependent on the examiner. We might observe an increased Kappa Index after prevalence adjustment. Nevertheless, prevalence-adjusted Kappa is not free of bias. Simulation studies indicates that it would result in substantial overestimation of reliability.37

The VAE algorithm's diagnostic process is automated, and manual adjudication was performed by a single nurse as a secondary verification step. Since the ANVISA criteria are examiner-dependent and require manual evaluation, the adjudication process for this method was more robust. In contrast, the VAE criteria are originally designed for automation, which is why we developed a platform to standardize and automate the process. The manual review by the nurse served only as a double-check to ensure data accuracy. Furthermore, two validation steps were conducted with VAE algorithm before its implementation: The first involved an artificial simulation of events (VACs, IVACs, and PVAPs), while the second utilized formal patient data collected at the coordinating site. These tests confirmed the correct identification of all events without ambiguous classifications. Given these differences, we acknowledge that variations in the adjudication process may not impact the comparison of agreement between the two methodologies. We believe that applying a manual adjudication process similar to ANVISA for the VAE algorithm would not significantly alter the results, as the core of the VAE methodology relies on automated detection rather than subjective interpretation.

Finally, when we evaluate the outcomes of the events, for records with no event identified by CDC or ANVISA criteria, the mean duration of ventilation, ICU stay, and duration of antibiotic use were similar. However, for records identified as events by ANVISA, CDC, or by the intersection of the criteria, a higher mean duration of ventilation, ICU stay, and duration of antibiotic use were observed, along with a significantly higher mortality rate. These results highlight the association between the occurrence of adverse events and unfavorable outcomes in mechanically ventilated patients and are supported by previous studies in the literature.25,29,32,38

In conclusion, the results highlight the challenges in identifying and classifying VAPs and VAEs, as well as their association with potentially unfavorable outcomes in mechanically ventilated patients. Regardless of the criteria adopted for identifying these events and despite their discrepancies, these are potentially preventable scenarios that worsen patient outcomes. Therefore, the necessity of preventive measures and effective management protocols is emphasized to reduce their incidence and consequently improve clinical outcomes. Despite this research tried to avoid lack of representativeness by addressing ICUs from all the five macroregions in Brazil, some clinical and administrative factors could have impaired its full generalization. Limitations regarding hospital sampling could comprise: lack of Hospital Infection Control Committee specialized personnel, variability in healthcare quality standards, variability in ICU diaries completeness, heterogeneity in clinical research knowledge, Anvisa VAP reporting capacity, different nosocomial profile, ICU size and complexity. Our study provides valuable insights into the challenges and potential solutions for VAP surveillance in Brazil. Based on the above data, we can infer the need to revisit the identification and impact of VAPs and VAEs in patients admitted to public and private hospitals in Brazil.

The local Institutional Review Board (IRB) approved this study (CAAE: 52354721.0.1001.0070). All national ethical requirements were met to support this research. The IRB waived the need for informed consent for retrospective data collection (ICU medical records). This study was registered in clinicaltrials.gov (NCT05589727). The study was first registered on 21 October 2022.

Anonymized raw data can be provided after reasonable request to the authors.

HAOJ, AA and GMN conceived and designed the study, GMN, DLGR, MYC and KCCB drafted the manuscript, which was edited and revised by all authors. GMN wrote the first draft of the report with input from DLGR and HAOJ. MYC, and KCCB made a substantial contribution to the conception and the design of the manuscript. LBOA was responsible for the statistical analysis. DLGR, GMN, HAOJ, have directly accessed and verified the underlying data reported in the manuscript. All authors had full access to all the data in the study, read and approved the final version of the manuscript to submit for publication. HAOJ was responsible for the decision in submitting this manuscript.

The study received funding from the Unified Health System ‒ Institutional Development Support Program (PROADI-SUS) of the Ministry of Health of Brazil. The sponsor had no role in study design, data collection, data analysis, data interpretation, or writing of the report.