The publication of a study on the effectiveness of COVID-19 vaccines has garnered attention after facing initial rejection from a government health journal. The study, now published by JAMA Network Open, found vaccines to be approximately 55% effective in preventing hospitalizations linked to COVID-19. Additionally, it showed a 50% reduction in visits to emergency departments and urgent care clinics due to COVID-19.
Although these findings align with previous research confirming the efficacy of COVID-19 vaccines, the paper became a point of public interest after political appointees from the Trump administration blocked its publication in a Centers for Disease Control and Prevention (CDC) journal. They argued that the study’s design was susceptible to incorrect assumptions that could lead to flawed conclusions. However, many public health experts argue that the design is dependable and has been used effectively for decades to assess vaccine effectiveness.
In an accompanying commentary published by the study, Natalie Dean, a biostatistics expert from Emory University, emphasized the importance of continuing to publish vaccine effectiveness estimates amid evolving immunity and viral strains.
“It is critical that we continue to characterize and publish estimates of vaccine effectiveness in populations with changing immunity against evolving viral strains.” — Natalie Dean
The study was originally set to appear in the CDC’s Morbidity and Mortality Weekly Report, having received approval from the agency’s Office of Science. However, acting CDC Director Jay Bhattacharya flagged concerns, prompting authors to address these issues. According to Althea Grant-Lenzy, the CDC’s chief science officer, the study’s publication was not completely halted, and researchers were allowed to approach external journals for dissemination.
The test-negative design used in the study focuses on patients in hospitals or emergency rooms with respiratory illnesses. Researchers evaluate the vaccination status of patients and calculate the probability of a positive COVID-19 test among those vaccinated versus unvaccinated. Despite critiques of the methodology, the design has been validated through publication in respected journals such as Pediatrics and the New England Journal of Medicine.
Critics, including Bhattacharya, argue that reliance on assumptions might skew results due to factors like prior infection and varied patient behavior. Study proponents counter this criticism, noting that prior infection is less of an issue given widespread COVID-19 exposure in the U.S. They argue no study design is without flaws, but an alternative for real-time vaccine effectiveness assessment is lacking from the U.S. Department of Health and Human Services.
The CDC convened a forum earlier this month to debate the pros and cons of the study’s methodology. A panel, mainly supporting the study, included Dean and two others addressing its strengths. Martin Kulldorff, a critic and biostatistician, also participated. Alongside Bhattacharya, he co-authored the Great Barrington Declaration, criticizing pandemic closures. Kulldorff argued against including varied diseases within studies with this design and questioned the absence of long-term studies to assess vaccine efficacy.
“We were in a pandemic! That’s why!” — Audience member
Kulldorff last year headed a federal vaccine advisory committee after appointment by U.S. Health Secretary Robert F. Kennedy Jr., later resigning to become chief science officer at the HHS planning and evaluation office.
Support for the Associated Press Health and Science Department comes from the Howard Hughes Medical Institute’s Department of Science Education and the Robert Wood Johnson Foundation, although AP is accountable for the entire content.