Introduction

The rapid development and deployment of COVID-19 mRNA vaccines have been instrumental in reducing the global burden of the COVID-19 pandemic. These vaccines have demonstrated substantial efficacy in reducing severe disease, hospitalizations, and mortality1,2. On September 11, 2023, XBB.1.5-containing COVID-19 mRNA vaccines were authorized for emergency use by the U.S. Food and Drug Administration (FDA)3. These mRNA vaccines from Moderna and Pfizer–BioNTech were updated for the 2023-2024 season to include the monovalent XBB.1.5 component, offering better protection against the SARS-CoV-2 Omicron XBB.1.5 subvariant4. Individuals aged six months and older are eligible to receive a single dose of an XBB.1.5-containing COVID-19 mRNA vaccine, as the enhanced transmissibility and immune evasion of Omicron XBB.1.5 subvariant have reduced the neutralizing antibody responses elicited by previous COVID-19 mRNA vaccines5,6,7. From September 2023 to March 2024, over 30 million doses of XBB.1.5-containing COVID-19 mRNA vaccines were administered in the USA8.

Extensive studies have been conducted on earlier vaccine formulations. Pharmacovigilance platforms, such as the Vaccine Adverse Event Reporting System (VAERS)9 in the USA and EudraVigilance10 in Europe, are examples of passive surveillance methods. These systems rely on self-reports from patients, which may introduce reporting bias and result in incomplete data representation across the vaccinated population11. In contrast, utilizing electronic health record (EHR) data for investigating vaccine-related adverse events has proven to be an effective active surveillance method12. The Vaccine Safety Datalink (VSD) project has been monitoring COVID-19 vaccine safety in large populations using member sites data13,14. Overall, the COVID-19 related vaccines showed good safety in previous studies2,15,16,17,18,19,20.

However, continuous safety monitoring remains essential to support public health efforts, particularly some rare but serious types of adverse events21. The National COVID Cohort Collaborative (N3C)22 is one of the largest centralized and harmonized collections of EHR data in the United States. Leveraging N3C enables comprehensive, real-world analyses that can provide robust insights into the safety of XBB.1.5-containing COVID-19 mRNA vaccines.

In this study, we utilized EHR data from the N3C to evaluate the safety of XBB.1.5-containing COVID-19 mRNA vaccines by assessing the risk of 15 adverse events of special interest (AESIs) following vaccination. A self-controlled case series (SCCS) design was employed to account for time-invariant confounders. We further examined risk stratification by sex, vaccine brand, and defined post-vaccination risk periods.

Results

Study population

Overall, the study population consisted of 244,494 adult individuals (mean age: 61.14 years, standard deviation [SD]: 16.93 years; 59.4% female, 72.6% white) who received the XBB.1.5-containing COVID-19 mRNA vaccines between September 11, 2023, and June 1, 2024, in the USA (see Table 1). Among the participants, 55.4% received the vaccines from Moderna (n = 141,495), while 44.6% received vaccines from Pfizer-BioNTech (n = 102,999). Hypertension (57.6%) was the most prevalent comorbidity, followed by cardiovascular disease (39.0%) and obesity (31.1%).

Table 1 Characteristics of Participants Who Received any Dose of XBB.1.5- containing COVID-19 mRNA Vaccine from September 11, 2023 to June 1, 2024 by Vaccine Brand in N3C

Risk of adverse events

The risk of anaphylaxis was found to be significantly associated with the XBB.1.5-containing COVID-19 mRNA vaccine, with incidence rate ratios (IRRs) of 17.35 (95% confidence intervals [CI]: 9.32–30.03) on day 0, 9.35 (95% CI: 5.12–15.95) on day 1, and 6.20 (95% CI: 3.40–10.57) on day 2 (see Fig. 1). Subgroup and secondary analyses were consistent with the primary analysis, showing elevated relative risks across different groups: males (IRR: 25.68; 95% CI: 10.14–56.85), females (IRR: 12.98; 95% CI: 5.31–27.19), Moderna recipients (IRR: 13.27; 95% CI: 5.03–29.15), Pfizer-BioNTech recipients (IRR: 22.41; 95% CI: 9.43–47.39), and individuals without infection records during the period over 90 days prior to the start of the observation period and continuing through the end of the observation period (IRR: 15.34; 95% CI: 7.54–28.50) (see Fig. 1).

Fig. 1: Risk of Anaphylaxis Following XBB.1.5-containing COVID-19 mRNA Vaccination, Adjusted for COVID-19 Infection and Temporal Trends.
figure 1

Notes: Anaphylaxis was studied separately. Event counts fewer than 20 were replaced with “< 20” in accordance with N3C regulation. IRR indicates incidence rate ratio, calculated using conditional Poisson regression. The primary analysis included patients who received either brand of the XBB.1.5-containing COVID-19 mRNA vaccine and experienced anaphylaxis. The model was adjusted for COVID-19 infection and temporal trends.

For other AESIs in the primary analysis, no statistically significant increase was observed within the 28-day risk period after vaccination compared to the reference periods (see Fig. 2). Specifically, the confidence intervals for acute myocardial infarction (IRR: 0.94; 95% CI: 0.78–1.14), Non-hemorrhagic stroke and transient ischemic attack (IRR: 1.18; 95% CI: 0.95–1.44), narcolepsy or cataplexy (IRR: 1.69, 95% CI: 0.90–3.04), Guillain-Barré syndrome (GBS) (IRR: 0.24; 95% CI: 0.01–1.24), seizure (IRR: 0.92; 95% CI: 0.76–1.12), coagulopathy (IRR: 0.91; 95% CI: 0.74–1.11), multisystem inflammatory syndrome (IRR: 0.73; 95% CI: 0.46–1.12), Bell’s palsy (IRR: 0.95; 95% CI: 0.58–1.49), transverse myelitis (IRR: 3.15; 95% CI: 0.74–12.08), appendicitis (IRR: 0.91; 95% CI: 0.56–1.43), and pulmonary embolism (IRR: 0.99, 95% CI: 0.79–1.22) crossed 1. In contrast, IRRs for hemorrhagic stroke (IRR: 0.71; 95% CI: 0.50–0.98) and myo/pericarditis (IRR: 0.43; 95% CI: 0.16–0.95) had upper confidence interval limits below 1. Notably, encephalitis cases were not estimable due to an insufficient number of observed events.

Fig. 2: Risk of Adverse Events within 28 Days Following XBB.1.5-containing COVID-19 mRNA Vaccination, Adjusted for COVID-19 Infection and Temporal Trends.
figure 2

Notes: Each outcome was studied separately. Encephalitis was omitted because it was not observed with significant cases in this study. Events fewer than 20 were replaced with ‘< 20’ in accordance with N3C regulation. IRR indicates incidence rate ratio, calculated using conditional Poisson regression. The primary analysis included patients who received either brand of the XBB.1.5-containing COVID-19 mRNA vaccines and experienced specific outcomes. The model was adjusted for COVID-19 infection and temporal trends.

Subgroup analysis

In the subgroup analysis stratified by vaccine brand, the Pfizer-BioNTech XBB.1.5 COVID-19 mRNA vaccine subgroup revealed that narcolepsy or cataplexy was observed with fewer than 20 events in 1871 person-days during the risk period, compared to 65 events in 29,728 person-days during the reference period. This resulted in a higher incidence risk (IRR: 2.92; 95% CI: 1.10–7.37) within the 28-day risk period after vaccination compared to the reference periods (see Fig. 3). No significant associations with increased risk were observed for the other 13 AESIs in the Pfizer-BioNTech subgroup or any AESIs in the Moderna subgroup.

Fig. 3: Risk of Adverse Events within 28 Days Following XBB.1.5-containing COVID-19 mRNA Vaccination by Vaccine Brand, Adjusted for COVID-19 Infection and Temporal Trends.
figure 3

Notes: Each outcome was studied separately. Encephalitis was omitted because it was not observed with significant cases in this study. Event counts fewer than 20 were replaced with ‘< 20’ in accordance with N3C regulation. IRR indicates incidence rate ratio, calculated using conditional Poisson regression. The primary analysis included patients who received either brand of the XBB.1.5-containing COVID-19 mRNA vaccines and experienced specific outcomes. This subgroup analysis was stratified by vaccine brands. The model was adjusted for COVID-19 infection and temporal trends. NE, not estimable.

Secondary analysis

When analyzed by sex, the XBB.1.5-containing COVID-19 mRNA vaccines were not associated with a statistically significant increase in any of the 14 AESIs within the 28-day risk period compared to the reference periods, consistent with the primary analysis. (see Supplementary Fig. 1).

A secondary analysis, which involved altering the risk period to 14 days and 70 days, was consistent with the primary analysis, revealing no significant associations with an increased risk for any of the 14 AESIs after vaccination (see Supplementary Fig. 2). After exclusion of patients with infection effect in observation period, no significant associations with increased risk of any of the 14 AESIs following vaccination with XBB.1.5-containing COVID-19 mRNA vaccines were found (see Supplementary Fig. 3). We found no increased risk associated with colonic diverticulitis in the primary analysis (IRR: 1.94; 95% CI: 0.95–3.74) following exposure to the XBB.1.5-containing COVID-19 vaccine. The results regarding colonic diverticulitis in the subgroup analysis and the secondary analysis of altering the risk period to 14 days and 70 days were consistent with the primary analysis. We also examined co-administration with influenza vaccines on the same day, which accounted for only a small proportion of cases (19,563 out of 244,494 in the overall cohort; 35 out of 531 in the anaphylaxis cohort).

Discussion

In this retrospective observational study using the N3C dataset, we investigated the safety of XBB.1.5-containing COVID-19 mRNA vaccines employing the SCCS study design, involving a total of 244,494 adult participants. We found that anaphylaxis was associated with XBB.1.5-containing COVID-19 mRNA vaccines in the day 0–2 risk window, while no significant associations were observed for other AESIs within the 28-day risk period following vaccination.

Our findings suggest a significantly increased risk of anaphylaxis following XBB.1.5-containing COVID-19 mRNA vaccination within a very short risk window. Anaphylaxis is a rare but serious and potentially life-threatening allergic reaction that has been reported with various vaccines, including the XBB.1.5-containing COVID-19 mRNA vaccines23,24. Our findings support CDC surveillance data indicating that anaphylaxis remains a rare but serious adverse event following COVID-19 vaccination, with a reported incidence of approximately 5 cases per million doses administered21. A previous study examining the BA.4-5 and BA.1 bivalent COVID-19 vaccines found no association with an increased risk of anaphylaxis25. However, they used a 28-day risk window for analysis, which may have masked the acute onset of reactions that occur within a much shorter time frame. Other studies on this topic have mostly relied on case reports or surveillance platforms, further underscoring the need for more comprehensive investigations in this area24,26,27.

The current study found an increased risk of narcolepsy or cataplexy (IRR: 2.92, 95% CI: 1.10–7.37) in the Pfizer-BioNTech vaccine subgroup. However, the result should be interpreted with caution. First, the lower bound of the 95% confidence interval was close to 1, and the confidence intervals were wide, indicating a high degree of uncertainty. Second, these events were rare during the study period (< 20 events in the risk period and 65 events in the reference period out of 29,728 person-days). The small number of cases could lead to variability in the risk estimate. Narcolepsy and cataplexy are recognized neurological adverse reactions that have been reported infrequently after COVID-19 vaccination28. Previous case reports, as well as our study, suggest that these conditions may occur following vaccination with Pfizer-BioNTech’s COVID-19 vaccine29. Given the limited number of observed cases, this safety signal warrants further studies.

Our study contributes to the growing evidence supporting the safety profile of COVID-19 vaccines. Previous studies have reported occurrences of AMI following vaccination, including cases with fatal outcomes30,31,32. However, a large cohort study found no increased risk of AMI across mRNA and non-mRNA COVID-19 vaccines, including first, second, and booster doses33. Conflicting conclusions had been made by previous studies regarding risks of coagulative disorders, such as thrombocytopenia, following mRNA vaccines targeting earlier COVID-19 strains34,35. However, in our study, we did not observe an increased risk of AMI and coagulative disorders following vaccination with XBB.1.5-targeting mRNA vaccines. A prior study found no significant association between COVID-19 vaccines (including ChAdOx1 nCoV-19, BNT162b2, mRNA-1273, and Ad.26.COV2.S across first and second doses) and immune-mediated neurological disorders36. Similarly, our primary analysis of XBB.1.5-containing mRNA vaccines showed no increased risk of Bell’s palsy, encephalitis, Guillain-Barré syndrome, and transverse myelitis. Myo/pericarditis, particularly in individuals under 40, has previously been regarded as a serious adverse event associated with earlier COVID-19 vaccines37,38. Studies demonstrated an increased risk of myo/pericarditis following both the BNT162b2 and mRNA-1273 vaccines targeting earlier strains39,40. However, we did not find any evidence of an increased risk of myo/pericarditis associated with either brand of mRNA vaccines targeting the XBB.1.5 subvariant. Furthermore, preliminary safety signals for ischemic were noted by the Vaccine Safety Datalink (VSD) after vaccination41. Two studies using the SCCS design to investigate showed no significantly elevated risk for stroke within 1–21 days and 1–42 days of receiving bivalent COVID-19 vaccines42,43, which aligns with our findings regarding the XBB.1.5-containing vaccines.

While previous randomized clinical trials have concluded that the vaccines are generally safe44,45,46, their limited sample sizes and follow-up durations may reduce the likelihood of detecting rare adverse AEs. Real-world data is a robust enhancement for a comprehensive and continuous safety assessment of XBB.1.5-containing COVID-19 mRNA vaccine safety. Consequently, our findings provide post-marketing evidence that complements data from clinical trials. A prior research found no increased risk of 28 AEs following vaccination with a monovalent XBB.1.5-containing vaccine, it was based solely on EHR data of the elder population in Denmark and lacked sensitivity analyses due to the brief nature of the research letter47. Our study expands on these findings by analyzing U.S. data with a more robust study design.

Furthermore, our study leveraged the N3C database, which includes data from over 2 million patients across more than 70 institutions nationwide, to construct a cohort of over 240,000 patients—ensuring a sufficient sample size to detect rare vaccine-associated adverse events. This expansive, nationally representative cohort enhances the generalizability of our findings. The N3C data undergoes rigorous quality control measures, standardized harmonization protocols of OMOP CDM and institutional review board approvals at all participating sites, ensuring high reliability and reproducibility for safety surveillance analyses22,48. The identification of vaccine records employed EHR, claims data, written prescriptions, and self-reported medication information integrated into the N3C dataset, which ensured relatively comprehensive vaccination information for this study. In addition, using EHR data mitigates the reporting bias and incomplete information associated with adverse event reporting in pharmacovigilance platforms, allowing us to calculate IRRs where data from these platforms cannot. Our findings offer an active and supplementary perspective for post-marketing vaccine safety surveillance, further helping to address vaccine hesitancy. We employed an SCCS study design, which is effective for studying vaccine safety concerning rare AEs, and the within-person comparison eliminates non-time-varying confounders.

Due to the inherent limitations of EHR data, the N3C database may contain inaccuracies, missing values, misclassifications in EHR coding, or incomplete records. These issues could impact the reliability of vaccination and adverse event records. Specifically, vaccination records and adverse events may not be fully captured or may contain erroneous information. Despite the extensive data available in N3C, it represents only a portion of the country rather than its entirety. We excluded patients based on certain criteria, which may have resulted in the loss of potentially eligible participants. The fixed 28-day risk window may not be applicable to all populations or all types of adverse reactions. By focusing on only the first event, there is a possibility of missing subsequent events during the follow-up. Causal relationships between outcomes and exposures cannot be established, given the retrospective and observational study design. The multiple comparisons conducted in our analysis may lead to an increased risk of Type 1 error, which refers to falsely rejecting the null hypothesis when it is actually true. Subgroup analyses were not conducted throughout the study due to sample size limitations. Further investigation into specific subpopulations is warranted to explore potential differential risks and to better understand how these factors may influence the outcomes. Given these limitations, our findings need to be interpreted with caution.

In conclusion, we found that anaphylaxis was associated with XBB.1.5-containing COVID-19 mRNA vaccines in the day 0–2 risk window, while no significant association was observed for other AESIs within the 28-day risk period following vaccination.

Methods

Data source and ethics approval

This study utilized Level 3 data from the N3C dataset, which aggregates EHR data from over 84 medical institutions across the United States. The dataset includes EHR information from over 22 million individuals, encompassing clinical diagnoses, laboratory test results, medication histories, demographics, physician notes, and more19. The N3C dataset is standardized using the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) 20 for integration, storage, and representation. The N3C Data Enclave is managed under the authority of the National Institutes of Health (NIH). Data transferred to the National Center for Advancing Translational Sciences (NCATS) from N3C is governed by a Johns Hopkins University Reliance Protocol (IRB00249128) or through individual data partner agreements with the NIH. Data usage for this study was authorized by N3C (DUR-36ED2AE) and reviewed and approved by the Medical School Institutional Review Board (IRB) at the University of Michigan (HUM00243962). All data usage was in compliance with N3C’s data use policies and obligations.

Study design

Our study adopted the SCCS design to study the safety of XBB.1.5-containing COVID-19 mRNA vaccines using data from September 11, 2023, to June 1, 2024. The SCCS design is a method commonly utilized in vaccine safety studies, involving within-person comparison of risk and reference periods to eliminate non-time-varying confounders such as sex, race/ethnicity and genetic characteristics49,50.

The exposure was defined as the administration of either brand of XBB.1.5-containing COVID-19 mRNA vaccines. The day of vaccination was considered day 0, with different risk periods determined based on the nature of the diseases25,51,52. For anaphylaxis, which requires a much shorter risk window due to its acute onset, we defined 0, 1, and 2-day risk periods to accurately capture anaphylaxis events. Day 0 was included in the risk window for the following reasons: anaphylaxis typically occurs within minutes or hours after vaccination53, so day 0 is the best window to capture events. In addition, patients who experience anaphylaxis are unlikely to receive further vaccination on the same day, thereby resolving the uncertainty of whether the vaccination or the adverse event occurred first. For other AESIs, the primary risk period extended from day 1 to day 29 after vaccination. (see Fig. 4).

Fig. 4: Overview of the Self-controlled Case Series Study Design.
figure 4

Notes: The main risk period was days 0–2 for anaphylaxis and days 1–29 for other outcomes following XBB.1.5-containing COVID-19 mRNA vaccination. The reference period included both a pre-vaccination and post-vaccination time frame. The pre-vaccination period started 29 days after a previous non-XBB.1.5-containing COVID-19 vaccine and ended 14 days before the XBB.1.5-containing vaccination. The 28-day gap was included to minimize the influence of prior vaccinations. A 14-day pre-exposure window was excluded to prevent bias from event-dependent exposure. The post-vaccination period began immediately after the defined risk period and continued until the end of the study.

The reference period consisted of both pre- and post-vaccination intervals. The pre-vaccination period extended from 28 days after prior non-XBB.1.5 COVID-19 vaccinations to 14 days before the current vaccination, with the 28-day period implemented to eliminate potential confounding effects from prior vaccines. A 14-day pre-exposure window was excluded to control for event-dependent exposure25,36,50. The post-vaccination reference period began 30 days after XBB.1.5 vaccination and continued until the earliest occurrence of either a discontinuous observation in OMOP CDM, death, or the end of the study. Each patient contributed data to both the risk and reference periods (see Fig. 4).

Study participants

We included patients who had received any documented dose of the XBB.1.5-containing COVID-19 mRNA vaccine in the N3C dataset. Patients were excluded if they (1) were not continuously observed for the 365 days prior to the vaccination date through the end of follow-up (June 1, 2024), as defined by the observation period of the OMOP CDM; (2) were younger than 18 years old as of the XBB.1.5 COVID-19 mRNA vaccination date; (3) had received a prior COVID-19 vaccination less than 28 days before the XBB.1.5 dose; or (4) lacked any previously documented COVID-19 vaccination in the N3C dataset dating back to December, 2020, which was considered a potential indicator of incomplete vaccine records or non-continuous observation; (5) with missing any data in demographics. Figure 5 illustrates the process of patient selection and the corresponding numbers.

Fig. 5: Flowchart of patient selection and numbers in the N3C dataset (September 11, 2023 – June 1, 2024).
figure 5

Note: 1: The washout period for anaphylaxis was 30 days, and for other outcomes,  365 days. 2: According to N3C data policy, any cell size less than 20 must be redacted and replaced with ‘< 20’.

Exposure

The exposure was documented XBB.1.5-containing COVID-19 mRNA vaccination. The sources of N3C vaccination records included EHR, claims data, and self-reported medication information. Vaccination data were extracted from September 11, 2023, the national rollout date of XBB.1.5-containing COVID-19 mRNA vaccines, to June 1, 2024, the study end date. Both Pfizer-BioNTech and Moderna XBB.1.5-containing COVID-19 mRNA vaccines were included. Vaccination records were identified with related concept ids of OMOP CDM from the N3C dataset (See Supplementary Table 1). Only the first dose was included for each patient because either brand of XBB.1.5 vaccine was one-dose formulation.

Outcomes

The list of AESIs for outcomes was adjusted from the list of AEs in COVID-19 vaccine safety surveillance protocols and previous study25,51. In total, there were 15 kinds of AESIs, including Guillain-Barré syndrome (GBS), seizure, hemorrhagic stroke, non-hemorrhagic stroke and transient ischemic attack, narcolepsy or cataplexy, anaphylaxis, acute myocardial infarction, myo/pericarditis(either or both conditions), coagulopathy (including thrombocytopenia, disseminated intravascular coagulation, and deep venous thrombosis), multisystem inflammatory syndrome, Bell’s palsy, transverse myelitis, appendicitis, pulmonary embolism, and encephalitis. We identified these events using standard concept IDs for specific disease codes (see Supplementary Table 1)25,47. We included only the first event of each outcome during the whole observation to maintain the SCCS event-independence assumption. Patients who experienced specific AESI within 365 days (30 days for anaphylaxis) before the study period started were excluded from each analysis25. Records of specific outcomes from outpatient, inpatient, and emergency department settings were all retrieved from the dataset. Each outcome was analyzed separately, and patients contributed as a single case in the analysis of each AESI.

Statistical analysis

We evaluated the demographic characteristics and comorbidity status of the study population during the one-year period before the study’s start date (January 20, 2020). Only vaccinated individuals who experienced any AESIs were included in the study according to the SCCS design. The conditional Poisson regression model with an offset of the length of observed days was used to estimate the incidence rate ratios (IRRs) with corresponding 95% confidence intervals (CI), comparing the event incidence rate of the risk period and reference periods for each outcome. The model was adjusted for COVID-19 infection and background rates of disease temporal trends. COVID-19 infection was treated as a confounder using a time-dependent covariate. Specifically, we calculated the total number of overlapping days between the 90-day post-infection period and each risk or reference period and incorporated this into the model. Temporal trends in background disease rates were adjusted on a weekly basis in the model. To address concerns about multiple comparison issues while evaluating 15 different AEs simultaneously, we took several measures. First, we carefully selected these 15 AEs based on the study protocol and prior research25,51, which identified them as potential AEs of COVID-19 vaccines. In addition, we applied a more conservative significance threshold of 0.01 to reduce the likelihood of false positives54. These steps helped ensure the robustness of our findings.

Secondary analysis

(1) Subgroup analyses stratified by sex and vaccine brands were conducted to assess the potential association. (2) To test the robustness of our findings, we conducted secondary analyses by altering the follow-up period after exposure to 14 and 70 days. (3) To better account for the potential impact of COVID-19 infection on AESIs, we conducted an analysis by excluding any patients who had a COVID-19 infection in the 90 days prior to the start of the observation period and continuing until the end of the observation period. (4) Other than 15 AESIs, we additionally examined the associations of exposures with colonic diverticulitis as a negative control outcome, as it is considered to have no association with exposure to COVID-19 vaccination24.

All analyses were conducted within the N3C enclave using Structured Query Language (SQL) and R programming language (version 4.2.2).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.