Introduction

The time-dependent exposures in observational cohort studies pose a substantial analytical challenge. Incorrectly, handling these exposures can introduce immortal time bias, leading to distorted and potentially misleading estimates of treatment effects1,2. Typically, while investigating a specific exposure, individuals are considered at risk from the beginning of the study. However, with time-dependent exposures, individuals only enter the at-risk period when the exposure begins, causing a shift in exposure status over time3. Considering patients who develop exposure during the follow-up as at-risk from study inception is methodologically incorrect. For example, consider a study examining whether statin use reduces the risk of a heart attack. In a time-fixed exposure model, individuals are classified as statin users if they were ever exposed to the medication at any point during follow-up, regardless of the timing of exposure. This means that a person is considered a user even if they started statins late in the follow-up period. In contrast, a time-varying exposure model updates each patient’s exposure status over time, allowing for a more accurate representation of the individual’s true exposure status. If a patient initiates statin therapy during follow-up, their status changes from non-user to user at the time of initiation, better capturing the dynamic nature of medication use and its temporal relationship with the outcome. These individuals, by definition, will survive until the exposure status changes, leading to the allocation of person-time to incorrect exposure groups and subsequent biased analysis results4. To prevent immortal time bias, it is essential to account for the time-varying nature of exposures employing a counting process framework5. This approach accurately reflects dynamic changes in exposure status over time, ensuring that individuals contribute person-time to the appropriate exposure category. While existing literature largely addresses the analysis of single TVE, with statistical software offering corresponding tools, there remains a gap in research focusing on clinical contexts involving multiple concurrent TVEs.

This study aimed to investigate appropriate statistical approaches for analyzing data with multiple TVEs, using a real-world pediatric dataset as an example. Specifically, we compared the results of a biased model, which treated both influenza vaccination and antibiotic treatment (i.e., exposures of interest) as time-fixed variables, with those of several other models that incorporated one or both exposures as time-varying. The outcome of interest was the incidence of influenza/influenza-like illness.

Methods

Study population and setting

The study cohort included Italian children aged 6 months to 14 years as of October 1, 2017, and who were followed throughout the 2017–2018 influenza season, from October 1, 2017, to May 31, 2018. The data were obtained from Pedianet (https://pedianet.it), a population-based Italian database that compiles patient-level information from family pediatricians participating in the Pedianet network. These data, collected during routine medical care, are anonymized in accordance with Italian privacy laws by assigning a unique numeric code to each patient record. The anonymized information is securely transmitted on a monthly basis to a central database in Padua, where it undergoes validation. Participation in the Pedianet system is entirely voluntary, and informed consent is obtained from parents or legal guardians to allow the storage and use of their child’s anonymized data for research purposes.

Only children followed by one of the family paediatricians from the Pedianet network adhering to the influenza vaccination program (i.e., who provided influenza vaccination to their patients during the influenza season in agreement with the National Health System) were included in the study, similarly to what was done in a previous study6. Study participants were considered at risk from October 1, 2017. Follow-up continued until the onset of influenza/ILI (i.e., the outcome of interest), the end of the study (i.e., May 31, 2018) or until participants dropped out due to unrelated reasons, whichever came first.

We adhered to the STROBE guidance for reporting on observational studies7.

This study adhered to the principles outlined in the Declaration of Helsinki and followed the methodological standards recommended by the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP). Approval for the study and database access was granted by the Internal Scientific Committee of Società Servizi Telematici Srl, the legal entity responsible for Pedianet. In accordance with Italian legislation, retrospective, observational, and non-interventional studies are exempt from the requirement for ethics committee approval.

Definition of exposures

Vaccination status was defined based on the electronic health record from the Pedianet database and assumed to provide protection throughout the influenza season, consistent with evidence of influenza vaccine effectiveness8. Children under 6 months were excluded due to age-related ineligibility for influenza vaccination. Antibiotic exposure was identified using the prescription registry within the Pedianet database. Five exposure scenarios were considered to model antibiotic exposure: one assuming full-season coverage, and three others assuming coverage for five, seven, or ten days after each prescription. These timeframes reflect treatment durations commonly recommended by national and European pediatric guidelines for acute respiratory tract infections. Such infections, including pneumonia, acute otitis media, and bacterial pharyngitis, are typically treated with antibiotics for five to ten days. While these durations represent simplified assumptions and do not account for individual pharmacodynamics, they provide clinically grounded estimates for defining exposure windows. Our use of multiple windows also allowed us to assess the sensitivity of estimates to different definitions of exposure9,10,11,12,13,14.

Definition of outcome

The outcome of interest was influenza/influenza-like illness (ILI), identified using a machine-learning algorithm applied to electronic health records from the Pedianet database. The algorithm was trained to detect clinical diagnoses of influenza/ILI based on pediatricians’ ICD codes, clinical notes, and visit patterns. To ensure accurate case detection, a custom-developed and validated Natural Language Processing (NLP) algorithm trained on a comprehensive, gold-standard labeled dataset was employed. This algorithm demonstrated excellent performance, with an accuracy of approximately 99%15. Laboratory-confirmed influenza was not systematically available, as testing was only performed and reported at the discretion of the family pediatrician. Therefore, the outcome reflects clinically diagnosed influenza/influenza-like illness (ILI), rather than laboratory-confirmed influenza.

Statistical analysis

As a first step, the main characteristics of the study cohort were presented and summarized. Categorical variables are reported with numbers and proportions, while continuous variables are reported with median and quartiles. Cox regression mixed effects models with random intercept for the region of residence were fitted to estimate the hazard ratios (HRs) and 95% confidence intervals (CIs) for influenza/ILI associated with influenza vaccination and antibiotic use. Both time-fixed and time-dependent effects of these exposures were evaluated. When exposures were treated as time-fixed variables, the proportional hazards assumption was assessed using Schoenfeld residuals (see Appendix—Models Summaries section). Models were adjusted for age, sex, deprivation index16, comorbidity, and the number of visits and antibiotic prescriptions in the previous year (except for those under one year of age), as proxies for the clinical condition. Children were classified as having comorbidity if they had at least one disease-based healthcare exemption for chronic complex condition (i.e., cystic fibrosis, diabetes, chronic obstructive pulmonary disease, and asthma), immunodeficiency or immunosuppressive therapy, neurological and neurocognitive conditions (including Trisomy 21), prematurity (less than 37 weeks of gestation), renal failure, congenital cardiac disease (including heart failure), or chronic liver conditions15. The number of visits and antibiotic prescriptions were categorized as < 8 or ≥ 8, and as 0, 1–2, or ≥ 3, respectively, following the classification used in a previous study based on the same data source15. Bootstrap resampling was conducted to assess model stability (see Appendix, Model Fit and Stability sections). Six different model specifications were compared to assess the optimal approach (Fig. 1). Specifically:

Fig. 1
figure 1

Different time modelling approaches adopted. Panel A, B and C refer to Model 1,2 and 3, respectively. Panel D refers to Model 4,5 and 6, depending on the length of coverage assumed for the prescribed antibiotics.

Full size image

Model 1: vaccine and antibiotic prescriptions as time-fixed

$$\begin{aligned} Influenza & \sim Age + Gender + Dep.Index + Comorbidity \\ & \;\;\; + N.Visits + N.Antibiotics + Antibiotics + Vaccine \\ \end{aligned}$$

Model 2: vaccine as time-varying exposure (TV subscript), antibiotics as time-fixed

$$\begin{aligned} Influenza & \sim Age + Gender + Dep.Index + Comorbidity \\ & \;\;\; + N.Visits + N.Antibiotics + Antibiotics + Vaccine_{TV} \\ \end{aligned}$$

Model 3: vaccine and antibiotics as time-varying exposures (TV subscript)

$$\begin{aligned} Influenza & \sim Age + Gender + Dep.Index + Comorbidity \\ & \;\;\; + N.Visits + N.Antibiotics + Antibiotics_{TV} + Vaccine_{TV} \\ \end{aligned}$$

Model 4: vaccine and antibiotics as time-varying exposures, coverage of the antibiotics prescribed ends after 10 days (TV10 subscript)

$$\begin{aligned} Influenza & \sim Age + Gender + Dep.Index + Comorbidity \\ & \;\;\; + N.Visits + N.Antibiotics + Antibioitcs_{{TV_{10} }} + Vaccine_{TV} \\ \end{aligned}$$

Model 5: vaccine and antibiotics as time-varying exposures, coverage of the antibiotics prescribed ends after 7 days (TV7 subscript)

$$\begin{aligned} Influenza & \sim Age + Gender + Dep.Index + Comorbidity + N.Visits \\ & \;\;\; + N.Antibiotics + Antibiotics_{{TV_{7} }} + Vaccine_{TV} \\ \end{aligned}$$

Model 6: vaccine and antibiotics as time-varying exposures, coverage of the antibiotics prescribed ends after 5 days (TV5 subscript)

$$\begin{aligned} Influenza & \sim Age + Gender + Dep.Index + Comorbidity \\ & \;\;\; + N.Visits + N.Antibiotics + Antibiotic{\text{s}}_{{{\text{TV}}_{5} }} + Vaccine_{TV} \\ \end{aligned}$$

A time-fixed baseline model approach was adopted in Model 1 (Fig. 1.A), considering both exposures as time-fixed. This implied that all children receiving vaccination, antibiotics or both were considered exposed from the beginning of the follow-up. Model 2 (Fig. 1.B) evaluated influenza vaccination as time-varying exposure, utilizing a counting process structure that duplicated observations for vaccinated individuals. Conversely, antibiotic exposure remained time-fixed. Model 3 considered both antibiotics and influenza vaccination as time-varying exposures. This represents an extension of the conventional approach, where typically, only one variable is treated as time-varying, mainly because restructuring the dataset to such a format might be overly complex. This issue is addressed by the mtvc R package (version 1.1.0)17(see Appendix—Multiple Time-Varying Covariates (MTVC) in R—survival package section). This tool accepts one or more time-varying variables, along with their corresponding exposure change dates. It generates a restructured dataset where each individual has a row representing their exact exposures during each time window (Fig. 1.C). In addition, as it might not be appropriate to consider that exposure to antibiotics lasts for several months, three different models were developed (Fig. 1.D), in which each individual who received a prescription for antibiotics would be considered no more exposed after a coverage period of five, seven, or ten days.

The analyses were conducted using R version 4.4 (R Foundation for Statistical Computing, Vienna, Austria), with packages mtvc, survival, coxme, dplyr and forestplot. The code to reproduce this analysis is publicly available on GitHub18.

Results

A total of 56,069 children aged 6 months to 14 years (median age: 7.3 [IQR: 4.4–10.3], Table 1) were identified. Of these, 4,572 (8.2%) received influenza vaccination, and 13,975 (25%) had at least one antibiotic prescription during the follow-up. Most patients resided in Veneto (62%) or Campania (12%). In total, 20,466 had a low deprivation index (37%), whereas 8,396 (15%) did not report their socio-economic status. The remaining 27,207 (49%) had a high deprivation index. The number of patients with at least one comorbidity was 2,434 (4.3%).

Table 1 Characteristics of the study cohort.
Full size table

Figure 2 summarizes the estimated impact of antibiotic and vaccination exposures on the risk of influenza/ILI for each model. The time-fixed baseline model , which treated both influenza vaccination and antibiotic exposure as time-fixed, estimated HRs of 0.47 (95% CI: 0.38;0.58) for influenza vaccination and 0.57 (95% CI: 0.53;0.62) for antibiotic exposure. These results would suggest a protective effect of both exposures against influenza/ILI infections. A similar estimate (HR = 0.47 [95% CI: 0.38;0.59]) was obtained in the second model, where influenza vaccination was considered time-varying. Indeed, this was explained by the fact that vaccines are dispensed to patients a few months after the study begins, meaning that the amount of person-time misallocated did not significantly affect the estimate. Conversely, the observed association for antibiotics remained unchanged, indicating a consistently protective effect against influenza/ILI. This finding is unexpected, given that antibiotics primarily target bacterial infections, while influenza/ILI is caused by a virus. This unforeseen result may indicate potential misspecification in the initial models, particularly those that treated antibiotic exposure as time-fixed.

Fig. 2
figure 2

Forest plot of the different time modelling approaches adopted.

Full size image

In contrast, the third model, which considered both exposures as time-varying, returned a different estimate for the effect of antibiotic treatment, with an HR of 1.18 (95% CI: 1.10; 1.28), indicating a positive association with influenza/ILI.

The last three models, which used different antibiotic coverage protection periods (5, 7, and 10 days), reported an even stronger positive relationship between antibiotic use and influenza/ILI, with increased risks ranging from 1.58 to 2.99, respectively, for 10 to 5 days of coverage.

Discussion

This study investigated the impact of multiple time-varying exposures on clinical outcomes by comparing different statistical approaches, including treating exposures as time-fixed or as single- or multiple-time-varying variables. Our findings underscore the novel importance of accounting for the dynamic interplay of multiple time-varying exposures when conducting cohort studies, highlighting the need for researchers to properly consider the impact of misclassification of exposures.

This study used real-world data as an application to assess the association between antibiotic treatments and influenza vaccination on influenza/ILI infections. The first two models, which treated antibiotic use as a time-fixed variable, paradoxically suggest a protective effect against influenza/ILI. However, this is not reasonable and in contrast with medical knowledge18. In contrast, subsequent models, which more accurately captured the time-varying nature of antibiotic exposure, revealed a positive association between antibiotic use and the risk of influenza/ILI.

Indeed, antibiotic treatment has been previously associated with an increased risk of viral infections19. The relationship may be explained by the disruption of the gut microbiome balance, which plays a critical role in immune system regulation, induced by antibiotic exposure, which has been identified as a predisposing factor to viral infections, including influenza/ILI20. Moreover, antibiotic treatment may be considered as a proxy for acute infections that can transiently impair the immune system, thereby increasing susceptibility to subsequent infections, including influenza/ILI. This hypothesis could account for the positive association observed in this study between antibiotic exposure and influenza/ILI risk when using time-varying models, which are widely recognized as the appropriate methodological choice for such analyses.

As already known, the impact of considering a time-varying exposure as time-fixed may change considerably depending on the nature of that variable. In our application, considering the vaccination as time-fixed does not significantly impact the estimates, meaning that even a more straightforward approach, considering only antibiotics therapy as time-varying, would have returned reasonable estimates. Vaccination was administered to patients only a few weeks after the beginning of the study, and its coverage lasted until the end of the study. Consequently, the impact of misclassification is minimal for this exposure.

On the other hand, this is not true for antibiotics, which could be prescribed at any time and lasted only for a few days. Indeed, classifying such exposure as fixed in time, or for longer periods than actual, would significantly bias the respective HR.

This study shows that the bias generated by wrongly defining the exposure window cannot be underestimated, especially when multiple time-varying exposures are considered. More complex approaches that accurately allocate person-time to the correct exposure categories are necessary for valid conclusions.

When it comes to interpreting our results from a clinical perspective, it is important to note that this study has several limitations. First, antibiotics were not further classified based on their spectrum of activity or diagnosis, leading to a potential misinterpretation of the results, with children presenting acute respiratory infections being at higher risk of developing subsequent influenza/ILI. Furthermore, misclassification bias might be introduced by using electronic health records, with the assumption of full compliance with prescribed antibiotics regimens. Moreover, the decision to model antibiotic exposure for five, seven, or ten days may oversimplify complex biological processes. However, the selected durations are supported by national and European pediatric clinical guidelines and consensus, recommending 5–7 days of treatment for community-acquired pneumonia and up to 10 days for acute otitis and pharyngitis in younger or high-risk children. These evidence-based choices strengthen the rationale for the exposure windows tested in our models and allow for a structured sensitivity analysis of the effect estimates. Additionally, as our outcome was defined using a machine-learning algorithm applied to clinical records, and laboratory-confirmed influenza data were not systematically available, as testing was only performed and reported at the discretion of the family pediatrician, our results refer to clinically diagnosed influenza/ILI. This may have introduced some degree of misclassification, which we acknowledge as a limitation. Lastly, unmeasured confounders, such as healthcare-seeking behaviour or concurrent bacterial infections, could also influence the associations between antibiotic use and influenza/ILI risk.

Conclusions

The overall estimates can be substantially influenced by the statistical approach adopted, particularly when the exposure under investigation is subject to immortal time bias. The impact of misspecification is particularly pronounced for exposures with a dynamic, time-varying nature.

In this study, influenza vaccination was administered a few weeks after the beginning of the study, with coverage lasting until the end of the study, resulting in minimal misclassification impact for this exposure. On the other hand, antibiotic treatment could be prescribed at any time and lasted for only a few days. Misclassifying such exposure as time-fixed or extending the duration beyond its actual periods would have led to substantial bias in the HR estimates.