Table 1 Aspects related to the four main attributes of the estimand in trials investigating IBIs

Treatment

►Describe all components of the IBI. IBIs are treatment packages that differ in several dimensions, such as (a) the theoretical foundation, (b) the number of modules, (c) the psychotherapeutic techniques used, (d) the sequence and/or the timing of treatment modules, (e) the rules for how clients access new content, (f) the amount and type of guidance, or (g) the type of personalization. Trial protocols should describe all components. This description should also include aspects of technical implementation, such as (i) the type of platform, (ii) the form of the application (mobile vs. web-based), and (iii) the nudging mechanisms used (e.g., gamification, reminders).

►Define what “being treated” means. In principle, various meaningful definitions exist, such as (a) gaining access to the treatment material or (b) completing all or a certain proportion of the modules. However, IBI participants often choose the dose and intensity of the IBI treatment for themselves. This IBI characteristic makes some estimands more plausible (e.g., the treatment effect of individuals gaining access to an IBI with a non-standardized and self-selected dose; treatment policy strategy) than others (e.g., the effect that would have been observed with perfect adherence). Moreover, protocol deviations are difficult to define and assess. This also concerns treatment discontinuation. For example, in an IBI trial where participants enter modules sequentially, a treatment could be considered discontinued if individuals stop logging in to the modules. However, in other IBIs, individuals decide for themselves how many and which modules they use; defining discontinuation and handling it appropriately can become much more challenging. Consequently, many IBI trials likely focus on the effect of making an IBI accessible.

►Define if the treatment regimen under investigation accepts treatments in parallel. IBIs are low-threshold interventions that are not regularly accompanied by visits to study centers. Therefore, there is little control over what individuals do in parallel. In the eligibility criteria, IBI often limits the parallel treatments with which participants are allowed to enter the trial. However, the use of parallel treatments initiated after randomization is usually not restricted. Consequently, it should be explicitly stated which parallel treatments are allowed. In most instances, especially in clinical samples, all parallel treatments are allowed and a treatment policy strategy (e.g., collect data even if individuals use treatments in parallel) will be employed. However, this should be stated explicitly, and the frequency and intensity of parallel treatments should be assessed and reported.

Population

►Excluded post-hoc changes the population attribute. IBI trials may exclude randomized patients post hoc (modified intention-to-treat analysis⁵³). In IBI trials, this may concern individuals who have at least started working with the program or completed at least two modules. However, such exclusions change the population attribute, may introduce bias⁵⁴, and must be justified. Consider the exclusion of individuals who commit suicide immediately after randomization. This exclusion is only justifiable if the researcher is absolutely certain that the observed adverse event has nothing to do with the outcome of the randomization (e.g., if individuals commit suicide because they were assigned to a treatment arm that they consider useless); if applied inappropriately, one throws away evidence against the specific treatment regimen that caused the adverse event. This also applies to per-protocol analyses that define strata based on observed completion status (see Table 3).

Endpoint

►Consider the time-point of assessment as an endpoint attribute. Describe when the endpoint will be assessed. If possible, define a time frame that is considered acceptable and how observations collected outside this time frame are handled.

►Discuss whether linking measurement to progress in the IBI is sensible. IBI trials may align assessments with specific events, such as the start of a new module or completion of all modules. However, this creates a correlation between adherence and endpoint availability, which raises methodological challenges. First, assessment times can vary widely when participants complete the program at their own pace, making it difficult to control for time effects. Second, if the assessment of the outcome is linked to the completion of the intervention, participants control when the endpoint is assessed; for example, participants may complete the endpoint assessment when they feel ready rather than at the intended point in time⁵⁵. If the missing assessments at the time of interest depend on the endpoint itself, the estimate could be biased^55,56.

►Assessing endpoints outside the IBI could be useful. While conventional clinical trials often involve on-site assessments conducted in study centers, this is less common in IBI trials. IBIs may assess the endpoint within the application. However, this can result in a correlation between endpoint availability and adherence. Especially if no information is available about how endpoints developed among individuals who discontinued the IBI, problems can arise. Researchers must rely on a hypothetical strategy that imposes assumptions about the post-ICE symptom development. Therefore, alternative assessment modalities (e.g., telephone interviews) for the primary endpoint should be considered. This method allows for standardization of the assessment time-point for all participants, reduces reliance on IBI participation patterns, and could increase the sense of commitment to the evaluations.

►Apply the same principles to endpoints different from symptom change. In some studies, other variables, such as treatment discontinuation, could be the endpoints. To derive meaningful treatment effects, the same principles discussed for defining appropriate estimands should be applied.

Statistical summary measures

► Select an effect size measure that is aligned with the estimand. If a trial is concerned with the mean difference, the statistical summary measure should reflect this mean difference. Therefore, specifying effect sizes that reflect the proportion of variance explained is not informative (e.g., R²).

►State your standardizer. Many IBI studies report between-group effects in standardized mean differences. If standardization is required, the standardizer should be specified. For a given mean difference, different standardizers lead to differences in the size of the effect sizes⁵⁷ and, more importantly, represent different estimands⁵⁸. Standardizing a mean difference by the standard deviation of an untreated reference population has a different meaning than standardizing by the standard deviation of endpoint among all randomized individuals at the endpoint assessment⁵⁹. It may be useful to report effects in units of the scale, as raw, unstandardized scale values do not depend on the standardizer and are often informative to clinicians.