Introduction

Blended therapy is a promising treatment approach for mental disorders because it combines the strengths of face-to-face psychotherapy and (self-guided) digital health interventions: it offers the flexibility and accessibility of digital tools - such as structured online modules or app-based exercises that can be completed independently by patients - while still preserving the personal, responsive aspects of in-person sessions1. According to systematic reviews, blended therapy could potentially save clinicians time, support lasting changes from psychotherapy, and may even reduce treatment dropout rates1,2.

In face-to-face psychotherapy, dropout is defined as premature termination of the treatment, and usually operationalized as not attending all planned sessions3. Meta-analytic findings estimate dropout rates in face-to-face psychotherapy to range from 12% to 27%4,5,6,7,8. Already in 2005, Eysenbach emphasized the need for more specific research on dropout in digital health interventions, or so-called internet-based interventions (IBI), in his “law of attrition,” highlighting that disengagement is a common and complex phenomenon in such contexts9. This is also reflected in more variable definitions of dropout in the field of IBI: For example, “finished all available modules” or “completed number of recommended modules”, which could be less than the total numbers of modules10. Accordingly, dropout rates are highly variable, ranging from 15% to 65%8,11,12,13,14. Forbes et al.10 give a systematic overview of dropout operationalizations in IBI. Given the heterogeneous definitions applied, they conclude a need for a more standardized way to report treatment dropout metrics. Blended therapy, with combined face-to-face and digital components, adds another layer of complexity: dropout may involve discontinuing in-person sessions, disengaging from digital tools, or both. However, a closer examination of the included studies reveals significant inconsistencies in how dropout is operationalized. Some studies, such as Christensen et al.15, do not provide an explicit operationalization of dropout, while others, such as Campbell et al.16 and Kiluk et al.17, define treatment completion narrowly, based only on attendance of a minimum number of face-to-face sessions.

Blended therapy might help to reduce treatment dropout through more consistent engagement and guidance18, but the inconsistencies in definitions hinder the ability to interpret findings, compare results across studies, and draw generalizable conclusions about dropout rates and their implications for patient outcomes. This lack of standardization creates challenges for clinicians, who struggle to translate inconsistent findings into practical retention strategies, for researchers, who face difficulties in synthesizing evidence to advance the field, and it complicates decision-making for policymakers regarding the allocation of healthcare funding for specific interventions.

Treatment dropout in blended therapy might be associated with treatment outcomes. Patients who dropped out of face-to-face psychotherapy for depression showed poorer treatment outcomes in comparison with completers19. Evidence from guided IBI highlights the critical role of treatment completion in determining outcomes, too. For instance, Moshe et al.20 demonstrated a strong dose-response relationship between session completion and effect size, emphasizing that treatment completion significantly improves therapeutic outcomes. Wright et al.21 also reported that higher completion rates were consistently associated with better outcomes. These findings from face-to-face psychotherapy and digital interventions suggest that the relationship between treatment dropout and outcomes likely extends to blended therapy as well, but more research is needed. In blended therapy, disengagement from one component (e.g., the digital component) may not equate to total treatment dropout. However, dropping out of one component could still diminish the overall effectiveness of the treatment. Conversely, consistent engagement across digital and face-to-face components may amplify therapeutic benefits. Thus, understanding and standardizing dropout definitions in blended therapy is essential not only for accurately measuring dropout rates but also for identifying how varying levels of engagement across components influence patient outcomes.

To move beyond conceptual distinctions and empirically capture the level of engagement, indicators that reflect actual patient behavior are required. Although meta-analytic evidence is lacking, some studies indicate that adding usage data improves the prediction of treatment dropout in IBI when combined with other predictors (e.g., sociodemographic variables). Bremer et al.22 reported that the average number of days needed to complete each module of an IBI was a strong predictor for dropout. Moshe et al.23 showed that adding usage data measures, such as days taken to complete a module and minutes spent online to complete each module, were useful in providing a more accurate prediction of treatment dropout. While these findings derive from IBI, they underscore the potential of usage data to improve dropout predictions in blended therapy. So far, few studies have investigated usage patterns in blended therapy. Krijnen-de Bruin et al.24 distinguished between “low” and “regular” users in a case-by-case, personalized intervention for anxiety and depression. Kemmeren et al.25 identified three distinct usage groups, emphasizing “mainly web-based,” “mainly face-to-face,” and “blended compliant” users in an integrated, alternate and standardized intervention for depression. Wu et al.26 found that poor early digital engagement, defined as completion of all assigned digital activities prior to the second face-to-face session, was a strong predictor of treatment dropout in an integrated, alternate and personalized intervention for depression and anxiety. However, no studies have yet investigated how usage patterns over the course of the entire intervention are associated with treatment dropout. This is a critical gap, as identifying patterns linked to dropout could guide the design of interventions to enhance engagement and adherence. By exploring how different usage patterns are linked to dropout, we can identify at-risk patients and design more personalized interventions to prevent dropout and improve outcomes.

The absence of standardized definitions and the lack of research connecting dropout to outcomes and usage patterns with dropout in blended therapy present significant barriers to the efforts of designing better interventions. To address these gaps, our study attempts to provide a comprehensive understanding of treatment dropout in blended therapy. Specifically, we focus on two key aims: first, to conduct a scoping review to identify operational definitions of dropout in the blended therapy literature, and second, to investigate the identified operational definitions using real-world data on blended therapy. For the second aim, the key objectives are to compare dropout rates for identified definitions (RQ.1), to investigate the association between treatment dropout and mental health outcomes (RQ.2), and to explore how different usage patterns relate to dropout (RQ.3).

Results

Scoping review of operational treatment dropout definitions (aim 1)

Figure 1 shows the detailed screening process for the scoping review. The literature search identified N = 194 publications. After screening, a total of n = 14 publications provided an operational definition of treatment dropout and were included in the scoping review. Table 1 gives an overview of the operational definitions identified in the scoping review.

Fig. 1: PRISMA flowchart for the scoping review.
Fig. 1: PRISMA flowchart for the scoping review.The alternative text for this image may have been generated using AI.
Full size image

The flowchart illustrates the process of identification, screening and including studies for the scoping review. N represents the number of studies.

Table 1 Operational definitions identified in the scoping review
Full size table

Synthesis of literature search results

Supplementary table 1 provides a detailed overview of all included studies. N = 1 study included only the digital component for operationalizing treatment dropout27, n = 4 studies included only the face-to-face component26,28,29,30, and n = 7 studies combined the digital and face-to-face component for their operational definition of dropout31,32,33,34,35,36,37. N = 2 studies were not clearly categorizable38,39 (included digital and face-to-face component but do not report if and how it is combined). In summary, three categories of definitions emerged: definitions that rely solely on the face-to-face component, definitions that rely solely on the digital component, and definitions that rely on some sort of combination of the two components (see Table 1 for a detailed overview). Most definitions are based on the percentage of content (digital, face-to-face, or both) completed, with criteria ranging from 50% to 100% of content completion. Only one definition comprised a more complex combination of components were “adherence was operationalized as at least 1 home screen interaction after a minimum of three therapy session33.”

Building on the classification by Ferrao Nunes-Zlotkowski et al.1, most studies can be categorized as employing an integrated interaction between digital and face-to-face components, typically following an alternating delivery pattern with standardized content assignment (see Supplement). Notably, only one study reported the use of personalized content assignment26. Regarding the role of the components in the intervention, only Kooistra et al.34 identified the face-to-face component as supplementary to the digital core component. In contrast, the other studies did not explicitly designate either component as core or supplementary.

Application of identified definitions to real-world data (aim 2)

The operational definitions of treatment dropout identified in the literature vary in scope and focus, including either only digital or face-to-face components, or a combination of both. Some of the identified operational definitions appear to rely heavily on the specific context of the blended therapy intervention. After synthesizing existing definitions from the scoping review, we applied five operational definitions to the PsyTOM dataset (see Table 2). The criteria for face-to-face components used in other studies, which are the amount of prescribed and then attended sessions, cannot be directly applied to the PsyTOM study. The PsyTOM study was conducted in a routine care setting; hence, there was no predetermined number of face-to-face sessions as part of the trial. Consequently, data on the number of intended face-to-face sessions were not available. Therefore, we relied on the therapist’s evaluation of treatment dropout (Was the therapy ended earlier than originally planned? (Yes/No)) to determine dropout in the face-to-face component and could not include information about the percentage of face-to-face sessions completed. For the digital component, we used a variable that tracked the percentage of assigned digital components completed by each participant.

Table 2 Identified definitions and operationalizations with PsyTOM data
Full size table

For all the following analyses, we used the PsyTOM trial data of participants in the blended therapy arm who either completed or dropped out of therapy during the study period (n = 147). The sample was mostly female, rather young and mostly well educated (see Table 3).

Table 3 Sample characteristics
Full size table

Consistency in dropout rates of different operational definitions of treatment dropout (RQ 1)

The dropout rate varied based on the definition applied (Table 4). For example, Definition 2 - completion of 100% of assigned digital content—produced a very high dropout rate (93.8%), while Definition 1—therapist-rated dropout—resulted in a much lower rate (67.3%). Definitions 1, 3, 4, and 5, which incorporate therapists’ evaluations, classified fewer participants as dropouts compared to Definition 2 (digital-only), which categorized most participants as dropouts. Although Definitions 3 (100% digital & therapist) and 4 (75% digital & therapist) identified a similar number of dropouts to Definition 1, they only applied to a subset of participants and resulted in missing values for about a third of the participants reflecting gaps in the operational criteria based on the definitions from the scoping review (for example, the combination “completed 100% of digital content” and “therapist evaluated participant as dropout” was not accounted for in the operational definition and thus resulted in a missing value.). Definition 5 (50% digital & therapist) classified the least participants as dropouts, but simultaneously, produced the highest number of missing values.

Table 4 Treatment dropout frequencies and rates for each definition
Full size table

Association of treatment dropout with treatment outcomes (RQ 2)

For definition 4 (75% digital & therapist) and definition 5 (50% digital & therapist), almost all group differences were significant, with participants who dropped out reporting higher scores for depressive symptoms and anxiety symptoms and lower scores of satisfaction with life at baseline and 6-month follow-up (see Table 5). For definition 1 (therapist-only), definition 2 (digital-only), and definition 3 (100% digital & therapist), there were no significant group differences for all scales and timepoints (see Table 5). In conclusion, the results show that significant group differences between dropouts and non-dropouts on certain measures are dependent on the definition of dropout applied, even though all other definitions showed descriptive trends (see Table 5). These results should be interpreted with caution, as group sizes varied across definitions.

Table 5 Associations between treatment dropout and depressive symptoms, anxiety symptoms and satisfaction with life
Full size table

Usage patterns of the digital component (RQ 3.1)

We used an elbow plot and silhouette scores to determine the optimal number of clusters for the clustering algorithm. The elbow plot initially suggested four clusters after 10 and 15 iterations. However, a five-cluster solution provided clearer and more interpretable patterns in the usage data (descriptives see Table 6), leading us to select this as the best fit. The silhouette score for the five-cluster solution indicated a moderate model fit (mean = 0.24) and alternative cluster numbers did not yield higher silhouette scores.

  • Cluster 1 (therapist contact) represented users with the highest amount of therapist interactions (n = 16), indicating stronger reliance on therapist support. This group had more assigned chapters and was characterized by above-average engagement and high completion rates.

  • Cluster 2 (selective usage) represented users with a below-average number of assigned chapters, moderate engagement and minimal therapist interaction (n = 12). However, participants in this cluster had the longest average session duration.

  • Cluster 3 (minimal usage) represented users with the lowest engagement across all metrics. Participants had a below-average of assigned chapters but showed low completion rates and low additional module requests (n = 52).

  • Cluster 4 (additional content) represented high module requests but average session metrics and little therapist interaction (n = 24). This group exhibited average login frequency and duration, but they stood out for requesting the most additional modules, suggesting a preference for extra content.

  • Cluster 5 (moderate usage) represents average usage (n = 40). Participants in this cluster showed moderate engagement across all metrics, but high completion rates. They had minimal therapist interactions and requested few additional modules.

Table 6 Descriptives usage data and z-transformed values for cluster centroids
Full size table

Impact of dropout definition (RQ3.2 and RQ3.3)

Across all clusters, the identification of treatment dropout cases was consistent across all definitions (see Table 7): in every cluster, more participants were categorized as dropouts than non-dropouts, regardless of the dropout definition. However, in comparison to Definition 1, the other definitions that relied more heavily on the digital component consistently classified more participants as dropouts in each cluster. The operational definitions applied classified almost all participants as dropouts, even if they showed moderate or high engagement metrics (Tables 6 and 7). Across all definitions, dropout was lowest in Cluster 1 and highest in Cluster 3.

Table 7 Usage behavior and dropout frequencies
Full size table

Discussion

This is the first study to investigate the impact of different operationalizations of treatment dropout in blended therapy. We first identified operational definitions in the literature and then compared them using real-world data.

We conducted a scoping review to identify existing definitions of treatment dropout in blended therapy. Some studies considered only the digital or face-to-face component for their operational definitions, while others used various combinations of both. By applying these definitions to our trial data, we found that, first, some definitions from the literature were too tailored to their specific study designs to be applied to our data, and second, treatment dropout rates in our data changed depending on the definition applied. The scoping review revealed that operational definitions depend very much on the context and overall structure of a blended therapy intervention. For some interventions, face-to-face sessions might be the core component, and the digital component supplementary. For other interventions it might be the other way around with the digital component as core component. For integrative set-ups, the different face-to-face and digital modules might build on each other, resulting on a higher importance of completing both components, especially if they follow an alternate structure. Moreover, assignment of digital components can be standardized or personalized, potentially resulting in different criteria for treatment completion1. The range of operational definitions identified in the literature search reflects this variability in blended therapy intervention setups. This heterogeneity complicates direct comparisons, highlighting the importance of reporting results in a manner that facilitates comparability across different studies. One recommendation for researchers is to at least report separate dropout rates for the digital and face-to-face components in addition to the overall rate derived from the study-specific definition of treatment dropout.

We used data from the PsyTOM study to compare dropout definitions, acknowledging that this data is also influenced by its contextual setting. PsyTOM is integrated in routine care, meaning participants are already in an outpatient therapy setting, so naturally, there is an emphasis on the face-to-face component and the contact with the therapist. This might explain why the results highlight the importance of the therapist’s evaluation in determining dropout status and why only relying on the digital component resulted in substantially higher dropout rates. This was demonstrated in differences in dropout rates by about 25% between only looking at digital module completion and the therapist’s evaluation only. The other definitions that included the therapist evaluation and the digital content completion yielded dropout rates closer to the therapist evaluation-only definition, suggesting that solely relying on digital engagement metrics may not be the best indicator of treatment dropout in this type of blended therapy. The fact that therapist-evaluated dropouts resulted in lower dropout rates than digital module completion alone indicates that therapists may consider additional factors when classifying someone as a dropout - perhaps especially in settings where face-to-face therapy is the core component of blended therapy setups. For example, therapists might assess patient engagement through interactions during face-to-face sessions and by evaluating the need for treatment adaptability to individual circumstances, aspects that cannot be captured by digital metrics alone. This perspective also aligns with core findings from psychotherapy research: factors such as the therapeutic alliance and the quality of interaction between patient and therapist are consistently predictors of treatment success40. In IBI, guidance has likewise been shown to significantly enhance adherence and outcomes41,42. It may therefore not be the amount of digital content completed that determines treatment success or continuation, but how well the digital elements are integrated into a therapeutic context that is supportive and responsive to the individual needs of the patient. This view aligns with research showing that both therapists and patients value the ability to tailor blended therapy protocols to personal circumstances and preferences43,44. Moreover, personal circumstances, such as time constraints due to other responsibilities or engagement with therapy content outside of digital modules, are difficult to assess when only relying on digital usage data or the number of intended face-to-face sessions. This suggests that in some settings, therapist and patient input might be crucial for understanding true disengagement and dropout risk in blended therapy. However, digital engagement can also play a crucial role, depending on the setting. Wu et al.26 found that poor digital engagement was associated with a higher likelihood of treatment dropout—specifically, clients who did not complete the digital activities assigned by their provider early in treatment were significantly more likely to drop out. This suggests that while therapist evaluations may provide a more comprehensive view of patient treatment engagement, early digital engagement may still be an important predictor of dropout risk, particularly in blended therapy interventions where digital components are more central. In the blended therapy model used in the PsyTOM study, the face-to-face component serves as the primary treatment, while digital modules function as supplementary exercises. Many participants did not complete all digital modules, but were not evaluated as dropouts by their therapists. While this finding may be relevant to other blended therapy interventions, its applicability likely depends on the relative emphasis placed on digital vs. face-to-face components. Future research should further explore how therapist evaluations and digital engagement metrics interact to provide a more comprehensive understanding of treatment dropout risk in blended therapy.

In addition to differences in dropout rates, we examined how different operational definitions of dropout influenced associations with key mental health outcomes, including depressive symptoms, anxiety, and satisfaction with life. Compared with completers, participants who had dropped out reported worse mental health outcomes post-treatment at least descriptively. This is in line with prior research that showed that treatment completion is associated with better outcomes20,21. Under definitions 4 (75% digital & therapist) and 5 (50% digital & therapist), there were significant differences in anxiety, depressive symptoms, and satisfaction with life between dropouts and completers, and all other definitions showed descriptive trends. These findings suggest that the association between treatment dropout and mental health outcomes remains consistent, regardless of the dropout definition applied. Higher baseline symptom severity and lower quality of life are associated with increased dropout rates23,45. Our findings align with this pattern: Across all dropout definitions, participants who dropped out had higher levels of depressive and anxiety symptoms and lower satisfaction with life at baseline compared to completers. These differences were primarily descriptive for definitions 1 (therapist evaluation), 2 (digital only), and 3 (100% digital & therapist) but reached statistical significance under definitions 4 and 5. Thus, participants with higher initial symptom burden may have been at greater risk of disengaging from treatment. This reinforces the importance of early identification of at-risk individuals to enhance retention in blended therapy interventions. The inconsistencies in associations with outcomes across dropout definitions present a challenge for ensuring the validity and generalizability of findings in blended therapy research. Depending on how dropout is defined, the strength of associations with treatment outcomes may vary, leading to differing interpretations of dropout’s impact on symptom change. One potential explanation is that participants who ultimately drop out may experience lower early treatment benefits, contributing to disengagement46. However, alternative explanations, such as external life factors or low initial motivation, should also be considered47,48,49. To better understand dropout dynamics in blended therapy, future research should adopt both quantitative and qualitative approaches. While meta-analyses can summarize research results on early engagement patterns and symptom trajectories as predictors of dropout risk, more qualitative studies are needed to explore patient- and therapist-reported reasons for disengagement. Existing work, such as that of Wilhelmsen et al.50 and Wells et al.51, has provided valuable insights into patient-reported reasons for dropout, for example, time constraints and attitudes towards treatment. However, studies that integrate both patient and therapist perspectives are still lacking. A next step in addressing this gap is to integrate both perspectives (see preregistration, https://doi.org/10.17605/OSF.IO/G84K9) to understand the different processes that lead to disengagement from blended therapy.

After identifying various usage patterns for the digital component in the PsyTOM data, our results show some variation in dropout classification based on usage patterns across different definitions. Definitions that relied more strongly on the digital component tended to classify a larger number of participants as dropouts, even among those with moderate or high usage levels, which suggests that relying on usage data alone may again overestimate dropout risk in our data. This points to a broader research question in the context of digital and blended therapy, that goes beyond the concept of treatment dropout: How can adherence to the intervention be effectively measured? The concept of adherence refers to “following the prescribed recommendations52.” In the context of digital interventions, Donkin et al.53 describe adherence as “the degree to which the user followed the program as it was designed”, aligning with the concept of intended use. As of right now, there are still many questions surrounding adherence: one important issue in this regard is understanding the right dosage in blended therapy, in order to establish a criterium for intended use. Sieverink et al.54 documented in a systematic review that only six studies reported a justified operationalization for the intended use of an intervention. Though most studies operationalized adherence as “the more use, the better”54. This idea seems to play into some operational definitions of treatment dropout, too, classifying participants as dropouts if they don’t complete 100% of the provided content31,32,37. However, the justification for this specific percentage remains unanswered. As described, dropout may involve qualitative differences, such as symptom improvement or the achievement of treatment goals55. However, these aspects are typically disregarded when applying criteria such as completing “100% of the provided content”. In their systematic review, Ferrao Nunes-Zlotkowski et al.1 identified the “good enough effect” as a barrier to intervention engagement in blended therapy. This phenomenon describes patients discontinuing therapy because they feel better or believe further intervention is unnecessary1,55. Therefore, not all dropouts are inherently negative. Especially in digital components, where flexibility and autonomy are higher, disengagement may reflect informed choices rather than failure. This underscores the importance of future research that examines not only when and how dropout occurs, but also why, ideally by incorporating patient-centered approaches. Additionally, a more fine-grained reporting is needed, including the timing of dropout within the intervention and concurrent symptom trajectories, to better understand the diverse reasons behind disengagement.

The application of the results of the scoping review to our real-world data revealed varying dropout rates, highlighting the importance of reporting consistent and comparable definitions across studies. Due to heterogeneity in blended therapy interventions, it might be difficult to recommend a universally applicable definition for all studies. However, our findings indicate that both digital and face-to-face components should be considered, and therapist evaluations included when reporting treatment dropout. To improve comparability and interpretability across studies, we recommend reporting dropout rates separately for digital and face-to-face components, along with the operational definition applied in the study context. Additionally, specifying the blended therapy classification according to Ferrao Nunes-Zlotkowski et al.1 and including the intended use of intervention components would improve transparency. Reporting the expected level of engagement and including dropout timing with symptom tracking would help differentiate between treatment dropout and termination of treatment due to symptom improvement. Adding this information would provide a more nuanced understanding of treatment dropout in a specific study context and facilitate comparisons across studies, even if the blended therapy setup differs. Looking ahead, as the boundaries between digital and face-to-face psychotherapy become increasingly fluid and interactive and will potentially be shaped by AI-supported components, static, percentage-based definitions of dropout may no longer be sufficient. We see this work as a starting point to encourage more flexible and context-sensitive operationalizations of dropout in future research.

This study highlights the significant heterogeneity between different blended therapy setups, such as alternate versus case-by-case and integrated versus sequential approaches or those focusing primarily on face-to-face versus digital components. While comparisons across these setups could provide more nuanced insights into dropout differences, this was beyond the scope of the present study. One limitation of this study is the incomplete dataset regarding information about the dropout status of the participants. At the time of the end of data collection, most participants (approximately 70%) had not yet completed their treatment, and only data from participants who had either completed or dropped out were included. Of this subset, therapists reported a higher proportion of dropouts compared to completers. This is why, compared to dropout rates found for face-to-face psychotherapy (16%–30%) and internet-based interventions (15%–65%), the dropout rates for the different operationalizations in our study appear quite high (48%–92%). Therefore, this dataset is not suitable for such a comparison of dropout rates between different settings. Given the large amount of missing data, imputation for treatment dropout was not feasible. Additionally, we had no information about the number of intended face-to-face sessions. As a reference for treatment dropout, we had to rely on the therapist-rated item, Was the therapy ended earlier than originally planned? (Yes/No). Although this was the only available indicator for dropout, it does not explicitly distinguish between treatment dropout and early termination due to the patient reaching treatment goals earlier than expected. Future research could benefit from more detailed measures to better capture these distinctions. Furthermore, the model fit for the usage data clusters was suboptimal, suggesting that engagement in blended therapy is more complex than the variables used in this study could capture. While the usage variables included were informed by existing literature23,24,25, and like other authors24,56 we found groups of minimal users and moderate users, future studies could benefit from exploring advanced methodologies, such as machine learning, to identify more relevant variables for informing usage patterns in blended therapy interventions. Lastly, the change in analysis method from survival analysis to logistic regression limits the interpretability of our findings with respect to our preregistered research question. In particular, we are unable to make statements about the timing of dropout, only about its association with usage patterns. Moreover, the interpretation of the logistic regression results is limited, as several models showed signs of separation and numerical instability. Overall, both the cluster analysis and the subsequent logistic regressions are exploratory in nature and should be interpreted with caution.

Treatment dropout definitions in blended therapy are heterogeneous. Including the therapist evaluation may offer a valuable perspective beyond considering dropout from the face-to-face or the digital components in blended therapy in isolation, especially if the digital components are integrated into a routine care setting. To draw comparisons with other studies, we recommend reporting the specific operational dropout definition applied in the study context, as well as separate dropout rates for digital and face-to-face components. We further recommend including a classification of the blended therapy approach, as well as information about the intended use and dropout timing when reporting treatment dropout rates for blended therapy interventions.

Methods

Scoping review

For aim 1, we conducted a literature review, following the methodology of a scoping review57 (PRISMA-ScR) to understand how treatment dropout in blended therapy for mental health is defined in the literature. This approach clarifies and identifies key concepts or definitions (here: treatment dropout) related to a concept (here: blended therapy) based on studies identified through a systematic literature search58. The literature search was conducted in January 2025 in PubMed with the search string: (“blended”) AND (“mental” OR “psychological” OR “psychotherapy”) AND (“dropout” OR “adherence” OR “attrition” OR “termination” OR “engagement” OR “usage”). Following the PCC criteria57 (participants, concept, context), two individuals screened the results separately. We included primary research (including study protocols) on individuals receiving psychotherapy (participants) that was delivered in a blended treatment approach (context, digital component and face-to-face component) and that reported treatment dropout (concept).

Practical application of derived dropout operationalizations with real-world data

We then applied the derived operationalization (aim 1) to the data from the PsyTOM trial (aim 2).

The PsyTOM trial, registered in the German Clinical Trials Register (DRKS00028536) on 07.06.2022, investigated the effectiveness of blended therapy (BT) compared to treatment as usual (TAU) in a naturalistic routine care setting in Germany. In BT, psychotherapists could use TONI, a comprehensive, transdiagnostic, and transtheoretical internet-based toolkit composed of modular therapeutic content. Psychotherapists working in routine outpatient care under public health insurance were eligible to participate if they had internet access in their practice. Patients were enrolled in the study by their therapists based on the following inclusion criteria: age over 18, sufficient German language proficiency, internet access, and the ability to read and write. All participants received routine psychotherapy provided by licensed psychotherapists. Treatment lengths and procedures varied in accordance with the naturalistic design, and no restrictions were placed on additional healthcare utilization. A more detailed description of the study design is available in Schaeuffele et al.59.

In the (BT) condition, psychotherapists could flexibly integrate TONI into routine psychotherapy. TONI is a digital toolkit comprising twelve transdiagnostic, transtheoretical modules (see Supplement), as well as self-monitoring tools. Therapists could individualize treatment by selecting specific modules, chapters, and usage formats, without predefined instructions regarding frequency, duration, or implementation. Patients could request access to modules, and therapists had the option to view module inputs, enable messaging, and incorporate other TONI components, such as symptom monitoring, into the therapeutic process. Additionally, participants and therapists could exchange messages. A more detailed description of the intervention is available in the corresponding publications59,60.

Sample

We investigated the different operationalization in the BT arm of the PsyTOM trial RCT (n = 583). Due to the naturalistic nature of the trial and the absence of a fixed treatment duration, many patients in the BT arm were still receiving psychotherapy at the end of the data collection period. Thus, we included only those BT participants who had either completed or dropped out of treatment during the study period (n = 147). The hypotheses and planned analyses were preregistered on the Open Science Framework (https://doi.org/10.17605/OSF.IO/G84K9).

Primary outcome

The primary outcome was treatment dropout. As we aimed to compare definitions, we conducted all analyses using different operationalizations of dropout as identified through the scoping review. As a reference, we used the therapist-rated item from the PsyTOM study, Was the therapy ended earlier than originally planned? (Yes/No).

Covariates

Depressive symptoms were assessed with the 8-item German version of the Patient Health Questionnaire61,62,63 (PHQ-8). Anxiety symptoms were assessed with the 7-item German version of the Generalized Anxiety Disorder Scale-764,65 (GAD-7). Satisfaction with life was assessed with the 5-item German version of the Satisfaction with Life Scale66,67 (SWLS). Participants completed questionnaires at baseline (t0), after 6 weeks (t1), after 12 weeks (t2) and after 6 months (t3). For the analysis, we will use the assessment at t0 and t3.

Statistical analysis

We performed all analyses using RStudio (Version 2023.9.0.463). During the pre-processing of the data, missing data for mental health outcomes were handled by using multiple imputation. For a full overview of data preparation, see Schaeuffele et al.68 and the corresponding OSF project (10.17605/OSF.IO/PRE87). The code for all following analyses is provided on OSF (https://osf.io/fh7ku/?view_only=b17783ddb00d4f4f9cd1e1c351d92d67).

To address RQ1 (compare dropout rates for identified definitions), we used descriptive analyses of treatment dropout rates. We calculated treatment dropout rates (absolute and relative frequencies) for therapists’ evaluation of dropout, and by applying the operational definitions identified from the literature review. Due to violation of the normality assumption, we relied on Wilcoxon-Rank-Test to evaluate differences in depressive symptoms, anxiety symptoms, and satisfaction with life between participants who dropped out and those who did not (RQ.2). As the Wilcoxon test is not directly supported by Rubin’s rules for pooling results across multiple imputations, we followed the recommendation by Eekhout et al.69 and calculated the median of the p-values obtained from each of the 30 imputed datasets as an overall pooled estimate of significance. To address the risk of false positives due to multiple comparisons across different dropout definitions and time points, we applied a Bonferroni correction to the pooled median p-values. All tests were conducted separately for each time point (baseline and follow-up), and for five operational definitions of dropout in a total of ten comparisons. Bonferroni-adjusted p-values were then used to determine statistical significance. The correction was applied by dividing the conventional alpha level (0.05) by the number of comparisons (10), yielding an adjusted significance threshold of p < 0.005.

To address RQs3 (usage patterns and association with dropout), we used K-means clustering to identify distinct usage patterns among participants based on their usage metrics (number of logins, total duration of logins, mean duration of logins, number of assigned chapters, number of requested modules, number of messages from therapist to patient, number of messages from patient to therapist, completion of assigned content). Instead of conducting survival analyses as preregistered, we relied on logistic regressions to examine the association between usage patterns and treatment dropout, with the different usage patterns (clusters identified in RQ3.1) as predictors. This change in analysis method was necessary due to our inability to identify a reliable time-to-event variable, as treatment dropout was only recorded at 6 months follow-up (t3) or had to be reported manually by the therapist. The results of this analysis are reported in detail in the supplement.