Introduction

Mental health disorders represent a prevalent set of clinical conditions associated with substantial personal and economic burdens. However, despite their prevalence and impact, there exists a conspicuous deficit in the provision of effective treatment1,2,3,4. Across Europe, estimates suggest that 15–40% of the population experiences some form of mental disorder, yet fewer than one-third of these cases receive treatment that meets the established standards of adequacy5,6,7,8,9.

One reason for the lack of adequate treatment of mental disorders are structural supply issues, for example caused by a shortage of mental healthcare providers in more rural areas10. Furthermore, negative attitudes towards mental health treatments hinder seeking help especially in mild to moderate cases11. Finally, prompt access to mental health treatment is paramount for its efficacy, yet mental health facilities and specialists often impose prolonged waiting periods spanning several months12. These extended waiting intervals amplify the economic strain of mental disorders13, exacerbate clinical manifestations14,15, diminish treatment adherence, and elevate dropout rates16,17. In summary, providing adequate mental health treatment is complicated by a variety of structural issues leading to several other problems like economic and patients’ personal costs.

E-mental health (EMH) interventions aim to provide adequate treatment of mental health disorders through technological means and channels, such as app- or web-based systems, text messages, videos, or digital monitoring. Here, the term EMH is used to describe any digitally delivered interventions with the goal of improving mental health outcomes. Due to the easy accessibility of EMH products, such interventions have many advantages: They can 1) fill structural supply gaps for rural areas, 2) bridge long waiting times for in-person mental health treatment, and 3) provide additional anonymity for those concerned about stigmatization18,19. Thus, EMH tools have the potential to be a viable method to overcome the various issues hindering adequate mental health treatment.

In outpatient settings, EMH interventions are effective tools to treat mental disorders according to several meta-analyses, including the treatment of anxiety and depression20,21,22 eating disorders23, posttraumatic stress24, or work-related stress25. Furthermore, EMH interventions find predominantly positive acceptance from both patients and mental health practitioners26,27,28. Thus, EMH interventions are viable and accepted tools in the treatment of mental disorders in outpatient settings.

Inpatient treatment signals an especially high need for timely and adequate intervention and is indicated for cases considered too severe for outpatient treatment29. Inpatient interventions can profit from supportive EMH procedures either to bridge waiting times, to blend with in-person interventions, or to ensure stabilization and relapse prevention in aftercare treatments. Especially the implementation of post-treatment aftercare improves the chances of a favorable and sustained development30,31,32. Thus, EMH treatment can be an important factor in the long-term success of inpatient treatments. Adequate aftercare enhances rehabilitation according to several reviews and meta-analyses33,34,35. Randomized controlled trials (RCT) exist on EMH treatment as add-ons to regular inpatient interventions36 and aftercare37. Furthermore, a systematic review38 found support for the efficacy of EMH aftercare treatments but was limited by the small number of studies. Yet as of now, there are to our knowledge no meta-analyses on the use of EMH in inpatient treatments, nor are there meta-analyses on EMH for inpatient aftercare. In addition, the systematic review on EMH inpatient care is several years old and does not incorporate the more recent research38.

The present study seeks to summarize the findings of previous RCTs on EMH treatments in inpatient settings in a meta-analysis. In addition, the risk of bias of the studies are assessed39. Specifically, this meta-analysis seeks to investigate 1) the total effect of EMH treatments on mental health outcomes in inpatient settings; 2) the effects of EMH treatments, divided into interventions and aftercare treatments; 3) the effect of EMH treatments in relationship depending on the mental health disorder; 4) the effect of type of therapy on EMH efficacy; 5) the effects of follow-up measures on EMH interventions are investigated to test long-term effects; and 6) the assessment of bias of the currently published RCT literature. Furthermore, post-hoc analyses were conducted to investigate 1) the role of EMH medium (e.g., app-based, web-based, SMS-based) on EMH treatment efficacy, and 2) the effect of the type of control group on EMH efficacy.

Results

Selected literature

A total of 26 research studies containing 123 effects and a sample size of n = 6112 (intervention group = 3041, control group = 3071) were included. A summary of the included studies is shown in Table 1. Five studies used blended treatment during inpatient stay while 21 studies conducted post-inpatient aftercare treatment. The most common patient groups (according to the number of studies) were eating disorders (k = 7) followed by mood disorders (k = 6), transdiagnostic (k = 4), psychotic disorders (k = 3), return to work treatments (k = 2), mental comorbidities with somatic disorders (k = 2), anxiety disorders (k = 1), and substance abuse (k = 1).

Table 1 Overview of studies
Full size table

Thirteen out of 26 studies utilized a passive control group for which participants did not receive any type of active treatment (e.g., waiting list); eight studies used an active control group with an active treatment alternative to the EMH treatment (e.g., aftercare e-mail reminders for mental health tools, psychoeducation, rehabilitation activities, or psychosocial support such as counsling); five studies used an active control group to which the EMH treatment as added to in the intervention group; finally, one study used both active and passive control groups.

Three studies used SMS-based EMH interventions. Eighteen studies used web-based interventions such as SUMMIT40, IN@41, HEINS42, Deprexis37,43,44, GSA Online45, and EDINA46. Five studies used app-based tools such as MCT & More47 and Mindshift36. Each specified tool was used by only one study except for Deprexis, which was used in three studies.

Out of all included studies, 17 were conducted in Germany, two in Sweden and USA respectively, and one in Hungary, Iran, Finland, Canada, and Australia, respectively.

Study search and selection flow is depicted in Fig. 1.

Fig. 1: Study selection process.
figure 1

Flowchart depicting study selection. The first selection of 726 studies was found in five different databases. Following the evaluation by exclusion criteria, 30 studies were selected for risk of bias evaluation. After four studies were excluded for risk of bias, 26 studies were included in the meta-analysis. EMH e-mental health, RCT randomized controlled trial.

Full size image

Risk of bias assessment

Four studies were rated as high risk of bias and excluded from the analysis. Out of the remaining studies, 19 were rated as medium risk of bias and seven as low risk of bias. Among the most common bias concerns were asymmetrical attrition rates in control and intervention groups, high attrition rates with unclear reasons, alternating allocations (rather than random allocation), and inadequate information on blinding procedures (e.g., no specifications for statements such as “the procedure was blinded”). All four high risk studies were excluded also due to unclear, high, or uneven attrition rates between groups.

The risk assessment is summarized in Table 1.

Publication bias analyses

Preliminary analyses were conducted to test for publication bias using funnel plot and p-curve analyses.

Funnel plot analysis

Funnel plots with effect sizes plotted against standard errors are depicted in Fig. 2a.

Fig. 2: Funnel plots of included studies.
figure 2

Funnel plot across all effects (a) and after excluding studies with the largest standard errors (b). The funnel plots depict the effect sizes (Hedges’ g) plotted against the studies’ (reversed) standard errors. Asymmetry analyses found a significant asymmetry (a), but not when excluding four effects with the largest standard errors (b). As the effect size remains unaltered, the results do not indicate publication bias.

Full size image

Publication bias would express itself in a preference for publishing significant compared to non-significant results. Because smaller studies need a higher effect size to reach significant effects compared to larger studies, an asymmetrical distribution with more smaller studies with larger effect sizes compared to larger studies would indicate publication bias. A regression analysis using standard error as a predictor of effect sizes suggests significant asymmetry (z = 3.6, p < 0.001, i = 123 effects).

Publication bias can be controlled by excluding the smallest studies48. After excluding studies with the largest standard errors (i = 4 effects, 3% of the total effects), another regression test showed no indicators of funnel plot asymmetry (z = 1.89, p = 0.058, i = 119, Fig. 2b). The total effect size remained unaltered (g = 0.33 [0.2, 0.46], p < 0.001), showing that the publication bias correction did not impact the results. Thus, the results do not indicate publication bias.

P-curve analysis

P-curve analysis was used to investigate publication bias further. A right-skewed p-curve would indicate an existing effect while a left-skewed p-curve would indicate publication bias or p-hacking as the latter curve would result from a tendency to acquire significant p-values of just below .05 despite the absence of a true effect indicated by a higher rate of results with smaller p-values. The p-curve is depicted in Fig. 3.

Fig. 3: P-curve of included studies.
figure 3

P-curve including the meta-analysis’ 109 significant effects, compared to a hypothetical null-effect curve and a hypothetical 33% power effect curve. Analysis shows a significant right skewedness, indicating the existence of a true effect.

Full size image

Out of all effects, i = 109 effects provided a significant effect size of p < 0.05, out of which i = 108 showed a p-value of p < 0.025. The significant right-skewedness test (pbinominal < 0.001, zFull = -65.51, zHalf = -64.65, pHalf < 0.001) suggested the existence of a true effect. Furthermore, the non-significant flatness test (pbinominal = 1, zFull = 64.13, zHalf = 65.88, pHalf = 1) provided no indicators that a true effect is not present.

In total, both funnel plot and p-curve analysis show no indicators of publication bias or p-hacking, and that the observed effect is true.

Effect size analysis

A summary of all results is presented in Fig. 4.

Fig. 4: Synthesized effect sizes.
figure 4

Effect sizes, confidence intervals, and number of effects across conditions, controlled for study. Note. Total = across all data; relevant effects = only effects of measures relevant to the mental condition are included; blended = treatment with EMH blended with inpatient care; aftercare = treatment after inpatient care. CBT cognitive-behavioural therapy, PD Psychodynamic therapy.

Full size image

Total effect

Total effect size with study as random effect revealed a significant positive effect of EMH intervention (g = 0.3 [0.2, 0.39], p < .001, k = 118). When only including effects of measures relevant to the mental disorder symptoms (e.g., Beck depression scores for depressive disorder patients) and removing measures not directly related to the mental disorder’s symptoms or clinical outcomes (e.g., social support, self-esteem), effect size increased (g = 0.36 [0.22, 0.5], p < 0.001, k = 83).

As expected given the variety of study designs and conditions, significant heterogeneity was observed for both the total effect (Q(117) = 408.25, p < .001) and when including only clinically relevant outcomes (Q(82) = 647.91, p < 0.001).

Treatment type

By-treatment type analysis revealed that both blended interventions during inpatient stay (g = 0.42 [0.27, 0.58], p < 0.001, k = 19) and aftercare treatments following inpatient stay (g = 0.29 [0.24, 0.34], p < 0.001, k = 99) showed significant effects.

Mental condition

By-condition analysis revealed significant effects of EMH interventions for eating disorder (g = 0.19 [0.07, 0.32], p = .003, k = 17), mood disorder (g = 0.38 [0.28, 0.49], p < 0.001, k = 22), psychotic disorder (g = 0.43 [0.27, 0.58], p < 0.001, k = 10), return to work (g = 0.21 [0.12, 0.3], p < 0.001, k = 24), and transdiagnostic patients (g = 0.4 [0.31, 0.49], p < 0.001, k = 34). No significant effects were found for anxiety disorders (g = 0.35 [−0.22, 0.93], p = 0.23, k = 3), mental comorbidity with somatic disorders (g = 0.19 [−0.02, 0.39], p = 0.072, k = 6), and substance abuse (g < 0.01 [−0.27, 0.28], p = 0.964, k = 2).

Type of therapy

Analysis by type of therapy revealed significant effects for cognitive behavioural therapy (CBT)-based treatments (g = 0.26 [0.18, 0.34], p < 0.001, k = 43) and psychodynamic (PD) treatments (g = 0.35 [0.27, 0.43], p < 0.001, k = 39).

Follow-up stability

To investigate potential effects of measurement time (e.g., a decrease of intervention efficacy for longer intervals after treatment), a linear mixed model with measurement time as the fixed effect and study as the random effect for effect sizes was calculated. Results showed no significant effect of measurement time (t(61) = −00.97, p = 0.337), showing no indication that the strength of the treatment effect is influenced by the time passed between intervention and measurement.

Post-hoc analyses

Post-hoc analyses were conducted to investigate differences between EMH medium/channel and effects of type of control group. EMH medium analysis revealed significant effects for EMH tools implemented as web-based tools (g = 0.32, CI [0.25, 0.37], p < 0.001) and multimedia interventions (g = 0.79, CI [0.29, 1.29], p = 0.002). Effects for app-based and SMS-based EMH tools were not significant. However, multimedia was used by only one study43. The only specific EMH tool used by multiple studies was Deprexis, which showed a significant effect (g = 0.61, CI [0.46, 0.77], p < 0.001).

Control group analysis revealed that EMH interventions significantly improve mental health outcomes compared to passive controls (no active treatment; g = 0.29, CI [0.19, 0.39], p < .001), active controls (active treatment alternative to EMH; g = 0.32, CI [0.24, 0.4], p < .001), and active controls to which the EMH intervention was added to in the intervention condition (g = 0.3, CI [0.22, 0.39], p < 0.001). Thus, EMH interventions show efficacy compared to active treatments and usual and when used in addition to usual treatments.

Few studies focused on patients affected by anxiety disorders, complicating interpretations of the presented results. Meanwhile, studies with transdiagnostic patients often included patients with anxiety disorders and measured anxiety symptoms (e.g., GAD-7). To gain further insight into the effects of EMH treatment on anxiety disoders, an additional post-hoc analysis has been conducted measuring the efficacy of EMH treatment on anxiety symptoms specifically. The analysis showed a significant effect on anxiety symptoms (g = 0.39, CI [0.18, 0.59], p < 0.001).

Discussion

EMH procedures have shown to be a viable tool for the treatment of mental disorders, yet research on EMH in inpatient settings is relatively sparse. The current work presents, to our knowledge, the first meta-analysis providing evidence for the efficacy of EMH in inpatient treatment and aftercare. We found a significant small effect of EMH treatment (g = 0.3).When focusing on disorder symptoms and clinically relevant outcomes, the effect size is further increased (g = 0.36), signalling that EMH procedures are suitable as interventions tailored to mental disorders in inpatient settings. A preliminary analysis further found no indicators of publication bias or p-hacking within the literature.

The effect remained significant when dividing the studies into the common implementation types of EMH, first when blended with in-person inpatient treatment (g = 0.42) and second as an aftercare treatment following inpatient intervention (g = 0.29). The majority of studies (21 out of 26) used an aftercare setting with the goal to ensure stabilization and prevent relapse of inpatient cases. Inpatient cases tend to be more severe compared to outpatient cases, with worse post-treatment outcomes when not sufficiently supported by aftercare following discharge30,31,32. The present results suggest that EMH can provide such an effective tool, closing an important mental health supply gap.

By-disorder analysis found that EMH was especially effective for psychotic disorders (g = 0.42), transdiagnostic patient groups (g = 0.4), and mood disorders (g = 0.38). The results are comparable to meta-analyses finding small yet significant effects of EMH in outpatient settings for mood disorders22, providing evidence that the effects are comparable to inpatient settings.

The positive effect of EMH treatment for psychotic disorders is surprising given that EMH interventions may worsen psychotic patients’ concerns about technology and being recorded due to psychopathological paranoid tendencies49. Furthermore, the effect contrasts the negative outcomes reported in studies investigating psychotic patients42,50,51. While the results complement previous research on the effectiveness of EMH outpatient treatments for schizophrenia and psychosis52, the usage of EMH interventions for psychotic disorders remains not well developed, and their efficacy cannot be reliably estimated with the current research. For inpatient settings, SMS-based aftercare reminders for medication adherence did not improve patient outcomes50. The HEINS web-based aftercare program containing multiple modules (including psychoeducation, crisis plans, contacts to psychiatrists, and supportive monitoring) meanwhile showed positive user acceptance and adherence42, and Horyzons, an online social therapy aftercare program containing multiple features (including psychoeducation, skill development support, peer-to-peer conversations, and expert support), improved patient employment and reduced emergency room visits compared to usual care51. Given that both Horyzons and HEINS are interactive support units containing multiple modules, the results suggest that more extensive EMH treatment is needed to ensure aftercare of patients with psychosis. Patients with severe illnesses such as psychosis may not be able to effectively utilize digital health tools. EMH tools are to be used with caution when treating patients with psychosis and should be used in addition to in-person treatment instead of an alternative.

For outpatient treatment, the efficacy of EMH treatment for anorexia nervosa is not well researched, potentially due to the severity of the disorder and the presumed necessity for face-to-face treatment by clinicians21. Out of seven studies investigating eating disorder patients, four focused mainly on bulimia nervosa41,46,53,54. When excluding a follow-up study55 and a pilot RCT56, only one proper RCT study focused on anorexia nervosa57. Although the initial results are promising, caution should be taken when transferring the results onto patients with anorexia nervosa given that the disorder leads to severe consequences including somatic complications that may be insufficiently tracked and treated through digital means.

Meanwhile, no significant effects for anxiety symptoms, comorbidity with somatic disorders, or substance abuse disorders were found. However, only one study investigated anxiety symptoms36. Meanwhile, multiple studies with transdiagnostic patient groups included patients with anxiety disorders37,39,58. A post-hoc analysis focusing on anxiety symptoms revealed a significant effect (g = 0.39). Inpatient treatment is typically not indicated for anxiety disorders, which may explain the low number of studies. Given that EMH interventions are effective in treating anxiety disorders in outpatient settings22, and that the post-hoc analysis revealed a significant improvement in anxiety symptoms, the current negative findings on EMH inpatient treatment for anxiety disorders are to be interpreted with caution. A similar caution can be expressed for the negative result on substance abuse patients, which has been investigated by only one study59. Furthermore, future research ought to differentiate effects of EMH for different anxiety diagnoses in inpatient care, as EMH outpatient treatment effectiveness has been found to differ across anxiety disorders22.

Analysis by type of therapy revealed the effectiveness of both CBT- (g = 0.26) and PD- (g = 0.35) based interventions, showing that EMH treatment is effective when based on either of these types of psychotherapy

Finally, the result that observation period did not affect outcomes indicates that EMH-based treatment effects do not deteriorate with time passed after treatment, indicating the long-term stability of the effects. However, the latest measurement used in this analysis was 24 months after treatment. Hence, results cannot be interpreted for longer periods.

In general, the meta-analysis shows the efficacy of EMH treatment across different mental health disorders and types of therapy. Hence, mental health treatment can profit from integrating EMH into the patient journey. Given that EMH add-on also significantly improves outcomes compared to a regular active control group (g = 0.3), adding EMH to regular practices can improve overall treatment outcomes. Since treatment as usual tends to be minimal for aftercare treatment, EMH can facilitate long-term improvements and remission prevention following inpatient treatment since other aftercare practices are lacking or minimal. Especially web-based EMH treatment has been shown to be effective throughout multiple studies (g = 0.32) compared to SMS- or app-based approaches. Hence, practitioners may use EMH tools both as additives and as alternatives to regular treatment, and especially for aftercare following inpatient treatment. Web-based EMH tools have shown efficacy in most studies.

The meta-analysis is limited by the small number of studies especially for subgroup analyses, as some subgroups (e.g., anxiety disorder or substance abuse patients, or whole health approaches) only include a single study each and can thus not be properly interpreted. Although a total effect was found with a sufficient number of trials, further RCT research is needed to conduct more conclusive meta-analyses for subgroup-related research areas.

The small number of studies precludes further interesting analyses relevant to the design and implementation of EMH methods. For example, a previous meta-analysis on outpatient settings found that specific EMH methods were more effective for certain disorders (e.g., chatbots for depression, mood monitoring features for anxiety). Such research questions may be tackled in future meta-analyses when an adequate number of RCTs have been conducted. Meta-analyses and reviews are generally limited by the terms used and search outputs when conducting literature searches. Even though two literature searches (February 2024 and July 2024) were done for this meta-analysis, it may still not include all relevant literature. Furthermore, this meta-analysis was not preregistered. However, all relevant documents are publicly available.

Specific neuropsychological and cognitive measures were excluded from this meta-analysis to focus the research on explicit mental health outcomes. However, mental health deficits often co-occur with cognitive deficits, for example in memory, concentration, or problem-solving tasks. Although disorder-specific questionnaire measures encompass the measurement of such deficits, future research can focus on the effect of EMH interventions for the improvement of cognitive skills in patients affected by mental health disorders specifically.

Out of all 26 included studies 20 were conducted in Western or Northern Europe (17 in Germany, two in Sweden, one in Finland), three were conducted in North America (two in the USA and one in Canada), one in Australia, one in Hungary, and one in Iran. Research from other regions, such as Africa or East Asia, was absent. This may be due to differences in healthcare systems in different regions, and treatments alternative to inpatient treatment for more severe health cases. Thus, the results of this meta-analysis are mainly derived from studies conducted in countries with populations majorly of European descent. In order to generalize the reported findings, future research may aim to investigate EMH tools in more diverse populations.

Engagement and adherence are major concerns when applying EMH tools60,61,62,63. Effects of EMH on attrition were mitigated in this analysis by including group attrition effects in the RoB assessment: in fact, all four high risk studies were excluded due to unclear or uneven attrition rates. Engagement can be defined as usage as intended, measured for example through use frequency or completion60. Various included studies excluded participants with low engagement despite completion41 and hence controlled for low engagement. Included studies mostly did not report direct effects on engagement on outcomes. One study found no effect of EMH tool use (assessed via logs) on symptom severity56. Similarly, other studies did not find a correlation between EMH use frequency and symptom improvement47, completed models and symptom improvement64, or differences between high- and low-frequency users59. Meanwhile, the number of completed EMH courses did significantly improve symptoms in patients with anorexia nervosa55. Although there are only few studies and results are not consistent, the results nevertheless indicate that use frequency or intensity does generally not affect the treatment efficacy. Finally, some studies report improved engagement in the intervention compared to a control group65,66, indicating that EMH intervention may improve engagement behaviour. Future research may investigate effects of such engagement when implementing EMH tools.

Given that various measurement outcomes were used and summarized to generalize a wider range of findings, results do not consistently reflect the most clinically relevant outcomes (e.g., remission or relapse rates) which were only reported by six studies for varying mental disorders. Instead, the majority of research studies relied on symptom questionnaires. In total, a majority of the studies included were assessed with some concerns regarding risk of bias. Due to the low number of high-quality research with low bias and large sample sizes, results should be interpreted with some degree of caution. EMH implementations furthermore involve certain risks67 such as a lack of quality standards68, data privacy issues18, or a lack of digital literacy by practitioners19. Despite promising results in this meta analysis, in the context of such risks, more high quality RCT research is necessary for a more rigorous assessment of EMH efficacy.

In conclusion, the results indicate that EMH procedures are an effective tool in the treatment and aftercare of inpatients, especially for psychotic, mood disorder, and eating disorder, and patient groups combining different diagnoses. EMH tools can be used both in addition to in-person treatment and when in-person treatment is not available, e.g., for aftercare. Future research should investigate effects of EMH tools for the inpatient treatment of specific disorders and the relevance of the specific tools used. Larger sample sizes and randomized trials are warranted to substantiate these effects.

Methods

This review was conducted in accordance to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines69 and Cochrane Handbook guidelines for meta-analyses and systematic reviews39.

Literature

Literature search

The literature databases SciencGov, PsycInfo, PubMed, and CENTRAL were searched for published literature. In addition, the ProQuest Database was searched for dissertation theses, and ICTRP and ClinicalTrials were searched for trial result registers.

To aim for high sensitivity according to Cochrane guidelines39, we used multiple search terms in relation to the following topics: e-mental health (digital, online, e-mental health, technology-based, web-based, internet-based, mobile-based), treatment setting (psychotherapy, psychiatric, psychosomatic), inpatient setting (inpatient, ward patient, hospitalized), and experimental design (RCT, randomized controlled trial). The search term used corresponds to the following: (“digital” OR “online” OR “e-mental health” OR “technology-based” OR “web-based” OR “internet-based” OR “mobile-based”) AND (“psychotherapy” OR “psychiatric” OR “psychosomatic”) AND (“inpatient” OR “ward patient” OR “hospitalized”) AND (“RCT” OR “randomized controlled trial”). Two researchers conducted the literature search in February 2024. Literature search was performed in English and German.

A secondary search was conducted in July 2024 by extending the search to use the terms “e-health”, “mhealth”, and “telemedicine”, and using the mesh terms “digital health”, “telemedicine”, “psychotherapy”, “psychosomatic medicine”, and “inpatients” if applicable, for the databases CENTAL and PubMed. The secondary literature search did not yield any new viable studies.

Literature selection

We included research studies providing EMH interventions during inpatient treatment or aftercare following inpatient treatment, and studies investigating psychiatric symptoms co-occurring in patients hospitalized for physical conditions (e.g., stress or depression symptoms in cancer patients). Cluster and pilot RCTs were included as well.

Studies were excluded if they 1) did not investigate the effect of EMH intervention or aftercare methods, 2) did not investigate inpatients (either during or after inpatient intervention), 3) did not investigate mental health measures as treatment outcomes (e.g., only focusing on somatic symptoms or acceptability of the intervention; specific neuropsychological or cognitive outcomes like problem-solving skills were also excluded), 4) were not randomized controlled trials, 5) did not provide sufficient information to extract the relevant data (e.g., outcome measures or sample sizes), and 6) showed a high risk of bias assessed via the Risk of Bias tool (see next section)39. Neuropsychological or cognitive were excluded to focus the meta-analysis on mental health treatment effects. Although cognitive or neuropsychological deficits can be symptoms of mental health disorders, symptom-focused measures of mental health deficits (e.g., depressiveness questionnaires for clinical depression) provide a more discriminative estimation of mental health deficits.

Three independent raters took part in the literature selection. In case of disagreements, the raters discussed the study until agreement was found.

Risk of bias assessment

Risk of bias was assessed using the Cochrane risk-of-bias tool for randomized trials (RoB 2)39. RoB 2 is designed to assess the risk of an RCT’s bias by classifying the level of risk for the following domains: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective reporting. Examples of risk of bias include non-random or semi-random participant grouping (incl. alternating allocation); high, uneven, or unexplained participant attrition between groups; lack of blinding; or unreported discrepancies between the study protocol and study. If no information on a domain was provided, the particular domain was assessed with medium risk.

Domains were rated on three levels: low, medium, or high risk of bias. Research studies with a high risk of bias were excluded from the analysis.

Measurement selection

For the total analyses, only measures related to clinical symptoms and psychosocial performance were included. These include: metric variables of disorder-related incidents (relapses, readmissions, abstinence, admissions); disorder-related symptom severity measurements; general psychopathology, well-being, or quality of life; employment-related measures (when relevant); and specific mental or psychosocial measures expected to correlate with symptom severity (e.g., self-esteem, positive and negative affect, stress). All relevant measures in a study were included in an analysis and controlled by treating study as a random effect.

Variable summarization

To investigate the relevant research questions, studies and measures were categorized by the following system.

EMH treatment type was categorized into either blended intervention (EMH was implemented into the inpatient setting) or aftercare treatment (EMH was provided after completing inpatient setting).

The variable Disorder type was classified into the following categories based on the patient group investigated in the study: anxiety disorders (ICD-10 diagnoses F40 and F41), eating disorders (ICD-10 diagnoses F50) mood disorders (ICD-10 diagnoses F3), psychotic disorders (ICD-10 diagnoses F2), substance abuse disorders (ICD-10 diagnoses F1x.2), or their DSM-5 diagnostic equivalents. A study treating patient groups from different categories was classified as transdiagnostic. A study was categorized as somatic comorbidity if the effects of EMH interventions on mental health outcomes in somatic inpatient groups were investigated (e.g., stress or anxiety symptoms in cancer patients). Finally, the category return to work was used for studies focussing on outcomes related to workplace reintegration following inpatient care.

The variable type of therapy was classified according to the type of therapy the EMH intervention was based on according to the authors. If no type of therapy was mentioned, the variable was valued as not available.

Data extraction

Data was summarized on multiple variables: author, title, year, country, type (aftercare, blended treatment), treated mental disorder, somatic illness (if present), digital method, type of therapy, type of control group (active, passive), outcome measure, follow-up, sample sizes, and outcome results (means, standard deviations, odds ratios, effect sizes). Data was extracted by one rater and verified by two other independent raters. Study characteristics were tabulated according to the planned subgroup analyses. Studies with insufficient data were excluded from the (sub-)analyses.

Data transformation

Hedges’ g was used to report effect sizes as it outperforms Cohen’s d for small sample sizes. Cohen’s d effect sizes and variances were transformed to Hedges’ g values and variances using the following formulas48:

$$g=d\,\left(1-\frac{3}{4{df}-1}\right)$$
(1)
$${vg}={vd}\,\left(1-\frac{3}{4{df}-1}\right)2$$
(2)

When a study reported odds ratio (OR) values, values were first transformed into Cohen’s d using the following formula48:

$$d=\mathrm{log}\left({OR}\right)\frac{\sqrt{3}}{\,\pi }$$
(3)

Cohen’s d values were then transformed into Hedges’ g according to Formula 1.

Data analysis

Heterogeneity was tested and pre-assumed given the variety of setups in research studies and subgroup analyses were therefore decided a priori. Fixed-random effects models were used with study as a random factor. To assess the results’ robustness, the total effect is analysed two times, first using the whole range of data, and second using only outcomes that are clinically relevant (limited to symptom severity and clinical outcomes). Effects’ certainty and confidence were assessed through risk of bias assessment according to Cochrane guidelines and by investigating publication bias using funnel plot and p-curve analyses. The meta-analysis was not preregistered. No protocol is available for the meta-analysis. Confidence was assessed by calculating confidence intervals from standard errors.

Post-hoc analyses were decided after the data was analysed for the main hypotheses. Post-hoc analyses included the effect of EMH medium, the role of control group, and the effect of EMH on anxiety symptoms specifically.