Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Letter to the Editor
  • Published:

Living in the best of all worlds. Despite premises approximating the truth with meta-analyses

International Journal of Impotence Research volume 18, pages 572–573 (2006)Cite this article

Access through your institution
Buy or subscribe

We thank Dr Rosen and Dr Shaw for the stimulating comments on our study. In their letter they claim that our analysis failed to consider several issues that might have an impact on the findings.

Performing a meta-analysis is never completely objective. One has to make some premises as anyone conducting a meta-analysis has to. Therefore, stating that some aspects are neglected is an argument that applies to all meta-analyses and, therefore, always true. In our article, we provided the reader with all information that is needed to be able to form his opinion on the (internal and external) validity of our analyses.

  • It is true that we included trials with different treatment durations. We are aware that discontinuation rates may increase with longer treatment, although most studies adapted an intention-to-treat model that accounts for this bias. To date, there is to our knowledge no evidence from the literature that disease progression influences trial findings in erectile dysfunction, even though it is possible. Although there is some evidence for sildenafil in clinical settings that the full effect of the medication is reached only after multiple dosages,1 we had no evidence from literature that in highest fixed dose trials the trial duration itself has a similar impact.

  • It is also possible that phosphodiesterase 5 (PDE-5) inhibitor-naive patients (as in most sildenafil studies) may show higher response than men with PDE-5 inhibitor experience. However, speculating one could argue that the control group (as well as the drug group) might also have been more responsive to treatment, so that the relative benefit should be approximately the same. We mentioned this aspect in the discussion (labeling it ‘treatment history’).

  • We also agree with Rosen and Shaw that broad standard deviation data imputation can distort findings. However, data extraction from figures or computation from standard error is not the same as data imputation. We imputed standard deviations from other studies in seven cases (only in tadalafil studies). As all the reported standard deviations were largely homogeneous (irrespective of agent), the performed imputation is unlikely to bias the results.

  • Concerning their point about heterogeneity of trials, we report here additionally that even with their proposed threshold of P=0.10, there is no evidence of heterogeneity between primary study results. The funnel plot (along with Higgins' I square) suggests the presence of moderate heterogeneity, but it is evident from the plot, that part of this heterogeneity is to be explained by the agent used.

  • We agree with Rosen and Shaw that sensitivity analyses with the assumption of random effects (REM) could be included if wanted. We certainly judge it obligatory in the cases with unclear real effect they argue with (soj proteins2 or passive smoking3) with a considerable heterogeneity of trial results. We ourselves applied this, for example, in case of herbal essences.4 We argue that the effect of a clearly chemically defined agent as sildenafil or vardenafil with an inconsiderable heterogeneity of trial results is different from those of soj proteins or hypericum extracts. As there is no substantial heterogeneity between findings on the same agent, it is not at all logical that REM leads to increased standard errors. In post hoc analyses, we here report that random effect procedures lead to the same results as fixed effect ones (in the argued cases of sildenafil and vardenafil exactly the same effect sizes and standard errors as in FEM), only the standard error of the pooled effect size of tadalafil studies increases marginally. An obligatory report of sensitivity analyses testing different a priori model assumptions (fixed vs random) is not regulated in the QUality Of Reporting of Meta-analyses (QUOROM) statement either.5

  • We have to correct Rosen and Shaw, the analysis was not ‘limited exclusively to change in EF domain scores from baseline’, rather ‘weighted mean differences between agent and placebo group in the EF domain score after treatment’ (citation from article text) were calculated. The baseline EF domain score for the study Padma-Nathan 2003 was extracted from Figure 2 of the published study report.6

  • Rosen and Shaw's comment on failing to control for possible confounders of treatment effect (such as baseline severity) is very well worth considering. We discussed this already briefly in the article. In a pilot study, we analyzed the effect of several covariates on the treatment effect as measured by post-treatment difference between agent and placebo group.7 We found that several different factors are likely to affect the treatment effect. However, controlling for all of these factors was impossible, and we did not have enough information to decide for one (or some) of them to control. Based on this pilot study, we performed a more sophisticated analysis in sildenafil flexible dose studies to find determinants of treatment effect, and, actually, we found that baseline severity, among others, plays a key role.8

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. McCullough AR, Barada JH, Fawzy A, Guay AT, Hatzichristou D . Achieving treatment optimization with sildenafil citrate (Viagra) in patients with erectile dysfunction. Urology 2002; 60(Suppl 2B): 28–38.

    Article  Google Scholar 

  2. Anderson JW, Johnstone BM, Cook-Newell ME . Meta-analysis of the effects of soy protein intake on serum lipids. N Engl J Med 1995; 333: 276–282.

    Article  CAS  Google Scholar 

  3. He J, Vupputuri S, Allen K, Prerost MR, Hughes J, Whelton PK . Passive smoking and the risk of coronary heart disease – a meta-analysis of epidemiologic studies. N Engl J Med 1999; 340: 920–926.

    Article  CAS  Google Scholar 

  4. Linde K, Berner MM, Egger M, Mulrow C . St John's wort for depression: meta-analysis of randomised controlled trials. Br J Psychiatry 2005; 186: 99–107.

    Article  Google Scholar 

  5. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF . Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Lancet 1999; 354: 1896–1900.

    Article  CAS  Google Scholar 

  6. Padma-Nathan H, Stecher VJ, Sweeney M, Orazem J, Tseng L-J, deRiesthal H . Minimal time to successful intercourse after sildenafil citrate: results of a randomized, double-blind, placebo-controlled trial. Urology 2003; 62: 400–403.

    Article  CAS  Google Scholar 

  7. Berner MM, Kriston L, Harms A . Covariates of treatment effect in trials with oral sildenafil in the treatment of erectile dysfunction – a meta-analytic regression analysis. J Sex Med 2005; 2(Suppl 1): 56.

    Google Scholar 

  8. Kriston L, Harms A, Berner MM . A meta-regression analysis of treatment effect modifiers in trials with flexible-dose oral sildenafil for erectile dysfunction in broad-spectrum populations. Int J Impot Res [E-pub ahead of print: 11 May 2006; doi:10.1038/sj.ijir.3901479].

    Article  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychiatry and Psychotherapy, University Hospital Medical Center, Freiburg, Germany

    M M Berner & L Kriston

Authors
  1. M M Berner
    View author publications

    Search author on:PubMed Google Scholar

  2. L Kriston
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to M M Berner.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Berner, M., Kriston, L. Living in the best of all worlds. Despite premises approximating the truth with meta-analyses. Int J Impot Res 18, 572–573 (2006). https://doi.org/10.1038/sj.ijir.3901489

Download citation

  • Published:

  • Issue date:

  • DOI: https://doi.org/10.1038/sj.ijir.3901489

Search

Advanced search

Quick links