Abstract
Some studies indicate that the blind in clinical trials of the efficacy of antidepressant drugs is less than perfect. It has been suggested that, as a consequence of this incomplete blind, biased raters inflate efficacy and that, in fact, these drugs are relatively ineffective. However, in the literature, we could find no prior attempt to quantify rater bias and, thus, measure its contribution to claims of antidepressant efficacy. We used the distribution of SCL-90 (Symptom Check List) depression scale scores to derive a patient-based effect size, and contrasted this with the clinician-based effect size. We propose the difference between these two effect sizes (patient self-rating and clinician-derived) to be an indirect measure of bias. If patients had a prodrug bias, this method would be invalid. However the response rate from studies with active placebo suggest a patient prodrug bias is unlikely. The effect sizes derived from patient self-ratings are smaller than those derived from clinician ratings. This allows for the possibility that some clinician ratings were biased. However, quantifying the effect of bias suggests that it was insufficient to invalidate the original study conclusions based on clinician ratings, because the proportion of responders, based on patient self-ratings, differed significantly between the two drugs and placebo. Their 95% confidence intervals (CI) did not overlap. This analysis allows that some clinician ratings may be biased. However, the extent of bias appears insufficient to alter conclusions based on clinician ratings regarding efficacy of antidepressant drugs in this trial. Application of our approach in other trials is necessary to establish generalizability.
Similar content being viewed by others
Log in or create a free account to read this content
Gain free access to this article, as well as selected content from this journal and more on nature.com
or
References
Blumenthal DS, Burke R, Shapiro AK . (1974): The validity of “identical matching placebos.”. Arch Gen Psychiatry 31: 214–215
Bunney WE, Brodie KH, Murphy DL, Goodwin FK . (1971): Studies of alpha-methyl-para-tyrosine, L-DOPA and L-Tryptophan in depression and mania. Am J Psychiatry 127: 48–71
Carroll KM, Rounsaville BJ, Nich C . (1994): Blind man's bluff: Effectiveness and significance of psychotherapy and pharmacotherapy. Blinding procedures in a clinical trial. J Cons Clin Psychol 62 (2): 276–280
Daneman E . (1961): Imipramine in office management of depressive reactions (a double blind clinical study). Dis Nerv System 22 (4): 213–217
Derogatis LR, Lipman RS, Covey L . (1973): The SCL-90: An outpatient psychiatric scale. Psychopharmacology Bulletin 9: 13–28
Fahy P, Imlah N, Harrington J . (1963): A controlled comparison of electroconvulsive therapy, imipramine and thiopentone sleep in depression. Neuropsychiatry 4: 310–314
Fisher S, Greenberg R . (1993): How sound is the double-blind design for evaluating psychotropic drugs? J Nerv Mental Dis 181: 345–350
Friedman AS . (1975): Interaction of drug therapy with marital therapy in depressive patients. Arch Gen Psychiatry 32: 619–637
Friedman AS, Granick S, Cohen HW, Cowitz B . (1966): Imipramine (Tofranil) vs. placebo in hospitalized psychotic depressives (a comparison of patients’ self-ratings, psychiatrists’ ratings and psychological test scores). J Psychiatric Res 4: 13–36
Gershon ES, Bunney WE, Goodwin FK, Murphy D, Henry GM . (1971): Catecholamines and affective illness. In Ho BT, McIsaac WM (eds), Brain Chemistry and Mental Illness. New York, NY, Plenum Press, pp 125–161
Goleman P . (1995): Psychologists dispute value of antidepressants. New York Times, November 29.
Greenberg RP, Bornstein RK, Greenburg MD, Fisher S . (1992): A meta-analysis of antidepressant outcome under “blinder” conditions. J Cons Clin Psychol 60 (5): 664–669
Greenberg RP, Bornstein RF, Zborowski MJ, Fisher S, Greenberg MD . (1994): A meta-analysis of fluoxetine outcome in the treatment of depression. J Nerv Mental Dis 182: 547–551
Greenberg RP, Fisher S . (1989a): Examining antidepressant effectiveness: Findings, ambiguities and some vexing puzzles. In Fisher S, Greenberg RP (eds), The Limits of Biological Treatments for Psychological Distress: Comparisons with Psychotherapy and Placebo. Hillsdale, NJ, Erlbaum, p 137
Greenberg RP, Fisher S . (1989b): A second opinion: Rethinking the claims of biological psychiatry. In Fisher S, Greenberg RP (eds), The Limits of Biological Treatments for Psychological Distress: Comparisons with Psychotherapy and Placebo. Hillsdale, NJ, Erlbaum, pp 309–336
Henker B, Whalen CK, Collins BE . (1979): Double-blind and triple-blind assessments of medication and placebo responses in hyperactive children. J Abn Child Psychiatry 7: 1–13
Hill LE, Nun AJ, Fox W . (1976): Matching quality of agents employed in “double-blind” controlled clinical trials. The Lancet 352–356
Hirschfeld R, Keller M, Panico S, Arons BS, Barlow D, Davidoff F, Endicott J, Froom J, Goldstein M, Gorman JM, Guthrie D, Marek RG, Maurer TA, Meyer R, Phillips K, Ross J, Schwenk TL, Sharfstein SS, Thase ME, Wyatt RJ . (1997): The National Depressive and Manic-Depressive Association consensus statement on the under-treatment of depression. JAMA 277: 333–340
Hollister L, Overall J, Johnson M, Pennington V, Katz G, Shelton J . (1964): Controlled comparison of amitriptyline, imipramine, and placebo in hospitalized depressed patients. J Nerv Mental Dis 139: 370–375
Hussain Z . (1970): Drugs in depressive illness. Br Med J 2: 482
Lambert MJ, Doxey RH, Kingston MD, Edwards BC . (1986): Zung, Beck, and Hamilton rating scales as measures of treatment outcome: A meta-analytic comparison. J Cons Clin Psychol 54 (1): 54–59
McLean PD, Hakstian AR . (1979): Clinical depression: Comparative efficacy of outpatient treatments. J Cons Clin Psychol 47: 818–836
Moher D, Pham B, Jones A, Cook DJ, Jadad AR . (1998): Does quality of reports of randomized trials affect estimates of intervention efficacy reported in meta-analysis? The Lancet 352: 609–613
Muñoz RF, Hollon SD, McGrath E, Rehm LP, Van den Bos GR . (1994): On the AHCPR depression in primary care guidelines. Am Psychol 49 (1): 42–61
Quitkin FM, Stewart JW, McGrath PJ, Tricamo E, Rabkin JG, Ocepek-Welikson K, Nunes E, Harrison W, Klein DF . (1993): Columbia atypical depression: A subgroup of depressives with better responses to MAOI than to tricyclic antidepressants or placebo. Br J Psychiatry 163(Suppl 21):30–34
Rabkin JG, Markowitz JS, Stewart J, McGrath P, Harrison W, Quitkin FM, Klein DF . (1986): Assessment of patient and doctor medication guesses in a placebo-controlled trial of imipramine and phenelzine. Psychiatric Res 19: 75–86
Schulz K, Chalmers I, Hayes R, Altman D . (1995): Empirical evidence of bias. JAMA 273 (5): 408–412
Thompson R . (1982): Side-effects and placebo amplifications. Br J Psychiatry 140: 64–68
Uhlenhuth E, Park L . (1963): The influence of medication (imipramine) and doctor in relieving depressed psychoneurotic outpatients. J Psychiatric Res 2: 101–122
Weintraub W, Aronson H . (1963/1964): Clinical judgment in psychopharmacological research. J Neuropsychiatry 5: 65–70
White K, Kando J, Park T, Waternaux C, Brown WA . (1992): Side-effects and the “blind ability” of clinical drug trials. Am J Psychiatry 149: 1730–1731
Wilson I, Vernon J, Sandiler M . (1963): A controlled study of treatment of depression. J Neuropsychiatry 4: 331–337
Young SN, Annable L . (1996): The use of placebos in psychiatry: A response to the draft document prepared by the TRI-Council Working Group. J Psychiatry Neurosci 21 (4): 235–238
Acknowledgements
Supported by USPHS grant MH30906.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Petkova, E., Quitkin, F., McGrath, P. et al. A Method to Quantify Rater Bias in Antidepressant Trials. Neuropsychopharmacol 22, 559–565 (2000). https://doi.org/10.1016/S0893-133X(99)00154-2
Received:
Revised:
Accepted:
Issue date:
DOI: https://doi.org/10.1016/S0893-133X(99)00154-2
Keywords
This article is cited by
-
Efficacy of selective serotonin reuptake inhibitors in the absence of side effects: a mega-analysis of citalopram and paroxetine in adult depression
Molecular Psychiatry (2018)
-
Randomized controlled trials of antidepressants: clinically and scientifically irrelevant
Debates in Neuroscience (2007)
