Introduction

Glaucoma stands as the primary cause of irreversible and preventable blindness globally1,2. While the pathogenesis of glaucoma is not well understood, reducing intraocular pressure remains the main modifiable risk factor to prevent vision loss, and the evolution of glaucoma therapy has seen exponential growth and numerous clinical trials are in progress3. As randomized controlled trials (RCTs) represent the gold standard methodology for evaluating pharmacological therapy of glaucoma4,5, data reported from these studies must exhibit completeness and consistency across various sources to guarantee the accuracy of evidence-based information suitable for use by both lay and professional populations6,7,8,9,10,11.

The documented problem of selective reporting of clinical trial results introduces apprehensions regarding the reliability of information derived from journal publications in guiding medical decisions12. Clinical trial registration serves as a mechanism to mitigate the influence of dissemination biases13. The reporting of adverse events (AEs) in ClinicalTrials.gov has been legally mandated since September 200914. Enacted through the Sect. 801 of the Food and Drug Administration Amendments Act (FDAAA 801) in 2007 and subsequently operationalized through the Final Rule in 201715, clinical investigators must provide tabular summaries encompassing all anticipated and unanticipated serious adverse events (SAEs), other adverse events (OAEs), and data related to all-cause mortality (ACM)16. The tabular summaries of adverse events (AEs) submitted to ClinicalTrials.gov are expected to mirror those reported in corresponding publications. However, there is strong evidence in other medical fields that publications provide incomplete and inconsistent reporting of AEs, which is also in contradiction to the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline7,17,18,19,20.

Considering the frequent and chronic use of topical antiglaucoma therapy, side effects such as conjunctival hyperemia, ocular stinging and burning, hypertrichosis, and short breath are common21,22. Furthermore, existing surgical interventions for glaucoma are more demanding than pharmacological interventions and come with many complications and side effects23,24. As there is little data on the trustworthiness of AE reporting in clinical trials of interventions for glaucoma, we aimed to evaluate the completeness of reporting of AE in clinical trial testing interventions for glaucoma, both in the ClinicalTrials.gov registry and in the corresponding journal publications.

Results

General characteristics

Out of 138 registered trials that met inclusion criteria (Fig. 1), 79 (57.2%) had a corresponding journal publication. The list of all registered trials and their characteristics are available in the Supplementary Note N1 and the Supplementary Table T1. For the trials included in the study, the median two-year impact factor for journals publishing these trials was 3.4 (IQR 2.2–4.2) (Table 1). Most of the registered trials were phase 3, completed before the FDAA Final Rule on mandatory reporting of adverse events and mortality15 industry sponsored and with last update on Clinical Trials.gov before publication. Most common registered interventions were fixed combination eye drops or prostaglandin analogues eye drops (63%).

Fig. 1
figure 1

Flow chart of the study.

Full size image
Table 1 General characteristics of trials on glaucoma interventions (n = 79) with results both in the ClinicalTrials.gov and the publication.
Full size table

We found discrepancies in reporting adverse events between the registry and corresponding publications for all 79 publications. While the overall reporting of OAEs was similar between the registry and the publications, SAEs were more often reported in the registry. On the other hand, more trials reported OAEs as zero in the registry, but had more different numbers of OAEs reported or number of patients with OAEs in the publications (Table 2). There were no overall statistically significant differences in the reporting of deaths (Table 2).

Table 2 Reporting adverse events for glaucoma trials (n = 79) in the ClinicalTrials.gov and the corresponding publication.
Full size table

Most publications (n = 74, 94%) explicitly stated the number of withdrawals of participants due to AEs, in contrast to data in the registry (n = 60, 75%) (χ2 = 9.629, P = 0.002). For 11 trials (14%) which reported on the withdrawal of patients due to AEs, the numbers in the publication differed from that in the registry; for 8 of these trials, the number in the publication was higher than that in the registry.

Most of the trials involved testing a pharmacological intervention. Only 4 trials were on surgical interventions for glaucoma (NCT04352660, NCT02006693, NCT03654885, NCT01228149). The most common intraoperative complication for surgery trials was bleeding into the anterior chamber of the eye (NCT02006693, NCT03654885), while postoperative adverse events included hypotonia in all trials, hyphema and elevated intraocular pressure (NCT02006693, NCT03654885), device fracture (NCT02006693) or reduced visual acuity (≥ 2 lines at any time) (NCT03654885). In the publication from 1 study (NCT02016898), adverse events were listed, but not their frequency, except for hypotonia.

Differences in SAEs reporting between the registry and publications

The reported numbers of patients with SAE (Table 3) differed in 16 publications (20%) compared to those in the registry. More SAEs were reported than in the publication (69% vs 31%). Among 14 (18%) publications in which SAEs were not mentioned (Table 2), 8 studies reported zero SAEs in the registry, while 6 studies did not report SAEs in the registry. In 26 of the publications (33%) a description of at least one specific SAE was omitted (Table 3).

Table 3 Reporting serious adverse events (SAEs) and other adverse events (OAEs) for glaucoma trials (n = 79) in ClinicalTrials.gov and the publications.
Full size table

Similar to the number of patients with SAEs, we found discrepancies in reporting of specific SAEs, with more SEAs reported in the registry. The descriptions of specific SAEs were different in 38% of the publication (Table 3).

Differences in OAEs reporting between the registry and publications

With regard to OAEs reporting (Table 4), all 79 trials reported OAEs in the registry, but in 69 publications (87%) the reported number of OAEs differed from the corresponding number in the registry. The same was true for the reported number of patients with OAEs (82% trials with the difference). For both, the number of reported OAEs was more often higher in the publication the in the registry. In 21 publications (27%), the description of OAEs was different from that in the registry.

Table 4 Reporting other adverse events (OAEs) for glaucoma trials (n = 79) in ClinicalTrials.gov and the publications.
Full size table

Differences in the description of OAEs between registry and publication were found for a significant number of trials, with most trials reporting OAEs as AEs and with frequency threshold larger in the publications than in the registry. The majority of the trials reported OAEs as AEs only, followed by a combination of the two types of reporting.

The median reported frequency threshold in the registry was 5% (IQR 2–5%) and 1% (IQR 1–3%) in the publications. Moreover, 51 publications (64.6%) had different frequency thresholds, with the majority (n = 45, 88%) reporting lower thresholds than in the registry (Table 4). It was not possible to determine the frequency threshold for 13 publications (16.5%) because it was not reported or specified. There was no statistically significant difference depending on whether the frequency threshold was equal to or smaller than in the publication (χ2 = 0.29, P = 0.58, and χ2 = 0.50, P = 0.48, respectively). The publications also had more specific OAEs reported, with no statistically significant differences regarding the type of reporting (χ2 = 2.789, P = 0.59). Similar results were found for the number of patients with OEAs (χ2 = 3.399, P = 0.49).

Differences in reporting all-cause mortality between the registry and publications

Out of 79 trials, 58 (73%) were completed before and 21 trials (27%) after the FDAA Final Rule15 implementation (Table 5). Overall, more trials reported on deaths in the registry than in the publication. Out of 58 trials completed before the Final FDAA rule, 27 reported all-cause mortality: 16 reported it in the registry and the publication and 18 in the publication. Only 7 studies reported all-cause mortality both in the registry and the publication. In contrast, all trials completed after the FDAA Final Rule reported adverse events in the registry, but this was true only for a third of the corresponding publications.

Table 5 All-cause mortality reporting in clinical trials for glaucoma (n = 79) in Clinical Trials.gov and publications.
Full size table

Discrepancies in reporting deaths were analyzed for 37 (46.8%) published trials that reported deaths in the registry. About a half of these (n = 15) reported deaths in the corresponding publications. Conversely, out of 26 trials that reported deaths in the publication, about a half (n = 15) reported deaths in the registry. The published number of reported deaths differed from the registered number in only 2 out of 19 publications with reported deaths in the registry (1 trial with 1 death more in the registry and another trial with 1 more death reported in the publication.

Discussion

Our study demonstrated significant inconsistencies in how adverse events and all-cause mortality for glaucoma interventions are reported in a trial registry and in journal publications. While there was no difference in overall reporting of OAEs and all-cause mortality, SAEs were reported in the registry for all trials, but not in the publication for about a fifth of the trials. On the other hand, OAEs were more frequently reported as zero in the registry then in the publication, but more trials reported higher number of OAEs and higher number of patients with OAEs in the publications than in the registry. The trials completed after the introduction of the FDAA Final Rule on mandatory reporting of AEs and all-cause mortality in a tabular summary format, all reported on this item in the registry but only a third of the publications.

Our results should be considered in view of several limitations. Firstly, we studied trials registered in a single registry. Whereas there are 20 registries included in the WHO International Clinical Trials Registry Platform25, ClinicalTrials.gov is the largest one, includes many of the trials from other registries, and has high data quality assurance protocols26. We could have missed some of the trials with our search strategy, the number of trials with published results was rather small, and it is possible that we did not identify all related relevant publications. However, the trials included in the study constituted 57% of the retrieved trials from the registry, which provides a realistic insight into the completeness and transparency of reporting adverse events for glaucoma trials. We acknowledge that discrepancies in adverse event reporting between the registry and the publications may arise from the differences in the definitions and criteria used for adverse events. While regulatory submissions to regulatory entities follow strict guidelines, authors may apply different thresholds or classifications in a journal article. However, even considering these potential differences, the inconsistency in reporting underscores the need for standardized and transparent safety reporting across all platforms to ensure accurate interpretation and to uphold the integrity of clinical trial publications. The strengths of our study are the study period of over 14 years, inclusion of trials under the FDAA mandate to report adverse event, and the inclusion of several bibliographical databases for searching for journal publications. Finally, the inter-rater reliability for data extraction was high in our study.

The discrepancy between the registered and reported adverse events and their general underreporting in journal publications presents a serious problem, and has been demonstrated for interventions from other medical fields. Firstly, the inconsistencies create mis-impressions about the safety of glaucoma interventions and distort the balance of benefit and risk during assessment by different stakeholders: health professionals, decision makers and patients. Secondly, as glaucoma is a disease with high health and economic burden27, complete, transparent and accurate results reporting is necessary to make evidence-based decisions in clinical practice. Finally, differences in adverse events reporting in registries and publications complicate evidence synthesis, such as those in systematic reviews and clinical practice guidelines, because it is not clear which information is accurate – that in the registry or that in the journal publication. These discrepancies cast doubts on the validity and trustworthiness of all clinical trials. Our experience from this study is that it is very difficult to verify individual adverse events between the publication and the registry. It is an extremely demanding additional task that would significantly prolong systematic review processes.

A common explanation for underreporting of AEs is limited space in journals28 and/or absence of AEs so they are not reported if zero28,29,30. Space limitation should not be a valid explanation, as most journals publish supplementary information with published articles, where the information on AEs can be reported, even if it is not the main result of the study31. Also, it is possible that the results may change from the time of reporting in the registry and writing of the article, but the authors should then revise the registered results. Most of the trials in our study were published in high-impact journals, which should follow standard reporting guidelines and check the consistency of reporting in the registry and the publication, according to the widely accepted standards from the International Committee of Medical Journal Editors32. On the other hand, checking the results in submitted manuscripts against the summary results in a registry is a demanding task, for which many journals do not have sufficient resources and which would significantly prolong the publication process.

Inconsistent reporting of AEs is not specific for glaucoma intervention trials. Discrepancies were found for trials on drug-drug interactions30, allergic rhinitis29, and antidepressant/antipsychotic drugs11. In our study, overreporting of OAEs did not depend on the type of reporting, unlike that described for allergic rhinitis interventions, with underreporting of OAEs if reported as treatment-related adverse events (TRAEs)29..

What can be recommended to address the problem of inconsistent reporting of adverse events in glaucoma intervention trials? As the problem is not specific for glaucoma intervention trials, there is a need for more serious engagement of all stakeholders to produce valid, reliable and trustworthy reporting of results from clinical trials. The usual recommendation is that journals should better enforce already existing reporting standard. Item 19 of the CONSORT reporting checklist33 requires reporting of “All important harms or unintended effects in each group,” with special extension to the CONSORT for harms developed in 202220. As long as this guidance is in the form of recommendations, it will probably not have an impact on the quality of publications, as demonstrated in our study. The importance of mandatory regulations is illustrated by the finding in our study that the trials completed after the FDAA Final Rule, all reported deaths in the registry, compared to only 28% reporting this in the trials completed before FDAA Final Rule. For trials before and after the Final Rule, the reporting of deaths was similar in publications (33% and 31% before and after the FDAA Final Rule respectively). It seems that the trialists followed the regulation for the registry but did not follow this with similar quality of reporting in journals. Journals cannot act alone to improve the trustworthiness of the published trial results. They can implement already developed and validated tools to assure structured reporting of information34, although the use of the tool in journals has failed to show beneficial effects in practice35. We believe that a good solution would be uniform reporting of most important safety parameters in the registry and the journal publication. As the best example from our study, the frequency threshold stands out, which was typically higher in the registry than in the associated publication, leading to significantly different numbers of OAEs between the publication and the registry. Other stakeholders, such as funders and research/academic organizations, may have greater power to operationalize the mandate for complete reporting of trial results: funders by developing tools to check publication output from a research grant, and research organization by providing oversight through their research ethics committees/institutional review boards or other relevant oversight bodies. Finally, patients and their families are a powerful voice in demanding honest reporting as the only way to ensure patient safety.

An area for future research is to examine whether trials with available protocols and initial registry entries pre-specified the safety reporting to be assessed and how they would be reported, and to assess the consistency with their final reports. This could reveal whether discrepancies in safety reporting arise from deviations from the original study plans or selective reporting practices, thereby enhancing our understanding of the integrity of safety reporting in clinical trials. Due to the low number of trials that studied surgical interventions for glaucoma, we could not make conclusions on possible differences in reporting in comparison to the trials testing pharmacological interventions. This should be followed up in future studies, as surgical innovation follows a different innovation and clinical testing than pharmacological interventions, with different innovation phases requiring different standards for registration and reporting36.

To conclude, the discrepancies in reporting adverse events for glaucoma intervention trials present a serious problem as they fail to provide a clear and trustworthy picture of the safety of glaucoma therapy and raise concerns about the credibility of the trial results. Concrete measures are needed to ensure reporting consistency for adverse events/harms in trials for glaucoma and other interventions. Trial registries, funders, and publishers should collaborate to develop digital tools to safeguard the concordance between the results reported in registries and journals.

Methods

Study periods and data sources

We conducted a retrospective analysis of completed RCTs related to glaucoma, registered in ClinicaTrials.gov from September 27, 2009, and updated and with results on or before November 1, 2023. We chose the ClinicalTrials.gov as the largest registry, with well structured and curated information for all stepts of the trial registration, as well as clear legal requirements for reporting safety data. The commencement date for the study aligned with the initiation of mandatory adverse events reporting, and the termination date encompassed more than 14 years of mandatory adverse events reporting. We followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines for reporting the study.

Sample

To ascertain that the trials met the criteria for the classification as Applicable Clinical Trials (ACT) as per the Food and Drug Administration Amendments Act Sect. 801 (FDAAA 801), we followed the guidelines outlined in the "Elaboration of definitions of responsible party and applicable clinical trial (ACT)" dated March 9, 2009, for trials initiated after September 27, 200737. For trials initiated after January 18, 2017, compliance with the criteria of mandatory reporting of all OAEs, SAEs and all-cause mortality was assessed using the checklist for evaluating whether a clinical trial or study is an ACT15. Researchers conducting ACT are legally mandated to report their results, including all adverse events, to ClinicalTrials.gov within 12 months after the trial’s primary completion date, regardless of whether the findings have been published in a journal. This requirement is stipulated by the FDA Amendments Act of 2007 (FDAAA 801) and reinforced by the Final Rule effective from January 2017 to include mortality reporting15.

We searched the registry for completed RCTs using the keyword "glaucoma” to make a sensitive rather than specific search, because this search includes “glaucoma” as a text word and “Glaucoms” as a disease concept in ClinicalTrials.gov, which includes different types of the disease. Further inclusion criteriea were implemented choosing the filters in the registry: 1) interventional studies, to to focus on trials testing specific interventions, excluding observational studies; 2) phase 3 or phase 4 trials, as these phases are crucial for confirming efficacy and monitoring safety in larger populations, providing more comprehensive data on adverse events reportin; trials without specified phase information were excluded to maintain consistency in our dataset; 3) trials with registered results, since this criterion was essential for comparing registry data with corresponding publications; and 4) trials marked as “Completed” to ensure that the results and safety data should have been fully collected and reported. For trials with more than one publication available at ClinicalTrials.gov, we included only the the first full publication reporting on the main trial results related to the primary outcome, ensuring our analysis reflected the initial and most relevant reporting of adverse events associated with each trial. We did not include secondary publications, such as ancillary or subgroup analyses, focusing solely on the primary report to assess the consistency and completeness of safety reporting between ClinicalTrials.gov and corresponding journal publications. In cases where the publications were not provided in the registry, we conducted searches on PubMed, Web of Science, Scopus, and Google Scholar databases using the National Clinical Trial (NCT) identifier from the ClinicalTrials.gov record, typically found in the abstract or main text of published articles. If the initial search yielded no results, we conducted additional searches using the principal investigator’s name and study title, medication types, concentration, length of the research, and country where trial is conducted. The comparison between registered data and publications was performed on full publications, including supplementary data.

Data extraction and comparisons

Complete reporting in the registry included tables summarizing the number of affected participants out of those at risk for each AE, as mandated by the Final Rule16. Notably, the "All-cause Mortality" item was not mandatory before the Final Rule16, so it was not analyzed for trials with a primary completion date before the rule implementation. For these trials, we examined the reporting of deaths primarily through other elements of the outcome data, particularly SAEs.

In publications, complete reporting of AEs constituted an explicit statement regarding the occurrence of SAEs, deaths, or OAEs, following CONSORT extensions for improved reporting of harms in randomized trials20. While the CONSORT extension for reporting safety data prefers the use of the term “harm” as “(t)he totality of possible adverse consequences of an intervention or therapy; they are the direct opposite of benefits, against which they must be compared” vs the terms adverse events or “adverse reactions and adverse drug reaction,” we opted to used the term “Adverse Event” as described in the ClinicalTrials.gov’s glossary, because it matched the definition of “harm” in the CONSORT extension: “An unfavourable change in the health of a participant, including abnormal laboratory findings, that happens during a clinical study or within a certain amount of time after the study has ended. This change may or may not be caused by the intervention/treatment being studied38. Any disparities in the completeness, the number of affected participants, total AEs, or the description of AEs between ClinicalTrials.gov and publications were classified as inconsistent reporting of AEs. An example is available in the Supplementary Note N2. We did not use standardized terminology systems like MedDRA when comparing description of reported adverse events in the registry and publications because multiple MedDRA versions were released over the 14 years of the study time-span, and trials varied in which version they used. We focused on exact matches in terminology between the registry and the publication, rather than mapping terms to standardized categories. For example, even if different terms are synonyms within MedDRA (e.g., “nausea” and “queasy”), we categorized that as inconsistent reporting between the registry and the publication for the same trial.

Two investigators (AK and MG) independently extracted the data concurrently from the entire trial cohort and corresponding publications to mitigate potential bias from subjective interpretation. Inter-rater reliability was high for SAEs reporting in ClinicalTrials.gov and publications (kappa range 0.83 to 1.00). For OAEs reporting, the inter-rater reliability was high for some elements (kappa range 0.69 to 1.00). For elements with lower agreement (differences in description of OAE between ClinicalTrials.gov registry and in publications – kappa = 0.69, 95%confidence interval (CI) 0.38 to 1.00, and for number of different OAEs in publication – kappa = 0.74, 95% CI 0.57 to 0.92), we resolved the disagreements through consensus discussion, particularly for the interpretation of differences.

Statistical analysis

Data extracted from the registry were entered into a spreadsheet and coded. We reported the percentages, medians, and their respective inter quartile range (IQR) for nonparametric variables. Categorical binary variables were used to present the data differences in AE descriptions. If we could not categorize the result from the available data, we indicated these as not categorized (“n/c”), and stated the reason for non-categorization.

The differences between the registry and publications were tested using chi-square test/Fisher exact test or Mann–Whitney U test. The statistical analyses were conducted using JASP39. The significance was determined at P < 0.05.