Introduction

Retinitis pigmentosa (RP) is the most common hereditary retinal degeneration, affecting one in every 4000–5000 people worldwide1,2. The photoreceptor degeneration causes night blindness, visual-field narrowing, and vision loss. Although various treatment strategies, including gene therapy3,4,5,6regenerative medicine7,8optogenetics9artificial retinas10,11and drug development12,13,14,15,16,17,18,19,20have been studied, there is no established treatment to prevent photoreceptor cell degeneration regardless of the causative gene21,22.

We previously showed that neuronal cell death is attenuated by maintaining intracellular ATP levels by inhibiting the ATPase activity of valosin-containing protein23,24,25,26,27,28 or by promoting ATP production. Branched chain amino acids (BCAAs) promote glucose transporter translocation to the plasma membrane, and increase glucose uptake and ATP production29. BCAAs protect cultured cells against cellular stresses, including endoplasmic reticulum and mitochondrial respiratory chain inhibition stresses30. Furthermore, BCAAs attenuate photoreceptor cell degeneration in mouse models of RP even when the administration is started after the disease has progressed considerably30.

Thus, we conducted an exploratory, placebo-controlled, double-blind, comparative study to assess the efficacy and safety of TK-98, an investigational drug consisting of BCAA combination granules containing l-isoleucine, l-leucine, and l-valine, in patients with RP31. In the clinical trial, it was not shown that the administration of BCAA combination granules resulted in a slow visual-field progression in the patients. However, the evaluation of morphological tests suggested that the administration may have a beneficial effect on progression. Although the study was conducted with random allocation of the patients to TK-98 or placebo, with allocation adjustment factors of age at the time of obtaining consent and mean deviation (MD) value of Humphrey visual-field tests (HFA) before registration, there were factors with substantial differences between the groups.

Herein, we report the results of a post hoc analysis of the clinical trial investigating the efficacy and safety of BCAAs in patients with RP. We conducted an analysis of the HFA 10-2 data with covariate adjustment to compensate for baseline factors. In addition, we assessed the reliability of the HFA 10-2 screening test results, which were used for patient inclusion, randomization, and baseline visual-field assessment in the clinical trial. We also analyzed the HFA 10-2 data from 13 weeks after the start of administration, considering the time it takes for the investigational drug to start working.

Methods

We conducted a post hoc analysis on a placebo-controlled, double-blind, comparative clinical trial that assessed the efficacy and safety of TK-98 in patients with RP (https://jrct.niph.go.jp/en-latest-detail/jRCT2051180072). This clinical trial was conducted as described previously31. Briefly, this study included 70 patients with RP, who were aged 20 years or older. The eyes with MD value between − 5.0 dB and − 25.0 dB in the HFA (10 − 2 Swedish interactive threshold algorithm standard program; HFA 10-2) before the registration were eligible to be included to the clinical trial. The included patients were assigned to either the treatment or placebo group with allocation adjustment factors of age (less than 53 years old, or 53 years and older) and MD value at the beginning of the clinical trial (less than − 15.0 dB, or equal to or more than − 15.0 dB). Forty-five and twenty-five patients were assigned to the treatment and placebo groups, respectively. The patients were administered 4.15 g of the study drug (l-isoleucine, 952 mg; l-leucine, 1904 mg; and l-valine, 1144 mg) or a placebo three times a day for 78 weeks. The patients underwent comprehensive ophthalmic examinations including HFA 10-2 and macular ellipsoid zone (EZ) length measurements using optical coherence tomography (OCT, Spectralis HRA + OCT system, Heidelberg Engineering, Heidelberg, Germany), as described previously31. MD value and total point score (TPS), which is the sum of visual sensitivities (in decibels) for all 68 test points32were assessed using the HFA 10-2. EZ length was measured using macular cross-sectional images, and the average of the measured values on horizontal and vertical images was used for assessment31.

In addition to the data obtained during the observation period of the clinical trial, the data obtained before the patients were included in the trial were analyzed. The study adhered to the principles of the Declaration of Helsinki. The clinical trial was approved by the Kyoto University clinical trial review committee (K051) and was conducted in accordance with the Law Concerning the Quality, Efficacy and Safety of Drugs, Medical Devices and Other Products, the Ministerial Ordinance on Standards for the Conduct of Clinical Trials of Drugs (GCP Ordinance). Usage of patient data before inclusion in the clinical trial was approved by the Institutional Review Board and Ethics Committee of the Kyoto University Graduate School of Medicine (R2198), and the trial was conducted according to the Ethical Guidelines for Medical and Health Research Involving Human Subjects of the Ministry of Health, Labour, and Welfare of Japan. Written informed consent was obtained from each patient separately for the clinical trial and for the use of data before inclusion in the trial.

Baseline characteristics and analysis with covariate adjustment

The characteristics at the time of allocation were analyzed in patients in per protocol set (PPS) who received the study drug for at least 53 weeks. The MD slope before the start of the clinical trial was calculated using linear regression, with MD values on HFA 10-2 observed up to 3 years before inclusion in the clinical trial, including the result at the time of allocation (Fig. 1). Intergroup differences in annual changes of TPS and MD values in HFA 10-2 were evaluated using a linear mixed-effects model, with eye laterality (right or left) treated as a random effect, and treatment group and other covariates treated as fixed effects. Covariates were chosen based on the following criteria. The first criterion was variables for which the absolute difference between groups exceeded the absolute value of the 1-year change in the placebo group31 (e.g., MD value at the time of allocation or EZ length at the time of allocation; see Table 1). This criterion was based on the rationale that such baseline differences represent clinically meaningful imbalances that could confound group comparisons, particularly in a 1.5-year study period. The second criterion was variables that showed significant intergroup differences at baseline (P < 0.1), such as MD slope just before allocation and EZ length at the time of allocation (see Table 1). In addition, time from the onset of the first subjective symptoms (mean difference: 1.6 years, see Table 1) was included as a covariate based on its clinical relevance and the magnitude of the difference, which was also considered likely to impact group comparisons over the study period. These covariates (MD value at the time of allocation, EZ length at the time of allocation, time from the onset of the first subjective symptoms, and MD slope just before the allocations) were incorporated into the linear mixed model to adjust for potential confounding effects.

Fig. 1
figure 1

Schematic diagram of HFA 10-2 tests used for analysis in the clinical trial and in the current post hoc analysis. *1The screening Humphrey visual-field (HFA 10-2) test before registration was used for eligibility decisions at enrollment, for allocation to study drug groups, and as baseline data at the start of the clinical trial. The HFA10-2 test, conducted within 30 days of drug administration, was eligible for inclusion in the clinical trial screening and allocation and for baseline data. *2Analysis of data before the start of the clinical trial. The number and ratio of eyes with paradoxically increased MD or TPS values were evaluated using the HFA10-2 tests before the start of the clinical trial. HFA10-2 results at the start of the clinical trial (0, 0 W) as well as one time before (− 1), two times before (− 2), and three times before (− 3) the start of the clinical trial were analyzed. *3MD slope before the start of the clinical trial was calculated using MD values of HFA10-2 tests observed up to 3 years prior to inclusion in the clinical trial, including the result at the time of enrollment/allocation. *4HFA10-2 results from 13 weeks after starting the study drug administration until the end of the clinical trial were analyzed separately, excluding the HFA10-2 data at the eligibility screening.

Full size image
Table 1 Baseline characteristics and intergroup differences at the time of allocation (per protocol set).
Full size table

Stratified analysis of the visual-field test with pseudophakia population

The stratified analysis of the visual-field test in the pseudophakia population was conducted in the PPS population.

Analysis of visual field before and during the clinical trial

We evaluated three consecutive HFA 10-2 tests in eyes in the PPS population before inclusion in the clinical trial. The eyes diagnosed with cataract progression during the clinical trial period were excluded. The number and ratio of eyes whose MD and TPS improved, that is, the MD or TPS value increased in the latter compared to the former between consecutive HFA 10-2 tests, were evaluated. In addition, the absolute change between consecutive HFA 10-2 tests was divided by the length of the period between tests to calculate the annual change amount (Fig. 1). In addition, we analyzed data from 13 weeks after the start of administration to the end of administration, excluding the visual-field test results at the time of inclusion in the clinical trial, which seemed to be affected by psychological factors and was suspected to be less reliable. Additionally excluding the test results at the time of inclusion would provide sufficient time for the drug to start showing its effect. The TPS and MD change rates were analyzed in the PPS population.

Results

Baseline characteristics between the groups compared to the overall trend of change

The clinical trial was conducted with random allocation of patients to the TK-98 or placebo group with allocation adjustment factors of age at the time of obtaining consent and the MD value before registration31. Even though the differences between the groups at the time of allocation in age (51.0 ± 11.3 and 52.1 ± 14.4 in TK-98 and placebo groups, respectively, P = 0.63) and HFA 10-2 MD values (− 14.8 ± 4.9 and − 14.2 ± 5.1 dB, in TK-98 and placebo groups, P = 0.55) were not significant, some measurement with substantial differences were observed between groups, including MD slope just before the allocation in the PPS population (− 0.11 ± 0.92 and − 0.42 ± 0.89 dB/year in TK-98 and placebo groups, respectively, P = 0.07, Table 1).

The baseline differences between groups at the time of allocation were assessed relative to the expected annual change. The annual changes observed in the placebo group from the analysis of the clinical trial (− 0.52 dB, − 35.4 dB, − 21.9 dB, − 95.6 μm for HFA 10-2 MD, TPS, HFA 30-2 TPS, and EZ length, respectively)31 were used as reference values. Consequently, the baseline differences in the MD value and EZ length between groups were larger than or equivalent to the absolute values of these annual changes (Table 1), suggesting clinically meaningful imbalances.

TPS and MD changes analyzed with covariate adjustment

Based on these findings, we performed covariate-adjusted comparisons of the annual changes in TPS and MD values in HFA 10-2. Covariates were as follows: MD value at the time of allocation, EZ length at the time of allocation (selected owing to absolute intergroup differences exceeding the expected annual change), MD slope just before allocation, and time from the onset of the first subjective symptoms (selected owing to significant or clinically relevant baseline differences). With covariate adjustment, the annual decreases in the TPS and MD of HFA 10-2 were slower in the TK-98 group than in the placebo group, although this was not statistically significant (Table 2).

Table 2 Intergroup comparison of change in total point score and mean deviation adjusted for baseline covariates (per protocol set).
Full size table

Stratified analysis of visual-field change with pseudophakia population

The ratio of phakia/pseudophakia eyes was significantly higher in the TK-98 group than in the placebo group (Table 1; P = 0.002). Considering the possibility that cataract progression in phakic eyes may affect the visual-field tests, a stratified analysis of the visual-field test with the pseudophakia population was conducted.

The stratified analysis of the visual-field test with the pseudophakia population included 7 patients (12 eyes) in the TK-98 group and 10 patients (17 eyes) in the placebo group. Baseline characteristics of the pseudophakia population are shown in Supplementary Table S1. Although the sample size was small and the difference was not significant, the decreases in the TPS and MD of HFA 10-2 were smaller in the TK-98 group than in the placebo group (Table 3).

Table 3 Stratified analysis of change in total point score and mean deviation in pseudophakia eyes.
Full size table

Pre-trial visual-field trends and potential bias at baseline

Owing to baseline differences in some parameters, we were concerned about the reliability of the HFA 10-2 test results at screening, which were utilized for patient inclusion, randomization, and baseline visual-field assessments in the clinical trial. Therefore, we assessed the reliability by examining three consecutive HFA10-2 tests prior to the inclusion of eyes in the PPS population. The number and ratio of eyes whose MD and TPS paradoxically improved between the consecutive HFA10-2 tests increased as the beginning of the clinical trial approached (Fig. 1, Supplementary Tables S2 and S3). The TPS and MD values improved in a greater number of eyes in the TK-98 group than in the placebo group from one time before to the time of the clinical trial start, while no significant intergroup differences from three times before to two times before the clinical trial start were observed (Supplementary Tables S2 and S3). The annual change between the visual-field tests conducted two tests prior to the start and one test prior to the start showed no significant differences between the two groups. However, when examining the annual change amount between the tests at the start of the trial and the one just prior to it, a significant difference was found; only the TK-98 group and not the placebo group showed paradoxically positive increased values for both TPS and MD (Supplementary Tables S4 and S5).

Visual-field tests from week 13 till the end of the clinical trial

The visual-field test results from week 13 were analyzed for three reasons. First, the visual-field test results at the time of allocation, which were used as baseline data at the start of the clinical trial, were suspected to be less reliable as stated above. Second, the visual-field test at inclusion was performed up to 30 days prior to the start of the investigational drug administration. Third, it may have taken some time for the study drug to start showing its effects. Therefore, we analyzed the HFA 10-2 data 13 weeks after the start of the drug administration, excluding the visual-field test at the time of allocation (Fig. 1). This analysis showed a slower decrease in TPS and MD in the TK-98 group than in the placebo group, although the difference was not statistically significant (Table 4).

Table 4 Intergroup comparison of change in total point score and mean deviation from 13 weeks after the start of the treatment till the end of the observation period.
Full size table

Discussion

In this study, we conducted an additional analysis of a clinical trial to evaluate the efficacy of BCAAs in patients with RP31 to compensate for baseline differences between groups. Although the clinical trial was conducted as a randomized placebo-controlled test, certain factors showed significant differences between the groups at baseline. With covariate adjustment, the visual-field deterioration in the HFA 10-2 test tended to be slower in the TK-98 group than in the placebo group, although the difference was not statistically significant. This tendency was consistent with slower morphological deterioration observed in the TK-98 group than in the placebo group, as reported previously31.

We analyzed the patients’ visual-field test results conducted before inclusion in the clinical trial and found that the results of the visual field improved in an increasing number of patients as the beginning of the clinical trial approached. Given the nature of the disease, MD and TPS in visual-field tests should always decrease in RP. Thus, increased MD or TPS is caused by inter-inspection fluctuation, learning effect, placebo effect, or motivation of the patient for the test. This analysis indicated that the visual-field test at the inclusion of the clinical trial was less reliable. Most of the patients included in the clinical trial were waiting eagerly to try new possible treatments or new clinical trials as there are no established available treatments to stop or slow the disease progression. The visual-field test improvement in an increasing number of patients as the beginning of the clinical trial approached was possibly caused by the motivation or degree of concentration of the patients for the test. Moreover, in this clinical trial, the patients were informed beforehand that if their visual-field test at the time of allocation was worse than the MD value of -25 dB, they would not be included in the clinical trial. This might result in improved visual-field test at the time of allocation because of the subjective feature of the test. If a patient performs unusually well as a result of heightened psychological focus or temporary motivation, the subsequent test result—reflecting the patient’s typical condition—are likely to show a decline compared to the initial performance. The fact that the ratio of increased visual-field tests at the time of inclusion was significantly higher in the TK-98 group than in the placebo group may have influenced the main outcome results of the clinical trial. Considering that knowledge of the visual-field test results affecting eligibility might influence the paradoxical improvement of the visual field as the start of the clinical trial approaches, it is recommended to use morphological parameters as the inclusion criteria rather than subjective functional tests.

The tests results at inclusion were also used as data for allocation as well as the data for the baseline of the clinical study to make the clinical trial more feasible with fewer visual-field tests for included patients. However, the fact that the patients knew that there was a possibility that they would be excluded from the clinical trial if the visual-field test result was worse than the criteria may have affected the subjective test at the time of inclusion. The analysis of visual-field tests conducted from 13 weeks after the start of the treatment until the end of the observation period, excluding the inclusion data that were considered unreliable, showed a slower decline in the visual field in the TK-98 group than in the placebo group: a contradictory result from the analysis including the inclusion screening data. This result suggests that it is strongly recommended not to use subjective data at the eligibility screening as the baseline data for analysis or as data for group allocation in future clinical trials.

Thus, this study identified the limitations of using subjective functional tests as the main outcome measures, main inclusion criteria, or main allocation factor in clinical trials. Even though a direct measurement of functional efficacy is desired and the visual-field tests have been widely used as the main outcome measurement in clinical trials in RP13,16,17,18,20fluctuations, learning or phycological effect may hinder the sensitive detection of the efficacy of the treatment32,33,34,35. Morphological examinations, including EZ length or thickness of retinal layers on OCT images and ring-shaped hyperautofluorescence area or abnormal hypoautofluorescent area on fundus autofluorescence images, have been shown to well reflect the retinal function36,37,38,39,40. Moreover, it has been shown that objective morphological examinations have less fluctuations and can more sensitively detect the disease progression compared to subjective functional examinations32. Using objective morphological examinations as primary outcomes in future clinical trials would help in the sensitive and accurate assessment of the efficacy of the treatment.

In addition, although the patients were randomly assigned to the treatment or placebo group with allocation adjustments (age and MD), certain factors were significantly different at the time of allocation between the groups, including the ratio of phakia/pseudophakia. As cataracts progress gradually, the different ratios of phakic eyes might affect the visual-field test results, which were the main outcome measurements in this clinical trial. It would be worth considering the adjustment of the phakia/pseudophakia ratio between the groups when using functional tests, such as visual-field or visual-acuity tests in future clinical trials.

In summary, the current study conducted a post-hoc analysis showing that visual-field deterioration tended to be slower in the TK-98 group than in the placebo group. This study also highlighted an important methodological issue: the limitations of using subjective functional tests such as the visual-field tests as the main outcome measures, inclusion criteria, or allocation factors in clinical trials. However, some limitations of this clinical trial should be also acknowledged. First, even with the post-hoc analysis, the difference between groups did not reach statistical significance. This lack of significance may be due to the limited sample size of the study. The clinical trial was conducted with a limited number of patients, with the sample size calculated with a power of 75.3% to detect significance at α = 20%31, which may have contributed to the insufficient statistical power. Second, although we observed that the visual-field result at inclusion may not have been entirely reliable—likely influenced by psychological factors such as increased concentration or motivation—we did not perform a formal test–retest variability assessment in this clinical trial. In response to this concern, we examined the correlation between two consecutive HFA 10-2 tests—the test conducted immediately prior to the trial and the one preceding it—and compared this correlation with correlations from earlier timepoints and from an independent naturalistic observational cohort (Supplementary Fig. S1). While the correlations were relatively high, particularly near trial initiation, these analyses do not equate to a direct measurement of test–retest variability. Therefore, we acknowledge that the absence of a structured test–retest assessment limits our ability to fully characterize the extent and implications of psychological or motivational effects on test performance. Including such evaluations in future studies would provide more robust evidence regarding the reliability of subjective functional outcomes in clinical trial settings. Third, although the paradoxical improvements in visual-field test results as the start of the clinical study approaches were likely to be caused by psychological factors, such as patients’ increased concentration or motivation for the test, these factors were not directly evaluated during the trial period. Finally, we did not perform a qualitative assessment or evaluation of patient-reported outcomes, such as using a visual function questionnaire in the clinical trial. Inclusion of such data would have provided deeper insights into the efficacy and safety of the study drug.

In conclusion, the analysis with covariate adjustment and the analysis after a period until the drug was supposed to take effect suggested that the visual-field deterioration tended to be slower in the TK-98 group than in the placebo group. The data analyzed before and during the clinical trial in this study, demonstrating the limitations of subjective functional visual-field tests, will be useful for conducting future clinical trials on RP.