Introduction

A thymic cyst is the most prevalent nonneoplastic lesion in the anterior mediastinum1 which is usually incidentally detected and shown as a nodule measuring less than 30 mm2,3. Despite their benign nature, the proportion of unnecessary thymectomy for thymic cysts ranges from 24.3% (17/70)4 to 62.7% (32/51)3, posing a significant concern with the increasing use of chest CT for lung cancer screening.

Concern for a thymic epithelial tumor is the main cause of unnecessary thymectomy. Distinguishing thymic cysts from solid tumors requiring surgical resection on CT can be difficult: approximately three-quarters of thymic cysts show hyperattenuation to water owing to internal hemorrhage or proteinaceous or calcium-containing fluid4,5, mimicking soft tissue features on a non-contrast scan. Furthermore, the pseudo-enhancement phenomenon6 of thymic cysts could simulate a solid tumor on a contrast scan. In contrast, thymic epithelial tumors with cystic degeneration7 mimic the characteristics of thin-walled thymic cysts.

MRI has advantages over the limited diagnostic ability of CT for differentiating thymic cysts from solid lesions due to its superior tissue characterization8. The sensitivity of MRI for diagnosing thymic cysts ranges from 71% with conventional MRI5 to 100% with a normalized apparent diffusion coefficient (ADC)9. As a result, the International Thymic Malignancy Interest Group1 recommended MRI for the diagnosis of suspected thymic cysts. Considering that MRI-based diagnosis of thymic cysts indicated no malignant potential of lesions during follow-up2,10, distinguishing thymic cysts from other anterior mediastinal lesions on MRI has become more clinically important.

However, reliable diagnosis of thymic cysts on MRI is challenging because not all thoracic radiologists are familiar with chest MRI, and standardized criteria for diagnosis are lacking. The various internal signal intensities and presence or absence of wall enhancement of thymic cysts can lead to inter-reader variability. To date, the inter-reader agreement in diagnosing thymic cysts with MRI has been evaluated only in limited populations, showing moderate agreement (κ = 0.50)9. Given that MRI has become the modality of choice for thymic lesions and is being more widely used11, it is important to assess the inter-reader agreement to determine its practical value. Moreover, it is still uncertain which MR sequences are essential for acceptable agreement. Therefore, the purpose of our study was to investigate the inter-reader agreement for diagnosing thymic cysts using various combinations of MR sequences and to assess the effect of the addition of CT on inter-reader agreement.

Methods

This retrospective study was approved by the institutional review boards (IRB Numbers: 2022 − 1157 [Asan Medical Center, Institution A] and 2021-02-156 [Samsung Medical Center, Institution B]), and informed consent was waived by the institutional review board of both institutions due to the retrospective nature of this observational study. This study was performed in accordance with the Helsinki Declaration.

Study population

From January 2013 to June 2021 (Institution A) and from January 2010 to October 2020 (Institution B), consecutive patients who underwent contrast-enhanced chest MRI scans for anterior mediastinal lesions were included. The inclusion criteria were as follows: (a) lesion size of ≤ 30 mm; (b) acquisition of diffusion-weighted imaging (DWI), ADC maps, and subtracted T1-weighted images; and (c) available pre- and post-contrast chest CT scans within six months of MR acquisition. The exclusion criteria were as follows: (a) multiple lesions and (b) the presence of artifacts obscuring the lesion. Of the included patients, a total of 15 patients (nine patients from Institution A and six patients from Institution B) have been reported in a prior study that assessed longitudinal changes in thymic cysts10. The patients’ clinical information (age, sex), duration of imaging follow-up, and available pathologic findings from biopsy or resection were recorded.

Chest MRI and CT protocols

Institution A acquired MRIs with a 3.0-T scanner (MAGNETOM Skyra, Siemens Healthineers, Erlangen, Germany), while Institution B acquired MRIs with 3.0-T scanners from different vendors (Ingenia, Philips Healthcare, Best, the Netherlands or MAGNETOM Skyra, Siemens Healthineers, Erlangen, Germany). The MRI protocol consisted of at least the following sequences: (1) axial T2-weighted images (T2WIs) with fat suppression, (2) pre- and post-contrast axial T1-weighted images (T1WIs) with fat suppression, (3) subtracted T1WIs, and (4) DWIs with ADC maps. The details of the MRI protocol are described in Supplementary Table 1.

Both pre- and post-contrast CT scans were obtained using the following parameters using a 16 or 64 multi-channel CT scanner. The post-contrast scans were obtained 40 s (Institution B) or 50 s (Institution A) after injection of contrast media. The details of the CT protocol are described in the Supplementary Material.

Reader study

Before the reading session, one thoracic radiologist (Y.A. with 2 years of experience in chest MRI) reviewed and selected chest MRIs according to the inclusion and exclusion criteria. Then, MR and CT images were anonymized and prepared in a different randomized order for each session and each reader. A total of six sessions were evaluated to determine the optimal image sequence combination (four internal reading sets from Institution A, where all four readers work) and externally validate it with different MR vendors from Institution B (two external reading sets). To determine whether experience affects interpretation, four readers with varying levels of chest MRI reading experience (0–11 years) were enlisted.

The internal reading set (from Institution A) consisted of four separate review sessions to optimize the combination of image sequences and modalities. Session 1 consisted of the basic sequences recommended for mediastinal lesion evaluation11,12. Subtraction imaging, known to aid in differentiating cystic lesions11, was evaluated by replacing the DWI/ADC map in Session 2 and adding it to the sequence combination from Session 1 in Session 3. Finally, the added value of CT, typically the initial examination and used for follow-up, was also evaluated in Session 4.

Session 1 = T2WI + pre- and post-contrast T1WI + DWI/ADC map.

Session 2 = T2WI + pre- and post-contrast T1WI + subtracted image.

Session 3 = T2WI + pre- and post-contrast T1WI + DWI/ADC map + subtracted image.

Session 4 = T2WI + pre- and post-contrast T1WI + DWI/ADC map + subtracted image + pre- and post-contrast CT.

The external reading set (from Institution B) consisted of two separate review sessions to validate and optimize the MR sequence combination and role of CT:

Session 5 = MR sequence showing the highest inter-reader agreement in Sessions 1–3.

Session 6 = Session 5 + pre- and post-contrast CT.

Image review was independently performed by four board-certified thoracic radiologists (S.M.L., J.C., and two non-authors with 11 [Reader 1], 6 [Reader 2], 0.5 [Reader 3], and 0 [Reader 4] years of experience in chest MRI, respectively) from Institution A, with 4-week intervals between evaluation of internal and external reading sets. All readers participated in evaluation of the internal reading set, and three readers (Reader 1, Reader 2, and Reader 4) participated in evaluation of the external reading set. During each session, the readers were asked to rate their diagnosis for thymic cysts using a five-point scale: 5 = definitive cyst; 4 = probable cyst; 3 = indeterminate; 2 = probable non-cyst; and 1 = definitive non-cyst. To minimize recall bias, the cases were presented in random order for each session and each reader. The reading time was not limited. The criteria for thymic cysts are provided in the Supplementary Material.

Statistical analysis

Continuous variables were compared using an independent t-test or the Mann–Whitney U test, and categorical variables were compared using Pearson’s chi-square test or Fisher’s exact test, as appropriate. The inter-reader agreement was analyzed for both a three-category scale (definite cyst or probable cyst vs. indeterminate vs. probable or definite non-cyst) and a two-category scale (definite or probable cyst vs. others). To determine whether size of lesion affects inter-reader agreement, the lesions were divided into two groups based on their median diameters (2 cm). Fleiss’ κ values were used for agreement among all four readers, while weighted κ values were used for agreement in each pair of readers. Interpretation of κ values was performed as follows: ≤0.20, poor; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and ≥ 0.80, almost perfect agreement13. To compare inter-reader agreements between different sessions, a difference in the dependent-κ estimates was tested by bootstrapping approach. The proportions of lesions diagnosed as a cyst (i.e., ratings for a definite cyst or probable cyst) by each reader were compared using a generalized estimation equation model. All statistical analyses were performed using R ver. 3.6.3 (R Foundation for Statistical Computing, Vienna, Austria) with P < 0.05 defined as statistically significant.

Results

Patient characteristics

Of 62 eligible patients from Institution A and 25 patients from Institution B, a total of 55 patients (mean age, 59 ± 11 years; range, 32–79; 26 men) from Institution A and 21 patients (mean age, 56 ± 10 years; range, 38–76, 13 men) from Institution B were finally included (Fig. 1). The mean longest diameters of the lesions on axial T1WIs were 18.0 ± 5.9 mm (range, 8–29) in Institution A and 19.8 ± 4.1 mm (range, 13–29) in Institution B. Based on the radiologic reports, 46 of 55 patients (83.6%) from Institution A were diagnosed with thymic cysts, while 19 of 21 (90.5%) patients from Institution B were diagnosed with thymic cysts. In Institutions A and B, the mean follow-up duration was 2.9 ± 2.9 years (range: 0.0–15.8) and 2.4 ± 2.5 years (range: 0.0–8.9), respectively (Table 1). During follow-up, among the radiologically diagnosed cystic lesions, the size of seven lesions from Institution A (15.2% [7/46]) and two lesions from Institution B (10.5% [2/19]) increased, while the remaining lesions remained stable or decreased in size. No enhancing solid portion was identified. For radiologically non-cystic lesions, four lesions from Institution A (44.4% [4/9]) and one lesion from Institution B (50% [1/2]) were surgically confirmed to be thymomas. The size of the remaining five lesions from Institution A remained unchanged, whereas that of the remaining lesion from Institution B increased.

Fig. 1
figure 1

Flowchart for patient inclusion and exclusion. ADC = apparent diffusion coefficient, DWI = diffusion-weighted imaging.

Full size image
Table 1 Characteristics of the study population.
Full size table

With regard to imaging findings, most lesions (72.7% [40/55] from Institution A and 90.5% [19/21] from Institution B) showed high signal intensity on T2WIs (Supplementary Table 2). Diffusion restriction was observed only in a few lesions (12.7% [7/55] from Institution A and 9.5% [2/21] from Institution B). At Institution A, 21.8% [12/55] of patients showed eccentric or nodular enhancement, suggesting a non-cyst, whereas at Institution B, 33% [7/21] of patients showed these enhancements.

Inter-reader agreement according to imaging sequences

In Institution A, for the three-category scale, the inter-reader agreement was the highest (κ = 0.53) in Session 4, in which pre- and post-contrast CT scans were included, followed by Session 3 (κ = 0.51); however, no statistically significant difference in inter-reader agreement was observed (p = 0.699) (Table 2, Supplementary Table 3). The inter-reader agreement was the lowest (κ = 0.42) in Session 2, which included the subtracted image without DWI and the ADC. The majority of discrepancies were attributed to suspicious eccentric wall thickening on the subtracted image (Fig. 2). For the two-category scale, the highest inter-reader agreement (κ = 0.57) was observed in Session 3, in which all available MR sequences were provided (Fig. 3), followed by Session 1 (κ = 0.53), in which only fundamental MR sequences were provided. No significant differences in inter-reader agreement were observed between sessions, even with the addition of CT (all, p-values > 0.05).

Table 2 Inter-reader agreement for diagnosing cysts according to combinations of imaging sequences.
Full size table
Fig. 2
figure 2

Example of discordance caused by a subtracted image in a 53-year-old woman. (A) A T2WI shows an 18-mm nodule with internal T2 hyperintensity in the anterior mediastinum (arrow). (B) The nodule showed hyperintensity on pre-contrast T1WIs (C) but did not show internal enhancement on post-contrast T1WIs. However, (D) the subtracted image showed subtle hyperintensity mimicking solid enhancement. (E) The lesion shows subtle high intensity on DWI, (F) but the corresponding ADC value was high, indicating that diffusion restriction was not present. Therefore, all readers diagnosed the lesion as definite cyst in Session 1 (T2WI + pre- and post-contrast T1WI + DWI/ADC map). However, in Session 2 (T2WI + pre- and post-contrast T1WI + subtracted image), two readers (Readers 1 and 3) changed the diagnosis based on the enhancement being suspicious (indeterminate) or solid enhancement being real (definite non-cyst), whereas the others (Readers 2 and 4) consistently diagnosed the lesion as a definite cyst based on the enhancement on the subtracted image being a misregistration artifact.

Full size image
Fig. 3
figure 3

Example of improved agreement with both the subtracted image and DWI/ADC map in a 58-year-old woman. (A) A pre-contrast T1WI demonstrated a 21-mm nodule with internal T1 hyperintensity in the anterior mediastinum (arrow). (B) No internal enhancement is present on a post-contrast T1WI, but (C) the subtracted image demonstrated eccentric enhancing wall thickening (arrowheads) on the left side of the nodule. (D) The nodule showed T2-hyperintensity. Consequently, the results of two readers (Readers 1 and 3), who suspected eccentric wall thickening, diagnosed the lesion as indeterminate for a cyst in Session 2, whereas the results of the other two readers (Readers 2 and 4), who determined that no true eccentric wall thickening was present, diagnosed the lesion as a definitive cyst. In Session 3, where DWI/ADC maps (E,F) were presented, all readers diagnosed the case as a definite cyst considering the homogeneously high ADC value (F).

Full size image

The pairwise inter-reader agreement ranged from fair (κ = 0.24 between Readers 1 and 2 in Session 1) to almost perfect (κ = 0.84 between Readers 2 and 4 in Session 2) at Institution A (Table 3). In terms of the mean pairwise inter-reader agreement, both Sessions 3 and 4 reached substantial agreement (κ = 0.65) for the three-category scale, and Session 3 was the highest for the two-category scale (κ = 0.59).

Table 3 Pairwise agreement between readers.
Full size table

Based on the results in the reading set from Institution A, Session 5 at institution B consisted of the same combination of MR sequences as Session 3. The inter-reader agreement was fair (κ = 0.37) for the three-category scale and moderate (κ = 0.48) for the two-category scale (Table 2, Supplementary Table 3). After adding CT, the inter-reader agreement increased to moderate (κ = 0.53) for the three-category scale and substantial (κ = 0.74) for the two-category scale despite; however, these findings were not statistically significant (p = 0.227 and 0.241, respectively) (Fig. 4).

Fig. 4
figure 4

Example of improved agreement with addition of CT in the external reading set in a 67-year-old woman. (A) A pre-contrast T1WI demonstrated a 17-mm nodule with internal T1 hyperintensity in the anterior mediastinum (arrow). (B) No internal enhancement is present on post-contrast T1WI, but (D) the subtracted image demonstrated eccentric enhancing wall thickening (arrowheads) on the left side of the nodule. (D) The nodule showed T2 hyperintensity. (E, F) The DWI/ADC maps did not show diffusion restriction. However, one reader (Reader 1) was suspicious of eccentric wall thickening on the subtracted image and thus diagnosed the nodule as indeterminate for a cyst, whereas the other three readers diagnosed the lesion as a definite cyst. After the addition of CT (G, H), all readers, including Reader 1, diagnosed the lesion as a cyst based on the lack of enhancement on the post-contrast CT image (H) and the consistent low attenuation of the lesion on both pre- and post-contrast CT.

Full size image

Regarding pairwise inter-reader agreement, Readers 2 and 4 reached almost perfect agreement (κ = 0.93, both) in both Sessions 5 and 6 (Table 3; Fig. 5). The lowest level of inter-reader agreement was observed between Readers 1 and 4 for the three-category scale (κ = 0.24). After the addition of CT, all pairs of readers reached substantial to almost perfect agreement in the external reading set. The mean pairwise inter-reader agreement was almost perfect for the two-category scale (κ = 0.85) in Session 6.

Fig. 5
figure 5

Line graphs showing pairwise agreement between readers. Inter-reader agreement assessed with a three-category scale in the (A) internal and (B) external reading sets. In the second line, line graphs show inter-reader agreement assessed with the two-category scale in the (C) internal and (D) external reading sets.

Full size image

Inter-reader agreement according to lesion size

Regarding the size subgroup, which was divided using a cut-off value of 20 mm, the inter-reader agreement ranged from fair to substantial (κ = 0.40–0.61) in each group, and there were no significant differences between the subgroups (all p-values > 0.05; Table 4). The difference in κ was the greatest in Session 1 for the three-category scale (κ = 0.41 for < 20 mm vs. κ = 0.57 for ≥ 20 mm), while it was the smallest in Session 4 for the two-category scale (κ = 0.51 for < 20 mm vs. κ = 0.49 for ≥ 20 mm).

Table 4 Inter-reader agreement according to the imaging sequence and size of the lesion (based on the long-axis diameter).
Full size table

Proportions of lesions diagnosed as a thymic cyst

The number of lesions diagnosed as a thymic cyst by all readers was 37, 27, 32, and 32 out of 55 in Sessions 1, 2, 3, and 4, respectively, and 16 and 17 out of 21 in Sessions 5 and 6. All readers diagnosed lesions as a thymic cyst most frequently in Session 1 (81.4%), whereas the smallest proportion of thymic cysts was diagnosed in Session 2 (69.5%, p < 0.05; Supplementary Table 4). With regard to each reader, both Readers 1 and 3 diagnosed cysts in significantly fewer patients when a subtracted image was provided first (from Session 1 to Session 2: 78.2–58.2% for Reader 1; 78.2–61.8% for Reader 3; and all p-values < 0.05), whereas Readers 2 and 4 consistently diagnosed cysts in a similar proportion of patients regardless of the image sequence combination (all p-values > 0.05).

Discussion

Despite the substantial role of chest MRI for the diagnosis and follow-up of thymic cysts, information about inter-reader agreement and optimal MR sequences is still limited. We found moderate inter-reader agreement (κ = 0.50–0.57) for diagnosing thymic cysts measuring ≤ 3 cm, with no significant differences between MR sequence combinations or with the addition of CT (all p-values > 0.05).

Thymic cysts usually manifest as well-defined saccular or oval lesions with signal hyperintensity on T2WIs and without contrast enhancement on pre- and post-contrast T1WIs. Recently, Ackman et al. reported that most thymic cysts (98%, 39/40) showed smooth wall enhancement on MRI and that the wall thickness ranged from 0.9 to 3 mm2. Considering these features, the diagnostic criteria for thymic cysts were established for this agreement study. In our study, the inter-reader agreement was moderate across MR sequence combinations, which indicates that diagnosing thymic cysts with MRI may be prone to substantial inter-reader variability. Furthermore, this variability could be attributed to the subjective nature of evaluating contrast enhancement, particularly when it is peripheral and eccentric. Indeed, more than one-third of lesions (21/55) from Institution A and half of those (11/21) from Institution B showed wall enhancement that was thin, smooth, or eccentric. In addition, the pseudo-enhancement phenomenon on MR14 may also hamper assessment of enhancement9.

Given the difficulty in determining contrast enhancement, especially of lesions with signal hyperintensity on T1WIs15, we initially hypothesized that subtracted images would aid in increasing inter-reader agreement by providing a more objective assessment of enhancement. However, the agreement did not increase after the addition of subtracted images (κ = 0.53–0.52), and the proportion of lesions diagnosed as thymic cysts significantly decreased (81.4% [179/220] to 69.5% [153/220]). The most significant pitfall of subtracted images is their susceptibility to misregistration artifacts, which may mimic eccentric wall enhancement and lead to an incorrect diagnosis of a tumor such as cystic thymoma. Moreover, when a lesion exhibits subtle enhancement, the readers’ determination of enhancement tends to be more subjective15. These weaknesses of subtracted images resulted in a wide range of Cohen’s κ (0.36–0.84) in Session 2, which included subtracted images. Fortunately, the concurrent availability of a subtracted image and DWI/ADC map compensated for the wide range of inter-reader agreement and achieved the highest mean pairwise agreement (Session 3; κ = 0.65). Based on our results, MR sequences including both a subtracted image and DWI/ADC map may be optimal in terms of inter-reader agreement. In addition, given that the agreement was only moderate despite the technical advances in MR imaging, it seems necessary to develop a more effective algorithm for interpreting these lesions.

The addition of CT increased the inter-reader agreement from moderate (κ = 0.48) to substantial (κ = 0.74) only in the external reading set (internal reading set, κ = 0.57–0.50). However, interestingly, the proportion of diagnosing cyst with CT did not significantly change even in the external reading set. This finding implies that the readers determined the diagnosis of a thymic cyst based on MR characteristics and that the role of CT was very limited. Chest CT characteristics such as lesion shape or wall calcification2,16,17 may aid only in images from an unfamiliar MR vendor or protocol with ambiguous MR signal intensity or indeterminate enhancement. The multicenter design of our study reflects various clinical settings and facilitated evaluation of the effects of these different settings and their protocols on inter-reader agreement in the diagnosis of a thymic cyst. Nevertheless, given the routine clinical circumstance that most thymic lesions are incidentally identified on CT and information obtained from CT, interpretation of available CT and MRI scans together may be reasonable.

There are several limitations to our study. First, a diagnostic accuracy evaluation was not performed because histologic or clinical reference standards were not available for all cases. One of the reasons is that surgical resection is rarely performed once a lesion is suspected to be a thymic cyst. Indeed, only one of 76 patients in our study was pathologically diagnosed with thymic cyst. Therefore, the pathologic diagnosis as an inclusion criterion for all study population may introduce selection bias or spectrum bias. Instead, we have attempted to include as many lesions as possible, thereby reflecting real clinical practice. Second, the study population was small, and selection bias may have been present because only patients with all required MR sequences and CT scans were included. Nonetheless, we evaluated the inter-reader agreement in a larger number of patients than in a previous study, which included 15 patients9. Third, discrepancies in image quality, due to the use of different MRI machines, may have affected interpretation or agreement. Although this also needs to be assessed, our goal was to evaluate readers’ interpretation or agreement in real clinical practice, which involves various MRI protocols and machines.

In conclusion, the overall agreement for the diagnosis of thymic cysts on MRI was moderate. MR sequences including both the subtracted image and DWI/ADC map may be optimal in terms of inter-reader agreement. Further validation with histologic findings or a clinical reference is necessary to verify our results.