Introduction

Obsessive-compulsive disorder (OCD) stands as a prevalent and serious mental health condition, affecting approximately 1–3% of adults over their lifetime [1]. This disorder exhibits remarkable heterogeneity, with patients presenting diverse clinical manifestations and variable responses to pharmacotherapy [2, 3] to extent that cases with OCD can exhibit non-overlapping symptom profiles [4]. To address the heterogeneity, clinical psychiatrist usually stratifies OCD patients into subtypes based on clinical symptoms, such as washing and checking [5]. However, while this approach has been successful, it also possesses limitations. Patients across different subtypes may still share similar aberrant patterns [6, 7], and a universally accepted classification system remains elusive [2]. Furthermore, clinical symptoms interact intricately with biological mechanisms and may evolve throughout the course of the illness [8, 9]. The identification of neurophysiological subtypes within OCD holds promise for gaining a more objective comprehension of the disorder’s underlying biological mechanisms and enhancing personalized diagnostic and therapeutic strategies. Nonetheless, reliably pinpointing such subtypes remains a challenge. In recent years, researchers have increasingly turned to objective neuroimaging data to tackle this issue within mental health disorders [10].

Previous structural neuroimaging investigations have explored group-level gray matter morphological disparities in OCD, highlighting abnormalities in brain regions including the medial frontal gyrus, hippocampus, precuneus, and insula [11,12,13,14]. However, the considerable intersubject heterogeneity poses challenges in identifying reproducible neuroimaging biomarkers for clinical decision-making [15, 16]. Despite this heterogeneity, many neuroimaging studies in OCD continue to utilize traditional case-control methodologies, primarily aimed at capturing group-level effects [17]. Yet, it’s been demonstrated that such approaches overlook neuroimaging characteristics unique to individuals [18,19,20,21]. To address this heterogeneity, the normative model has emerged as a potential solution [20]. This model determines individualized neuroimaging anomalies by evaluating deviations from the normal distribution [20]. Leveraging the normative model, neuroimaging inquiries have successfully pinpointed subject-level structural brain alterations in various mental disorders such as autism, schizophrenia, and depression [8, 18, 22,23,24]. However, whether individualized structural deviations could reveal OCD subtypes remains to be elucidated.

Recent neuroimaging studies have shown that neuropsychiatric disorders affect intrinsic brain networks. Structural brain abnormalities, originating in specific local brain regions known as ‘disease epicenters’, spread to other brain regions along with the pathways defined by normal brain network architecture, supporting the network-based spreading hypothesis [25,26,27,28]. Different psychiatric disorders and their subtypes have distinct disease epicenters, reflecting their unique biological underpinnings and progression patterns [29, 30]. In OCD, a similar association between structural brain abnormalities and normal brain network has also been observed [31, 32]. Furthermore, the availability of a whole-brain atlas of neurotransmitter receptors and transporters has enabled researchers to explore the molecular substrates underlying these structural and functional brain abnormalities [33]. Both local molecular factors and the global brain network architecture shape cortical abnormalities in psychiatric disorders [29]. Understanding the molecular bases of these structural abnormalities helps connect neuroimaging findings to the biological mechanisms underlying OCD.

In this study, we aimed to identify distinct OCD subtypes based on individualized gray matter morphological abnormalities compared to normative expectations. We recruited 100 first-episode, untreated patients diagnosed with OCD and 106 healthy controls, who underwent structural imaging scans. First, we created individualized deviation maps to capture gray matter morphological abnormalities using normative modeling and used these maps to determine OCD subtypes. We then performed a series of sensitivity analyses to assess the reproducibility and generalizability of the identified subtypes. Subsequently, we examined the disease epicenters of these subtypes, hypothesizing that they would exhibit divergent disease epicenters. Additionally, to gain deeper insights into the molecular underpinnings of these structural abnormalities, we explored the associations between the structural brain abnormalities in these subtypes and the distribution of neurotransmitter receptors and transporters.

Methods

Participants

This study adhered to the principles outlined in the Helsinki Declaration and received approval from the research ethics committee of the First Affiliated Hospital of Zhengzhou University. Before participating, each individual provided informed consent.

A total of 100 first-episode, untreated patients diagnosed with OCD were recruited from outpatient services within the Department of Psychiatry at the First Affiliated Hospital of Zhengzhou University. Diagnosis was established independently by two experienced psychiatrists according to the criteria outlined in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) for OCD. The severity of symptoms was evaluated using the Yale–Brown Obsessive Compulsive Scale (Y-BOCS) [34]. Exclusion criteria ensured that none of the patients had a history of neurological disorders, brain trauma, concurrent mental or psychotic disorders, or first-degree relatives with similar conditions. Healthy controls (HCs, n = 106) were enlisted from the community via poster advertisements. All participants were of Han Chinese ethnicity, right-handed, and underwent screening to ensure they met specified exclusion criteria. These criteria included abstaining from medications such as anesthetics and analgesics within the preceding month, no history of substance abuse, brain tumors, traumatic brain injury, surgeries, or organic body diseases, as well as the absence of cardiovascular diseases, diabetes, or hypertension. Additionally, individuals with contraindications for MRI scanning or other structural brain abnormalities were excluded from the study.

Scan acquisition

High-resolution T1-weighted anatomical images of participants were acquired using on 3-Tsela GE Discovery MR750 scanner (General Electric, Fairfield Connecticut, USA) using the following parameters: repetition time = 8164 ms, echo time = 3.18 ms, inversion time = 900 ms, flip angle = 7 degrees, resolution matrix = 256 × 256, slices = 188, thickness = 1.0 mm and voxel size = 1 × 1 × 1 mm3.

Imaging data processing

Voxel-wise gray matter volume (GMV) for each participant was derived using voxel-based morphometry in the CAT 12 toolbox (http://dbm.neuro.uni-jena.de/cat12/). The standard CAT 12 pipeline was followed, as detailed in reference [35]. This pipeline encompassed bias-field correction, brain segmentation into gray and white matter and cerebrospinal fluid, correction for partial volume effects, normalization to Montreal Neurological Institute space, resampling to 1.5 mm³, and nonlinear modulation [35, 36]. Subsequently, the gray matter maps underwent smoothing with a 6-mm full width at half maximum Gaussian kernel. Additionally, the total intracranial volume (TIV) of each participant was computed. To ensure data quality, the Image Quality Rating (IQR) was recorded [24].

Subtyping analysis based on individualized gray matter morphological abnormalities

First, we estimated individualized gray matter morphological abnormalities by employing the normative modeling. Following methods used in previous studies [18, 20, 24], we trained a Gaussian process regression model to estimate the normative range of regional GMVs from age and sex in HCs according to the brain connectome atlas [37]. This trained model was then applied to each patient to generate predicted GMVs. Subsequently, Z scores were calculated to quantify deviations of the predicted GMVs from normal distributions. Positive Z scores indicated higher GMVs in patients compared to HCs, and vice versa. This process resulted in a Z-score vector (246 × 1) for each patient, representing their individualized gray matter morphological abnormalities.

Next, we employed the k-means algorithm to identify OCD subtypes, with the Z scores serving as features and the squared Euclidean distance as the distance metric. The optimal number of subtypes, ranging from 2 to 10, was determined using silhouette values [38]. For each number, the k-means algorithm was repeated 100 times to avoid local minima during the initialization of centroid positions [39].

Subsequently, for each subtype, we examined the gray matter morphological abnormalities relative to HCs using a two-sample t-test, with sex, age, and TIV as covariates.

Reproducibility analysis

A series of sensitivity analyses were undertaken to evaluate the reproducibility of the clustering outcomes. Firstly, to mitigate the influence of a small subset of patients, we randomly selected 90% of the patients and conducted k-means clustering on this subset. We then computed the Adjusted Rand Index (ARI) between the subtype assignments based on this subset and those derived from the entire patient cohort. This process was repeated 1000 times. Secondly, we validated clustering outcomes by employing another brain atlas with different resolutions, specifically the Automated Anatomical Labeling (AAL) atlas. The AAL atlas contains 90 cortical and subcortical regions (excluding the cerebellum) [40]. This validation step ensured that our results were not contingent on the specific brain parcellation scheme used in the analysis, thereby enhancing the reproducibility of our findings. Thirdly, we validated our results using another distance metric in the k-means algorithm, namely correlation distance (1 min Pearson’s correlation coefficient).

Disease epicenter mapping for each subtype

Subsequently, we explored whether structural brain abnormalities of the identified subtypes exhibited distinct disease epicenters. We evaluated the normal brain network using the structural covariance network. This network characterizes the coordination of regional volumes among brain regions, potentially reflecting shared developmental pathways [41,42,43] and has been implicated in the pathology of OCD [43, 44]. Notably, our previous research has indicated that the structural covariance network demonstrated a stronger association with gray matter morphological abnormalities compared to the functional network in mental disorders including OCD [32, 45].

The normal structural covariance network was constructed using another extensive dataset from a single site, comprising 492 healthy individuals aged 19–80 years. Further details about this dataset are available elsewhere [46]. The structural covariance network was constructed by computing pairwise correlation coefficients of regional gray matter volumes across subjects, resulting in a 246 × 246 structural covariance (SC) network where negative correlations were set to zero [30]. We examined the relationship between the SC network and the gray matter morphological abnormalities of the identified subtypes, employing methodologies established in prior studies [28, 30]. For each subtype, the normalized collective abnormalities/differences of structural neighbors of region i (\({D}_{i}^{{sc}}\)) were obtained as follows:

$${D}_{i}^{{sc}}=\frac{1}{{N}_{i}}\mathop{\sum }\limits_{j=1,j\ne i}^{{N}_{i}}{A}_{i}\times {{\rm{SC}}}_{i,j}$$

Where \({D}_{i}^{{sc}}\) represents the normalized collective abnormalities of the structural neighbors of region i, Di is regional gray matter morphological abnormalities (unthresholded t statistic) of region i, Ni is the number of neighbors of region i with a structural covariance connection, SCi,j is the strength of structural covariance between region i and region j. For each subtype, the \({D}_{i}^{{sc}}\) was predicted using regional abnormalities of the neighboring regions. The Pearson’s correlation coefficient between true abnormalities (Di) and predicted ones (\({D}_{i}^{{sc}}\)) across all brain regions was calculated.

A brain region was considered as the disease epicenter if, along with its connected neighbors, it exhibited pronounced atrophy than other regions [28, 30]. Brain regions were ranked based on their atrophy, and SC-informed abnormalities (\({D}_{i}^{{sc}}\)) in ascending order. Then, the average ranking values were considered as the disease epicenter likelihood rankings, and significance was determined through permutation testing (10,000 times).

Association between neurotransmitter receptors/transporters with gray matter morphological abnormalities of the identified subtypes

We then investigated the link between neurotransmitter receptors/transporters profiles and gray matter morphological abnormalities of the identified subtypes. These profiles were obtained from PET-derived neurotransmitter receptors/transporters atlas shared by Hansen et al. [33]. The neurotransmitter receptors/transporters include serotonin (5HT1A [47], 5HT1B [47,48,49,50,51,52,53], 5HT2A [54], 5HT4 [54], 5HT6 [55, 56], 5HTT [54]), norepinephrine (α4β2 [53, 57], M1 [58], VAChT [59, 60]), cannabinoid (CB1 [61,62,63,64]), dopamine (D1 [65], D2 [66,67,68,69], DAT [70]), GABA (GABAa [71]), histamine (H3 [72]), glutamate (mGluR5 [73, 74], NMDA [75, 76]), opioid (MOR [77]), and norepinephrine (NET [78,79,80,81]). PET images were averaged across participants within each study, normalized to the MNI-ICBM 152 non-linear 2009 template, and then parcellated into 246 brain regions. The average regional neurotransmitter receptor/transporter densities were then Z-scored [29, 33].

For each subtype, we developed a multilinear model linking neurotransmitter receptors/transporters profiles with gray matter differential pattern. The significance of these models was assessed through permutation testing (10,000 times), false discovery rate (FDR) correction.

Subsequently, to better understand the relative importance of each predictor (in this case, neurotransmitter receptor/transporter profile) in contributing to the model’s overall predictive power, we conducted a dominance analysis for each multilinear model. Dominance analysis aids in determining how each predictor influences the model’s fit by considering all possible predictor combinations within the model [82]. This involves applying the multilinear model to every potential subset of predictors, resulting in 2n–1 subset models for a model with n predictors. In our study, we utilized the total dominance to assess each predictor’s relative significance [29, 82]. Total dominance is the mean increase in explained variance (R2) contributed by each predictor across all subset models in which it is included [82, 83].

Results

Clinical demographics

Patients with OCD did not demonstrate significant differences from HCs regarding age, sex, TIV and image quality (IQR). However, in comparison to HCs, individuals with OCD showed significantly fewer years of education (p < 0.001). These results are outlined in Table 1.

Table 1 Demographic and clinical characteristics of participants.
Full size table

Two subtypes manifesting opposite gray matter morphological abnormalities are identified

Using individualized gray matter morphological abnormalities as features, we identified two OCD subtypes, exhibiting contrasting patterns of gray matter morphological abnormalities relative to HCs, as shown in Fig. 1. Subtype 1 had significantly increased gray matter volume in the frontal gyrus, precuneus, insula, hippocampus, parahippocampal gyrus, amygdala, and temporal gyrus (FDR corrected p < 0.05). Conversely, subtype 2 showed significantly decreased gray matter volume in the frontal gyrus, precuneus, insula, superior parietal gyrus, temporal gyrus, and fusiform (FDR corrected p < 0.05). When considering all patients together, there were no significant gray matter volume differences compared to HCs.

Fig. 1: Subtyping results.
figure 1

A Silhouette values for each number of subtypes. B Gray matter morphological abnormalities of the identified subtypes (FDR corrected p < 0.05).

Full size image

Regarding demographic and clinical variables, subtype 1 had a higher proportion of male patients than subtype 2 (χ2 = 16.161, p < 0.001). Other variables, such as age of onset, illness duration, and symptom severity, did not differ significantly between the subtypes (p > 0.05). Additionally, there were no significant differences between the subtypes in TIV and IQR, indicating that these factors did not influence our results. These findings are summarized in Table S1.

Reproducibility analysis results

To assess the reproducibility of the clustering results, we validated them in randomly selected subsamples and using another brain atlas with different resolutions. These results confirmed the robustness and reproducibility of our results. Specifically, the average ARI values (Fig. 2A) subtyping results from the entire patient cohort and those from randomly selected subsets were 0.98 (±0.04). This high ARI alleviates concerns about biases potentially introduced by individual patients. Furthermore, the optimal number of subtypes remained two when using an alternative brain atlas (AAL), as evidenced by silhouette values in Fig. 2B. The two subtypes identified with the AAL brain atlas also exhibited contrasting gray matter morphological abnormalities (FDR corrected p < 0.05) compared to HCs, as shown in Fig. 2C. The ARI between subtype assignments using different brain atlases reached 0.74, demonstrating a good level of consistency across different parcellation schemes. Thirdly, when using correlation metric in the k-means clustering, the optimal number of subtypes was still two for both brain atlases. The corresponding silhouette values are presented in Fig. S1. The ARI between subtype assignments using different distant metrics was 0.62 for brain connectome atlas and 0.62 for AAL. Overall, these results confirm the reproducibility and robustness of our subtyping outcomes.

Fig. 2: Reproducibility results.
figure 2

A Adjusted Rand Index (ARI) values between the subtyping results based on randomly selected subsamples and those based on all patients. B Silhouette values for each number of subtypes using the AAL brain atlas. C Significant gray matter morphological abnormalities (FDR corrected p < 0.05) relative to healthy controls for the identified subtypes using the AAL brain atlas.

Full size image

Subtype exhibit divergent structural covariance network-informed disease epicenters

In both subtypes, we found significant correlations between regional gray matter morphological abnormalities and SC-informed abnormalities (subtype 1: r = 0.44, permutation pFDR < 0.001, as shown in Fig. 3A; subtype 2: r = 0.50, permutation pFDR < 0.001, see Fig. 3A). These results indicate that the observed gray matter morphological abnormalities in both subtypes are influenced by the normal SC network.

Fig. 3: Association between gray matter morphological abnormalities of the identified subtypes and normal structural covariance network (SC).
figure 3

A Pearson’s correlation coefficients between regional differential patterns and the normalized collective abnormalities/differences of structural neighbors (SC-informed differential pattern) in the identified subtypes. B Distributions of putative disease epicenters for each subtype.

Full size image

Additionally, we observed distinct disease epicenters in the two subtypes (see Fig. 3B). Specifically, subtype 1 exhibited disease epicenters mainly in the middle frontal gyrus (permutation p < 0.01). In contrast, subtype 2 displayed disease epicenters in the striatum, thalamus and hippocampus (permutation p < 0.01).

Gray matter morphological abnormalities of subtypes exhibit distinct associations with neurotransmitter receptors/transporters

We further investigated the connections between neurotransmitter receptors/transporters and gray matter morphological abnormalities in the identified subtypes. This investigation involved constructing two multilinear models that accounted for the spatial distributions of receptors/transporters and the differential patterns of gray matter morphological abnormalities in each subtype. The goodness-of-fit of these models (adjusted R2) was 0.43 (F-statistic (246,226) = 10.80) for subtype 1 and 0.34 (F-statistic (246,226) = 7.76) for subtype 2, as illustrated in Fig. 4A. Importantly, all FDR-corrected permutation p-values were less than 0.001 (Fig. 4B). Moreover, our analysis utilizing dominance analysis revealed key neurotransmitter receptors associated with each subtype. Specifically, 5HT1A emerged as significant for subtype 1, whereas 5HT2A was identified as crucial for subtype 2 (see Fig. 4C).

Fig. 4: Association between gray matter morphological abnormalities of the identified subtypes and distribution of neurotransmitter receptors/transporters.
figure 4

A Multilinear models are constructed to analyze the association between neurotransmitter receptors/transporters and gray matter morphological abnormalities of the identified subtypes. The bar plot displays the corresponding model goodness-of-fit (adjusted R2), with all FDR-corrected permutation p-values being less than 0.001. B Permutation testing results. C Dominance analysis is employed to determine the relative importance of predictors for each multilinear model. Total dominance values, indicative of the predictors’ relative importance, are depicted.

Full size image

Discussion

To address the heterogeneity of structural brain abnormalities in OCD, we identified two robust subtypes using subject-level gray matter abnormalities, assessed using the normative model of gray matter. These subtypes demonstrated opposite patterns of gray matter abnormalities relative to HCs, but shared indistinguishable clinical profiles. Interestingly, when combined, all patients showed no significantly differential gray matter abnormalities. Furthermore, subtype 1 exhibited disease epicenters in the frontal gyrus, while subtype 2 showed them in the striatum, thalamus and hippocampus. Additionally, we found distinct associations between gray matter morphological abnormalities in these subtypes and neurotransmitter receptors/transporters. Specifically, the abnormal pattern in subtype 1 was primarily linked to 5HT1A, whereas that in subtype 2 was mainly associated with 5HT2A. These findings provide insights into the underlying molecular mechanisms of these subtypes. In summary, our results reveal two reproducible subtypes characterized by distinct gray matter morphological differential patterns and underlying molecular bases. These insights contribute significantly to the nosology of OCD, enhancing our understanding of its heterogeneous nature.

Psychiatric disorders, including OCD, are well-known for their heterogeneity among cases. Patients with OCD exhibit a wide range of symptom profiles, disease trajectories and treatment responses [2, 3]. This intersubject variability leads to conflicting findings in neuroimaging studies [84, 85]. For instance, while the hippocampus is a key region in OCD pathology, studies report both increased and decreased gray matter volumes in this area [86,87,88]. Inconsistent findings about structural abnormalities hint the existence of OCD subtypes. In the current study, we unveiled two distinct OCD subtypes with contrasting anormal patterns. Subtype 1 only demonstrated increased gray matter volume, while subtype 2 only decreased gray matter volume in widespread brain regions. Most of these brain regions showing structural abnormalities are overlapped (e.g. hippocampus). However, when combining all patients, no significant structural abnormalities emerged. These results, indicating that previous inconsistent findings may stem from differing subtype proportions [86,87,88]. These subtypes align with previous studies using various clustering algorithms and neuroimaging metrics [87, 89]. In spite of tremendous differences in structural brain abnormalities, these two subtypes demonstrated indistinguishable clinical profiles. Despite structural differences, the subtypes had similar clinical profiles, suggesting that symptom-based subtyping alone may miss underlying neuroanatomical distinctions. Moreover, sensitivity analysis results excluded the potential factors (e.g. imaging quality) on our results and verified the reproducibility of the identified subtypes.

Another significant finding is the observation of divergent disease epicenters within the identified subtypes, as informed by the structural covariance network. Previous neuroimaging researches suggest that structural brain abnormalities in psychiatric disorders tend to adhere to the normal connectome architecture rather than being randomly distributed [28, 32]. Typically, these abnormalities originate from specific disease epicenters and then propagate to other brain regions, likely reflecting shared developmental or maturation trajectories [25, 27, 28, 42, 43]. Our previous studies have established a link between normal brain networks and gray matter morphological abnormalities, supporting the network-based spreading model in OCD [31, 32]. In the current study, we found that the identified subtypes exhibited divergent disease epicenters in regions such as the hippocampus, striatum, and insula, which are consistently implicated across various psychiatric disorders [90]. These regions display structural abnormalities early in the course of illness and may exert causal effects on the structural abnormalities observed in other brain regions [91]. Divergent disease epicenters in these subtypes hint distinct pathological progression trajectories in the identified subtypes. This hypothesis could be confirmed in future studies using longitudinal data.

To explore the biological mechanisms underlying these structural abnormalities, we examined their associations with neurotransmitter receptor/transporter profiles. Neurotransmitter dysfunction is a key aspect of psychiatric disorders, with current antidepressants and antipsychotics targeting neurotransmitter functions [29]. The serotonergic system, in particular, plays a crucial role in OCD pathogenesis, with serotonin reuptake inhibitors being the first-line pharmacotherapy [92, 93]. While neurotransmitter receptor profiles are linked to structural and functional dysfunctions in psychiatric disorders [29, 33], up to half of all OCD patients do not benefit from this combination treatment in a clinically meaningful way [93]. This prompts efforts aimed at identifying diagnostic subgroups with specific pathogenic mechanisms that may mediate treatment resistance, as well as improving treatment options. Our study found that subtype 1 abnormalities were predominantly linked to 5HT1A, while subtype 2 was mainly associated with 5HT2A, providing insights into the distinct molecular mechanisms of these subtypes. These receptors are implicated in the neuropathophysiology of OCD. For instance, increased 5HT2A availability in the caudate nucleus has been reported in OCD patients [94]. Second-generation antipsychotics may induce or worsen obsessive-compulsive symptoms through 5-HT2A receptor blockade in regions like the anterior cingulate cortex, the dorsal posterior prefrontal cortex and the orbitofrontal cortex [95, 96]. Modulation of 5HT1A and 5HT2A is also critical to the antidepressant efficacy of 5-HT reuptake inhibitors [97, 98]. These findings suggest that the identified subtypes may exhibit distinct treatment responses, which could be explored in future research.

This study has several limitations. First, it was conducted using a cross-sectional sample, raising questions about the stability and generalizability of the identified subtypes. Future studies should validate our results with independent and longitudinal datasets to confirm their robustness. Second, the limited clinical information available restricted our ability to explore potential differences in other clinical aspects between the subtypes. Comprehensive clinical data in future studies could provide deeper insights into subtype characteristics. Especially, the multivariate correlation analysis between neuroimaging feature and clinical characteristics, and the potential of individualized abnormalities to predict the prognosis can be further investigated [99,100,101,102]. Third, individual variations in neurotransmitter systems may affect the observed associations between structural abnormalities and neurotransmitter systems, which were based on publicly available datasets [103]. Finally, patients with OCD in our study had lower educational levels compared to healthy controls. However, we found no significant differences in educational level between the subtypes, suggesting minimal impact on the clustering results. Further investigation into the effects of educational level would still be valuable.

In conclusion, we identified two robust OCD subtypes based on individualized gray matter morphological abnormalities. These subtypes exhibit contrasting patterns of gray matter morphological abnormalities relative to normal population and have divergent structural covariance network-informed disease epicenters. Additionally, these abnormalities are distinctly associated with neurotransmitter receptors/transporters, indicating distinct molecular underpinnings. These findings offer novel insights into nosology and heterogeneous nature of OCD.