Abstract
Race is rapidly and effortlessly extracted from faces. Previous fMRI studies have reported race-related modulations in the bilateral Fusiform Face Areas (FFAs) and Occipital Face Areas (OFAs) during the categorization of faces by race. However, our recent findings revealed a comparable Other-Race Categorization Advantage between a well-studied case of pure acquired prosopagnosia—patient PS—and healthy controls. Notably, PS demonstrated faster categorization by race of other—compared to same-race faces, similar to healthy participants, despite sustaining lesions in the right OFA (rOFA) and left FFA (lFFA). This observation suggests that race processing can occur effectively even with damage to core face-sensitive regions, challenging the functional significance of race-related activations in the rOFA and lFFA observed in healthy individuals with fMRI. To address this apparent contradiction, we tested PS and age-matched controls during the categorization by race of same- to other-race morphed faces. Our data showed that PS required more visual information to accurately categorize racially ambiguous faces, indicating that intact rOFA and/or lFFA are crucial for extracting fine-grained racial information. These results refine our understanding of the functional roles of these key cortical regions and offer novel insights into the neural mechanisms underlying the perception of face race and prosopagnosia.
Similar content being viewed by others
Introduction
Race-related information is rapidly extracted from faces and can significantly influence interpersonal and social dynamics. Interestingly, when participants are asked to explicitly categorize faces by race, they are faster and, in some cases, more accurate to Other-Race (OR) than to Same-Race (SR) faces1,2,3,4,5,6. This robust perceptual phenomenon is often referred to as the Other-Race Categorization Advantage (ORCA). Valentine7,8, inspired by the categorization literature9,10 proposed an exemplar-based theoretical framework to explain the recognition and race categorization findings of the other-race effect. In this model, a face is encoded as a point in an n-dimensional space constructed through experience with encountered faces. The dimensions represent factors (e.g., physiognomic features such as hair color and eye color, and width) that maximize discrimination between faces. The origin of the face space corresponds to the mean of all dimensions, where typical faces, encountered more frequently, are located. Distinctive faces, in contrast, are positioned farther from the origin. In this face-space model, visual experience shapes the dimensions, and limited exposure to OR faces clusters their exemplars in a high-density area, while SR faces are more widely distributed across dimensions. This high-density clustering for OR faces arises from less variation in dimensions optimized for SR face discrimination. For example, hair and eye color are diagnostic for Caucasian faces but not for Asian faces, leading to a broader distribution for Caucasian faces. This differential density results in the perceived similarity of OR faces, often expressed as the belief that OR faces “all look alike“11,12. The model explains the poor recognition performance for OR faces due to the increased difficulty in discriminating exemplars in high-density clusters, with all the other-race faces sharing a high similarity to each other. Simultaneously, the classification advantage for OR faces stems from faster activation of individuals within the smaller cluster representing OR faces. Thus, this framework provides an elegant account of how visual experience shapes face space representations and behavioral performance8. This theoretical model has been statistically validated by neural network simulations (Caldara and Abdi13).
Functional neuroimaging studies investigating the neural representation of the ORCA have revealed the critical involvement of two prominent face-sensitive regions: the Fusiform Face Areas (FFAs) located in the middle bilateral fusiform gyri, and the Occipital Face Areas (OFAs), located in the ventral lateral part of the occipital cortex. More specifically, during a face race categorization task, Feng et al.14 observed greater neural responses in both the FFAs and OFAs to SR compared to OR faces. Similarly, Contreras and colleagues15 observed that the right and left FFAs were more strongly correlated for same- than other-race faces, which overall indicates that the FFAs play a critical role in processing race from faces. Hughes et al.16 asked participants to detect the presence of an oddball (i.e., a face outline with a noisy pattern) among a stream of faces using an fMRI adaptation paradigm, while testing how face-selective brain areas responded to variations in physical similarity among races. The results revealed greater release from adaptation –an increased neural response– in the bilateral FFAs with increasing morph levels disparity between two SR morphed faces as compared to OR ones. In other terms, the FFAs exhibited stronger responses to subtle variations in familiar SR faces as opposed to less familiar OR faces. Finally, a larger signal difference in the left FFA between SR and OR faces was also found to be associated with poorer SR categorization performance14. Altogether, these results suggest that the left FFA plays a critical role in the magnitude of the ORCA.
Although previous neuroimaging studies suggest a significant relationship between race processing and the neural responses in face-sensitive areas, none has yet explored the existence of a causal involvement of those areas in the processing of face race. To address this gap in the literature, we recently investigated whether extensive brain lesions to the left FFA and right OFA would abolish the ORCA17. We tested patient PS, a well-studied case of pure acquired prosopagnosia who sustained a closed-head injury in 1992 leading to extensive bilateral occipito-temporal lesions encompassing the left FFA, the right OFA, as well as a small region in the right Anterior Temporal Lobe (ATL) (see, Sorger et al.18, for a complete anatomical report of her lesions; Rossion19,20, for a review). As a result of her lesions, patient PS is unable to recognize previously seen familiar faces, including the faces of her close relatives. To examine whether face-sensitive areas play a causal role in the ORCA, patient PS, healthy age-matched controls and young adults performed three categorization by race tasks with a range of naturalistic Western European (WE) and East Asian (EA) face stimuli (i.e., with/without external facial features, with/without background). As expected, patient PS was slower to categorize SR and OR faces by race than the controls. Yet, critically, she exhibited a distinct other-race categorization advantage in all the tasks, comparable to that of the control participants. These findings demonstrate that prosopagnosia does not abolish face categorization abilities, but also that an intact face-cortical network is not necessary to perceive and categorize visual race signals in faces, as this information might be effectively encoded in extended brain areas. Crucially, we thus hypothesized that the right OFA and left FFA are not causally engaged in face by race categorization. This hypothesis inherently questions the functional role of the neural activations14,15,16 and the respective correlations with the behavioral performance observed in healthy observers during this perceptual task14.
It is worth noting that the ORCA is typically characterized as a response time categorization advantage for other race faces, where accuracy in control participants tends to be close to ceiling when utilizing typical face stimuli5,14,21. However, when putting additional pressure on the visual processing system by using visually degraded face stimuli (by systematically adding random visual noise to the face images) a significant accuracy effect in control participants becomes obvious where at a critical threshold of noise, other-race faces are still able to be perceived and categorized while same-race visual signals are no longer perceived3. This effect highlighted the specific enhancement of other-race visual signals in terms of saliency, which only became apparent through taxing the visual system and making the task more difficult. Such an effect should be considered in relation to the question of how race categorization might manifest in an impaired face processing system such as in prosopagnosia, where the ability to combine both fine-grained and comprehensive face representations is compromised15. A standard race categorization task that requires discriminating between same- and other-race faces might not tap into the specific impairments that characterize prosopagnosia. However, visually degrading face stimuli might also not represent an optimal experimental approach to observe a deficit in fine-grained analysis, as the process intrinsically erodes such visual features. Accordingly, to provide a better understanding of how PS perceives and categorizes race when judging more nuanced race differences, we employed the use of face stimuli that contain systematically controlled proportions of same- and other-race visual signals. We have previously used a similar psychophysical approach manipulating the quantity3,5,22,23,24,25,26 and intensity24 of face signal during the decoding of facial expressions of emotion to successfully overcome the presence of ceiling effects from using full-face signals27. Henceforth, here we used the same logic and postulated that these face sensitive regions might be causally engaged in the categorization of subtle, more elaborated face race representations.
To probe this hypothesis, we examined the categorization by race of SR (i.e., Western European) and OR (i.e., East Asian) faces in patient PS and healthy controls using a range of ambiguous face stimuli. These stimuli were created by averaging pairs of East Asian and Western European faces at varying weights to obtain a linear continuum of Asian-to-European morph faces. These stimuli were used in a face race categorization paradigm to assess the fine-grained categorization abilities across the morphed continuum. Differences in the psychophysical response profiles between PS and healthy controls may then be used to illustrate the relative contribution of the rOFA and lFFA during fine-grained race perception and analysis.
Results
Response frequency
We compared the proportion of “Western European” (“WE”) responses given by PS at each morph level with those of the age-matched (AM) controls (Table 1). One-tailed modified t-tests revealed that the difference between PS and the AM controls was not significant for the morph levels from 100% East Asian (EA) to 50/50 (Fig. 1; Table 1). Patient PS however showed significantly lower percentages of “WE” responses than 94.9% of the AM participants for the morph level 40/60, 93.2% of the AM controls at 30/70, 91.3% of the AM controls at 20/80, 95.2% of the AM controls at 10/90, and 98.7% of the AM controls at 100% WE. Importantly, at these morph levels from 40/60 to 100% WE, patient PS also fell outside the 99% bootstraps confidence interval boundaries of the AM controls. PS consistently categorized the faces with a higher proportion of WE morph as belonging to the EA category compared to the controls, while her categorization of the morphs with higher percentage of EA was not significantly different.
Average percentage of “Western European” responses for each morph level. Significant differences between PS (red) and the control group (yellow) as computed by one-tailed modified t-tests are depicted on the figure with black stars. The error bars represent standard deviation, the ribbons reflect the 99% normal bootstrap confidence intervals with 1000 replicates. “ * ” p < .05, “ ** ” p < .025, “***” p < .01.
PSE and JND
To evaluate the perceptual bias of PS and controls, we compared their Point of Subjective Equality (PSE) defined as the morph level at which participants were equally likely to respond “WE” or “EA”. We observed that PS exhibited a higher PSE (MPS = 6.2) compared to 82.2% of the AM controls (MAM = 5, SDAM = 0.8). However, a two-tailed modified t-test revealed that this difference was not statistically significant, t(20) = 1.4, p = .18 (Fig. 2). This observation suggests that the morph level at which PS and the control group have an equal likelihood of responding EA or WE is comparable.
To evaluate the discrimination sensitivity of PS and the AM controls, we used the Just Noticeable Difference (JND) measure. PS exhibited a significantly higher JND, indicating lower response precision (MPS = 3.3) compared to 91.7% of the AM controls (MAM = 2.3, SDAM = 0.6). This difference was statistically significant as revealed by a one-tailed modified t-test, t(20) = 1.82, p = .04 (Fig. 2). This observation indicates that PS requires a larger change in racial information to shift from chance-level (50%) responses to 75% “WE” responses compared to controls.
Plot of the psychometric functions for patient PS (red) and the AM controls (yellow) according to their computed PSE (points) and JND (segments’ distance).
Diffusion model analysis
We then compared the estimated diffusion model parameters of PS to those of the control group using two-tailed modified t-tests. Note that the z value returned by the model is normalized, ranging from 0 to 1 and taking into account the dimension of the thresholds. A value of 0.5 indicates the absence of a priori bias. PS’s z and sz values were similar to those of the control group (Table 2; Fig. 3), indicating that her perceptual bias and inter-trial variability are comparable to those of the controls. PS and AM controls have z values close to 0.5, indicating no a priori bias. However, a significant difference was observed in the a-values, representing threshold separation, between PS and the AM controls. PS’s a value was higher than 96.79% of the AM participants, suggesting that she requires more information to reach a decision. She was not significantly different from the AM controls in terms of drift rate v at the morph levels with prominent EA information, from 100 to 50% EA. However, significant differences emerged at the 40/60 morph level, where PS was lower than 96.9% of the AM controls, and persisted across the subsequent morph levels: 30/70, where PS was lower than 96.8% of the AM controls, 20/80, where PS was lower than 95.8% of the AM controls, 10/90, where PS was lower than 97.3% of the AM controls and 100% WE, where PS was lower than 96.2% of the AM controls (Table 2; Fig. 3). These results indicate that PS shows lower performance and/or greater difficulty in reaching a decision than AM controls at the morph levels with greater amount of WE signal. The change of drift rate v over trials was comparable between patient PS and the AM controls, as evidenced by a similar inter-trial variability of the drift rate sv. Finally, neither the component d nor τ significantly differed between PS and the AM controls. PS’s d value shows that she did not exhibit a preference to respond more quickly to either race, and the results of the statistical analysis suggest that the magnitude of her response pattern aligns with those of the AM controls. In addition, she took a comparable amount of time for nondecision processes as the AM controls, as revealed by the comparison of their τ values.
Representation of the main components z (starting point), a (threshold), v (drift rate), and τ (non-decision time) derived from the Ratcliff Diffusion Model based on participants data from the categorization task (PS and AM controls in red and yellow, respectively) at each morph level (from 100% EA to 100% WE). Please note that for display purposes, the z value in the graphs corresponds to the normalized z obtained from the diffusion model, subsequently multiplied by the value a. Black stars indicate a significant difference between patient PS and controls. Additionally, the bottom-right panel displays the original representation as featured in “Stability and change in diffusion model parameters over two years” by M. von Krause, 2021, Journal of Intelligence, 9(2), p.2. Copyright 2021 by the authors. “*” p < .05, “***” p < .01.
Discussion
Previous neuroimaging studies in healthy young adults have reported the involvement of the FFAs and/or OFAs in race processing during face race categorization14,15,16. However, by testing a well-studied case of pure prosopagnosia – patient PS – we recently demonstrated that the lFFA and rOFA are not causally engaged in the Other-Race Categorization Advantage (ORCA), as the magnitude of PS’s ORCA was within the normal range of healthy observers17. To reconcile this apparent discrepancy and account for the potential impact of ceiling effects from using unambiguous racial faces (i.e., 100% Western European – WE or East Asian – EA) observed in our previous study, the present study utilized stimuli that required more sensitive fine-grained race categorization. We thus aimed to investigate the impact of lesions in these brain regions with the same patient in the categorisation of racially ambiguous faces and concurrently the effects of prosopagnosia.
Patient PS and age-matched (AM) controls categorized by race Asian-to-European morphs. We first analyzed the percentage of “Western European” responses. In addition, to better understand the cognitive mechanisms underlying race perception, we examined the Point of Subjective Equality (PSE) and the Just Noticeable Difference (JND) of our participants and compared their estimated diffusion model parameters. We found that patient PS responded “I see a WE face” less frequently than the controls at morph levels objectively containing more “WE” information. Consistent with this finding, the examination of the drift rate v for the same morph levels revealed that PS required more time to accumulate information compared to the AM participants, reflecting poorer discrimination performance. The analysis of both the PSE and the parameter z of the diffusion model demonstrated that this difference was not due to a general perceptual or a priori bias. Likewise, the analysis of the diffusion model parameters d and τ, suggests this difference does not reflect variations in non-decisional processes or response execution speed for either race. Rather, the difference seems to stem purely from unequal difficulties in extracting race information. This assumption is supported by the analysis of the Just Noticeable Difference (JND) and the diffusion model parameter a. They revealed that PS exhibited lower race categorization sensitivity than the control group, as evidenced by a larger JND. This reduced sensitivity required PS to use a larger amount of information to reach a decision about the race (larger a value). In other words, PS required more changes in the morphs to elicit a shift from 50 to 75% of “WE” responses compared to the AM controls.
These findings provide novel insights on prosopagnosia, but overall reconcile the inconsistencies between previous fMRI studies on race perception and our recent results with patient PS, which demonstrated the absence of a causal involvement of the lFFA and rOFA in other-race effects with full face race signals17. In the present experiment, we demonstrated that PS exhibits lower sensitivity and greater difficulties in extracting subtle racial cues compared to age-matched controls. Importantly, our results indicate that this difference cannot be attributed to a general perceptual bias but instead reflects a genuine need for more race information to achieve correct categorization. As such, the present findings suggest that the left FFA and/or the right OFA are necessary for accurate fine-grained race categorization, an ability that might be lost in acquired prosopagnosia. This need for a greater amount of race information in PS is also consistent with previous findings showing sensitivity in the right OFA to fine-grained structural face changes. Specifically, this region exhibits neural adaptation to the succession of two identical pictures but not to the presentation of two different morphs28,29. The OFAs (Occipital Face Areas) exhibit sensitivity to any structural changes in the morphed faces, whether related to identity or expression28. As such, the lesion in PS’s right OFA may compromise her sensitivity to the detection of the structural changes present in the morphed faces, thereby impairing her ability to accurately discriminate subtle morph variations. This could account for PS’s overall higher need for race-related information to perform race categorization.
It is important to highlight a specific aspect of these findings: our results showed that PS’ lower categorization sensitivity is not uniform across all morph levels. PS’ performance was relatively preserved in categorizing by race morphs containing higher proportions of EA information, yet was critically diminished for morphs containing higher proportions of WE information. Our analysis demonstrated that the differences we observed could not be explained by a perceptual bias, that is, a systematic shift in PS’s perception leading her to consistently perceive morphs as containing more EA features. This categorical impairment is better accounted for by a neurofunctional explanation. In healthy participants, neural fine-grained race representation may consistently be performed for familiar SR faces in healthy subjects, as evidenced by stronger neural activations in the lFFA for SR faces, significantly correlating with slower categorization of these faces14. Such a pattern of results suggests that this region plays a critical role in processing SR faces in healthy participants. Further support for this hypothesis comes from the results of Hughes et al.’s study16, finding a higher sensitivity of the bilateral FFAs to morph changes for SR compared to OR faces and suggesting a fine tuning of the FFAs for SR features. However, in patient PS such automatic fine-grained processing of SR faces is no longer possible, as her lFFA is damaged. It is, therefore, not surprising to observe a decreased sensitivity to subtle SR face information in PS.
This specific difficulty with the categorization of race in SR faces suggests a deficit in the way these faces are processed. Notably, such effect appears to be an amplification of a similar effect observed in healthy participants by Michel et al.30, who also used systematically morphed face stimuli from two different racial groups. In a face-composite task, participants had to judge whether consecutive faces had the same upper part, while the lower part could either be aligned or misaligned with it. Typically, the composite effect is defined by the tendency of participants to judge two identical upper parts as different if they are aligned with different lower parts. Such an effect tends to disappear if the parts are misaligned. The authors observed a greater composite effect for SR compared to OR faces. They interpreted this pattern as reflecting greater holistic processing for SR faces and, most interestingly, as driven by different types of processing applied to SR vs. OR faces. A similar conclusion was drawn by Zhao and Bentin31 who employed a face categorization task using faces with either “normal” or “distorted” configural organization of their internal features. They observed the typical ORCA in both distorted and normally configured faces, but that it was twice as large for distorted faces. Consequently, the categorization of SR faces was disproportionately affected by distortion compared to OR ones. The authors suggested that the ORCA stems from delayed classification of SR faces (in which configural computations are automatically applied) rather than from facilitated processing of OR faces. Regardless of these potential theoretical interpretations, our results point to a particular deficit that PS exhibits for fine-grained processes required for categorizing SR faces in more ambiguous race faces.
While the present study provides an explanation on the functional role played by the lFFA and rOFA in face categorization by race, we should mention we also previously found similar inconsistencies with patient PS for the other-race effect in face recognition17. More precisely, fMRI studies reported broader and stronger activations for SR compared to OR faces in the FFAs specifically32,33 or in broader areas of the ventral occipitotemporal cortex, where the FFA and OFA are located34. Additionally, these studies also found correlations between the lFFA and the recognition performance for SR and OR races faces32,33. However, PS still better recognized SR than OR faces17. Given the present findings, a future study comparing fine-grained identity discrimination for same- and other-race faces in PS and age-matched controls is necessary to confirm whether a similar impairment will be observed for face identity discrimination.
Conclusions
Acquired prosopagnosia typically results from lesions in the bilateral occipito-temporal regions35 and selectively affects several face processing abilities. Yet, we previously demonstrated that patient PS still exhibits the other-race categorization advantage17 despite extensive lesions to the lFFA and rOFA. To further understand the role of these face-sensitive regions in race processing, patient PS and age-matched controls categorized racially morphed faces. Our findings revealed a severe decrease in sensitivity towards race information in patient PS. Specifically, PS required a greater amount of race-related information than controls to change her perception of the morphs for same-race faces. These observations reveal that prosopagnosia affects the ability to perceive fine-grained race cues in faces. These findings more specifically highlight the critical role of the lFFA and rOFA in the perception of race, suggesting that an intact face cortical network is necessary to detect subtle changes in face racial information.
Methods
Participants
Patient PS’s case report
Patient PS was born in 1950. She is a pure case of acquired prosopagnosia with normal object recognition. She underwent a closed head injury in 1992, at the age of 42 years old. The accident resulted in major lesions to her right inferior occipital gyrus and left middle fusiform gyrus – encompassing the right OFA and left FFA –, as well as damage to part of her left posterior cerebellum and a small region in the right lateral middle temporal gyrus (see, Sorger et al.18, for an exhaustive anatomical description of her lesions). With the help of neuropsychological rehabilitation, she recovered remarkably well in the months following her injury and was even able to return to her work as a kindergarten teacher.
However, despite general visual recognition abilities within the average, the patient exhibits a selective impairment in facial identity recognition (for a comprehensive review, see, Rossion19,20). Although she is slightly slower than the normal range, her visual acuity remains within the standards36. Similarly, her color discrimination ability falls within the lower but normal spectrum range18. With the exception of higher frequencies, her contrast sensitivity is well preserved, indicating intact low-level visual processing36. Her ability to discriminate object categories36, recognize objects37 or visually similar (morphed) objects and nonface artificial objects (3D geometric shapes)38 is also spared.
Nevertheless, patient PS performs below the normal range in face discrimination, gender identification36, and recognizing whether a face is familiar or not36,38. She also exhibits a deficit in judging the resemblance of two morphed faces, particularly at lower levels of discrimination when the differences between faces are minimal38. Furthermore, her ability to recognize static facial expressions is severely compromised, while she shows intact performance with dynamic faces39. Her specific deficits with static expressions and identity recognition are rooted at the level of facial information use with an overreliance on the mouth region40,41,42. Along the same line, Rossion19 proposed that her marked deficits in facial identity recognition arise from a global impairment in holistic processing, which involves integrating and analyzing facial features as a whole rather than relying on individual features alone. In other words, while she demonstrates proficiency in recognizing individual face components, she lacks the ability to holistically integrate facial identity in a comprehensive fine-grained manner. Despite her severe impairments in facial identity and static expression recognition, patient PS retains some preserved face processing abilities. For example, she is able to categorize a face as a face43, discriminate pictures with and without faces37, or identify the age of a face36. Importantly, we recently demonstrated that patient PS can still determine the race of a face and more specifically exhibits Other-Race Categorization Advantage (ORCA) with similar amplitude as controls17. That is, as any other Western European participant, patient PS recognizes stereotypical Western European (WE) faces better than East Asian (EA) ones, and categorizes by race stereotypical EA faces faster than WE ones.
Participants: age-matched controls
We tested a total of 24 age-matched controls (AM), aged 61 to 79 years (M = 67.3, SD = 5.4, 17 women) who voluntarily participated in our study in total. All participants reported having normal or corrected-to-normal vision, no neurological or psychiatric history, and were not taking any medication that could slow down their reaction time. They identified themselves and their parents as Western European, and reported not having previously resided in a non-Western European country.
Patient PS and the control participants were informed of the main goals and risks of our study, and all provided written consent prior to their participation. The ethical committee of the Department of Psychology of the University of Fribourg approved the study reported here. The experiment was performed in accordance with relevant guidelines and regulations.
Stimuli
To assess the ability of our participants to categorize European-to-Asian morphed faces by race, we extracted 30 WE (15 women) and 30 EA faces (15 women) from the Radboud44, KDEF45, Stirling46, and CUFS47 databases. These black and white faces contained no distinctive features (e.g., jewelry, glasses, scars, beard, etc.), expressed neutral facial expressions, were outlined around the edge of the forehead, chin, and cheeks to present the internal features exclusively, and were displayed on a grey background. Each WE face was paired with an EA according to its gender (i.e., each male face with another male face, each female face with another female face). These pairs were then averaged together on a linear continuum to create faces with very subtle perceptual changes using the FantaMorph software48. We kept constant 60 key points for the facial features of each EA and WE face. In the software, we modified the distance along the vector connecting each point of the first face (the first parent face) with the corresponding point of its paired face (the second parent face), by steps of 10% of the total distance. This process allowed us to obtain a gradient of face morphs of 0/100, 10/90, 20/80, 30/70, 40/60, 50/50, 60/40, 70/30, 80/20, 90/10, 100/0, where the numerator represents the percentage contribution from the EA parent face, and the denominator represents the percentage from the WE parent face (Fig. 4). We obtained a total of 330 stimuli (30 pairs * 11 morph levels) all of which were presented twice to each participant. The contrast, luminance, and spatial frequency of all stimuli were normalized with the default options of the SHINE toolbox49. The stimuli were 400 × 400 pixels in size. For the experiment, we ensured that each participant viewed the stimuli at a visual angle of 8.5° horizontally.
Examples of the continuum of morph faces created. The faces originate from the following databases: Radboud database (https://rafd.socsci.ru.nl), KDEF database (https://kdef.se), CUFS database (https://mmlab.ie.cuhk.edu.hk/archive/cufsf/), Stirling database (https://pics.stir.ac.uk).
Procedure
All participants took part in the study in a quiet room at their home, monitored by an experimenter. Participants were presented with individual faces and instructed to categorize the race of each face (East Asian or Western European) as quickly and as accurately as possible, using the “a” and “l” keys (randomized across participants). Participants were not provided any feedback on their performance. Each face remained on the screen until the participant responded by pressing the “a” or “l” key, and the faces were presented one after another until the end of the task without any breaks (Fig. 5). The task completion percentage was continuously displayed in black in the upper left corner of the screen, providing participants with a sense of progress and motivation throughout the experiment. The participants completed the experiment in approximately 40–45 min. Participants maintained a distance of approximately 57 cm from the laptop screen to ensure consistent viewing conditions. Stimulus presentation was controlled using Matlab 202050 and the Psychophysics Toolbox (PTB-351,52) displaying the faces on laptops with a refresh rate of 60 Hz and a resolution of 1440 × 900 pixels.
The paradigm of the categorization task. The faces originate from the following databases: Radboud database (https://rafd.socsci.ru.nl), KDEF database (https://kdef.se), CUFS database (https://mmlab.ie.cuhk.edu.hk/archive/cufsf/), Stirling database (https://pics.stir.ac.uk).
Data analysis
We conducted the data analysis with the use of Rstudio 2022.12.0 Build 353. Prior to analysis, we filtered our data by removing the trials with outlier response time (i.e., falling below the first quartile or above the third quartile by more than 3 times the interquartile range) for each participant individually. Six responses with a response time greater than 13 s were excluded for patient PS. For the AM controls, an average of 20.1 trials (SD = 12) per participant were removed. We also excluded three AM controls whose overall accuracy for unmorphed EA and WE faces (100%) was lower than 67% in either the EA or WE conditions, indicating a lack of task engagement. The remaining datasets from PS and the 21 AM controls were analyzed for response frequency, Point of Subjective Equality (PSE), and Just Noticeable Difference (JND).
Response frequency
We first computed the proportion of times each participant responded “I see a WE face” at each morph level. We then compared the proportion of WE responses from patient PS to those of the AM controls, using modified t-tests, a common approach for assessing the abnormality or rarity of a single case in comparison to a control sample53. Since our prior study demonstrated a decreased ability of patient PS to categorize 100% EA and WE faces, we used one-tailed t-tests across all morph levels, as we expected her to be less precise than the AM controls17.
PSE and JND
To further examine PS’ race perception, we conducted psychophysical analysis with the R package MixedPsy54. We first fitted a psychometric function to the data of each individual participant using a Generalized Linear Model. We then extracted two well-known psychophysics parameters for each participant: the Point of Subjective Equality (PSE) and the Just Noticeable Difference (JND), which estimate perceptual bias and response precision, respectively.
More precisely, the PSE represents the level of morph at which the participant is equally likely to categorize the face as WE or EA. In other words, it marks the point at which the categorization of the face becomes entirely random. No perceptual bias is observed, and accuracy is considered high when the PSE is close to the value of the 50% EA – 50% WE morph level. On the other hand, the JND is a measure of discrimination sensitivity, referring to the smallest detectable difference in intensity between two stimuli. In the present study, it refers to the difference between the morph levels that elicits 50% and the one that generates 75% of “Western European” responses. In other terms, it assesses the steepness of the psychometric curve. The smaller the JND, the higher the response precision, as the participant is more sensitive or precise at detecting small changes in the morph. We compared the PSE values obtained by PS with those of the control group using two-tailed modified t-tests. This approach was chosen because no preliminary hypothesis regarding the direction of the perceptual bias was defined, making the two-tailed test appropriate for detecting differences in either direction. For the JND, we employed one-tailed modified t-tests. This choice was based on the expectation that PS would exhibit lower response precision compared to the controls, consistent with her documented impairment in race categorization17. Additionally, Rossion19,20 suggested that PS’s fine-grained holistic processing of faces is less efficient compared to controls, suggesting that her sensitivity to face race may also be impaired. The one-tailed test was specifically selected to evaluate the hypothesis that PS’s JND values would be higher, reflecting reduced precision.
Diffusion model analysis
To further elucidate the cognitive mechanisms underlying race perception, we conducted an exploratory analysis using the Ratcliff diffusion model55,56. This model effectively translates behavioral measures obtained from two-alternative force-choice tasks, namely response time (RT) and response type, into parameters of cognitive processing. As it uses the full range of responses, both correct and incorrect, it allows for a comprehensive examination of the latency distributions of the decision-making process. This model has been empirically validated57,58,59 and used in a wide range of studies investigating face processing60,61,62,63,64,65, but also face race categorization processing more specifically66. The diffusion model assumes that two-choice decisions are made by accumulating cues over time from an initial starting point until one of two thresholds is reached, after which a response is made (“East Asian” or “Western European”).
The diffusion model is illustrated in Fig. 6, where z commonly refers to the starting point, 0 and a to the thresholds. More concretely, a z value close to the values of 0 or a is representative of an a priori bias, and sz indicates the variability of this bias across trials (i.e., much z changes over trials). A large distance between the two 0 and a boundaries (i.e., large a value) indicates that a large amount of information is required to make a decision. The information accumulation process is usually noisy as illustrated by the scrambled lines, suggesting that the decision-making process fluctuates towards both thresholds during its progression. However, an average slope, represented by the drift rate (v), can be extracted from this noisy process. The inter-trial variability of the drift rate is called sv. A large drift rate v (i.e., a steep slope) reflects a qualitative extraction of stimulus information and, therefore, faster and more efficient processing. Participants may also be prone to respond faster to one or the other answer, the component d represents this contingency. For example, in our study, a positive d value would indicate that participants responded more quickly to the « EA » option compared to the « WE » one. Additionally, non-decision processes that occur before and after a decision is made (e.g., encoding the stimuli or executing the response) also contribute to the RT. These processes are accounted for in the diffusion model by the τ parameter. Note that each of these parameters can vary across trials, just like any other behavioral measure. As such, for each participant, the mean value of these parameters is extracted.
Illustration of four parameters of the diffusion model: the non-decision time (τ), the starting point (z), the boundaries “0” and “a” associated with Response A (e.g., in our study, “I see an EA face”) and B (e.g., “I see a WE face”), and the mean evidence accumulation drift rate “v”. From “Stability and change in diffusion model parameters over two years” by M. von Krause, 2021, Journal of Intelligence, 9(2), p.2. Copyright 2021 by the authors.
To estimate the parameters of the diffusion model, we used the R package rtdists. We first fitted separate models for each participant and morph level, with two response boundaries “East Asian” or “Western European”. Consistent with Benton and Skinner, we allowed only the drift rate (v) to vary across morph levels within each participant, as it directly linked to the amount of race-related information available. The other parameters, the starting point z, the threshold a, and the non-decision time value τ were kept constant. We used a total of 2500 maximum number of evaluations of the objective function (default was 200), as well as 1500 maximum number of iterations (default: 150) and a tolerance at 1*10− 6 (default: 1.5*1018). The model failed to converge for one AM participant in the categorization task, so this participant was excluded from the diffusion model analysis for the categorization task. Two-tailed modified t-tests were employed to assess parameter differences between PS and AM controls.
Data availability
The datasets of the current study are available in the Open Science Foundation (OSF) repository, https://doi.org/10.17605/OSF.IO/UWHSE.
References
Anzures, G., Quinn, P. C., Pascalis, O., Slater, A. M. & Lee, K. Categorization, categorical perception, and asymmetry in infants’ representation of face race. Dev. Sci. 13, 553–564 (2010).
Caldara, R., Rossion, B., Bovet, P. & Hauert, C. A. Event-related potentials and time course of the ‘other-race’face classification advantage. Neuroreport 15, 905–910 (2004).
de Lissa, P. et al. Race categorization in noise. i-Perception 13, 1–14 (2022).
de Lissa, P. et al. Rapid saccadic categorization of other-race faces. J. Vis. 21, 1–17 (2021).
de Lissa, P., Schaller, P. & Caldara, R. The speed of race. Soc. Cogn. Affect. Neurosci. 19, 1–11 (2024).
Woo, P. J., Quinn, P. C., Méary, D., Lee, K. & Pascalis, O. A developmental investigation of the other-race categorization advantage in a multiracial population: contrasting social categorization and perceptual expertise accounts. J. Exp. Child. Psychol. 197, 1–12 (2020).
Valentine, T. A unified account of the effects of distinctiveness, inversion, and race in face recognition. Q. J. Exp. Psychol. A. 43, 161–204 (1991).
Valentine, T. & Endo, M. Towards an exemplar model of face processing: the effects of race and distinctiveness. Q. J. Exp. Psychol. A. 44, 671–703 (1992).
Medin, D. L. & Schaffer, M. M. Context theory of classification learning. Psychol. Rev. 85, 207–238 (1978).
Nosofsky, R. M. Attention, similarity, and the identification–categorization relationship. J. Exp. Psychol. Gen. 115, 39–57 (1986).
Feingold, G. A. Influence of environment on identification of persons and things. J. Am. Inst. Crim L Criminol. 5, 39–51 (1914).
Vizioli, L., Rousselet, G. A. & Caldara, R. Neural repetition suppression to identity is abolished by other-race faces. PNAS 107, 20081–20086 (2010).
Caldara, R. & Abdi, H. Simulating the ‘other-race’ effect with autoassociative neural networks: further evidence in favor of the face-space model. Perception 35, 659–670 (2006).
Feng, L. et al. The other face of the other-race effect: an fMRI investigation of the other-race face categorization advantage. Neuropsychologia 49, 3739–3749 (2011).
Contreras, J. M., Banaji, M. R. & Mitchell, J. P. Multivoxel patterns in fusiform face area differentiate faces by sex and race. Plos One. 8, 1–6 (2013).
Hughes, B. L. et al. Neural adaptation to faces reveals racial outgroup homogeneity effects in early perception. PNAS 116, 14532–14537 (2019).
Schaller, P., Caldara, R. & Richoz, A. R. Prosopagnosia does not abolish other-race effects. Neuropsychologia 108, 1–13 (2023).
Sorger, B., Goebel, R., Schiltz, C. & Rossion, B. Understanding the functional neuroanatomy of acquired prosopagnosia. NeuroImage 35, 836–852 (2007).
Rossion, B. Twenty years of investigation with the case of prosopagnosia PS to understand human face identity recognition. Part I: function. Neuropsychologia 173, 1–53 (2022).
Rossion, B. Twenty years of investigation with the case of prosopagnosia PS to understand human face identity recognition. Part II: neural basis. Neuropsychologia 173, 1–38 (2022).
Blais, C., Jack, R. E., Scheepers, C., Fiset, D. & Caldara, R. Culture shapes how we look at faces. PLoS ONE. 3, 1–8 (2008).
Bayet, L. et al. Fearful but not happy expressions boost face detection in human infants. Proc. R Soc. B. 284, 1–9 (2017).
Rodger, H., Vizioli, L., Ouyang, X. & Caldara, R. Mapping the development of facial expression recognition. Dev. Sci. 18, 926–939 (2015).
Rodger, H., Lao, J. & Caldara, R. Quantifying facial expression signal and intensity use during development. J. Exp. Child. Psychol. 174, 41–59 (2018).
Stoll, C. et al. Quantifying facial expression intensity and signal use in deaf signers. JDSDE 24, 346–355 (2019).
Wyssen, A. et al. Facial emotion recognition abilities in women experiencing eating disorders. Psychosom. Med. 81, 155–164 (2019).
Richoz, A. R. et al. Recognizing facial expressions of emotion amid noise: a dynamic advantage. J. Vis. 24, 1–22 (2024).
Fox, C. J., Moon, S. Y., Iaria, G. & Barton, J. J. The correlates of subjective perception of identity and expression in the face network: an fMRI adaptation study. NeuroImage 44, 569–580 (2009).
Rotshtein, P., Henson, R. N. A., Treves, A., Driver, J. & Dolan, R. J. Morphing Marilyn into Maggie dissociates physical and identity face representations in the brain. Nat. Neurosci. 8, 107–113 (2005).
Michel, C., Corneille, O. & Rossion, B. Race categorization modulates holistic face encoding. Cogn. Sci. 31, 911–924 (2007).
Zhao, L. & Bentin, S. The role of features and configural processing in face-race classification. Vis. Res. 51, 2462–2470 (2011).
Golarai, G., Ghahremani, D. G., Greenwood, A. C., Gabrieli, J. D. E. & Eberhardt, J. L. The development of race effects in face processing from childhood through adulthood: neural and behavioral evidence. Dev. Sci. 24, 1–17 (2020).
Golby, A., Gabrieli, J., Chiao, J. & Eberhardt, J. Differential fusiform responses to same- and other-race faces. Nat. Neurosci. 4, 845–850 (2001).
Natu, V., Raboy, D. & O’Toole, A. J. Neural correlates of own-and other-race face perception: spatial and temporal response differences. NeuroImage 54, 2547–2555 (2011).
Damasio, A. R., Tranel, D. & Damasio, H. Face agnosia and the neural substrates of memory. Annu. Rev. Neurosci. 13, 89–109 (1990).
Rossion, B. et al. A network of occipito-temporal face‐sensitive areas besides the right middle fusiform gyrus is necessary for normal face processing. Brain 126, 2381–2395 (2003).
Schiltz, C. et al. Impaired face discrimination in acquired prosopagnosia is associated with abnormal response to individual faces in the right middle fusiform gyrus. Cereb. Cortex. 16, 574–586 (2006).
Busigny, T., Graf, M., Mayer, E. & Rossion, B. Acquired prosopagnosia as a face-specific disorder: ruling out the general visual similarity account. Neuropsychologia 48, 2051–2067 (2010).
Richoz, A. R., Jack, R. E., Garrod, O. G. B., Schyns, P. G. & Caldara, R. Reconstructing dynamic mental models of facial expressions in prosopagnosia reveals distinct representations for identity and expression. Cortex 65, 50–64 (2015).
Caldara, R. et al. Does prosopagnosia take the eyes out of face representations? Evidence for a defect in representing diagnostic facial information following brain damage. J. Cogn. Neurosci. 17, 1652–1666 (2005).
Orban de Xivry, J. J. O., Ramon, M., Lefèvre, P. & Rossion, B. Reduced fixation on the upper area of personally familiar faces following acquired prosopagnosia. J. Neuropsychol. 2, 245–268 (2008).
Fiset, D. et al. Mapping the impairment in decoding static facial expressions of emotion in prosopagnosia. Soc. Cogn. Affect. Neurosci. 12, 1334–1341 (2017).
Liu-Shuang, J., Torfs, K. & Rossion, B. An objective electrophysiological marker of face individualisation impairment in acquired prosopagnosia with fast periodic visual stimulation. Neuropsychologia 83, 100–113 (2016).
Langner, O. et al. Presentation and validation of the Radboud faces database. Cogn. Emot. 24, 1377–1388 (2010).
Lundqvist, D., Flykt, A. & Öhman, A. The Karolinska Directed Emotional faces (KDEF). APA PsycTests. 10, 1 (1998).
Hancock, P. Psychological Image Collection at Stirling (PICS).. http://pics.psych.stir.ac.uk (2008).
Wang, X. & Tang, X. Face photo-sketch synthesis and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31, 1955–1967 (2008).
Fantamorph. (Abrosoft Co, 2011).
Willenbockel, V. et al. Controlling low-level image properties: the SHINE toolbox. Behav. Res. Methods. 42, 671–684 (2010).
MATLAB (The MathWorks Inc., 2020).
Brainard, D. H. The psychophysics toolbox. Spat. Vis. 10, 433–436 (1997).
Kleiner, M. et al. What’s new in psychtoolbox-3. Perception 36, 1–16 (2007).
Crawford, J. R. & Howell, D. C. Comparing an individual’s test score against norms derived from small samples. Clin. Neuropsychol. 12, 482–486 (1998).
Balestrucci, P., Ernst, M. O. & Moscatelli, A. Psychophysics with R: The R Package MixedPsy. https://doi.org/10.1101/2022.06.20.496855 (2022).
Ratcliff, R. A theory of memory retrieval. Psychol. Rev. 85, 59–108 (1978).
Ratcliff, R. & Rouder, J. N. Modeling response times for two-choice decisions. Psychol. Sci. 9, 347–356 (1998).
Lerche, V. & Voss, A. Retest reliability of the parameters of the Ratcliff diffusion model. Psychol. Res. 81, 629–652 (2017).
Lerche, V. & Voss, A. Experimental validation of the diffusion model based on a slow response time paradigm. Psychol. Res. 83, 1194–1209 (2019).
Voss, A., Rothermund, K. & Voss, J. Interpreting the parameters of the diffusion model: an empirical validation. Mem. Cogn. 32, 1206–1220 (2004).
Benton, C. Rapid reactions to direct and averted facial expressions of fear and anger. Vis. Cogn. 18, 1298–1319 (2010).
Carlson, C. A., Gronlund, S. D., Weatherford, D. R. & Carlson, M. A. Processing differences between feature-based facial composites and photos of real faces. Appl. Cogn. Psychol. 26, 525–540 (2012).
Li, H. et al. Processing of individual items during ensemble coding of facial expressions. Front. Psychol. 7, 1 (2016).
Mueller, C. J. & Kuchinke, L. Processing of face identity in the affective flanker task: a diffusion model analysis. Psychol. Res. 80, 963–973 (2016).
Powell, G. et al. Face processing in autism spectrum disorder re-evaluated through diffusion models. Neuropsychology 33, 445–461 (2019).
Ratcliff, R., Love, J., Thompson, C. A. & Opfer, J. E. Children are not like older adults: a diffusion model analysis of developmental changes in speeded responses. Child. Dev. 83, 367–381 (2012).
Benton, C. P. & Skinner, A. L. Deciding on race: a diffusion model analysis of race-categorisation. Cognition 139, 18–27 (2015).
Acknowledgements
We wish to express our sincere gratitude to PS for her invaluable contributions, which have been essential in progressing scientific knowledge. We also express our thanks to the control participants, whose commitment significantly enriched our understanding of face race processing. Funding: This work was supported by a grant from the Swiss National Science Foundation [n◦100019_189018] awarded to R.C.
Author information
Authors and Affiliations
Contributions
J.D. designed the experiment; P.S. conceived the idea, conducted the experiments, and analyzed the results; P.S. created the figures and drafted the manuscript with additional design, analysis, and writing support from A.-R.R. and P.D.L.; A.-R.R. and R.C. supervised the project and provided writing-reviewing; R.C. provided the necessary resources and secured the funding; All authors edited the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Schaller, P., Richoz, AR., Duncan, J. et al. Prosopagnosia and the role of face-sensitive areas in race perception. Sci Rep 15, 5751 (2025). https://doi.org/10.1038/s41598-025-88769-9
Received:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1038/s41598-025-88769-9
