Abstract
Social cognition, the perception and processing of social information, is adversely affected in multiple psychiatric, neurological, and neurodevelopmental disorders, and these impairments negatively impact quality of life for individuals across the globe. Despite the clear importance of social cognition, efforts to advance research via harmonization of data across cultures and diagnoses has been stymied by the lack of uniformly used and suitable assessments. To address this issue, the current study conducted an expert survey and consensus process to identify social cognitive assessments that are best suited for cross-cultural and transdiagnostic use among adults. A large group of experts in social cognition were surveyed to gather nominations for cross-culturally and transdiagnostically appropriate measures. These measures were then critically evaluated by a smaller group of experts using a Delphi consensus process to identify the best existing tasks for each use. Ninety-eight experts, representing 25 countries, responded to the initial survey and nominated a total of 81 tasks. Initial rounds of the Delphi process identified 50 tasks with adequate psychometric properties that were then subdivided into social cognition domains. For each domain, members ranked the five best tasks, once for cross-cultural use and once for transdiagnostic use, and rated the suitability of those tasks for the intended use. No tasks were identified as ideally suited for either use; however, within each domain, 4–5 tasks emerged as the most consistently selected, and all were ranked as having “good” or better suitability for use. While there is still a critical need for social cognitive assessments that are specifically designed for cross-cultural and transdiagnostic use, there does appear to be a handful of existing tasks that are currently available and likely informative. Caution is warranted however, as these still require comprehensive evaluation in cross-cultural and transdiagnostic studies.
Similar content being viewed by others
Introduction
Social-cognitive deficits and biases are common among individuals with schizophrenia spectrum disorders1,2,3,4,5. These difficulties are related to various negative functional outcomes such as poorer community functioning, underdeveloped social skills, and less effective social problem-solving6. Recently, two important expert reviews supported a key role of social cognition in the assessment and treatment of psychosis. First, the consensus statement from a group of world leading researchers and clinicians identified social cognition as one of the domains that is crucial for precise clinical characterization and treatment planning for individuals with schizophrenia7. In addition, the schizophrenia section of the European Psychiatric Association also recently formulated a guidance paper with recommendations for optimal assessment of, among others, social cognition8. In their paper, they recommend assessment of social cognition mainly for the characterization of patients, as well as for personalized treatment planning.
Despite these calls for the routine assessment of social cognition, measurement continues to be a significant challenge, particularly regarding which constructs to consider and how to best measure them9. Many social cognition tasks were originally developed in the context of autism spectrum disorder research (involving false beliefs and verbal/visual mentalizing tasks) but subsequent research has demonstrated that these tasks show poor psychometric properties and/or construct validity in adult populations (e.g.,10,11). Likewise, assessments derived from social neuroscience studies in healthy adults have also shown relatively weak psychometric properties in schizophrenia12. Most notably, the Social Cognition Psychometric Evaluation project (SCOPE13,14), a NIMH-funded, multi-round study focusing on the identification of sound social cognitive measures for use in clinical trials of schizophrenia, identified just three measures with sufficient psychometric properties that could be recommended for further use: the Hinting Task15, the Penn Emotion Recognition Task (ER-40)16, and the Bell-Lysaker Emotion Recognition Task (BLERT)17.
While the SCOPE recommended tasks have been heavily used in the United States and United Kingdom, two of them show limited utility for large-scale international trials. The Hinting Task, for example, is a verbal task strongly influenced by social norms and knowledge, with vignettes that may not be applicable across cultures18. The BLERT utilizes low-quality videos of a single white male depicting various emotions that may not accurately represent diverse cultural contexts, particularly in light of the well-established other-race effect, which is also evident among individuals with schizophrenia and which may contribute to poorer performance in non-white individuals19. As these examples demonstrate, the role of culture in social cognitive performance is well established20, and thus individuals may appear to have more impaired social cognitive functioning when tasks are not matched to culture. Although SCOPE did not consider cross-cultural applicability in its evaluation criteria, the results underscore both the paucity of high-quality social cognitive tasks as well as the broad lack of tasks that can be used cross-culturally. The need for culturally sensitive social cognitive assessments has long been emphasized8,21,22, and the unavailability of such tasks renders harmonization efforts impossible.
In addition, interest in understanding social cognitive impairments is not limited to schizophrenia spectrum illnesses. Social cognition is adversely affected in numerous psychiatric (e.g., bipolar disorder, post-traumatic stress disorder, anorexia nervosa), neurological (e.g., traumatic brain injury, stroke, frontotemporal dementia, Parkinson’s Disease, Alzheimer’s Disease), and neurodevelopmental conditions (e.g., Autism, Attention Deficit Hyperactivity Disorder)23,24. By and large, these impairments are of moderate to large effect sizes and in many cases exceed the magnitude of other cognitive impairments that are also seen in these conditions. As such, experts are increasingly calling for the incorporation of social cognitive assessment into clinical practice and note that patterns of social cognitive impairment may inform differential diagnosis (e.g., parsing frontotemporal dementia from primary psychiatric disorder) and disease progression25. Harmonization efforts such as these require measures that can be utilized transdiagnostically, in which task psychometric properties remain similar across different clinical groups and meaningful comparisons between clinical groups can be made. Successful identification of such tasks would facilitate efforts to examine the possibility of shared vs. distinct etiologies of social cognitive impairments across disorders and to identify disorder specific difficulties that would inform clinical decisions and treatment planning.
To address these measurement limitations and challenges within social cognitive research, a group of experts was convened to form the Schizophrenia International Research Society (SIRS) Social Cognition Research Harmonization Group (RHG). The goals of the RHG were twofold: (1) conduct a wide-ranging expert survey to gather nominations for tasks of social cognition that may be well-suited to cross-cultural and transdiagnostic use among adults, and (2) use the Delphi Method within our RHG to identify a consensus set of social cognitive measures from these nominations for use in future data collection. Although cross-cultural and transdiagnostic applicability are separate concepts and usually examined individually, the current study sought to leverage the expertise of the field and the RHG to advance social cognitive assessment on both fronts. In doing so, it was also hoped that some tasks would be identified as suitable for both cross-cultural and transdiagnosic use, which could provide an immediate foundation for collaborative projects. This paper reports the outcomes of these efforts.
Results
Expert survey
Fifty-two tasks were nominated for cross-cultural use, and 77 tasks were nominated for transdiagnostic use. Forty-seven tasks were nominated as potentially suitable for use pending further development. Many tasks were nominated across uses, resulting in a total of 81 unique tasks across all categories. These tasks are listed alphabetically in Table 1.
Delphi Round 1
Twenty-one members of the RHG provided complete ratings for each of the 81 tasks. Forty-seven tasks received mean ratings ≥2 (adequate or better). Eleven of these tasks had an IQR > 1 and were included in Round 2. The remaining 36 tasks receiving consensus ratings of ≥2 were retained for further evaluation in Round 3. Thirty-three tasks scored in the inadequate range, and of these, 26 had consensus average ratings of <2 (i.e., average score of <2 and IQR of ≤1) and were therefore removed from further consideration. The remaining 7 were carried forward to Round 2. IQR could not be calculated for one task due to the lack of non-zero scores, and this task was also omitted from further consideration. Average Round 1 ratings and IQR values for each task are provided in Table 1.
During this round, 10 additional tasks were also nominated, resulting in 28 tasks that still required consensus ratings and a total of 64 tasks remaining under consideration. New tasks are listed at the bottom of Table 1, and a flow chart of the complete rating process is depicted in Fig. 1.
Flow Chart of the Delphi Rating Process.
Delphi Round 2
Seventeen members of the RHG provided complete ratings. Consensus was reached for 16 of the original tasks and 8 of the additional tasks, resulting in 4 tasks for which consensus was not reached. In either Round 1 or 2, 50 tasks received ratings of ≥2 indicating “adequate” or better quality, and these tasks advanced to Round 3. Tasks that failed to achieve a rating of 2 or more in either Round 1 or 2 were dropped from consideration (n = 14). Average ratings and IQR values from Round 2 are presented in Table 1.
Delphi Round 3
For this round, tasks were categorized according to social cognitive domain, and tasks assessing multiple domains (e.g., OSCARS) were listed within each applicable domain. This process resulted in 23 tasks for emotion processing, 6 for social perception, 17 for mental state attribution, 4 for attributional style/bias, and 6 for empathy.
Eighteen members of the RHG provided task rankings and ratings for suitability of use. Within each domain, a top tier of tasks emerged as the most consistently selected, and all were rated as having “good” or better suitability for use. The top six ranked tasks for cross-cultural use are listed in Table 2, and the top six ranked tasks for transdiagnostic use are listed in Table 3. Average suitability ratings for these tasks are also presented in the corresponding tables. Information for the remaining emotion processing and mental state attribution tasks are provided in Supplementary Tables 3, 4.
Discussion
The current study utilized an expert survey and Delphi consensus process to identify social cognitive tasks that may be appropriate for cross-cultural and/or transdiagnostic use. Within the domain of emotion processing, the ER-40 ranked as the top task for both cross-cultural and transdiagnostic use. This task shows 40 static images of individual’s faces and asks participants to label the emotion shown from the following choices: happy, sad, anger, fear, no emotion. The ER-40 uses age, gender, and ethnically diverse stimuli and has minimal verbal demand, which likely contributed to its high ranking.
Far fewer tasks assessing social perception were available; however, the OSCARS emerged as the most promising for cross-cultural use and the second most promising for transdiagnostic use. The OSCARS can be used as a self- and/or informant-report assessment and asks how much difficulty someone has decoding verbal cues. While an early study indicated that informant reports showed stronger convergent and external validity than self-reports26, a more recent, larger study suggests equivalent validity between the two modalities27. The interview-based format provides the option to combine across sources of information, including self- and informant-report or multiple informants, and may provide a more sensitive indicator of real-world change than performance-based tasks, thus potentially serving as a valuable coprimary measure for clinical trials28. Translation of this task is likely necessary for cross-cultural use; however, the general, non-performance based assessment approach may make it widely applicable and more easily adaptable to other cultures.
Several tasks evaluating mental state attribution were nominated, with both the SAT-MC and TASIT being ranked highest for cross-cultural use and the TASIT ranked highest for transdiagnostic use. The SAT-MC shows short animations of geometrical shapes enacting a social drama, and participants answer multiple choice questions about the actions and intentions of the shapes. The minimal verbal and memory demands of this task make it highly feasible for both uses, and it is noteworthy that it has one of the highest suitability ratings of all tasks for cross-cultural use. The TASIT involves short videos of everyday social interactions involving two or more people and evaluates the ability to detect lies and sarcasm in these interactions. The videos include English-speaking actors with Australian accents, which may somewhat limit its cross-cultural applicability; however, this should not impede transdiagnostic use, and an abbreviated format and some translations are available (e.g.,29,30).
For attributional style/bias, the AIHQ received the highest rankings for both uses. This task presents hypothetical, negative situations with ambiguous causes and asks participants why the scenario occurred. Participants also rate whether the other person acted intentionally, how angry it would make them feel, and how much they would blame the other person. Finally, they are asked how they would respond to the situation. The high ranking of this task for cross-cultural use is somewhat surprising given the high dependence on verbal language; however, the situations used are likely to be considered negative and ambiguous across most cultures, which may explain its high ranking. Further, only four tasks were nominated in this category, and the modest suitability ranking for cross-cultural use, as compared to transdiagnostic use, likely reflects some hesitancy regarding the task.
Finally, within the domain of empathy, the QCAE was highly ranked for both uses. This self-report scale asks participants to rate how much statements pertaining to cognitive and affective empathy pertain to them. Importantly, despite being ranked highly, suitability ratings for this task were modest, again suggesting that while it is a “good” task, additional development would be needed for optimal use.
Overall, there was considerable overlap in the top ranked tasks within each domain for cross-cultural and transdiagnostic use. This may be due to the somewhat limited number of generally “good” tasks from which to choose or to the possibility that the same characteristics make a task appealing for either transdiagnostic or cross-cultural use (e.g., sound basic psychometrics like validity and reliability). Nevertheless, there were a few instances, for example the Hinting Task, in which a task was ranked highly for one use and not another. Thus, we suggest care when selecting tasks for each potential use.
Additionally, while these measures, and others, exhibited potential and were rated relatively favorably, no task emerged as ideally suited for the intended use. Of tasks considered in Round 3, suitability ratings were primarily in the “good” range, with no tasks scoring in the highest range, and from Rounds 1 and 2, almost half of the tasks received ratings in the inadequate range. This may be because very few social cognitive tasks have been specifically designed with cross-cultural considerations in mind. As excellently reviewed by Bourdage and colleagues31 who focused on tasks appropriate for Global South communities, there is a considerable lack of multicultural assessment tools, and attempts to modify existing tasks have been primarily of low quality. Further, many existing modifications have been conducted with a single culture in mind (i.e., translating a measure to be used in a specific location) rather than with the goal of making a truly cross-cultural tool that could be readily used across several cultures.
It is also noteworthy that several tasks scoring below adequate in Round 1 or 2 still ranked highly in Round 3. This phenomenon was most common for domains with fewer nominated tasks and may reflect a lack of viable options. Alternatively, this may also indicate a willingness to use less than ideal tasks, either out of necessity as just mentioned, or because of easy availability and increased familiarity. The RMET is a good example of this. It is among the most widely used and translated social cognitive tasks32 and received the overall highest rating of suitability for cross-cultural use. However, translation does not guarantee cultural appropriateness, and recent years have seen a sharp increase in concern regarding its validity11 and growing evidence of cultural bias33,34,35. As such, we encourage critical evaluation of existing measures and thoughtful consideration of measurement when designing studies.
We also encourage continued development of novel tasks that are proactively designed to be used cross-culturally. Sensitive cultural adaptation of any cognitive measure is difficult and requires due process, in which researchers need to pay close attention to concept equivalence across cultures. For example, results from previous studies on functional capacity and interview-based measures of cognitive impairment showed that these adaptations usually require substantial edits and that the content of the tasks needed to be adapted36. Thus, applying these same principles to social cognitive tasks, that often use complex and nuanced social stimuli, is an even greater challenge but one that should not be neglected (see refs. 37,38 for suggested guidelines). It also bears noting that work of this type is resource intensive, and that limited funding has previously been cited as a primary barrier31. We therefore encourage investment on the part of funding agencies and foundations.
In addition to the general challenge of developing and validating culturally sensitive tasks, another important element will be to ensure measurement invariance across adaptations. Most social cognitive measure validation studies analyze only basic psychometric properties, such as internal consistency or concurrent and criterion validity. Relatively few studies have applied advanced statistical modeling to estimate measurement properties (see ref. 39 for an exception). It may therefore be helpful to use archival data to test whether some widely used measures are invariant across cultures and to consider measurement invariance in future validation efforts.
Moving forward, it may also be fruitful to emphasize continued development of paradigms rather than specific tasks. For example, the basic structure of the TASIT or the Hinting task, in which participants must interpret interactions between characters, is quintessential to social cognition and could be retained while the specific stimuli or scoring criteria could be adapted to apply more broadly to multiple cultures. A recently developed multiracial version of the RMET also provides a good example of this idea. Here, Kim and colleagues retained the structure of the RMET but updated the stimuli and answer choices to produce a more inclusive version of the task that may mitigate some of the bias introduced by only using white, European faces. Future work may also benefit by adapting novel paradigms from social and experimental psychology. Social cognitive research in the general population has been moving from static paradigms based on the perception and interpretation of social stimuli to more active, dynamic paradigms40 such as dyadic interactions (e.g., 41) that may allow for the assessment of social cognitive processes, like emotion recognition, in real time and in more naturalistic ways. These paradigms have substantial technical and analytical demands but might bring novel insights about the nature of social cognitive difficulties observed across clinical conditions.
As our study represents a consensus-based effort to identify cross-cultural and transdiagnostic social cognitive assessments, some limitations require consideration. First, our RHG included individuals with varying expertise including early career individuals, industry representatives, and individuals with lived experience. While this significantly increased the diversity of perspectives in the RHG, these forms of expertise were not equally represented within the RHG and not all members were as familiar with the breadth of existing social cognitive assessments, which could have skewed our results toward tasks that are more widely used and therefore more familiar. Services users were also underrepresented in the expert survey and RHG, which prevented us from broadly capturing their viewpoints. Accessibility of social cognitive tasks, particularly for service users, should be prioritized in future consensus-based work. Second, just over 25% of the experts invited to the initial survey responded. These experts represented a wide range of countries, but the majority were from North America and Europe and expertise in schizophrenia was most common. Broader representation may have yielded a different set of tasks for consideration by the RHG. Likewise, our RHG lacked representation from Spanish- and Arabic-speaking countries as well as African countries, which account for significant portions of the global population. Future work should strive for broader representation, and when possible, consider multiple languages, including indigenous languages. Third, the data generated here may be viewed as being primarily schizophrenia-focused. Expertise in schizophrenia was disproportionally represented compared to other specific disorders (e.g., autism); however, over half of the experts who responded to the initial survey (53%) identified as having primary expertise in areas other than schizophrenia. Similarly, of the 22 academic and clinical RHG members, 40% worked in populations other than schizophrenia. Thus, we believe this work is still widely applicable and relevant to fields outside psychosis but acknowledge that the results of the Delphi Process may reflect perspectives more heavily weighted by schizophrenia spectrum disorders. Finally, we focused on assessments that are appropriate for adults. Additional work will be needed to identify the most suitable assessments for children and adolescents.
Conclusion
Harmonization of social cognitive research has been significantly limited by a lack of cross-culturally and transdiagnostically valid assessment tools. Results of the global expert survey and subsequent consensus process reported here underscore the relative dearth of suitable measures but do identify a small selection of assessments that appear to be appropriate for current use. These tasks should be explicitly evaluated in cross-cultural and transdiagnostic studies. Additional efforts should also be made to continue adapting existing measures and to develop novel measures that can be used in each of these capacities. Notwithstanding these issues, the tasks identified here represent multiple social cognitive domains beyond the traditionally considered core processes of mental state attribution and emotion processing. In addition to guiding future research, identification of these tasks may provide a much needed springboard for increasing assessment of social cognition in clinical practice, where it remains under-utilized42.
Methods
This study has been approved by the Institutional Review Board (IRB) of The University of Texas at Dallas (IRB-23-177).
Expert survey for task nomination
Survey content was drafted by the conveners of the RHG (AP, MH, and TZ) and then further refined and augmented with input from RHG-members. The final survey consisted of two primary parts: (1) Background information; and (2) Task nominations for future use. For part 2, respondents were prompted to nominate social cognition tasks that they believed were suitable for: (a) International and/or cross-cultural studies; and/or (b) Transdiagnostic studies. In a separate question, this section also allowed respondents to nominate tasks that may not currently be suitable for either of the intended uses but that may show promise with continued adaptation and further development (see Supplementary Table 1). A final portion of the survey queried current use of social cognitive assessments and perceived measurement-related barriers within social cognitive research, the results of which will be reported elsewhere. A copy of the survey is available in Supplementary Materials.
In parallel, the definition of “expert” was formulated first by the conveners and then edited via group discussion with the RHG.
The term “expert” for academic researchers was defined as follows:
-
research experience (either academic or industry) in the field of psychology, psychiatry, social neuroscience, or an allied discipline for at least 4 years and currently active in one of those fields, AND at least 2 peer-reviewed publications on social cognition, of which at least 1 is as first, second, or senior author, and of which at least 1 has been published in the last 5 years.
-
for researchers from non-English speaking countries, articles written in languages other than English qualified if they were published in a peer-reviewed journal.
Non-academic expertise (e.g., clinicians, students, industry team members, or service users) was defined as:
-
hands-on experience with, or intricate knowledge of, at least 2 social cognition paradigms.
The online survey was implemented in REDCap43 (hosted at The University of Texas at Dallas) for data collection and management, and subsequently distributed via emailed invitations through the RHG-members’ familiarity with experts, as well as supplementary literature searches by graduate students. All RHG members qualified as experts according to the definitions above and were therefore encouraged to complete the survey as well. The estimated duration time for filling out the survey was 10 min. Data for the expert survey were collected between late October 2022—early January 2023.
In total, 381 experts were invited to participate anonymously via an emailed survey-link and asked to share the link with other potential experts meeting the criteria. Ninety-eight experts residing in 25 countries across five continents responded to the invitation and provided digital consent. 50% of respondents identified as men, and age was normally distributed (range: 20–70+; mode = 40–44 y). Most experts (70%) identified professionally as professor/lecturer (any level), 36% as researcher, 18% as a clinician, and 6% as service user/other. Schizophrenia/psychosis or high-risk for psychosis was checked by 46% of experts as their main study population of interest, followed by general population (13%), and autism, bipolar disorder and neurodegenerative disease (all 7%). For additional detail on respondent characteristics, see Supplementary Table 2.
The Delphi methodology
The Delphi method is a structured communication technique used to achieve consensus among a group of experts by soliciting their opinions through an iterative series of questionnaires and providing them with controlled feedback44. The method is based on the concept of collective wisdom, which assumes that the combined opinion of multiple people is closer to the truth than a single individual’s perspective45. To obtain consensus, group members complete a series of online, anonymous questionnaires from which results are aggregated in a systematic manner and then presented back to the larger group. This process of responding and receiving/incorporating feedback (i.e., a “round”) is repeated until group consensus is reached. New information can be introduced at any point or during any round, and ensuring anonymity is thought to reduce undue influence from any (especially more influential) group members and the pressure to conform. This process has previously been used in psychological assessment research (e.g., 46), including studies focused on cross-cultural assessment47 and one study that sought to identify social cognitive assessments for use in Japanese individuals with schizophrenia48. The Delphi portion of this study consisted of 3 consecutive rounds of online questionnaires, further outlined below.
Delphi expert panel selection
Panels with 10 to 50 members are recommended for Delphi studies49. As such, the original RHG membership, comprised of 13 international members of the Schizophrenia International Research Society, as well as two service users and two industry representatives (note: one of the industry partners was unable to continue their participation due to time constraints leaving only one industry representative in the final RHG.), was expanded to include experts from clinical specializations other than schizophrenia/psychosis based on RHG member recommendations. Clinical fields of expertise included: autism, bipolar disorder, neurodegenerative disorders, pediatrics and acquired brain injury. The final group of 26 experts consisted of residents from: USA (8), the Netherlands (4), United Kingdom (3), Australia (2), Canada (2), South Korea (2), China (1), France (1), Italy (1) Slovakia (1), and India (1). Members of the RHG were diverse in gender, race, ethnicity, and career stage (see Table 4).
Delphi procedure and data analysis
After the expert survey was completed, task nominations were collated and used to build a database containing a description of each task as well as a summary of the currently available psychometric data for that task. To the extent possible, information pertaining to reliability (e.g., test-retest reliability, internal consistency), validity (e.g., convergent and discriminant [including consideration of overlap with cognitive performance assessments], criterion, distribution of scores, sensitivity to group differences), practicality of administration, and tolerability (e.g., pleasantness or unpleasantness of completing the task) was included. This database was distributed to RHG members at each round to aid their evaluations.
For each nominated task the overarching goal was to reach a consensus score and establish its utility for cross-cultural and transdiagnostic research. At the beginning of each round, relevant information (i.e., the database) and/or feedback from the previous round was provided to allow the experts to modify their opinions with the aim of reaching group consensus. All tasks were rated on a 5-point scale, ranging from 0–4: 0=not possible to rate or insufficient information, 1 = inadequate, 2 = adequate, 3 = good, 4 = excellent. Our operationalization of consensus was an interquartile range (IQR) ≤ 1. For a four- to five-point Likert scale, an IQR of 1 or less is considered a high level of consensus50,51. Ratings were conducted anonymously while unique user IDs of participants were collected to monitor variation in expert responsiveness across all three rounds. As noted above, off-line and new information could be suggested at any point and/or during any round. After each round, the tasks for which consensus was not achieved moved into the subsequent round for re-rating. Data collection for round 1 started in May 2023 and round 3 was finalized in January 2024.
Round 1
Eighty-one different social cognition tasks were nominated in the expert survey and included in Round 1. The objective of the first round was to rate the overall quality of each nominated task based on the three considerations listed below:
-
1.
Does the task really measure social cognition, and does it tap into at least one social cognitive domain?
-
2.
Is the task generally a “good” task given what you know about its psychometric properties?
-
3.
Is this task relatively easy to administer and take or it is too onerous (e.g., burdensome) to be useful?
Individual scores of 0 (“not possible to rate or insufficient information”) were excluded before calculating the mean and IQR for each task. To maintain high responsivity, it was deemed necessary to limit the number of included tasks for subsequent rating rounds. Thus, tasks receiving a consensus average score of less than 2, indicating a rating of less than “adequate” were omitted from further consideration.
Round 2
Eighteen tasks that did not reach consensus in Round 1 were rated once more in a similar manner as for Round 1. These tasks were presented along with information on the average panel rating, each expert’s own previous rating (observable with the unique, anonymous user ID for each participant), and an overview of comments that were offered by experts in support of their ratings in the first round. In addition, 10 newly suggested tasks from round 1 were included to collect an initial rating. As only two old tasks and two newly suggested tasks did not reach a consensus score after Round 2, it was determined ad hoc not to request our RHG members for an additional rating for these tasks.
Round 3
After the consensus procedure, the goal of the third round was to identify the best tasks for cross-cultural and transdiagnostic use. Before this final round, tasks were categorized according to social cognitive domain by the conveners, after independent classification and a consensus meeting. The domains adhered to the four domains distilled from the SCOPE study52: emotion processing, social perception, theory of mind/mental state attribution, and attributional style/bias. A fifth domain was added specifically for empathy tasks, which were not included in the SCOPE study. Tasks assessing multiple domains were included in each applicable domain.
In Round 3, RHG members were then asked to identify and rank their top 5 tasks within each social cognitive domain and provide a rating of the overall suitability of that task for the intended use (1 = poor, 3 = fair, 5 = good, 7 = very good, 9 = superb). This was first requested for cross-cultural use and then for transdiagnostic use. To identify top tasks, mean rank was reverse coded so that higher scores indicated better tasks and then weighted by the number of times a task was selected within its social cognitive domain (e.g., (mean rank × number of times the task was selected)/18 raters). Thus, a task with a mean rank of 3.0 based on 10 rankings would have a weighted rank of 1.67 and would be preferred to a task with mean rank of 5.0 (the highest possible) based on just two rankings, which would have a weighted rank of 0.55.
Data availability
The datasets generated and analyzed for the current study are available from the corresponding author upon reasonable request at amy.pinkham@utdallas.edu.
References
Hajdúk, M., Harvey, P. D., Penn, D. L. & Pinkham, A. E. Social cognitive impairments in individuals with schizophrenia vary in severity. J. Psychiatr. Res. 104, 65–71 (2018).
Rocca, P. et al. Social cognition in people with schizophrenia: a cluster-analytic approach. Psychol. Med. 46, 2717–2729 (2016).
Vaskinn, A., Sundet, K. & Haatveit, B. Social cognitive heterogeneity in schizophrenia: a cluster analysis. Schizophr. Res. Cogn. 30, 100264 (2022).
Lewandowski, K. E., Pinkham, A. E. & Van Rheenen, T. E. Social cognition across the schizophrenia–bipolar disorder spectrum. Nat. Rev. Psychol. 3, 91–107 (2024).
Green, M. F., Horan, W. P. & Lee, J. Social cognition in schizophrenia. Nat. Rev. Neurosci. 16, 620–631 (2015).
Halverson, T. F. et al. Pathways to functional outcomes in schizophrenia spectrum disorders: Meta-analysis of social cognitive and neurocognitive predictors. Neurosci. Biobehav. Rev. 105, 212–219 (2019).
Maj, M. et al. The clinical characterization of the patient with primary psychosis aimed at personalization of management. World Psychiatry 20, 4–33 (2021).
Vita, A. et al. European Psychiatric Association guidance on assessment of cognitive impairment in schizophrenia. Eur. Psychiatry 65, e58 (2022).
Etchepare, A. & Prouteau, A. Toward a two-dimensional model of social cognition in clinical neuropsychology: a systematic review of factor structure studies. J. Int. Neuropsychol. Soc. 24, 391–404 (2018).
Higgins, W. C., Kaplan, D. M., Deschrijver, E. & Ross, R. M. Construct validity evidence reporting practices for the reading the mind in the eyes test: a systematic scoping review. Clin. Psychol. Rev. 108, 102378 (2024).
Higgins, W. C., Ross, R. M., Langdon, R. & Polito, V. The “Reading the Mind in the Eyes” test shows poor psychometric properties in a large, demographically representative US sample. Assessment 30, 1777–1789 (2023).
Kern, R. S. et al. Adapting social neuroscience measures for schizophrenia clinical trials, part 2: trolling the depths of psychometric properties. Schizophr. Bull. 39, 1201–1210, https://doi.org/10.1093/schbul/sbt127 (2013).
Pinkham, A. E., Harvey, P. D. & Penn, D. L. Social cognition psychometric evaluation: results of the final validation study. Schizophr. Bull. 44, 737–748 (2018).
Pinkham, A. E., Penn, D. L., Green, M. F. & Harvey, P. D. Social cognition psychometric evaluation: results of the initial psychometric study. Schizophr. Bull. 42, 494–504 (2016).
Corcoran, R., Mercer, G. & Frith, C. D. Schizophrenia, symptomatology and social inference: investigating “theory of mind” in people with schizophrenia. Schizophr. Res. 17, 5–13 (1995).
Kohler, C. G. et al. Facial emotion recognition in schizophrenia: intensity effects and error pattern. Am. J. Psychiatry 160, 1768–1774 (2003).
Bryson, G., Bell, M. & Lysaker, P. Affect recognition in schizophrenia: a function of global impairment or a specific cognitive deficit. Psychiatry Res. 71, 105–113 (1997).
Lim, K., Lee, S.-A., Pinkham, A. E., Lam, M. & Lee, J. Evaluation of social cognitive measures in an Asian schizophrenia sample. Schizophr. Res. Cogn. 20, 100169 (2020).
Pinkham, A. E. et al. The other-race effect in face processing among African American and Caucasian individuals with schizophrenia. Am. J. Psychiatry 165, 639–645 (2008).
Koelkebeck, K., Uwatoko, T., Tanaka, J. & Kret, M. E. How culture shapes social cognition deficits in mental disorders: a review. Soc. Neurosci. 12, 102–112 (2017).
Hajdúk, M., Achim, A. M., Brunet–Gouet, E., Mehta, U. M. & Pinkham, A. E. How to move forward in social cognition research? Put it into an international perspective. Schizophr. Res. 215, 463 (2020).
Mehta, U. M., Thirthalli, J., Gangadhar, B. N. & Keshavan, M. S. Need for culture specific tools to assess social cognition in schizophrenia. Schizophr. Res. 133, 255–256 (2011).
Cotter, J. et al. Social cognitive dysfunction as a clinical marker: a systematic review of meta-analyses across 30 clinical conditions. Neurosci. Biobehav. Rev. 84, 92–99 (2018).
Henry, J. D., Von Hippel, W., Molenberghs, P., Lee, T. & Sachdev, P. S. Clinical assessment of social cognitive function in neurological disorders. Nat. Rev. Neurol. 12, 28–39 (2016).
Van den Stock, J. et al. Current potential for clinical optimization of social cognition assessment for frontotemporal dementia and primary psychiatric disorders. Neuropsychol. Rev. 33, 544–550 (2023).
Silberstein, J. M., Pinkham, A. E., Penn, D. L. & Harvey, P. D. Self-assessment of social cognitive ability in schizophrenia: association with social cognitive test performance, informant assessments of social cognitive ability, and everyday outcomes. Schizophr. Res. 199, 75–82 (2018).
Halverson, T. F. et al. Psychometric properties of the Observable Social Cognition Rating Scale (OSCARS): self-report and informant-rated social cognitive abilities in schizophrenia. Psychiatry Res. 286, 112891 (2020).
Keefe, R. S., Poe, M., Walker, T. M., Kang, J. W. & Harvey, P. D. The Schizophrenia Cognition Rating Scale: an interview-based assessment and its relationship to cognition, real-world functioning, and functional capacity. Am. J. Psychiatry 163, 426–432 (2006).
Jarsch, M. et al. Development of the Basel Version of the Awareness of Social Inference Test–Theory of Mind (BASIT‐ToM) in healthy adults. J. Neuropsychol. 17, 125–145 (2023).
Westerhof-Evers, H. J., Visser-Keizer, A. C., McDonald, S. & Spikman, J. M. Performance of healthy subjects on an ecologically valid test for social cognition: The short, Dutch Version of The Awareness of Social Inference Test (TASIT). J. Clin. Exp. Neuropsychol. 36, 1031–1041 (2014).
Bourdage, R., Narme, P., Neeskens, R., Papma, J. & Franzen, S. An evaluation of cross-cultural adaptations of social cognition testing: a systematic review. Neuropsychol. Rev. 34, 1048–1094 (2024).
Eddy, C. M. What do you have in mind? Measures to assess mental state reasoning in neuropsychiatric populations. Front. Psychiatry 10, 425 (2019).
Dodell-Feder, D., Ressler, K. J. & Germine, L. T. Social cognition or social class and culture? On the interpretation of differences in social cognitive performance. Psychol. Med. 50, 133–145 (2020).
Van Staden, J. G. & Callaghan, C. W. An evaluation of the reading the mind in the eyes test’s psychometric properties and scores in South Africa—cultural implications. Psychol. Res. 86, 2289–2300 (2022).
Alvarez, R. et al. Reading the mind in the eyes and cognitive ability in schizophrenia-and autism spectrum disorders. Psychol. Med. 53, 7913–7922 (2023).
Gonzalez, J. M., Rubin, M., Fredrick, M. M. & Velligan, D. I. A qualitative assessment of cross-cultural adaptation of intermediate measures for schizophrenia in multisite international studies. Psychiatry Res. 206, 166–172 (2013).
Gregoire, J. ITC guidelines for translating and adapting tests. Int. J. Test. 18, 101–134 (2018).
Cruchinho, P. et al. Translation, cross-cultural adaptation, and validation of measurement instruments: a practical guideline for novice researchers. J. Multidiscip. Healthc. 17, 2701–2728 (2024).
Quesque, F. et al. Does culture shape our understanding of others’ thoughts and emotions? An investigation across 12 countries. Neuropsychology 36, 664 (2022).
Bolis, D., Dumas, G. & Schilbach, L. Interpersonal attunement in social interactions: from collective psychophysiology to inter-personalized psychiatry and beyond. Philos. Trans. R. Soc. B 378, 20210365 (2023).
Kramer, M. et al. Video analysis of interaction in schizophrenia reveals functionally relevant abnormalities. Schizophr. Res. 274, 24–32 (2024).
Quesque, F. et al. Social cognition in neuropsychology: a nationwide survey revealing current representations and practices. Appl. Neuropsychol. Adult 31, 689–702 (2024).
Harris, P. A. et al. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J. Biomed. Inform. 42, 377–381 (2009).
Dalkey, N. & Helmer, O. An experimental application of the Delphi method to the use of experts. Manag. Sci. 9, 458–467 (1963).
Habibi, A., Sarafrazi, A. & Izadyar, S. Delphi technique theoretical framework in qualitative research. Int. J. Eng. Sci. 3, 8–13 (2014).
Hutten, J. C. et al. Neuropsychological assessment of aggressive offenders: a Delphi consensus study. Front. Psychol. 15, 1328839 (2024).
Franzen, S., Papma, J. M., van den Berg, E. & Nielsen, T. R. Cross-cultural neuropsychological assessment in the European Union: a Delphi expert study. Arch. Clin. Neuropsychol. 36, 815–830 (2021).
Okano, H. et al. Evaluation of social cognition measures for Japanese patients with schizophrenia using an expert panel and modified Delphi method. J. Personal. Med. 11, 275 (2021).
Turoff, M. The design of a policy Delphi. Technol. Forecast. Soc. Change 2, 149–171 (1970).
Raskin, M. S. The Delphi study in field instruction revisited: Expert consensus on issues and research priorities. J. Soc. Work Educ. 30, 75–89 (1994).
Rayens, M. K. & Hahn, E. J. Building consensus using the policy Delphi method. Policy, Politi. Nurs. Pract. 1, 308–315 (2000).
Pinkham, A. E. et al. The social cognition psychometric evaluation study: results of the expert survey and RAND panel. Schizophr. Bull. 40, 813–823 (2014).
Smith, M. J. et al. Performance-based empathy mediates the influence of working memory on social competence in schizophrenia. Schizophr. Bull. 40, 824–834, https://doi.org/10.1093/schbul/sbt084 (2013).
Combs, D. R., Penn, D. L., Wicher, M. & Waldheter, E. The Ambiguous Intentions Hostility Questionnaire (AIHQ): a new measure for evaluating hostile social-cognitive biases in paranoia. Cogn. Neuropsychiatry 12, 128–143 (2007).
Benton, A. L., Sivan, A., deS, K., Varney, N. R. & Spreen, O. Facial Recognition: Stimulus and Multiple Choice Pictures: Contributions to Neuropsychological Assessment (Oxford University Press, Incorporated, 1983).
Brüne, M. Theory of mind and the role of IQ in chronic disorganized schizophrenia. Schizophr. Res. 60, 57–64 (2003).
Thibaudeau, É., Cellard, C., Legendre, M., Villeneuve, K. & Achim, A. M. Reliability of two social cognition tests: the combined stories test and the social knowledge test. Psychiatry Res. 262, 63–69 (2018).
Moritz, S., Woznica, A., Andreou, C. & Köther, U. Response confidence for emotion perception in schizophrenia using a Continuous Facial Sequence Task. Psychiatry Res. 200, 202–207 (2012).
van der Gaag, M. et al. Development of the Davos assessment of cognitive biases scale (DACOBS). Schizophr. Res. 144, 63–71 (2013).
Van Pelt, B. et al. Dynamic interactive social cognition training in virtual reality (DiSCoVR) for adults with autism spectrum disorder: a feasibility study. Res. Autism Spectr. Disord. 96, 102003 (2022).
Muros, N. I. et al. Facial affect recognition by patients with schizophrenia using human avatars. J. Clin. Med. 10, 1904 (2021).
Young, A., Perrett, D. I., Calder, A., Sprengelmeyer, R. H. & Ekman, P. Facial expressions of emotion: Stimuli and Test (FEEST). (2002).
Scherer, K. R. & Scherer, U. Assessing the ability to recognize facial and vocal expressions of emotion: construction and validation of the Emotion Recognition Index. J. Nonverbal Behav. 35, 305–326 (2011).
Kessels, R. P. C., Montagne, B., Hendriks, A. W., Perrett, D. I. & de Haan, E. H. F. Assessment of perception of morphed facial expressions using the Emotion Recognition Task: normative data from healthy participants aged 8–75. J. Neuropsychol. 8, 75–93, https://doi.org/10.1111/jnp.12009 (2014).
Fray, P. J., Robbins, T. W. & Sahakian, B. J. Neuorpsychiatyric applications of CANTAB. Int. J. Geriatr. Psychiatry 11, 329–336 (1996).
Phillips, L. K., Deldin, P. J., Voglmaier, M. M. & Rabbitt, S. Emotional Stroop performance predicts disorganization in schizophrenia. Schizophr. Res. 77, 141–149 (2005).
Zaki, J., Bolger, N. & Ochsner, K. It takes two: the interpersonal nature of empathic accuracy. Psychol. Sci. 19, 399–404 (2008).
Derntl, B. et al. Generalized deficit in all core components of empathy in schizophrenia. Schizophr. Res. 108, 197–206 (2009).
Kerr, S. L. & Neale, J. M. Emotion perception in schizophrenia: specific deficit or further evidence of generalized poor performance? J. Abnorm. Psychol. 102, 312 (1993).
Mehta, U. M. et al. Validation of Social Cognition Rating Tools in Indian Setting (SOCRATIS): a new test-battery to assess social cognition. Asian J. Psychiatry 4, 203–209 (2011).
Stone, V. E., Baron-Cohen, S. & Knight, R. T. Frontal lobe contributions to theory of mind. J. Cogn. Neurosci. 10, 640–656 (1998).
Bowers, D., Blonder, L. & Heilman, K. Florida Affect Battery (Center for Neuropsychological Studies, Department of Neurology Florida, USA, 1998).
Ambady, N., Hallahan, M. & Rosenthal, R. On judging and being judged accurately in zero-acquaintance situations. J. Personal. Soc. Psychol. 69, 518 (1995).
Happé, F., Brownell, H. & Winner, E. Acquiredtheory of mind’impairments following stroke. Cognition 70, 211–240 (1999).
Heider, F. & Simmel, M. An experimental study of apparent behavior. Am. J. Psychol. 57, 243–259 (1944).
Gibson, C. M., Penn, D. L., Prinstein, M. J., Perkins, D. O. & Belger, A. Social skill and social cognition in adolescents at genetic risk for psychosis. Schizophr. Res. 122, 179–184 (2010).
Klein, H. S. et al. Measuring mentalizing: a comparison of scoring methods for the hinting task. Int. J. Methods Psychiatr. Res. 29, e1827 (2020).
Rosset, E. It’s no accident: our bias for intentional explanations. Cognition 108, 771–780 (2008).
Kinderman, P. & Bentall, R. P. A new measure of causal locus: the internal, personal and situational attributions questionnaire. Personal. Individ. Differ. 20, 261–264 (1996).
Davis, Mark H. A multidimensional approach to individual differences in empathy. JSAS Catalog of Selected Documents in Psychology (1980).
Lane, R. D., Quinlan, D. M., Schwartz, G. E., Walker, P. A. & Zeitlin, S. B. The Levels of Emotional Awareness Scale: A cognitive-developmental measure of emotion. J. Personal. Assess. 55, 124–134 (1990).
Bazin, N. et al. Quantitative assessment of attribution of intentions to others in schizophrenia using an ecological video-based task: a comparison with manic and depressed patients. Psychiatry Res. 167, 28–35 (2009).
Mayer, J. D., Salovey, P. & Caruso, D. R. Mayer-Salovey-Caruso emotional intelligence test (MSCEIT) users manual. (2002).
Lysaker, P. H. et al. Metacognition amidst narratives of self and illness in schizophrenia: associations with neurocognition, symptoms, insight and quality of life. Acta Psychiatr. Scand. 112, 64–71, (2005).
Drury, V. M., Robinson, E. & Birchwood, M. Theory of mind’skills during an acute episode of psychosis and following recovery. Psychol. Med. 28, 1101–1112 (1998).
Bertoux, M., Delavest, M., de Souza, LC., Funkiewiez, A., L‚pine, JP., Fossati, P., Dubois, B. & Sarazin, M. Social Cognition and Emotional Assessment differentiates frontotemporal dementia from depression. J. Neurol. Neurosurg. Psychiatry 83, 411–416 (2012).
Bänziger, T., Scherer, K. R., Hall, J. A. & Rosenthal, R. Introducing the MiniPONS: A short multichannel version of the Profile of Nonverbal Sensitivity (PONS). J. Nonverbal Behav. 35, 189–204 (2011).
Barnby, J. M., Mehta, M. A. & Moutoussis, M. The computational relationship between reinforcement learning, social inference, and paranoia. PLoS Comput. Biol. 18, e1010326 (2022).
Dziobek, I. et al. Introducing MASC: a movie for the assessment of social cognition. J. Autism Dev. Disord. 36, 623–636 (2006).
Dziobek, I. et al. Dissociation of cognitive and emotional empathy in adults with Asperger syndrome using the Multifaceted Empathy Test (MET). J. Autism Dev. Disord. 38, 464–473 (2008).
Manera, V. et al. The multilingual CID-5: a new tool to study the perception of communicative interactions in different languages. Front. Psychol. 6, 152558 (2015).
Bänziger, T., Grandjean, D. & Scherer, K. R. Emotion recognition from expressions in face, voice, and body: the Multimodal Emotion Recognition Test (MERT). Emotion 9, 691 (2009).
Cyranowski, J. M. et al. Assessing social support, companionship, and distress: National Institute of Health (NIH) toolbox adult social relationship scales. Health Psychol. 32, 293 (2013).
Healey, K. M. et al. Observable social cognition–a rating scale: an interview-based assessment for schizophrenia. Cogn. Neuropsychiatry 20, 198–221 (2015).
Gur, R. C. et al. Age group and sex differences in performance on a computerized neurocognitive battery in children age 8–21. Neuropsychology 26, 251 (2012).
Powers, K. E., Worsham, A. L., Freeman, J. B., Wheatley, T. & Heatherton, T. F. Social connection modulates perceptions of animacy. Psychol. Sci. 25, 1943–1948 (2014).
Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y. & Plumb, I. The “Reading the Mind in the Eyes” Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. J. Child Psychol. Psychiatry Allied Discip. 42, 241–251 (2001).
Rutherford, M. D., Baron-Cohen, S. & Wheelwright, S. Reading the mind in the voice: A study with normal adults and adults with Asperger syndrome and high functioning autism. J. Autism Dev. Disord. 32, 189–194 (2002).
Grisdale, E., Lind, S. E., Eacott, M. J. & Williams, D. M. Self-referential memory in autism spectrum disorder and typical development: Exploring the ownership effect. Conscious. Cogn. 30, 133–141 (2014).
Lombardo, M. V., Barnes, J. L., Wheelwright, S. J. & Baron-Cohen, S. Self-referential cognition and empathy in autism. PloS ONE 2, e883 (2007).
Dodell-Feder, D., Lincoln, S. H., Coulson, J. P. & Hooker, C. I. Using fiction to assess mental state understanding: a new task for assessing theory of mind in adults. PloS ONE 8, e81279 (2013).
American Psychiatric Association, D. & Association, A. P. Diagnostic and Statistical Manual of Mental Disorders: DSM-5, Vol. 5 (American Psychiatric Association Washington, 2013).
Bell, M. D., Fiszdon, J. M., Greig, T. C. & Wexler, B. E. Social attribution test—multiple choice (SAT-MC) in schizophrenia: comparison with community sample and relationship to neurocognitive, social cognitive and symptom measures. Schizophr. Res. 122, 164–171 (2010).
Skuse, D. H., Mandy, W. P. & Scourfield, J. Measuring autistic traits: heritability, reliability and validity of the Social and Communication Disorders Checklist. Br. J. Psychiatry 187, 568–572 (2005).
Corrigan, P. W., Davies-Farmer, R. M. & Stolley, M. R. Social cue recognition in schizophrenia under variable levels of arousal. Cogn. Ther. Res. 14, 353–361 (1990).
Tyrer, P. et al. The Social Functioning Questionnaire: a rapid and robust measure of perceived functioning. Int. J. Soc. Psychiatry 51, 265–275 (2005).
Birchwood, M., Smith, J., Cochrane, R., Wetton, S. & Copestake, S. The social functioning scale the development and validation of a new scale of social adjustment for use in family intervention programmes with schizophrenic patients. Br. J. Psychiatry 157, 853–859 (1990).
Lin, A., Adolphs, R. & Rangel, A. Social and monetary reward learning engage overlapping neural substrates. Soc. Cogn. Affect. Neurosci. 7, 274–281, https://doi.org/10.1093/scan/nsr006 (2011).
Langdon, R., Connors, M. H. & Connaughton, E. Social cognition and social judgment in schizophrenia. Schizophr. Res. Cogn. 1, 171–174 (2014).
Behrens, T. E., Hunt, L. T., Woolrich, M. W. & Rushworth, M. F. Associative learning of social value. Nature 456, 245–249 (2008).
Diaconescu, A. O., Hauke, D. J. & Borgwardt, S. Models of persecutory delusions: a mechanistic insight into the early stages of psychosis. Mol. Psychiatry 24, 1258–1267 (2019).
Kramer, J. H. et al. NIH EXAMINER: conceptualization and development of an executive function battery. J. Int. Neuropsychol. Soc. 20, 11–19 (2014).
Okruszek, Ł. & Chrustowicz, M. Social perception and interaction database—a novel tool to study social cognitive processes with point-light displays. Front. Psychiatry 11, 496448 (2020).
Patterson, T. L., Moscona, S., McKibbin, C. L., Davidson, K. & Jeste, D. V. Social skills performance assessment among older patients with schizophrenia. Schizophr. Res. 48, 351–360 (2001).
Murray, K. et al. A new test of advanced theory of mind: the “Strange Stories Film Task” captures social processing differences in adults with autism spectrum disorders. Autism Res. 10, 1120–1132 (2017).
Happé, F. G. An advanced test of theory of mind: Understanding of story characters’ thoughts and feelings by able autistic, mentally handicapped, and normal children and adults. J. Autism Dev. Disord. 24, 129–154 (1994).
Gaudelus, B. et al. Measuring impairment of facial affects recognition in schizophrenia. Preliminary study of the facial emotions recognition task (TREF). Encephale 41, 251–259 (2014).
McDonald, S., Flanagan, S., Rollins, J. & Kinch, J. TASIT: a new clinical tool for assessing social perception after traumatic brain injury. J. Head. Trauma Rehabil. 18, 219–238 (2003).
Long, E. L., Cuve, H. C., Conway, J. R., Catmur, C. & Bird, G. Novel theory of mind task demonstrates representation of minds in mental state inference. Sci. Rep. 12, 21133 (2022).
Erwin, R. J. et al. Facial emotion discrimination: I. Task construction and behavioral findings in normal subjects. Psychiatry Res. 42, 231–240 (1992).
Sui, J., He, X. & Humphreys, G. W. Perceptual effects of social salience: evidence from self-prioritization effects on perceptual matching. J. Exp. Psychol. Hum. Percept. Perform. 38, 1105 (2012).
Aykan, S. & Nalçacı, E. Assessing theory of mind by humor: the humor comprehension and appreciation test (ToM-HCAT). Front. Psychol. 9, 382586 (2018).
Desgranges, B. et al. TOM-15: A false-belief task to assess cognitive theory of mind. Rev. Neuropsychol. 4, 216–220 (2012).
Abell, F., Happe, F. & Frith, U. Do triangles play tricks? Attribution of mental states to animated shapes in normal and abnormal development. Cogn. Dev. 15, 1–16 (2000).
Canty, A. L., Neumann, D. L., Fleming, J. & Shum, D. H. Evaluation of a newly developed measure of theory of mind: the virtual assessment of mentalising ability. Neuropsychol. Rehabil. 27, 834–870 (2017).
Moritz, S. et al. Whodunit–a novel video-based task for the measurement of jumping to conclusions in the schizophrenia spectrum. Psychiatry Res. 317, 114862 (2022).
Ratajska, A., Brown, M. I. & Chabris, C. F. Attributing social meaning to animated shapes: a new experimental study of apparent behavior. Am. J. Psychol. 133, 295–312 (2020).
Kelly, M. & McDonald, S. Assessing social cognition in people with a diagnosis of dementia: development of a novel screening test, the Brief Assessment of Social Skills (BASS-D). J. Clin. Exp. Neuropsychol. 42, 185–198 (2020).
Sarfati, Y., Hardy-Baylé, M.-C., Besche, C. & Widlöcher, D. Attribution of intentions to others in people with schizophrenia: a non-verbal exploration with comic strips. Schizophr. Res. 25, 199–209 (1997).
Ratto, A. B., Turner-Brown, L., Rupp, B. M., Mesibov, G. B. & Penn, D. L. Development of the contextual assessment of social skills (CASS): a role play measure of social skill for individuals with high-functioning autism. J. Autism Dev. Disord. 41, 1277–1286 (2011).
Baksh, R. A., Abrahams, S., Auyeung, B. & MacPherson, S. E. The Edinburgh Social Cognition Test (ESCoT): examining the effects of age on a new measure of theory of mind and social norm understanding. PloS ONE 13, e0195818 (2018).
Montagne, B., Kessels, R. P., De Haan, E. H. & Perrett, D. I. The emotion recognition task: a paradigm to measure the perception of facial emotional expressions at different intensities. Percept. Mot. Skills 104, 589–598 (2007).
Baron-Cohen, S. & Wheelwright, S. The empathy quotient: an investigation of adults with Asperger syndrome or high functioning autism, and normal sex differences. J. Autism Dev. Disord. 34, 163–175 (2004).
Costanzo, M. & Archer, D. Interperting the expressive behavior of others: the Interpersonal Perception Task. J. Nonverbal Behav. 13, 225–245 (1989).
Douglas, J. M., O’Flaherty, C. A. & Snow, P. C. Measuring perception of communicative ability: the development and evaluation of the La Trobe Communication Questionnaire. Aphasiology 14, 251–268 (2000).
Reniers, R. L., Corcoran, R., Drake, R., Shryane, N. M. & Völlm, B. A. The QCAE: a questionnaire of cognitive and affective empathy. J. Personal. Assess. 93, 84–95 (2011).
Acknowledgements
We would like to thank Linlin Fan, Ph.D., Sara Carrico, Lauren Bennett-Leleux, M.S., and Sarah Berretta for their assistance with various stages of the project. The study was supported by Schizophrenia International Research Society – Research Harmonization Award. TVR (group member) was supported by an Al and Val Rosenstrauss Fellowship from the Rebecca L Cooper Medical Research Foundation.
Author information
Authors and Affiliations
Consortia
Contributions
A.E.P., M.H., and T.Z. contributed to study conceptualization, methodological design, project administration, acquisition of study resources and funding, and reviewing and editing drafts of the manuscript. A.E.P. conducted the analyses and drafted the manuscript. Research Harmonization Group members participated in the expert survey and Delphi process, provided feedback at all stages of the study, edited drafts of the manuscript, and provided approval of the final version.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Pinkham, A.E., Hajdúk, M., Ziermans, T. et al. Harmonizing cross-cultural and transdiagnostic assessment of social cognition by expert panel consensus. Schizophr 11, 25 (2025). https://doi.org/10.1038/s41537-024-00540-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41537-024-00540-7
