Introduction

Scientific funding agencies like the U.S. National Science Foundation (NSF) and National Institutes of Health (NIH) have increasingly invested in research collaborations over the past few decades (National Research Council, 2015; König & Gorman, 2017). Demand for these transdisciplinary scientific teams is driven by scholarly assertions that transdisciplinary collaboration accelerates scientific discovery and addresses complex problems more effectively than individual-led efforts (e.g., Fiore, 2008; Wuchty et al., 2007; Hall et al., 2012). Consequently, these claims are often used to justify the funding of large-scale, long-term initiatives like NSF Science and Technology Centers (STCs), which convene researchers from various disciplines, career stages, and geographies to generate new knowledge, promote technological innovation, and recruit and retain students from diverse backgrounds (NSF, n.d.).

NSF currently contributes approximately $50 million over an STC’s 10-year lifetime. However, the internal metrics and external studies used to evaluate these programs largely focus on a Center’s scientific output (e.g., publications, awards, grants), which says little about whether the sum of scientific research conducted by these transdisciplinary teams is actually greater than what would have been achieved by an individual scientist or laboratory (Hall et al., 2018; König & Gorman, 2017). Furthermore, science of team science (SciTS) literature suggests that the effectiveness of a transdisciplinary research team depends not only on its assemblage of scientific knowledge experts but also on its relational capacity to exchange and strengthen this expertize over time (e.g., Love et al., 2021; Bammer et al., 2020; Woolley et al., 2010). As a result, understanding the effectiveness of transdisciplinary research collaboratives can be informed by examining the interpersonal behaviors and activities that impact team member engagement and participation over a team’s lifespan (Hall et al., 2018).

This study applies a SciTS approach to an NSF STC called the Center for Oldest Ice Exploration (COLDEX) to examine how relational connections among a transdisciplinary research team facilitate or hinder pathways for effective knowledge exchange, scientific innovation, and broadening participation in science. Inspired by Love et al., 2021 evaluation of an exemplary transdisciplinary team, this article uses a method known as social network analysis (SNA) to explore the interactions that catalyze connectivity between members of the COLDEX team and takes a longitudinal approach to understand how interpersonal connections within the COLDEX network change over time. This study builds upon Love et al., 2021 work by proposing “participation typologies” that describe how COLDEX participants engage and connect across the Center. These typologies will be tracked and repeatedly assessed over the project’s anticipated ten-year duration to gather real-time, data-driven insights about the Center’s relational evolution and advance the understanding of how certain activities and STC structures either impede or enhance transdisciplinary connectivity (e.g., Borgatti et al., 2009).

This paper is the first in a series of studies that aim to understand and improve how STCs such as COLDEX function and collaborate across an NSF-defined common structure. This initial research established a quantitative typological methodology to describe Center participation and engagement using a set of “participation typologies” and presents the results of a typological baseline assessment against which future SNA survey results will be compared. This first step to quantitatively establish “participation typologies” creates the foundation for a longitudinal mixed-methods study that will examine the ways in which interdisciplinary teams develop and evolve over time and assess the Center activities that catalyze or hinder participation and connectivity among participants. This article contributes a typological framework that can be used to provide near real-time insights to evaluate and inform the function and effectiveness of large transdisciplinary teams.

The goals of this paper are to:

  1. 1.

    Examine the initial transdisciplinary structure of a NSF STC.

  2. 2.

    Develop a methodology to quantitatively assess a transdisciplinary research team’s interactional evolution across project groups, career stages, and institutions.

  3. 3.

    Establish a typological baseline of participation from which to measure the transdisciplinary team’s evolution.

Background

SciTS is a transdisciplinary field that seeks to understand how multidisciplinary teams produce higher-impact science that effectively answers complex societal problems. One field of SciTS scholarship seeks to understand what and how team science approaches (often multi-disciplinary) bring more added value to scientific research compared to more traditional (commonly uni-disciplinary) research teams (Hall et al., 2018). As team science becomes an expectation for scientific funding agencies, developing methodologies and evaluation structures that facilitate the documentation and evaluation of team science approaches can help provide quantitative evidence rather than anecdotal evidence of the value and effectiveness of team science (Fiore, 2008; König & Gorman, 2017). Traditionally, science teams have been evaluated based on the quality of their scientific outputs by way of the proposal review process and summative publications (Trochim et al., 2008). However, large, often multi-year teams may benefit from ongoing interventions to strengthen their collaborative ties (Roelofs et al., 2018). Formative and longitudinal evaluation offers a way for science teams to assess effectiveness throughout the lifespan of a collaborative effort rather than at the end, providing opportunities for iterative and adaptive management to support and enable transdisciplinary collaboration (Roelofs et al., 2018).

Social network analysis as methodology for understanding dimensions of team science

SNA is an analytical method for identifying, describing, and interpreting the relational connections that exist between members of a social network (Gorczyca et al., 2012; Wellman, 1983). The methodology evolved out of 1930s social network theory, which describes the many ways individuals, organizations, or groups—defined in network theory literature as nodes—are connected to one another across a social structure (Wellman, 1983; Moreno, 1934). Today, SNA is a largely computational social science tool used to map the many roles, relationships, functions, and groupings within particularly complex networks like STCs (Borgatti et al., 2009).

The primary value of SNA is that it theoretically and analytically describes the many interactions, or dyadic links, that occur between nodes within a defined network, such as an STC. In a defined network, these dyadic links include similarities, such as a shared institutional affiliation or career status between nodes; social relations, such as a node’s collaborator or advisor role; discrete interactions, such as having “provided mentorship to,” “collaborated with,” or “helped” one another; and flows, or the information and resources exchanged between nodes via a dyadic link (Borgatti et al., 2009). In addition to revealing how nodes are connected, or tied, together, SNA can also be used to examine how a node’s individual position within a network may affect “the opportunities and constraints it encounters… [which] plays an important role in the node’s outcomes” (Borgatti et al., 2009). Finally, SNA can also be used to measure density, or the number of observed dyadic links divided by the total number of possible connections (Westaby et al., 2014).

Although SNA has faced criticism for being “merely descriptive” or “just methodology,” the method’s value is supported by a large body of literature that demonstrates its application across disciplines including psychology, criminology, medical science, risk analysis, and animal behavioral studies (Borgatti et al., 2009; e.g., Westaby et al., 2014; Faust & Tita, 2019; Valente et al., 2004; Ager et al., 2015; Wey et al., 2008). The methodology is particularly suited to SciTS (e.g., Ryan et al., 2014; Okamoto et al., 2015), where identifying and tracing relational connections between team members can help to assess the added value these connections may have on individuals, the larger team, and its fulfillment of scientific objectives. Yet despite this fit, applications of SNA in transdisciplinary research collaboratives like STCs remain limited (e.g., Love et al., 2021; Okamoto et al., 2015; Yang & Heo, 2014; Aboelela et al., 2007). Conducting an SNA of an ongoing STC therefore provides a critical opportunity to understand how relational dynamics within large-scale, long-term research collaboratives evolve over a project’s lifespan.

Overview of Science and Technology Centers and Center for Oldest Ice Exploration

Large-scale, long-term scientific collaboratives have become a prominent fixture of modern problem-solving, with funding agencies like NSF investing over $360 million dollars on transdisciplinary centers in 2022, nearly $56 million of which was used to fund 18 STCs (Wuchty et al., 2007; NSF, 2023b). These multidisciplinary and often geographically dispersed research teams are united by four pillars—Research, Education, Knowledge Transfer (KT), and Diversity, Equity, and Inclusion (DEI) – to help teams advance scientific research and innovation, broaden participation in STEM fields, share knowledge with external audiences and facilitate technology transfer.

Activities of the four STC pillars are intended to be well-integrated to improve Center-wide connectivity across pillars, and by extension, promote innovation and knowledge production. However, SciTS research suggests that the characteristic diversity STCs pride themselves on can also challenge scientific problem-solving, as team members must navigate the internal “social, organizational, political, and technological milieu that heavily influence how that work occurs” (Hall et al., 2018; National Research Council, 2015; Falk-Krzesinski et al., 2010; Stokols et al., 2008). For example, research has shown that interpersonal attributes such as relational history, emotional intelligence, and intensity of interaction can both negatively and positively impact a team’s effectiveness (Cummings and Kiesler, 2008; Love et al., 2021; Zhang et al., 2020; Boix Mansilla et al., 2016). In extreme cases, conflicting relational dynamics can impede the transfer of knowledge and advancement of science, causing some to question whether these research collaboratives are worth the monetary investment (Hall et al., 2018). Given these concerns, this research takes a SciTS approach to an NSF STC in order to provide data-driven insight as to how relational connections in these transdisciplinary research teams may facilitate or hinder pathways for effective knowledge exchange.

COLDEX is an NSF STC hosted by Oregon State University (OSU). The Center is funded for five years and involves 16 research institutions, liberal arts colleges, and professional organizations across the United States. As of April 2024, COLDEX included over 100 faculty, staff, and early career researchers (ECRs; defined here as undergraduate students, post-baccalaureates, graduate students, and postdoctoral scholars), and represented over 20 disciplinary affiliations, from glaciology and geochemistry to climate change communication and evaluation. Together, team members collaborate on efforts to “extend the ice-core record of past climate to at least 1.5 million years by drilling and analyzing a continuous ice core in East Antarctica, and to much older times using discontinuous ice sections at the base and margin of the ice sheet” (National Science Foundation (2023a)). In addition to this primary objective, COLDEX participants also pursue a multi-pronged secondary objective of broadening ECR participation in STEM fields, exchanging knowledge with external audiences (e.g., media, decision-makers, other research groups), and facilitating technology transfer with government and industry partners (COLDEX, 2024; Roop et al., 2024).

For COLDEX, the activities of the four pillars are undertaken by eight project groups operating both collaboratively and independently to accomplish goals at group, pillar, and Center-wide levels. The Center’s largest pillar by both funding and capacity, Research, is supported by four project groups: Exploration, Ice Analysis, Ice Coring, and Ice Dynamics & Modeling. Together, these project groups advance COLDEX’s primary objective through activities such as surveying the East Antarctic interior for viable old ice recovery sites, collecting and analyzing ice samples along the Antarctic margin, and developing new laboratory techniques and “rapid access” tools for examining and dating old ice (COLDEX, 2023).

The Center’s other three pillars each have one self-named affiliated project group. Education provides professional development opportunities for K-12 and college-level students and instructors. DEI addresses systemic barriers within polar science by engaging and recruiting students from underrepresented communities. KT supports internal and external communication and research such as this SNA research project. The Center’s eighth project group, Management and/or Other, operates separately from the pillar structure and oversees budgets, hiring and human resources, and monitoring of the Center’s activities.

Methods

This study uses SNA to measure and map existing and emergent relational connections reported by COLDEX participants during the Center’s first year (2021–2022). Although NSF defines an STC “participant” as someone who annually contributes 160 h or more to a project, this study uses the term “participant” to describe any individual identified as engaging in the COLDEX network via self-reported survey responses or responses reported by others, including the author team of this study. Initial survey data were analyzed to develop quantitative thresholds and typologies that characterize COLDEX participants’ varying forms of engagement in the Center’s activities. These typologies will serve as the foundation for monitoring the Center’s relational evolution over its intended ten-year duration.

Survey data collection

Connectivity data were collected using a Qualtrics (Qualtrics Labs, 2005) survey disseminated in September (hereafter the survey) of the Center’s first year (Appendix A). The survey was distributed to all COLDEX participants listed in the Center’s internal directory, which includes COLDEX-affiliated faculty, staff, and ECRs. The survey was distributed both in person during an annual meeting attended by most COLDEX participants and by email to reach those not attending. The survey had a 70% response rate (n = 67).

The survey asked respondents to provide professional demographic information, including their home institution, career status, and primary project group affiliation. Respondents cycled through the names of all participants listed in the Center directory and were asked to categorize their connection with each person as either 1) Direct (“I have directly communicated with this person 1:1 or in a small group”), 2) Indirect (“I recognize this person but can’t recall our direct interaction or communication”) or 3) Do Not Know (“I am not yet familiar with this person”). If Indirect was selected, respondents were asked to specify how they recognized that person. If Direct was selected, respondents were asked to detail further information about the reason and frequency for the interaction, cause of initial meeting, and nature of connection (i.e., mentorship and/or supervision, personal and/or professional). An optional final write-in encouraged respondents to correct any naming errors or add participants who may have been overlooked.

Development of quantitative thresholds and typologies

We used the survey data to consider two ways in which COLDEX participants interact and connect within the broader team: their integration within the wider Center and the “quality” of their connections to others (Table 1). We developed a set of five “Integration Thresholds (IT)” that describe the types of connectivity that an individual could have throughout the COLDEX network and a set of five “Quality Proxy Thresholds (QPT)” that use quantitative proxies to describe information about the strength, recurrence, and quality of connectivity across the Center’s network. Survey respondents had a quantitative value associated with each of these ten thresholds. Through analysis and normalization, we established three Center-wide ranges of “below average,” “average,” and “above average” for each IT, and “weak,” “average,” or “strong” for each QPT, which corresponded to a numeric value of one, two, or three (Table 2). For each respondent, the five IT and five QPT values were averaged to generate overall IT and QPT results, designating typological placement (Fig. 1; See Sections “Integration threshold development” and “Proxies for quality of connectedness in COLDEX ”).

Table 1 The ten quantitative thresholds that were developed to assess the different ways a participant could be connected with other team members (Integration Thresholds) and the quantitative proxies to understand the strength of those connections (Quality Proxy Thresholds).
Full size table
Table 2 The scalar values (0, 1, 2, or 3) assigned to each of the ten quantitative thresholds identified in our analysis.
Full size table
Fig. 1: Ranges for the ten Quantitative Threshold values across the Center.
figure 1

The Integration (A) and Quality Proxy (B) Threshold values have been separated. Below-average and weak values can be found to the left of each average range (in yellow), and above-average and strong values to the right of the average range. The Center-wide average (denoted in gray) for each threshold is also included.

Full size image

There were 29 individuals who were identified as nodes in the COLDEX network but did not complete the survey themselves. Rather than assign these individuals a typology, we elected to identify them as “Non-Respondents,” as there was not enough connectivity data available to formally characterize their level of participation or engagement. However, select professional demographic data—including institution, career stage, and primary project group—was available for these Non-Respondents via the Center directory and was incorporated when possible.

Although use of self-reported data meant that Non-Respondents did not impact the quantitative threshold calculations for 1) Total Connections, 2) Length of Connections, 3) Frequency of Interaction, 4) Mentorship and 5) Personal versus Professional, Non-Respondents did need to be considered in the development and analysis of the five thresholds that combined self-reported data and data reported by others: 1) Internal Institutional, 2) External Institutional, 3) Discipline, 4) Reciprocity (Quantity) and 5) Reciprocity (Percentage). To account for this, Non-Respondent survey data were not included in calculations that determined averages across the Center and career stages or to set the values of those quantitative threshold ranges. Additionally, two COLDEX participants were not listed in the Center directory during the survey’s distribution and therefore others were unable to report connectivity data to those survey respondents. Like Non-Respondents, these two respondents have been excluded from threshold calculations.

Coding was developed in R Studio Team (2020) to clean, organize, and facilitate data visualization in Kumu (2024), a software “platform for mapping systems and relationships using large data” that is standard in many social network analyses (e.g., Davies & Calma, 2018; Koon et al., 2022; Zucca et al., 2023). Non-respondent nodes were removed from the network maps provided in this article unless otherwise stated in order to better visualize relevant data.

Integration threshold development

Five IT describe the types of connectivity across the Center, including 1) Internal Institutional, 2) External Institutional, 3) Discipline, 4) Total Connections, and 5) Reciprocity. Together, these five thresholds identify the ways Center participants are connected within the COLDEX network.

Internal Institutional is a measure of the percentage of possible incoming and outgoing connections an individual could have with COLDEX participants within their home institution (e.g., OSU). To calculate this threshold, we determined each respondent’s percentage of possible institutional connections (PIC) using the calculation below, yielding an individual PIC value for each institution. Because survey respondents were able to report connections to Non-Respondents, this value was normalized by subtracting the total number of Non-Respondents at the institution from the PIC. Individual network maps were visualized using Kumu, and the platform’s SNA metrics were used to determine each participant’s total internal connections. This value was then used to calculate each participant’s percentage of Internal Institutional connections. COLDEX participants who were identified as the only individuals at their home institution were considered “Not Applicable” for this threshold.

$${\rm{PIC}}=2\left({{\rm{Participants}}}_{{\rm{INTERNAL}}}-1\right)$$
$${{\rm{PIC}}}_{{\rm{Normalized}}}={\rm{PIC}}-{{\rm{NonRespondents}}}_{{\rm{INTERNAL}}}$$

External Institutional is a measure of the percentage of possible incoming and outgoing connections an individual could have with COLDEX participants outside of their home institution. Possible external connections (PECs) were calculated by subtracting an individual’s unnormalized PIC (from Internal Institutional) and the total number of Non-Respondents outside of their home institution from the total number of possible Center connections (PCC) within the COLDEX network (190 possible connections). The total number of external connections a participant had was calculated by subtracting their internal connections from their total connections, which was then used to calculate the percentage of PEC.

$${\rm{PEC}}={\rm{PCC}}-{\rm{PIC}}-{{\rm{NonRespondents}}}_{{\rm{EXTERNAL}}}$$

Discipline examines the percentage of incoming and outgoing connections an individual has within their self-identified primary project group (e.g., Ice Analysis or DEI). Possible disciplinary connections (PDCs) were calculated by subtracting the number of Non-Respondents within a project group from the total number of possible connections for each project group. Much like the PIC values calculated for Internal Institutional, PDC values were calculated and normalized by subtracting the number of Non-Respondents within the group from that value. Kumu software was used to visualize project group networks and gather metrics for each individual’s number of connections. COLDEX participants who were identified as the only individuals at their home institution were considered “Not Applicable” for this threshold.

$${\rm{PDC}}=2\left({{\rm{Participants}}}_{{\rm{DISCIPLINE}}}-1\right)$$
$${\rm{Normalized}}={\rm{PDC}}-{{\rm{NonRespondents}}}_{{\rm{DISCIPLINE}}}$$

Reciprocity looks at the total number of reciprocated Direct and Indirect connections a survey respondent has with other COLDEX participants in their network. Reciprocity serves as a measure of Integration that accounts for the number of mutual connections a survey respondent could have across the Center. To assess Reciprocity, Kumu was used to visualize the network of each node and filter out unilateral connections so that only reciprocated connections remained. COLDEX participants who responded to the survey but were not listed in the COLDEX directory were considered “Not Applicable” for either the IT or QPT Reciprocity threshold, as other survey respondents could not report connections to these individuals (see 3.2.2).

Total Connections is the only fully self-reported IT. This threshold accounts for the total number of COLDEX participants that survey respondents reported directly interacting with during the surveying period. Because Indirect connections identified participants that a respondent was only familiar with but had not interacted with, Indirect connections were excluded from this threshold calculation.

The five IT include four thresholds that use both self-reported data and data reported by others and one threshold that uses only self-reported data. As a result, the IT average acts as a check against self-reported response bias, such as a respondent overestimating their number of connections at their home institution. Together, these five quantitative thresholds identify some of the ways Center participants are connected within the COLDEX network.

Proxies for quality of connectedness in COLDEX

While our survey did not directly assess the strength of a connection, data related to the longevity, recurrence, and the nature of interaction were used as proxies for understanding connection quality. These proxies led to the identification of five “Quality Proxy Thresholds (QPT),” including 1) Length of Connection, 2) Frequency of Interaction, 3) Mentorship, 4) Personal versus Professional, and 5) Reciprocity.

QPT includes four thresholds that use only self-reported data and one that uses both self-reported and data reported by others about that same individual. Whereas IT relies on data reported by others as it is necessary to minimize response bias in participants’ perceptions of their integration, QPT relies on self-reported data to more accurately assess the quality of connections, as only COLDEX participants themselves can speak to the strength of their relationship or sense of connection to another individual in the Center. Together, these five thresholds quantitatively characterize the nature and depth of the connections between Center participants in the COLDEX network.

Length of Connection uses self-reported data to ask respondents to categorize their connection length with another participant as 1) Less than 6 Months, 2) 6 Months to 2 Years, 3) 2 to 5 Years, or 4) More than 5 Years. For this threshold, we calculated the percentage of a participants’ total connections that have lasted two years or longer.

Frequency of Interaction utilizes self-reported data to assess how often respondents interacted with other COLDEX participants during the surveying period. Like Length of Connection, respondents were asked to approximate the frequency of their interaction with their reported Direct connections as one of the following: 1) Once or twice in the survey window, 2) A few times a month, 3) A few times a week, or 4) Almost every day. Reported values for “A few times a week” and “Almost every day” were then combined, as these frequencies were considered to be similar. For this threshold, we calculated the percentage of reported connections whose interactions occurred at this greatest combined frequency.

Mentorship identifies reported instances of receiving mentorship from, providing mentorship to, or a combination of providing and receiving mentorship to and from another COLDEX participant. For this threshold, we calculated the percentage of a respondent’s self-reported Direct connections that included a mentorship component.

Personal versus Professional examines self-reported data to determine whether respondents characterized the nature of each Direct and Indirect connection as personal, professional, and/or other. This threshold was calculated by determining the percentage of a respondent’s total connections that were identified as having a personal component.

Reciprocity in QPT calculates the percentage of possible reciprocated nodes (PRO) within an individual’s network. To normalize this calculation, the number of connections within each individual’s network that could not be reciprocated was calculated. These unreciprocated connections are defined as an outgoing connection to a Non-Respondent or an incoming connection from a participant not named in the survey (see Section 3.3). This value was then subtracted from the total number of nodes within an individual’s network map (excluding themselves) to determine the number of PRO. The number of reciprocated connections within an individual network was then divided by the PRO to determine the percentage of possible reciprocated connections.

Typology development

Following the process described above, participants individual threshold values were calculated and grouped into the below-average, average, or above-average ranges based on the values outlined in Fig. 1. These results were then assigned scalar values to be averaged to reach an overall IT and QPT final scaled result (Table 2), which were used to assign COLDEX participants to one of eight typologies (Table 3). The development of these typologies and the use of the language “above average” and “below average” suggests an inherent hierarchy within these typologies from Periphery to Connecting, Typical, Anchor, and Ingrained. However, one typology is not inherently “better” than another, nor is one quantitative threshold result better than another, and neither is the goal for any individual to progress to “higher” typologies (see 5.1).

Table 3 The eight participation typologies (including Non-Respondent) developed in this study, as well as the averaged Integration and Quality Proxy Threshold results and a description of a participant’s relational connections for each typology.
Full size table

Results

This study uses SNA to explore how relational connections evolve within a transdisciplinary research team. To do this, we first examine the initial connections between COLDEX team members based on key demographic characteristics of the COLDEX STC, including career status, institutional and project group affiliation, and self-identified pillar contribution. We then analyze the IT and QPT thresholds to sort COLDEX participants into scalar ranges and assign them to one of the eight “participation typologies.” Finally, we use the previously identified demographic characteristics to compare relationships between participants’ typological categorization and their position within the COLDEX network.

Network demographic characteristics

Survey data identified 96 total participants in the COLDEX network, including both survey takers (n = 67) and Non-Respondents (n = 29). In 2021, ECRs accounted for 46% of all COLDEX participants (n = 96), which included 11 undergraduate and 29 graduate students and four post-doctoral scholars. The remaining 54% (n = 52) of the COLDEX network was composed of faculty or staff. Of the Center’s 16 total institutions, OSU, University of Washington, and the University of Kansas had the highest number of COLDEX participants, positioning them as the Center’s three largest institutional hubs (Fig. 2).

Fig. 2: The institutional affiliations of all COLDEX participants in the Fall (n = 96).
figure 2

Oregon State U., U. Washington, and U. Kansas had the greatest number of participants and emerged as the Center’s largest institutional hub.

Full size image

Survey data also show that of the Center’s eight project groups, 37% of respondents identify primarily with Ice Analysis, and 31% identify primarily with Exploration (Table 4). The smallest project group identified was DEI, with only 1% of survey respondents identifying a primary association. Of the project groups remaining, 6% of respondents associated with Ice Coring and Management each, 10% with Education, and 4% with KT as their primary project group affiliation.

Table 4 Percentage of survey respondent (n = 67) participation in COLDEX’s four core pillars.
Full size table

Lastly, when asked to self-identify their engagement in COLDEX’s four core project pillars (Table 4), 88% of respondents identified participating in Research pillar activities, with 40% identifying Research as their only pillar contribution in the Center. By comparison, 21% of survey takers identified as participating in two COLDEX pillars, and 22% identified as participating in all four pillars.

Integration and quality proxy threshold results

The ten IT and QPT thresholds were analyzed to sort survey respondents into their respective scalar ranges (below average/weak, average, and above average/strong values; Fig. 1A). IT distribution reflects a relatively uniform spread across the Discipline, Total Connections, and Reciprocity thresholds, with approximately a third of participants falling into each of the three scalar ranges (Table 5). Internal Institutional and External Institutional maintained the largest IT distributional variations, with Internal Institutional yielding a skewed distribution of 51% of survey respondents at above average compared to 22% below and 28% average. The External Institutional threshold yielded a more concave distribution with 78% of survey respondents falling above and below average and 22% falling within the average range.

Table 5 The number of survey respondents within each threshold range (n = 67).
Full size table

Compared to IT, QPT distributions reflect greater range. Length of Connection, which describes how longstanding a connection is, has an average value range that spans from 11 to 50% (Fig. 1B). Within this threshold, 34% of respondents were weak, 28% were within the average range, and 37% of respondents were strong. By contrast, the most compact QPT, Personal v. Professional, identified an average value of one to eight percent, with the majority of survey respondents (54%) falling into the weak category. Frequency of Interaction identified 54% of survey respondents in the average range. Similar to the previously mentioned External Institutional threshold, the Reciprocity QPT yielded a more concave distribution, with 74% of survey respondents being categorized as strong or weak while 26% fell within the average range. Finally, within the Mentorship threshold, 34% of respondents fell within the weak range, 45% within the average range, and 21% within the strong range.

Center typology distribution

The IT and QPT results above enabled each COLDEX participant to be assigned a participation typology. Analysis of survey responses led to the classification of 10 Ingrained, 15 Anchor, 10 Typical, 18 Connecting, 11 Periphery, and three Networker participants. No participants were identified as Concentrated, and 29 COLDEX participants were classified as Non-Respondent (Fig. 3). Typologies with at least one average overall IT or QPT score (Typical, Connecting, and Anchor) accounted for 45% of COLDEX participants.

Fig. 3: Typological breakdown of COLDEX participants (n = 96) by career status.
figure 3

A combined 45% of participants had at least one average Integration Threshold or Quality Proxy Threshold score. The Ingrained (70%) and Anchor (73%) typologies are composed mostly of faculty/staff, while Networker (100%) is entirely faculty/staff. Typical (70%) and Periphery (82%) typologies, meanwhile, both have a majority composition of graduate and undergraduate students.

Full size image

Assessment of these typologies across career stages reveals that most of the participants within the Ingrained (70%) or Anchor (73%) participation typologies were faculty and staff whereas the Typical (70%) or Periphery (82%) were predominately comprised of ECRs (Fig. 3). The Networker typology, which represented the smallest number of Center participants, was entirely composed of COLDEX faculty and staff. The Connecting typology was the most heterogeneous typology, composed of 44% faculty and staff, 28% graduate students, 22% undergraduate students, and 6% postdocs.

Typological analysis

To better understand the impact of the QPT values across the five “hierarchical” typologies, we compared three QPTs—Mentorship, Personal versus Professional, and Length of Connection – as represented in five COLDEX participant network maps (Fig. 4). When viewed side by side, these maps reveal only minor differences in Mentorship across the typologies but significant differences in Length of Connection. Unlike ECRs, faculty and staff had relationships with COLDEX participants that existed prior to the Center’s formal convening: on average, 51% of faculty and staff connections were two or more years in duration, and 37% were five years or more, compared to 17% and 4% of ECR connections, respectively. However, Length of Connection is only one of five thresholds that are aggregated to form a participant’s overall QPT result, and QPT is only half of the calculation used to create these typologies, suggesting that there are other relational dynamics at play in participants’ perceived quality of connections.

Fig. 4: A comparison of mentorship, personal versus professional, and length of connection quality proxy thresholds for five COLDEX participants assigned to the periphery, connecting, typical, anchor, and ingrained typologies.
figure 4

Colored connections and elements correspond to connections that met the threshold value, whereas white connections and elements did not.

Full size image

One Anchor participant’s individual threshold maps show the importance of understanding these other relational dynamics (Fig. 5). Faculty and staff comprise 73% of the Anchor typology, and in the example maps provided, Length of Connection is the densest of the QPT maps (also including Mentorship, Frequency of Interaction, Personal versus Professional, and Reciprocity). Despite having a strong result for the Length of Connection threshold, this individual still had an average overall QPT result. While faculty members are more likely to have more connections of a longer duration than their ECR counterparts, there are other features of a connection to indicate quality that are incorporated into this methodology. Additional network maps for thresholds from each typology can be found in Appendix B.

Fig. 5: An example of an anchor participant’s network maps for six of the ten quantitative thresholds.
figure 5

Anchor participants had either a strong overall Quality Proxy Threshold (QPT) and an average Integration Threshold (IT) result, or an average QPT and above-average IT result. Within the Anchor typology, the average overall QPT result for participants was 2.1, and the average overall IT score was 2.6. This specific individual in this figure had an overall QPT of 2.2 and an overall IT of 2.6. All but 1 of the 15 Anchor participants had an above-average IT result and an average QPT value.

Full size image

Typologies, career status, and center participation

Data show a relationship between typological assignment and Center participation. Those in the Ingrained and Anchor typologies tend to be involved in three or four COLDEX pillars (Fig. 6B), which is a higher level of participation across Center pillars than the other typologies. This finding mirrors participation results segmented by career stage, which show that faculty and staff–primarily represented by Ingrained and Anchor typologies–tend to be involved in more activities across all four COLDEX pillars, whereas graduate and undergraduate students primarily represented by Connecting typologies–are more deeply engaged in only one or two pillars (Fig. 6A). While this finding does not negate instances of typologies engaging with pillar activities outside of these designated norms, it does provide tentative evidence that Center participants with higher levels of cross-disciplinary engagement generally have more numerous and deeper connections in COLDEX.

Fig. 6: A breakdown of the survey results (n = 67) examining the number of pillars respondents identify participating in based on career status and typology.
figure 6

A Depicting pillar engagement and career status, shows that while faculty/staff are reasonably well distributed across the total number of pillars, graduate students and undergraduates, in particular, tend to engage in fewer Center pillars. B Pillar engagement and typology, indicates that Peripheral and Connecting typologies tend to engage in only a single pillar, while Anchor, Ingrained, and Networker typologies interact in more than one.

Full size image

Discussion

Much of the SciTS research on transdisciplinary teams continues to retrospectively evaluate team effectiveness by its scientific output (e.g., awards, publications, additional grants funded; Hall et al., 2018; König & Gorman, 2017). However, these traditional measures of success are not always practical or representative, particularly in the case of STCs: funding agencies often invest in Centers for five or even ten years, despite the fact that “proof” of a Center’s merit (as measured by publications or awards) may not be available for years into their tenure. Additionally, these measures provide little value by way of diagnosing and guiding STC performance improvements over five years, or even a decade, of funding. Although there is a growing call to address this weakness using longitudinal approaches, limited attention has been given to developing a replicable methodology with which to quantitatively and qualitatively assess a team’s relational evolution throughout its lifespan (e.g., Love et al., 2021; Hall et al., 2018), and determine what activities, interventions and structures can support and accelerate innovation and effective team science.

SNA and the development of “participation typologies” and associated thresholds can help fill this gap by providing a method with which to establish baseline participation at the outset of a team science endeavor and track the team’s participation evolution over time. This formative approach offers a means for more timely and relevant data about the relational dynamics of large teams, allowing interventions and improvements to be offered more immediately, potentially yielding improved outcomes and stronger team connectivity (Roelofs et al., 2018; Stokols et al., 2008). Previous research shows that interpersonal relationships and strong collaborative ties can impact team effectiveness (Cummings and Kiesler, 2008; Love et al., 2021; Zhang et al., 2020; Boix Mansilla et al., 2016). COLDEX, as a team science effort with a decade of anticipated collaboration, will use these participation typologies to provide critical near real-time insights into Center effectiveness as it relates to activities and interventions that help to support the development and stewardship of these strong collaborative ties and interpersonal relationships by 1) the relationships between career stage, participation typology, and disciplinary or pillar affiliation, 2) the ongoing integration of ECRs into the COLDEX community, and 3) the influence of connection quality on typologies.

Anticipated evolution of typologies

The method by which the participation typologies were created leads to an implied hierarchical structure of typologies. A natural typological progression could exist in the form of a participant moving from Periphery to Connecting, then Typical, then Anchor, and finally Ingrained, as more connections are developed and strengthened over time. However, the goal should not be for a team to advance all of its members to “higher” typologies. In fact, we view this implied hierarchy as more of a spectrum, and we hypothesize that participants will move through the typologies in both directions, being more or less connected to others in the Center at different times. A combination of more time in the Center, engagement in Center activities, and barriers to participation such as a participant feeling as though they have nothing to contribute to a meeting, will likely influence how participants progress through the spectrum of typologies.

Existing SciTS literature suggests that there are relational and structural aspects, such as the availability and use of technology, effective communication structures, and how trust and accountability are built among collaborators, which influence how knowledge is transferred between members of effective interdisciplinary and geographically dispersed teams (Wu et al., 2007; Stokols et al., 2008; Cheruvelil et al., 2014). Wooley et al. (2010) found that teams whose members have good interpersonal skills, such as social sensitivity, best predicted a team’s ability to problem solve and team performance. We hypothesize that if the structural aspects to facilitate collaboration are established and remain in place, that participation in Center activities and typological progression will be determined by how a participants’ relational connections develop and evolve.

Understanding how these factors influence participation typologies and which Center activities support or hinder their development will be the focus of future work. Modifications to the quantitative survey have already been made to capture which offered activities COLDEX participants attend and report connecting to others. This work will also include interviews and focus groups aimed to deepen our understanding of the contexts and influences that lead to participation and resultant typological progression. Together, this work will help to inform what interventions, such as increasing the frequency of activities that the mixed-methods datasets suggest are successful in fostering relational connections, altering structural aspects of the Center, or introducing a formal mentorship program for ECR and early-to-mid career faculty, can be made to limit barriers to participation and increase engagement.

Connection quality influence on typological evolution

With the exception of Networker and Concentrated typologies, a respondent’s overall IT or QPT needs to increase in value to “advance” to the next typology. A closer look at Anchor participants showed that 15 of 16 had an above-average IT result and an average QPT (Fig. 5). That 94% of participants within this typology had a higher IT result indicates that a participant’s perceived quality of connections plays a crucial role in—and, in the case of the COLDEX network specifically, a major barrier to—their ability to evolve to the Ingrained typology. Establishing a better understanding of why participants do or do not have a strong sense of connection within the Center is a key objective of the upcoming qualitative interviews and focus groups that make this a mixed-methods approach.

These relational observations provide crucial evidence that transdisciplinary research collaboratives like STCs can stimulate Center-wide connectivity, even in the early stages of a Center’s development. Most notably, these insights suggest that the connection between a participant’s perceived quality of connection and their ability to progress through the typologies serves as a reminder that broadening and deepening participation over time necessitates building relational capacities and strengthening interpersonal relationships and collaborative ties beyond demographic silos such as project groups, career stages, and institutions. Activities and opportunities that catalyze participants’ engagement and connectivity beyond these bounds play a crucial role in providing justification for the continued funding of large-scale, long-term initiatives like STCs, and are the focus of future work.

A case study of ECR typologies and participation

A closer look at the baseline typological results for ECRs, who account for 46% of Center participants, shows that they are proportionally over-represented in the Periphery typology (82%). However, there is a path forward for ECRs to reach the Anchor and Ingrained typologies, as seven graduate students and postdocs were able to do through participation in multiple COLDEX pillars and the establishment of more cross-disciplinary connections (Fig. 3). Of these seven ECRs, four participated in two Center pillars, while the three remaining ECRs each participated in either one, three or all four pillars. The reported engagement by the postdoc and late-stage graduate student in three and four Center Pillars could signify a career transition point from the one- and two-pillar engagement exhibited by undergraduate and students to the three- and four-pillar engagement exhibited more commonly by faculty and staff (Fig. 6A). Tracking the pillar engagement and cross-disciplinary connections of a COLDEX participant as they transition from student to postdoc through the surveys and qualitative interviews could provide insight as to how an individual naturally moves through the typologies over the course of their career and the factors that lead to enhanced connectedness within the Center.

One of the case studies we will conduct in the future for examining typological movement will be following ECR career trajectories, as one of the objectives of an STC is to broaden participation and develop researchers in the field. We hypothesize that when an ECR begins their graduate program, they are likely to establish connections within their home institutions more quickly than externally throughout the Center and that those connections are likely to become stronger more quickly than External Institutional connections due to geographic proximity and an increased opportunity for interaction, which can lead to high levels of trust and collaboration (Roelofs et al., 2018). It could be expected that over time, through increased participation in Center activities, those connections would develop and deepen, moving the student from a lower spectrum typology like Periphery to a higher one, such as Connecting or Typical. As the student becomes engaged in more Pillar-spanning activities or takes on leadership roles within the Center, they may progress to an Anchor or even Ingrained participants. If that student participates in remote fieldwork, they may also move down on the typological spectrum, as they interact with fewer participants but have deeper connections to a smaller subset of participants.

Limitations

Although these findings establish an important relational baseline from which to measure the Center’s future growth and knowledge exchange, this baseline understanding still faces limitations. First, it is limited by unintended surveying errors. Gathering relational information via surveying makes data collection vulnerable to human error, as people may forget or misremember names, dates, and frequencies of meetings. Furthermore, relational connections do not adhere to clean temporal boundaries, and survey timing therefore affects which relationships are captured and represented across snapshots of the Center’s growth. Although this study’s combination of self-reported data and data reported by others in threshold calculations likely reduced the most egregious surveying errors (see section 3.3), some flawed or incomplete data is bound to remain.

Second, this baseline understanding is limited by the Center’s Non-Respondents, as many of these participants play pivotal roles across COLDEX (see section 3.1). Although response rate is a notable impediment for any survey, the COLDEX network is especially challenged due to definitional confusion over who “qualifies” as a COLDEX participant, particularly in cases where an individual is working at a COLDEX-affiliated lab or institution but not formally funded to produce COLDEX research. Section 3.2 articulates the ways in which Non-Respondents were accounted for during data analysis, and future surveys are working to address this limitation with an additional query related to the survey takers’ sense of belonging in COLDEX.

Finally, this baseline understanding is limited due to its current dependence on quantitative connectivity metrics. Thus far, our proposed methodology has used only quantitative data to create and analyze thresholds and typologies; however, in order to truly apply a mixed-methods research approach, qualitative data must also be collected. Future work is already underway to incorporate focus groups and interviews with COLDEX participants with the quantitative data sets already gathered. The aim of this qualitative data collection is to investigate the specific activities that COLDEX participants engage in and glean deeper insight as to how interpersonal behaviors within the COLDEX network change over time (e.g., Love et al. 2021; Kelman et al., 2016). Combining our pre-existing quantitative data with these supplemental findings will offer a richer understanding of how connections are formed and strengthened within the Center, of what specific interactions and interventions are facilitating or hindering pathways for effective knowledge exchange, and will validate or improve the QPT.

Conclusion and next steps

This study applied a SciTS approach to critically examine how relational connections in a transdisciplinary research team may facilitate or hinder pathways for effective knowledge exchange. Building off of Love et al., 2021 use of SNA to evaluate an exemplary transdisciplinary team, this study proposed a new methodology to quantitatively assess a research team’s interactional evolution across project groups, career stages, and institutions. This methodology was then applied to the COLDEX STC to establish a typological baseline and identify early changes that have emerged from the Center’s first year of operation.

Key relational observations were made that may implicate the team’s long-term pursuit of its scientific objectives. First, our data suggests that a participant’s perceived quality of connections plays a crucial role in – and, in the case of COLDEX specifically, a major barrier to – their ability to reach deeper typological levels within the research team. Second, a possible correlation exists between career stage, typology, and participation in Center pillar activities, which could indicate that individuals with senior career status or who have more cross-disciplinary participation are more likely to fall into the more advanced typologies. Finally, we see that the Center had early success in integrating ECRs into the network, which could continue to develop as ECRs participate in more Center activities and establish new relational connections. Together, these observations provide important data-driven evidence that transdisciplinary research collaboratives like STCs can and are achieving Center-wide connectivity, but also that conscious relational intervention must be taken if these connections are to broaden and deepen beyond professional demographic silos such as project groups, career stages, and institutions.

To further develop and continue this work, we have already implemented several other surveys to begin tracking the network longitudinally, and have begun conducting focus groups and interviews to provide rich data around which specific activities (e.g., webinars, standing virtual meetings, in-person gatherings, and professional development training, etc.) people engage in to understand what offerings from COLDEX people find valuable and enhance participation and connection. In building on this baseline for the Center, we plan to purposefully sample those ECRs within the Ingrained and Anchor typologies to better understand their journey to connectedness. We will take a robust and comprehensive approach to our qualitative inquiry and gain insights around each of the typologies developed in this study, as well as purposefully sampling case studies that emerge from the quantitative datasets to better understand their journey to connectedness. This added qualitative component to the research will augment the quantitative survey data that continues to be collected to better understand individual experience as COLDEX participants move throughout the different typologies and stages of connectedness.

This paper establishes an important and timely baseline for a longitudinal study intended to understand and improve how large transdisciplinary science teams such as Science and Technology Centers function and collaborate. As the COLDEX network evolves over its intended 10-year duration, this typological framework will be used to monitor changes in participation and engagement and recommend specific interventions in real time to help foster and deepen relational development for all participations. In doing so, this study offers researchers and scientific funding agencies an evaluative tool and framework for better understanding and quantitatively assessing the value and extent to which transdisciplinary interaction occurs across large-scale, research collaboratives from an array of scientific fields.