Introduction

The rapid development of artificial intelligence (AI) technology has brought unprecedented opportunities and challenges to the education sector. Particularly, under the impetus of human–AI collaboration, AI not only serves as an auxiliary tool to support teachers’ instructional tasks but also actively participates in classroom interactions in an intelligent manner, facilitating a profound transformation in education (Alier et al. 2024; Zhou and Hou, 2024; Hilpert et al. 2023). Traditional educational models have limitations in facilitating personalized learning and optimizing resource allocation. Human–AI collaborative teaching offers an efficient and sustainable lens to address these challenges through close collaboration between AI systems and teachers (Díaz and Nussbaum, 2024; Chen et al. 2022).

Human-AI collaboration in education, as an emerging interdisciplinary field, integrates cutting-edge theories and technologies from AI, education, cognitive science, and human–computer interaction. The Hybrid Intelligence Learning Environments design aims to develop and implement effective human–AI collaboration in education. Its core philosophy lies in the seamless integration of human intelligence and machine intelligence to achieve optimized teaching outcomes through their synergistic collaboration (Cukurova, 2024; Bredeweg and Kragten, 2022). Within this environment, AI not only functions as an assistant to teachers but also plays a vital role in personalized learning (Mittal et al. 2024), real-time feedback (Weber et al. 2024), cognitive intervention (Fan et al. 2024), and emotional engagement (Järvelä et al. 2023). In recent years, researchers started to look into the design of educational robots, the application of AI in teaching, and the cognitive and behavioural dynamics of human–AI collaboration (Schecter et al. 2022; Niu et al. 2024; Wu et al. 2024).

Existing studies have demonstrated significant advantages of human–AI collaboration in enhancing learning efficiency and supporting personalized learning (Huang et al. 2021). For instance, the integration of AI applications in real-time question answering (Fang et al. 2023) and homework grading (Duan et al. 2023) has effectively reduced teachers’ workload while improved instructional quality. The most extensive literature on AI applications in education focused on technological aspects, lacking systematic examination of human-AI collaboration during classroom instruction (Vössing et al. 2022; Yue and Li, 2023). Given that effective human-AI collaboration relies on effective collaboration between humans and AI systems, investigating the effectiveness of collaboration and the degree of synergy in classrooms becomes crucial. To address this research gap, this study aims to develop a framework to evaluate and enhance the synergy and orderliness of human–AI collaboration in classrooms.

The degree of collaboration and interaction between humans and AI can be analysed using the synergy degree model (SDM), which has been employed in various educational research. For example, Xue et al. (2020) applied the SDM to analyse the synergy between internal and external collaborative innovation in Chinese universities, comparing trends in collaboration development among three representative universities. Similarly, Cai and Wang (2022) utilized the SDM to assess the synergy of multiple teaching tools in blended classrooms and further calculated the orderliness of these tools. This study adopts the SDM approach to examine the efficacy of human–AI collaboration systematically, unveiling the key factors that influence its impact in educational contexts.

This paper is organized as follows: section “Theoretical background” introduces the theoretical background of this study by providing insights into the trend of human–AI collaboration and SDM. Section “Methods” describes the case study. Section “Results” presents the findings on the degree of order and synergy of human–AI collaboration to answer the research questions. Section “Discussion” discusses the framework of human–AI collaboration evaluation, order degree, and synergy degree of the current situation. Finally, the conclusion and limitations are presented in the section “Conclusion and limitations”.

Theoretical background

Human–AI collaboration in the classroom

The increasing use of AI in K-12 classrooms has significantly impacted education. Human intelligence and AI intelligence operate differently. Combining both intelligences in human–AI hybrid systems is critical for extending human cognition (Cukurova, 2024). In hybrid intelligence learning environments, human and AI work collaboratively. First, human–AI collaboration in classrooms helps teachers and students in various ways. For example, online educational resources can be recommended to students by AI for personalized learning (Ertmer et al. 2012), and AI-powered data collection and analysis can help human teachers and students monitor and track students’ progress (Yousufi et al. 2023). Järvelä et al. (2023) implemented human–AI collaboration through a novel trigger concept and a hybrid model of shared regulation (Human–AI shared regulation in learning), demonstrating the ways in which AI can enhance students’ self-regulated learning (SSRL). Furthermore, human–AI collaboration can assist in the management of classrooms. Paiva and Bittencourt (2020) used AI tools to help teachers make informed decisions regarding curriculum management, such as the development of partners. AI has also been used to predict future attendance and identify at-risk students (Sutjarittham et al. 2019).

There are different ways of AI integration in classroom teaching and learning. AI technologies can be embedded in computer systems and other advanced technologies, such as humanoid robots and online chatbots, to perform tasks typically assigned to instructors. Instructional design guidelines have been developed to employ an interactive robot as an assistant to enhance students’ English acquisition while simultaneously reducing the pressure and teaching load exerted on English instructors (Wu et al. 2015; Alemi et al. 2014). Wu and Yu (2023) suggested that educators could increase students’ learning outcomes by equipping AI chatbots with humans, like avatars, gamification elements, and emotional intelligence. Chatbots have been shown to improve classroom teaching and learning (Birenbaum, 2023; Haindl and Weinberger, 2024).

Human–AI collaboration in the classroom emphasizes the collaborative relationship between teachers and AI in which human teachers receive real-time support from AI tutors. AI tutors can analyse students’ personal situations and provide teachers with tailored support to improve their classroom teaching (Holstein et al. 2018). Sheridan et al. (1978) classified the machine role between humans and machines into four categories: extend, relieve, backup, and replace. The integration model of human–AI has three layers: embodiment, amplification, and interaction (Li et al. 2020).

In summary, numerous AI tools and applications have been developed, demonstrating significant potential for education. However, studies examining the mechanism of human–AI collaboration are rare. Investigating this collaboration within the classroom is crucial for understanding the effectiveness of AI integration in teaching and learning.

Towards an integrative evaluation of human–AI collaboration

Collaborative learning research focuses on the collaborative subject, process, and environment for facilitating effective teaching and learning (Moghaddam, 2008; Kim et al. 2022). Gupta and Pathania (2020) explored the collaborative subject, process, and environment within Google Classroom in the teacher education context. The learning environment fosters student–teacher and peer collaboration, promoting flexible learning and efficient communication. Lowell and Ashby’s (2018) study investigated the collaborative environment in online courses, where students, as collaborative agents, engaged in the design process by participating in feedback activities and utilizing feedback resources. Students enhanced their professional growth and developed their feedback skills by providing and receiving peer feedback. Human–AI collaborative classroom teaching necessitates a multifaceted approach to investigation. As AI transitions from an object to a subject, the human–AI collaboration should also be analysed through the lenses of collaborative subjects, processes, and environments, to better understand the dynamic and evolving relationship between humans and AI.

First, the subject in human–AI collaboration is the initiator of actions. Memarian and Doleck (2024) proposed a multidimensional taxonomy for human–AI interactions, in which a network of interactions may exist between the teacher, student, and AI entities. The alignment between the learner and AI may vary; therefore, the situation of the initiator of actions may also vary. For example, Farshad et al. (2023) conducted an empirical study examining the application of machine learning to study student engagement.

Second, human–AI collaboration supports educational teaching, with both humans and AI participating in teaching activities. Schoonderwoerd et al. (2022) stated that humans and AI have different kinds of collaboration models, with different effects on co-processing and co-performance. Specifically, the researchers designed interaction sequences and learning patterns to facilitate co-learning. A bidirectional evaluation approach was used to evaluate the quality of pedagogy in a human-robot collaboration situation (Wang et al. 2021).

Third, the learning environment and context are important to support human–AI collaboration in classroom teaching. Virtual reality (VR) and augmented reality (AR), collectively referred to as extended reality (XR), are reshaping human–AI interactions in today’s digital era (Christoff et al. 2023). The human–AI collaboration environment can incorporate a wide range of technologies and facility equipment; for example, a contextual learning-based robotics education pedagogy is developed for computer science students to help them learn robotics and develop their problem-solving abilities in situations involving human and robot interaction (Wang et al. 2021).

Informed by the above literature, this study proposes three elements of human–AI collaboration classrooms: collaboration subject, collaboration process, and collaboration environment, as illustrated in Fig. 1.

Fig. 1
Fig. 1
Full size image

An integrative perspective on human–AI collaboration.

Collaboration subject

In the triangular model of the sociocultural theory activity system, teachers from the community, students are the subjects, and the AI-assisted system serves as an intermediary (Sirisatit, 2010). In hybrid intelligence learning environments, robots are introduced into the classroom as a new AI-assisted system, and there is a synergistic effect between humans and AI (Dzedzickis et al. 2024; Baltrusch et al. 2021). Three elements—teachers, students, and AI machines, work synergized to achieve effective teaching and learning.

First, teachers establish learning designs and oversee AI machines. Although AI machines have some level of autonomy, human judgements should guide machine value judgements (Järvelä et al. 2023). In the classroom, teachers use AI machines to reduce their workload, which enables them to allocate more time to facilitate high-order thinking and 21st-century competencies of the students (Ley et al. 2023; Holstein and Aleven, 2022; Bredeweg and Kragten, 2022). Notably, teaching expertise and collaboration skills with AI are important for teachers when using AI technology (Jeon and Lee, 2023) during this process.

Second, technology-rich classrooms integrated with AI provide an enhanced learning experience for students (Dimitriadou and Lanitis, 2023). To acquire new knowledge in human–AI hybrid intelligence learning environments, it is essential that students cultivate the awareness and ability of human–AI collaboration and coexist harmoniously with machines (Kim et al. 2022). Access to immersive experiences and AI readiness can also improve learning quality and effectiveness.

Third, AI machines need to address human needs (Boardman and Butcher, 2019). AI machines provide valuable decision support to humans with their efficient and accurate sensing and processing capabilities. This not only compensates for human cognitive limitations but also significantly enhances the efficiency of task execution (Norlander, 2024). An example is computer vision which enhances human perceptual abilities. The computer vision techniques can recognize and interpret human gestures, emotions, and facial expressions, thereby not only augmenting human capabilities in interaction and collaboration but also paving the way for seamless human–AI synergy (Obaigbena et al. 2024). When interacting with humans, AI machines can also ‘think’ in a similar way to humans, and the machine’s capabilities can complement those of humans (Lesh, 2004). AI machines also assist teachers in completing teaching tasks, handling high-speed computations, and automating tasks with high precision (Singh and Mishra, 2021). The application of AI technology has changed teachers’ roles and reshaped their behaviour with students (Holstein and Aleven, 2022).

Collaboration process

Integrating AI into education brings significant changes to education goals, content, methods, and evaluation (Luan et al. 2020). Zhou et al. (2022) suggested that the six core elements of the process activity system are subject, object, community, rules, division of labour, and intermediary tools. Three factors that impact the collaboration process subsystem have shifted in the AI era: goal (Wang, 2009), process (Mora et al. 2020), and method (Christinck and Kaufmann, 2017).

First, teaching goals must align with the new requirements of human–AI collaboration, such as thinking skills (Bredeweg and Kragten, 2022). Moreover, human–AI collaboration should consider students’ learning context, including their location, available resources, and learning goals (Fan and Zhang, 2022), and should be adjusted and optimized based on dynamic changes that occur in the teaching and learning process (Järvelä et al. 2023).

Second, as for the teaching process, when selecting teaching content using human–AI collaboration, attention should be given to breaking the boundaries between knowledge production and dissemination, as well as diversifying content resources and shareability (Gabriel et al. 2021). In addition, teaching content should be presented in a manner that is reasonable and appropriate for students (Chiu, 2021). Accurate and timely knowledge transmission and feedback provision are essential for successful human–AI collaboration. This process also involves continuous adjustments and the selection of teaching methods to ensure that they are coherent and reasonable (Liang et al. 2022).

Third, teaching methods in classrooms and techniques should be versatile and extendable (Wienrich and Latoschik, 2021) so that teachers or AI machines can provide a variety of methods, strategies, and pedagogical models to accommodate learners’ goals and objectives to support them better (Kay, 2023). The chosen methods must be suitable for the teaching activities (Lee and Yeo, 2022).

Collaboration environment

Recent studies have suggested that AI infrastructure and technology are essential components of an intelligent teaching environment (Liang et al. 2022). The structural elements of such classrooms include external hardware foundations and internal software resources (Lugrin et al. 2016). In classrooms supported by human–AI collaboration, teaching, and learning become a cognitive process that involves collaboration between teachers, students, and machines in a complex and diverse ternary space (Fang et al. 2022). Such classrooms’ environment comprises sensor technology and resource platforms (Lu et al. 2020). Therefore, the collaboration environment subsystem is generally influenced by three main factors: the space–time structure, infrastructure, and key technology.

Regarding the space–time structure, cognitive interaction in human–AI hybrid intelligence learning environments occurs in different spaces, requiring different interaction and processing methods (Hakim et al. 2022). Additionally, the timing and cognitive transfer during the human–AI collaboration learning process vary, impacting its adoption and applicability (Shahab et al. 2021).

As for infrastructure, equipment is an essential tool to support the seamless connection of intelligent classroom spaces and the agile perception of the situation. Perception equipment enables the automatic perception of the learning context and data collection during the teaching process, which helps form a closed loop of data flow in a collaborative environment (Wang et al. 2019). The support of a server is necessary for data storage and analysis, and the terminal equipment for both teachers and students is essential to facilitate the interaction between humans and AI to generate the corresponding interaction data.

In terms of key technologies, human–AI hybrid intelligence techniques for environmental analysis and data processing include input, processing, and output techniques (Dellermann et al. 2021). This includes the integration of objective data and subjective perception and the combination of machine computing power and human cognitive co-processing. Moreover, the value effect of human decision-making and the synergetic integration technology of computer decision-making should also be considered.

Synergy degree model (SDM)

Haken (1983) proposed a synergetic theory in which a system consists of many subsystems, each containing different elements. These subsystems collaborate with each other, promoting the development of the system from disordered to ordered. The synergy degree indicates the level of interaction among subsystems, which are larger groups of parts that work together (Arnold and Wade, 2015). The order degree in subsystems refers to the level of collaboration among its elements, and it emphasizes the structural arrangement and hierarchical organization of subsystem elements, focusing on control and subsystematic arrangement. In contrast, the synergy degree concentrates on the effectiveness of interactions between subsystems, evaluating how collaboration enhances overall system performance beyond the individual contributions of its parts. These elements are the essential parts within subsystems, which can be achieved through self-organization and external management (Bickford and Vleck, 1997). A positive correlation exists between the synergy degree and the synergy effect. The greater the system collaboration, the stronger the overall function of the system, resulting in a better synergy effect, that is, 1 + 1 > 2. Therefore, the SDM focuses on the overall synergistic effects within and between subsystems, aiming to improve the collaborative level of the entire system rather than merely aggregating individual behaviours.

The SDM has been widely applied across various research fields, including multidisciplinary science (Xia and Bing, 2022), neuroscience, and sustainable science technology (Zhang et al. 2020). Although the SDM has been used less frequently in social sciences, the model has been found to be helpful in addressing relevant issues. For example, Tang and Gao (2022) employed the SDM to analyse 210 scientific and technological talent policies, measuring both the order degree of policy subsystems and the synergy degree of the composite system. Furthermore, the SDM captures the dynamic processes of interaction and collaborative evolution within a system, revealing the synergistic effects among its various components. As such, it is well-suited for analysing human–AI collaboration in classroom teaching. This model not only effectively assesses the synergy between subsystems (Bickford and Vleck, 1997), but also examines the synergistic effects among complex classroom elements, offering valuable theoretical insights into classroom synergy mechanisms and human–AI collaboration dynamics.

Research questions

An integrative perspective on human–AI collaboration encompasses a collaboration subject, collaboration process, and collaboration environment. By leveraging the SDM, the objectives of this study are: to study the order degree of collaboration subject subsystem, collaboration process subsystem, and collaboration environment subsystem respectively, and to examine the synergy degree of the human–AI collaboration classroom teaching system, comprising of the three subsystems, which is measured by the level of collaboration among these three subsystems. The two research questions are as follows:

RQ 1. What is the order degree among subsystems of human–AI collaboration in hybrid intelligence learning environments?

RQ 2. What is the synergy degree within the classroom teaching system of human–AI collaboration in hybrid intelligence learning environments?

Methods

Context and participants

This case study focuses on an eighth-grade English class on the topic ‘Will people have robots?’. The class lasted ~40 min and included 12 teaching activities, including robot importing, interaction dialogue, group discussion, group presentation, teaching and learning, material viewing, teaching and learning, knowledge exploration, video learning, teaching and learning, homework testing, and teaching and learning. The main ‘AI machine’ used in the collaboration subject was an educational robot, and the intelligent classroom was human–AI hybrid intelligence learning environment. Forty eighth-grade students and one teacher at a junior high school in Beijing, China, participated in this study.

Human–AI collaboration classroom procedure

A case study was designed by introducing a robot into hybrid intelligence learning environments. The teacher and the researchers developed the instructional design and wrote the robot scripts together in pre-class preparation. Once the preparations were ready, the teacher conducted the classroom intervention for 40 min. Figure 2 outlines the procedure of human–AI collaboration classroom teaching.

Fig. 2
Fig. 2
Full size image

Procedure of human–AI collaboration in classroom teaching.

The in-class implementation of human–AI collaboration follows a structured approach through six stages. Initially, the Teacher Introduction and AI Dialogue stage aligns with Robot Importing and Interaction Dialogue, where AI facilitates the introduction of teaching content. Subsequently, in stage 2, where AI assists students in answering questions, activities such as Group Discussion, Group Presentation, and Teaching and Learning enable AI to support students during discussions and presentations. In stage 3, AI provides learning materials during the material viewing activity, allowing students to engage in self-directed learning. In stage 4, AI collaborates with teachers in teaching, corresponding to teaching and learning, and knowledge exploration, where both AI and the teacher work together to guide students in constructing and exploring knowledge. In stage 5, AI plays videos, supporting the video learning activity and facilitating multimedia-based instruction. Finally, when AI generates and answers questions, AI collaborates with teachers in activities like teaching and learning, homework testing, and teaching and learning, addressing students’ test-related queries and providing comprehensive support.

Research procedures

The research design of this study consisted of six phases, as illustrated in Fig. 3. In Phase 1, the study collected data from a 40-min English classroom through an educational experiment and stored it in the database. In Phase 2, the study divided classroom activities based on the entire educational experiment’s time sequence and activity types. In Phase 3, the study designed the coding of the human–AI collaboration evaluation framework and coded data obtained in Phase 1. In Phase 4, three researchers independently scored the data using a scale from 0 to 3. In Phase 5, the researchers calculated the order degree and synergy degree of the human–AI collaboration classroom based on the SDM. In Phase 6, the study explored the research questions in consideration of the results. This study examined the human–AI collaboration classrooms according to the classification of the composite system synergy degree.

Fig. 3
Fig. 3
Full size image

Research Procedure.

Coding scheme

Phase 3 was based on the integrative perspective on evaluation of human–AI collaboration, as shown in Fig. 1. This study identified three subsystems for the composite systems of human–AI collaboration classroom teaching based on SDM and listed the currently available synergy evaluation frameworks and discussed their corresponding meanings, as shown in Table 1 to fully understand the correlation between human and AI to construct a hierarchical structure with progressive layers.

Table 1 Coding of the human–AI collaboration evaluation framework based on SDM.
Full size table

Scoring independently by researchers

In Phase 4, the classroom teaching in the case study was evaluated based on the sequence of teaching and learning activities carried out by both the teacher and students. Each teaching activity was scored on a scale of 0–3, where 2–3 was given when the content of each symbol was fully satisfied, 1–2 was given when it was partially satisfied, and 0 was given when the content was not at all satisfied.

To ensure the reliability of the data analysis, three coders independently coded the data based on the coding framework (Table 1). Prior to the formal coding, the researchers had a discussion session to have a good understanding of the coding framework and data. Questions about the coding framework and data were addressed at the discussion session. Researchers provided a detailed explanation of the scoring criteria and to address any discrepancies in understanding, leading to a consensus. Subsequently, trial coding with five randomly selected data was conducted among the three coders. The coders had satisfactory reliability in coding, with Cronbach’s alpha coefficient for the nine evaluation parameters are: teacher (α = 0.826), students (α = 0.845), AI machine (α = 0.807), teaching goal (α = 0.903), teaching process (α = 0.768), teaching method (α = 0.758), the structure of space and time (α = 0.759), infrastructure (α = 0.758), and supporting technology (α = 0.714).

The case study generated 324 sets of coded data that were saved in Excel for subsequent statistical analysis (Formula 1–6).

Calculation of order degree and synergy degree

In Phase 5, this study calculates the order degree of the subsystem and the synergy degree of the composite system using the following algorithm:

Order degree model of subsystem

According to the SDM of the composite system, the composition of the composite system (S) is S = {S1, S2, …, Sk, …,Sn}, k ∈ [1, n], where k is the kth subsystem of the composite system S and n is 3 in this study; that is, there are three subsystems in the human–AI collaboration classroom teaching system: the collaboration subject subsystem S1, the collaboration process subsystem S2, and the collaboration environment subsystem S3. Skj = {Sk1, Sk2, …, Skj, …, Skn}, Skj denotes the j element of the subsystem Sk, j ∈ [1, n]. Let the order parameter change be eij = (e1j, e2j, …, eij, …, enj), where eij is the first value of the j element, i ∈ [1, n], \({\beta }_{{ij}}\le {e}_{{ij}}\le {\alpha }_{{ij}}\), i ∈ [1,n]. In the synergy theory, the order parameter has two different functions in the order form of the system. One is a positive effect: the order degree increases with a change in the order parameter and its components. If j ∈ [1, m], the eij of the order parameter component is called a positive index. On the other hand, when j ∈ [m + 1, n], the order parameter component eij is called a negative index.

The order degree of eij is

$${u}_{k}\left({e}_{{ij}}\right)=\left\{\begin{array}{l}\frac{{e}_{{ij}}-{\beta }_{{ij}}}{{\alpha }_{{ij}}-{\beta }_{{ij}}},i\in \left[1,m\right],{e}_{{ij}}\,{{\rm {is}}\,{\rm{a}}\,{\rm {positive}}\; {\rm {index}}}\\ \frac{{\alpha }_{{ij}}-{e}_{{ij}}}{{\alpha }_{{ij}}-{\beta }_{{ij}}},i\in \left[m+1,n\right]{,e}_{{ij}}\,{\rm {{is}}\,a\; {\rm {negative}}\; {\rm {index}}}\end{array}\right.$$
(1)

In formula (1), the value of βij is determined by the minimum value of the jth element of subsystem Sk floating down by 5%, and the value of aij is determined by the maximum value of the jth element of subsystem Sk floating up by 5%.

The order degree of subsystem Sk is

$${u}_{k}\left({e}_{i}\right)=\mathop{\sum }\limits_{j=1}^{n}{w}_{j}\cdot {u}_{k}\left({e}_{{ij}}\right)$$
(2)

Among them, \({w}_{j}\ge 0,\,{\rm {{and}}}\,{\sum }_{j=1}^{n}{w}_{j}=1\).

The order degree of the subsystem is the degree of collaboration among the elements of each system and uk(ei) is the order degree of subsystem Sk with a threshold of [0,1]. Where Wj is the sum of the order degrees of the order parameter components used to obtain the weight of the order degree of the subsystem. The order degree of a subsystem reflects the degree of change from disordered to ordered. The greater the value uk(ei), the greater the influence and contribution of eij to the order degree of subsystem Sk, and the higher the order degree of subsystem; the smaller the value uk(ei), the less influence and contribution of eij to the order degree of subsystem Sk, and the lower the order degree of subsystem Sk.

In this study, the order of each parameter was obtained using the linear weighting method. The subsystem order parameter weights are:

$${w}_{j}=\frac{{p}_{j}}{{\sum }_{j=1}^{n}{p}_{j}}$$
(3)
$${p}_{j}={\sigma }_{j}\cdot \mathop{\sum }\limits_{i=1}^{n}\left(1-{r}_{i}\right)$$
(4)

In formula (4), pj denotes the degree of influence of the jth element on the subsystem, σj is the standard deviation of the jth element, and rij is the correlation coefficient.

Composite system synergy degree model

The synergy degree of the composite system indicates the order degree among the subsystems in the composite system. Synergy degree of the composite systems:

$${C}_{i}=\lambda \cdot \root{n}\of{\left|{\prod }_{k=1}^{n}\left[{u}_{k}\left({e}_{i}\right)-{u}_{k}\left({e}_{i-1}\right)\right]\right|}$$
(5)

Among them, i ≥ 2. Ci represents the synergy degree of a composite system, with values varying from −1 to 1.

$$\lambda =\left\{\begin{array}{l}1,\mathop{\prod}\limits_{k=1}^{n}\left[{u}_{k}\left({e}_{i}\right)-{u}_{k}\left({e}_{i-1}\right)\right] > 0\\ -1,{\prod }\mathop{\sum}\limits_{k=1}^{n}\left[{u}_{k}\left({e}_{i}\right)-{u}_{k}\left({e}_{i-1}\right)\right]\le 0\end{array}\right.$$
(6)

λ is used to judge the direction of collaborative development among subsystems. \(\left[{u}_{k}\left({e}_{i}\right)-{u}_{k}\left({e}_{i-1}\right)\right]\) indicates that the order degree of the kth subsystem is expressed by subtracting the order degree of the previous stage to determine the direction of change in the order degree of the kth subsystem. If the number is positive, the order degree increases and the synergy degree between the subsystems increases. If the number is negative, the order degree decreases, and the synergy degree between subsystems decreases. \({\prod }_{k=1}^{n}\left[{u}_{k}\left({e}_{i}\right)-{u}_{k}\left({e}_{i-1}\right)\right]\) is used to judge the changing trend of the subsystem, \({\prod }_{k=1}^{n}\left[{u}_{k}\left({e}_{i}\right)-{u}_{k}\left({e}_{i-1}\right)\right] > 0\) indicating that the synergy degree of the composite system is positive and \({\prod }_{k=1}^{n}\left[{u}_{k}\left({e}_{i}\right)-{u}_{k}\left({e}_{i-1}\right)\right]\le 0\) that of the composite system is negative.

Evaluation criteria

In Phase 6, evaluate the current situation according to the evaluation criteria and classification of the degree of composite system synergy (Qiao et al. 2017), as shown in Table 2.

Table 2 Classification of the composite system synergy degree.
Full size table

Data analysis

The order parameter values were derived from the original data of scores given by three independent coders, as presented in Table 3.

Table 3 The order parameter values of the classroom teaching system.
Full size table

According to Formulas (3) and (4), the order parameter weights of the collaboration subject, collaboration process, and collaboration environment subsystems were calculated, as listed in Table 4.

Table 4 Subsystem order parameter weights.
Full size table

According to formulas (1), (2), (5), and (6), the synergy degree and order degree of the subsystems of the human–AI collaboration classroom teaching system are shown in Table 5, along with the classroom system collaboration index data supported by the human–AI hybrid intelligence learning environments.

Table 5 Synergy degree index data.
Full size table

Results

What is the order degree among subsystems of human–AI collaboration in hybrid intelligence learning environments?

Figure 4 shows the trend of the order degree among the three subsystems. It is evident that the sequence of teaching and learning activities for both the teacher and students, extracted based on the chronological sequence, remains largely consistent throughout the development process, with some fluctuations. The order degree of the collaboration subject subsystem, collaboration process subsystem, and collaboration environment subsystem reached 0.9104, 0.9018, and 0.7306, respectively. The three subsystems are at a moderate level, exhibiting orderly development, though with some relative imbalance, according to Table 2.

Fig. 4: The order degree of the human-AI collaboration in the classroom subsystem.
Fig. 4: The order degree of the human-AI collaboration in the classroom subsystem.
Full size image

This figure illustrates the order degree of three subsystems across various educational activities in a class. The x-axis lists the educational activities, and the y-axis represents the order degree of collaboration, ranging from 0.0 to 1.0. Height of the data points: The height of each point indicates the level of collaboration for the corresponding educational activity. Shape of the data points: Diamond shape of the data point represents the collaboration subject system. Square shape of the data point represents the collaboration process system. Triangle shape of the data point represents the collaboration environment system. Educational activities include, in chronological order of the class schedule, robot importing, interaction dialogue, group discussion, group presentation, teaching and learning, material viewing, teaching and learning, knowledge exploration, video learning, teaching and learning, homework testing, and teaching and learning.

What is the synergy degree within the classroom teaching system of human–AI collaboration in hybrid intelligence learning environments?

The trend of the synergy degree of the compound system of human–AI collaboration in classroom teaching is shown in Fig. 5. The synergy degree of the system fluctuated within the range [−0.5, +0.5], with the maximum value reaching 0.4581. The overall level of collaboration fluctuates between low and moderate, reflecting relatively orderly development among the three subsystems, though one subsystem may exhibit disordered behaviour in contrast to the other two. There is a collaboration between humans and AI.

Fig. 5: The synergy degree of human–AI collaboration classroom composite system.
Fig. 5: The synergy degree of human–AI collaboration classroom composite system.
Full size image

This figure depicts the impact of different educational activities on synergy degree measured by a numerical scale from −1.0 to 1.0. The x-axis represents the educational activities(the educational activities are the same as in Fig. 4), while the y-axis indicates the synergy level.

The composite system’s synergy degree shows an upward trend, as it developed from ‘Group Discussion’ → ‘Group Presentation’, ‘Material Viewing’ → ‘Teaching and Learning’ → ‘Knowledge Exploration’, and ‘Teaching and Learning’ → ‘Homework Testing’ → ‘Teaching and Learning’. In particular, the values of the order degree of the subsystems were lower in the activities of ‘Group Discussion’, ‘Teaching and Learning’, and ‘Homework Testing’. This indicates an increasing trend, with the order degree values of the subsystems being relatively close throughout the development process. This suggests that collaboration among the three subsystems improved, and the collaboration of the composite system increased. Further, the collaboration of the composite system also increased during the activities of ‘Materials Viewing’ and ‘Teaching and Learning’. However, the order degree of each subsystem exhibits a downward trend and the order degree changes in the same manner. Despite this, the collaboration between subsystems improved, and the synergy degree of the composite system increased.

The level of collaboration in the composite system shows a downward trend from activities such as ‘Interaction Dialogue’ → ‘Group Discussion’, and from ‘Group Presentation’ → ‘Teaching and Learning’, and ‘Knowledge Exploration’ → ‘Video Learning’ → ‘Teaching and Learning’ activities. In the case of the ‘Interaction Dialogue’ to ‘Group Discussion’ activities, while the order degree of the collaboration subject, collaboration process, and collaboration environment subsystems decreases, the decreasing trend of the order degree of the subsystems in the collaboration process indicates an unbalanced synergy degree between subsystems. Similarly, from ‘Group Presentation’ to’ Teaching and Learning’, the order degree of the three subsystems varies in different directions in the first stage and is unbalanced in the second stage. The order degree of the collaboration subject and process subsystems first increases and then decreases, whereas that of the collaboration environment subsystem shows a downward trend. This suggests that the synergy degree of the subsystems was not balanced, resulting in a decrease in the synergy degree of the composite system. Overall, the collaborative development of the three subsystems differed, which led to a decrease in the synergy degree of the composite system.

The synergy of the composite system from ‘Teaching and Learning’ → ‘Material Viewing’ exhibits a declining trend that remains relatively stable. The order degree of the collaboration subject, collaboration process, and collaboration environment subsystems all decreased, but the rate of decline in the order degree of the collaboration environment subsystem was relatively slow. Overall, the synergy degree of the composite system remained almost unchanged, indicating a relatively balanced synergy degree of among the subsystems.

Discussion

This study utilizes the SDM to study the synergy degree and order degree of human–AI collaboration in hybrid intelligence learning environments. Through the analysis of classroom data, the research reveals that collaboration among teachers, students, and AI machines in collaborative teaching varies with different teaching contents and activity types. These findings contribute to the research in human–AI collaboration in an educational context.

First, this study used a robust quantitative method from synergetics theory to objectively evaluate the synergy degree between human and AI behaviours in hybrid intelligence learning environments. The study proposed a human–AI collaboration evaluation framework based on SDM, including three subsystems: collaboration subject, collaboration process, and collaboration environment. This evaluation framework is aligned with the findings of Mora et al. (2020), which showed that the collaboration assessment model was mainly elaborated from the three elements of collaborative intelligence, collaborative process, and collaborative platform and that the research is impactful due to its generation of collective intelligence and achievement of improved learning outcomes. In hybrid intelligence classroom environments, the study focused on the entire composite system of the human–AI collaboration classroom. The subjects of collaborative intelligence are humans (teacher and students in the classroom and AI. The process encompasses the entire sequence of actions conducted throughout the teaching activity, and the student learning environment is characterized by face-to-face instruction, which is enhanced by an array of facilities and technological tools that collectively create supportive external learning conditions. In this context, collaborative platforms can be integrated as pivotal components of hybrid learning environments. Therefore, the framework of this study differs from that of Mora’s research by developing these three elements into three subsystems based on the SDM. The human–AI collaboration evaluation framework proposed can help evaluate the effectiveness of human–AI collaboration in classrooms with hybrid intelligence environments and provide insights into the improvement of the human–AI collaboration.

The case study found that the overall degree of collaboration fluctuates between low and moderate, indicating unbalanced and orderly development among the three subsystems of collaboration subject, process, and environment. This has been confirmed by related studies (Zheng et al. 2017; Saha et al. 2023), which suggested that the collaboration mechanism has yet to be fully established in the application of human–AI collaboration in classroom teaching (Jarrahi et al. 2022), and the synergy becomes increasingly significant as AI continues to advance (Saha et al. 2023). Additionally, the synergy degree and order degree in human–AI machine collaboration is influenced by factors such as the hybrid intelligence learning environments, teaching content, and the types of activities involved. Hybrid intelligence learning serves as an enabling condition for human–AI collaborative teaching, as indicated by prior research (Pereira et al. 2023). It is crucial to note that variations in teaching content affect the organization of teaching activities. Studies have shown that the structure of these activities influences the mode of human–AI collaboration (Siirtola and Röning, 2019), and this research further demonstrates that these variations impact the synergy and order of human-AI collaborative classroom teaching under different types of teaching activities. Furthermore, the relationship in human–AI is not a simple linear one but rather a complex nonlinear interaction (Beer, 1995; Lundberg et al. 2020; Scibilia et al. 2023). The study further explained that the synergy among the subsystems reflects the overall synergy of the composite system, for example, in activities such as “interactive dialogue” the order degree of collaboration between subjects, processes, and environments is relatively high, but the synergy of the activity is relatively low. The order degree of a subsystem may be independent of time, whereas the synergy degree of the composite system is time-dependent. Further research is needed to address this issue.

With the continuous advancement and application of AI in the classroom, there has been growing interest in exploring the potential benefits of integrating human–AI intelligence to create a hybrid enhanced intelligence that can support and promote effective teaching and learning (Alfoudari et al. 2021; Brkić et al. 2024; Sharma et al. 2024). Taking human–AI teaching activities as the unit, the current level of human–AI machine collaboration in the classroom remains in dynamic change, which is closely related to the varying collaboration requirements of human–AI in different teaching activities. The extensive deployment of AI in educational settings poses significant challenges to teachers’ awareness, understanding and competencies in collaborating with AI (Son and Sampson, 2022). This challenge is particularly evident in the slow development and differing order degrees in processing subsystems of human–AI collaboration. This study suggests that enhancing the degree of collaboration among teachers, students, and AI in human–AI collaborative classroom teaching should not only focus on the synergy between the elements of the subsystems but also consider the temporal relationships within the overall system. Previous research has emphasized the importance of teachers’ and students’ human–AI collaboration literacy and abilities as key factors in improving collaboration (Lin et al. 2024; Kim and Cho, 2023). In conclusion, human–AI collaborative classroom teaching development should be approached from multiple levels to ensure effective coordination and optimal teaching outcomes.

Conclusion and limitations

Conclusion

This study examines the collaboration of human teachers and AI teachers in classrooms. It proposed a human–AI collaboration evaluation framework based on SDM that reveals the order degree and synergy degree of human–AI collaboration in hybrid intelligence learning environments. Through classroom data, this study measured the synergy degree and order degree in current human–AI collaborative classroom teaching, consisting of three interdependent and complementary subsystems: the collaboration subject, process, and environment. Furthermore, the teaching content and activities have a direct impact on both the synergy and order degrees, which exhibit dynamic variation over time. The findings of this study contribute to the development of more effective human–AI collaboration in the classroom and promote the integration of AI into teaching and learning. For example, an educational practitioner may use the human–AI collaboration evaluation framework based on SDM to develop a learning analytics dashboard for feedback and enhance the order degree between subsystems to better harness the potential of human–AI collaboration.

Limitations and future direction

This study has several limitations. Firstly, the hybrid intelligence learning environments in this study are composed of human teachers and AI teachers, supported by educational robots. This intelligent learning environment represents only one way of human-robot, and as such, the presentation of the research data results may be subject to bias. Findings from the current study suggest that the synergy degree of human–AI is generally moderate, potentially influenced by the characteristics of the learning environment. Future research could conduct human–AI collaborative classroom evaluations and measurements across different types of intelligent learning environments.

In addition, this study is based on manual analysis of classroom videos and has not achieved automated real-time analysis, which makes it less convenient for teachers to view results immediately after class, leading to delays. Future research could focus on developing tools or platforms for measuring human–AI collaboration. Thirdly, this study focused solely on evaluating the synergy degree and order degree in human–AI based on external behaviours. In addition to these external manifestations, other factors, such as cognition and emotion, should also be considered. Therefore, future research should analyse the synergy between cognition and emotion in hybrid intelligence learning environments and the impact of collaboration on learning outcomes. Last but not least, since the collaboration mechanism in human–AI of classroom teaching has not yet been fully identified, further research should aim to construct the internal mechanisms of such collaboration.