Introduction

The first six years of life lay the foundation for subsequent development and learning (Coelho et al. 2020; Rosas et al. 2019). Within the international context, and according to the International Standard Classification of Education (ISCED), these early years correspond to ISCED level 0: early childhood education. The purpose of early childhood education is to ensure the comprehensive and harmonious development of each student in all areas (physical, emotional, sexual, social, cognitive, and artistic), preparing them to be competent citizens in an ever-changing world. Early childhood education marks the beginning of the acquisition process of key competencies for lifelong learning; that is, the initiation of acquiring the skills necessary for students to progress successfully throughout their educational journey and address the main global and local challenges of a changing world. Key competencies constitute the ultimate goal of education and training at all educational stages, aiming to prepare students for future life in society once they leave the educational system (European Union, 2019). Among these key competencies is the ability to learn how to learn, which is closely linked to executive functions (EF) (Sala et al. 2020).

EF are cognitive and affective processes necessary for developing lifelong learning skills and for everyday functioning, as they allow us to coordinate our thoughts, actions, and emotions to achieve our goals, being especially necessary in novel, complex, changing situations that generate uncertainty and/or involve conflict (Diamond, 2020; Zelazo & Carlson, 2020). There are many models attempting to determine how many and which processes comprise EF. Although there are discrepancies among them, there is also some consensus in considering the following aspects as components of EF: the ability to initiate behaviors, inhibition, selection of relevant goals for the task, planning, organization of means, monitoring, flexibility, working memory, and regulation of responses of both emotional and behavioral nature (Diamond, 2013; Diamond, 2020).

A multitude of studies have demonstrated that EF are not only relevant for academic success but also for numerous areas of life: physical and mental health, work productivity, social competency, etc. (Coelho et al. 2020; Scerif et al. 2023). It has also been evidenced that the level of EF reached in early childhood is crucial for later development through middle childhood, adolescence, and adult life (Robson et al. 2020). Well-developed EF are a “kit” for life success (Diamond, 2014). Therefore, the acquisition of EF in the early years of life is critical for optimizing development. These issues justify that, while EF are of great interest in all educational stages—because, as mentioned earlier, they are linked to the key competence for lifelong learning of learning to learn—their approach is especially relevant in early childhood education (Diamond, 2016).

Although EF development is protracted, from birth to early adulthood, during the years corresponding to early childhood education, and especially between the ages of 3 and 6, EF show a significant peak of development (Poowanna et al. 2022). These changes in EF occur simultaneously with numerous brain changes (especially in the dorsolateral prefrontal cortex, but also in other interconnected areas) that underlie EF development. Thus, EF development occurs in parallel with the maturation of the dorsolateral prefrontal cortex—the main area underlying EF—and the refinement and specialization of its connections with other brain areas with which it is interconnected (Thompson & Steinbeis, 2020). However, it is well-known that brain development, and therefore EF development, is not innate, nor does it occur automatically. Brain and EF development depend on the dynamic interaction between environmental and genetic influences (Miguel et al. 2023; Veraksa et al. 2023). Therefore, optimal EF development requires adequate environmental input, especially since epigenetics emphasizes the importance of environmental influences on cognitive outcomes, as changes in the environment can lead to changes in heritability estimates of different skills such as EF (Freis et al. 2022). It is well-known that in the early years of life, the brain shows greater plasticity, meaning the brain is particularly sensitive to environmental influences—both positive and negative—making these early years a period of special malleability for brain and EF development (Miguel et al. 2023).

Early childhood education attendance is considered one of the most effective environmental inputs for promoting the development of children’s human capital and specifically, to promote EF development (Del Boca et al. 2023). The early childhood education curriculum framework is based on play, and more precisely, on “learning through play” (Parker et al. 2022). Play is an intrinsic activity for the child, it occurs spontaneously and not only provides pleasure and enjoyment but also significant benefits for development and learning. Young children naturally explore and learn many skills and knowledge through play. Children learn to make sense of the world around them through play. When they play, children are developing their cognitive, physical, and communication skills (Mardell et al. 2019; Gibb et al. 2021). Again, this is supported by neuroscience findings showing that the prefrontal cortex of the brain is refined by play (Parker et al. 2022).

Thereby, play is an essential element to enhance brain and learning development, and consequently, also for the development of EF (Gibb et al. 2021; Traverso et al. 2019; Veraksa et al. 2023). Consequently, it is an indispensable tool for the teaching-learning process of children (Zosh et al. 2018).

Among the different typologies of existing play, it has been demonstrated that role play, games with rules, and building blocks are the ones that most enhance EF development (Lillard, 2015; Veraksa et al. 2023; Yang et al. 2022). Characteristics such as intense engagement and excitement are common aspects to all these types of play that could determine their potential for EF development (Veraksa et al. 2023).

As previously indicated, play is a natural, fun, and meaningful way to learn. Consequently, systematic observation of a child’s behavior during play makes it possible to obtain relevant data to describe, explain, and understand fundamental aspects of the child’s development and learning, such as the development of EF (Pyle et al. 2020; Yogman et al. 2018). In this sense, both early childhood education curriculum and the scientific literature indicate that systematic observation (characterized by the study of spontaneous behavior that occurs naturally in everyday contexts) is the optimum tool for learning and development assessment of early children’s students (Peterson & Elan, 2020). Observing a child’s actions through play allows an educator or other professional to accumulate a record of the child’s development; to get information about what he knows and what he can do. It allows understanding the child’s strengths, challenges, and needs. With this information, educators can begin to plan appropriate curriculum and effective individualized instruction for each child. Assessment using systematic observation is a critical part of a high-quality early childhood program. Systematic observation brings together the inherent objectivity and rigor necessary in educational evaluation, with the flexibility needed to capture the many and often complex changes that occur in a child’s behavior in real-life settings (Belasko et al. 2023; Belza et al. 2020).

However, data from different research studies suggest that assessment of learning and development in early childhood education using systematic observation is still a challenge. The number of early childhood education teachers with specialized knowledge and methodological training on systematic observation is limited (Allen & Kelly, 2015). This leads them to frequently conduct assessments based on their subjective perceptions, which may overlook difficulties, limitations, or special educational needs of the students. As a result of this inadequate assessment, there could be children whose development is compromised by not receiving the stimulation, attention, and instruction they truly require. Early childhood teachers require more information about systematic observation (Cueto et al. 2017). More assessment instruments that provide ecological, accurate, and reliable information are also demanded by the scientific literature to adequately assess early children (McCoy, 2019; Silva et al. 2022; Souissi et al. 2022).

The scarcity of ecological measurement instruments that allow capturing valid and reliable information about childhood development, in general, and EF in particular, can be attributed to the rigorous work involved in constructing such instruments. Systematic observation lacks standard evaluation instruments, as this would limit the diversity of human behaviour and the contexts in which it occurs. Therefore, the construction of ad hoc instruments is required to not only capture the variability of human behaviour but also the diversity of contexts in which it occurs and may influence it. However, these ad hoc instruments must enable valid and reliable evaluation of the study object. This requires conducting pre-data analysis controls to ensure the quality of the recorded data, in order to account for possible errors or biases in these data, which could lead to erroneous conclusions about child development and learning, with the serious implications that entails. These necessary pre-data analysis controls include intraobserver reliability (ensuring that data obtained by an observer can generally be applied to an infinite number of intraobserver moments); inter-observer reliability (ensuring that data obtained by two or more observers can generally be applied to an infinite number of observers), and the validity of the constructed ad hoc instrument (ensuring that data obtained with the categories that compose the instrument cannot be generalized to an infinite number of other different categories). All of these processes to control the quality of observed data can be analysed using Generalizability Theory (GT), a multivariate structure developed by Cronbach et al. (1972) and adapted to observational designs by Mitchell, (1979). The aim of GT is to distinguish real variability from error variability in any measurement. GT provides information on sources of error that affect behavioural measurement. Therefore, GT is a theory of the multifaceted errors of behavioural measurement (Blanco-Villaseñor, 2001).

In relation to all of the above, the aim of this study was to construct and validate, using GT, an observational instrument aimed at assessing the EF employed by 4-5-year-old children while they engage in a playful activity.

Methods

Setting

The research took place in a public early childhood education center located in a city in the northeast of Spain (specifically, in the autonomous community of Aragon). The study was conducted in a classroom belonging to the final year of early childhood education.

The daily educational programming included a period of free play, during which children could engage in activities according to their interests using the materials available at each of the small learning stations called learning corners located inside the classroom. Each week, every child was required to play in all the corners.

Learning corners represent one of the current alternatives for organizing early childhood education classrooms, as stipulated by educational legislation in Spain for this stage. In each learning corner, numerous proposals are offered for engaging in activities of various types belonging to different educational domains. These corners are delimited spaces in the classroom where children work individually or in small groups simultaneously on different learning activities. Depending on the type of activity, the child’s action may require direction from the teacher or assistance from them, while in other activities, children may act autonomously. Working through corners allows for addressing the differences, interests, and learning rhythms of each child. Ultimately, this type of classroom organization promotes a diversity of learning options for students, integrating all educational domains in an integrated, playful, and enriching manner. The corners serve various educational purposes related to the constructivist conception of learning (Conde Vélez et al. 2019).

In this study, the focus was on the children’s activities while playing a game proposed in the logical-mathematical learning corner of the classroom. This game, called Camelot Jr., is explained later.

Design

An observational design was employed. Systematic observation is considered the optimal methodology for assessing childhood learning and development, particularly when studying children in a natural setting such as school. Specifically, following Anguera et al. (2008), the observational design was nomothetic, point, and multidimensional. It was nomothetic because multiple observation units were studied (precisely, 17 children were individually observed); it was point because a single session per participant was observed to assess their EF; and it was multidimensional because various domains of EF (working memory, inhibition, planning, and mental flexibility) were analyzed, in accordance with various theoretical models (Diamond, 2013; Miyake et al. 2000; Zelazo & Carlson, 2020), as well as other child and adult actions of interest for EF development, such as the assistance provided by the adult to children (Duncan & Tarulli, 2003). All these relevant aspects constituted the ad hoc observation instrument, which is elaborated in detail in a subsequent section. The observation conducted adhered to scientific criteria. It was active (as the aim was predetermined), non-participatory (the observers did not interact with the children), and direct (allowing complete perceptibility of behaviors). It was carried out through direct observation of recorded footage (Anguera et al. 2018). The guidelines outlined in the Guidelines for Reporting Evaluations based on Observational Methodology (GREOM) (Portell et al. 2015) and the Methodological Quality Checklist for Studies based on Observational Methodology (MQCOM) (Chacón-Moscoso et al. 2019) were followed.

Participants

The sample was composed by 17 children: 10 were female (58.8%) and 7 were male (41.2%). Their age range ranged from 4 to 5 years old (M = 4.74; SD = 0.26). Specifically, some students were 4 years old because the assessment was conducted at the beginning of the academic year, prior to the natural year change. The socioeconomic level of the participants ranged from low to medium-high. The inclusion criteria for the sample were: 1) belonging to the last academic year of early childhood education; 2) having informed consent from parents/legal guardians authorizing participation in the study; 3) having an adequate level of spoken Spanish; 4) not being students with special educational needs, i.e., not presenting disabilities, disorders in language and communication development, attention or learning difficulties, exceptional abilities, lack of language proficiency, or personal circumstances that hinder regular attendance at the educational center (Official Gazette of Aragon, 2017). The participant data were treated following the Declaration of Helsinki and the Organic Law 3/2018, of December 5, on the Protection of Personal Data and guarantee of digital rights. Additionally, this research was approved by the corresponding Ethics Committee (favorable opinion RAT 2022-484, from the Ethics Research Committee of the Autonomous Community of Aragon, Spain).

Instruments

*Game that promotes the activation of EF

To enhance the activation of EF in children, the Camelot Jr game was used, manufactured and marketed by the company Smart Games. It is a single-player game consisting of 48 challenges of progressive difficulty, a quality that makes the game suitable for players from 4 years old to 99 years old, according to the manufacturer’s own brand information. The objective of the game is, in each challenge, to build a path with wooden blocks that allows the two characters of the game (a knight and a princess), located each in different places of a castle, to meet. For each challenge, both the initial position (i.e., the type of castle where the characters appear and the location of each character within it) and the blocks to use for the construction of the path vary. Both aspects are determined in the game’s own instructions. Figure 1 illustrates, as an example, a challenge from the game (the initial situation of this challenge and the blocks with which to build the path are specified for better understanding).

Fig. 1
figure 1

Example of a challenge from the Camelot Jr. game 1) Starting situation. 2) Blocks with which to build the path for the characters to meet.

Full size image

In this study, the researcher provided the child with the initial position of each challenge and the blocks to use to construct the path. These blocks were offered to all children in the same orientation (a feature also determined in the game instructions). Thus, in each challenge, the child had to place these blocks in a specific location and orientation to build a path that could be traversed by the characters in order to meet. Additionally, this path construction had to be done following rules established—both in writing and graphically—by the game itself. Figure 2 outlines the rules to follow in constructing the path. Taking into account all these aspects, the Camelot Jr game can be considered both a game of rules and a wooden blocks game. As indicated in Introduction section, both types of games are related to EF development (Lillard, 2015; Veraksa et al. 2023; Yang et al. 2022). Specifically, to solve each challenge of the Camelot Jr game, the child had to plan where to place the pieces and in what orientation, remember the rules, and use them to solve the challenge (working memory); inhibit hasty responses during execution; monitor their execution and detect their own errors, as well as generate new alternatives if their initial solution was incorrect (cognitive flexibility). In conclusion, the game required the activation of different EF.

Fig. 2
figure 2

Rules to comply with in the build of the path for the characters to meet.

Full size image

* Recording instrument for activity

To record the activity of each child while playing the Camelot Jr game (and thus subsequently be able to carry out a comprehensive observation and evaluation of their EF), the recording camera of a Huawei P30 Lite mobile phone was used.

*Observation instrument

To evaluate the EF that the child put into action in each Camelot JR game challenge, an observation instrument was created ad hoc, as required by the observational methodology itself (Anguera, 2001). Since the construction and validation of the observation instrument was the objective of this work, its construction process and resulting characteristics are detailed in the Procedure and Results sections.

* Recording instrument for observational data

To carry out the coded recording of observational data (i.e., to code the child’s actions indicative of his/her EF), the free software LINCE PLUS was used (Soto et al. 2019).

*Analysis instrument

To calculate the validity and reliability of the constructed observation instrument using the GT, the freely accessible software SAGT v1.0 (Hernández-Mendo et al. 2016) was used.

Procedure

*Ethical Permissions

The research team held informative meetings with the management team of the educational center and with the families of potential participants, obtaining informed consent from both. In addition, the favorable opinion of the corresponding ethics committee was obtained (Ethics Research Committee of the Autonomous Community of Aragon, Spain; reference RAT 2022-484).

*Recordings of the play sessions

Before starting the recordings of the play sessions, the researcher immersed themselves in the classroom for one week, accompanying the teacher and children during this time. The aim was to familiarize the children with their presence and therefore, avoid reactivity bias (Anguera, 2001), i.e., to prevent the presence of the researcher from affecting the children’s usual behavior, so that it would lose spontaneity. Only the natural and spontaneous behavior of the children constitutes an adequate indicator of their real level of development and learning (Anguera et al. 2018). After this period of habituation by the children to the presence of the researcher, the play sessions began.

The play sessions took place in the participants’ own classroom, during school hours, and more specifically, at the usual times allocated for free play in the different corners of the classroom. The play session for each child was conducted when they freely went to the logical-mathematical corner, to which they had to go at least once a week, as mentioned previously.

For each child, the play session with Camelot Jr proceeded as follows. In the logical-mathematical corner, the researcher was present, who presented and explained the game to the child. Specifically, they explained the objectives of the game and its rules, using the graphic examples included in the game material itself (see Fig. 2). These rules involved the following aspects: (1) the characters could not jump to reach their destination, they could only climb up and down stairs, and walk on flat ground; (2) ladder-shaped pieces could be placed vertically and horizontally; (3) pieces had to be aligned with the board so that they did not go beyond it; (4) the placement of pieces had to be stable and they could not fall; (5) all the pieces provided by the adult had to be used in solving the challenge. (6) Additionally, the pieces forming part of the initial challenge situation could not be moved.

Once it was ensured that the child understood the rules of the game through example tests and questions about the rules, the game would begin. In each challenge or level, the researcher presented the challenge in the corresponding initial position and provided the child with the pieces needed to solve the challenge (both the initial situation and the pieces to be used following the instructions given by the game for each challenge).

Before the child began their action, that is, prior to the child’s execution of the challenge, the researcher asked the child to think about how they would solve the challenge (i.e., the adult promoted planning in the child). Thus, the child was given unlimited time to think about how they would execute the challenge. At the same time, once the child had thought about it, they were asked to explain what the execution process would be. Once they had thought about it, the child had to verbalize it. Only then could the child begin to place pieces to solve the challenge. During the execution of the challenge, if certain situations arose (such as the child making a mistake when placing a piece in violation of a challenge instruction), the adult could offer graduated assistance to guide the child towards successful resolution of the challenge. When the child indicated that they had completed the challenge, the researcher asked the child some questions aimed at enhancing their self-assessment. (These questions are found in Annex 1). In this way, in each challenge, three main moments or phases could be differentiated—planning, execution, and evaluation—phases closely linked to different EF.

After the child answered these self-assessment questions (regardless of the quality of their answers), the adult presented another challenge to the child, following the same procedure just explained. In this way, challenges were presented to each participant. The end of the game, and therefore, the session for each child, occurred when during the execution of two challenges (consecutive or not), the child had needed assistance from the adult (regardless of the amount of assistance required in the execution of each challenge). There was no time limit for solving each challenge or the total playtime for each child, although the average duration of the play sessions was 16 minutes.

Each and every child was recorded with a camera while playing to subsequently be able to evaluate their actions indicating their levels of EF through observation. In order to avoid children’s reactivity bias, the video camera was positioned in front of the child but away from them, making it possible to record children’s activity without their presence being able to alter it.

*Construction of the observation instrument

The process of constructing the observation instrument followed a continuous development and revision process until achieving its final version, following the contributions of Anguera, (2001) and Anguera et al. (2020). These contributions have been taken into account in the use of observational methodology in recent research (Fuentes-Moreno et al. 2020).

First, preliminary sessions were conducted through narrative records which, subsequently, through various semi-systematization processes, the last of which was a list of distinctive features, allowed for the discernment of the different actions carried out by the studied children. Some of these distinctive features were, for example: repetition of action without paying attention, adult demonstrates help action, anticipation of mistake, placing pieces in different locations, positions, and orientations…

Subsequently, a first provisional system of categories was developed. This system was adapted mainly through the confluence between our empirical records and the theoretical framework offered by the scientific literature on EF (Diamond, 2020; Zelazo & Carlson, 2020). Planning is understood as the ability to provide cognitive control and organize behaviors to achieve the desired goal (Deng et al. 2022). Inhibition involves the capacity to deliberately control or override our automatic or dominant behaviors and thoughts (Escolano-Pérez & Bravo, 2017). Regarding working memory, it focuses on the capacity to store and manipulate information temporarily (Byom et al. 2021). Finally, cognitive flexibility entails the ability to mentally shift between different tasks or novel sets (Escolano-Pérez & Bestué, 2021).

Also, it has been taken into account observational empirical works focused on early cognition and, specifically, on early infant EF (Blanco-Villaseñor & Escolano-Pérez, 2017; Escolano-Pérez, 2020; Escolano-Pérez & Acero-Ferrero, 2022; Escolano-Pérez et al. 2019; Escolano-Pérez & Sastre, 2010). In this way, an amalgamation was made between: a) theoretically defined and studied concepts in the specific scientific literature about early EF development; b) categories that were part of observation instruments constructed and used in other empirical works on the construction of child activity and executive functioning, and c) our empirical records that allowed us to know other types of actions not previously defined by other authors.

Through several pilot sessions, the first version of the observation instrument was refined, leading to multiple intermediate versions (up to 12 versions, in which different criteria and categories were adjusted, ensuring mutual exclusivity between the categories nested in the same criteria and refining their definitions, examples, and counterexamples) until the observation instrument considered suitable (version 13) was constituted for its use, that is, for carrying out the encoded recording of the play sessions recorded on video.

* Encoded recording

In order to perform the recording of observational data using the LINCE PLUS software program, the video recordings and corresponding observation instrument were imported into the program. Once both elements were imported, LINCE PLUS allowed the video recordings to be viewed, with the observation instrument simultaneously displayed on the screen. Whenever a child’s action corresponding to one or several categories of interest (and therefore, corresponding to one or several categories of the observation instrument) was detected, the video playback was paused and these categories were recorded. To do this, the corresponding categories of the observation instrument were clicked with the mouse. This process allowed for the encoded recording of the child’s actions.

In this study, this coding process was initially carried out by one observer, who recorded all the play sessions. After 2 weeks, this same observer re-coded all the play sessions again. Additionally, and independently, another observer coded all the play sessions. Both observers were experts in infant development, EF, and systematic observation.

* Data analysis: validation of the observation instrument

To calculate the validity and reliability (intra- and inter-observer) of the ad hoc constructed observation instrument, the principles of GT were followed. Previous studies have addressed the validation of observation instruments (Romero-Jara et al. 2023) in different fields, but validations of instruments based on GT are less frequent. More specifically, for the calculation of the instrument’s validity, a crossed design of two facets—Observer (O) and Categories (C)—was used, where Observer was the differentiation facet and Categories were the instrumentalization facet (O/C). Within the use of GT, and more specifically, for calculating intra-observer reliability, a crossed design of two facets (Categories and Time) was used, considering Categories as a differentiation facet and Time as an instrumentation or generalization facet (Categories/Time = C/T). For calculating inter-observer reliability, another crossed design of two facets was used, with Categories as the differentiation facet and Observers as the instrumentation facet (Categories/Observers = C/O).

As mentioned earlier, analyses corresponding to GT were conducted using the SAGT v1.0 software program.

Results

The ad hoc observational instrument designed to evaluate early EF through play was a mixed observational tool, comprising a field format integrated with category systems. Consistent with the methodological requirements of this type of observation instrument (Anguera, 2001; Anguera et al. 2018), categories nested within each criterion of the field format were exhaustive and mutually exclusive. It consisted of 17 criteria and 68 categories. The 17 criteria primarily represented different executive components in children (working memory, inhibition, planning, and mental flexibility), as well as the assistance provided by the adult to guide the child in their playful activity. Each criterion was specified by a minimum of two categories and a maximum of seven categories, resulting in a total of 64 categories, as previously mentioned. A summary of this observational instrument built ad hoc—and which was called Instrument for Evaluation of Executive Functions in Early Childhood Education (IEEFECH)—can be consulted in Supplementary material.

The 17 criteria were as follows:

  1. 1.

    Challenge Planning: the child takes time to think and explain where they will place the pieces.

  2. 2.

    Adherence to Planning: the initial placement corresponds to what was explained in the planning.

  3. 3.

    Correct Piece Placement: the child places the piece in the appropriate location and position to solve the challenge.

  4. 4.

    Mistake: either failure to comply with game instructions by the child or placing the piece in a location that does not solve the challenge.

  5. 5.

    Mistake Prediction and Correction: when facing a potential error, the child anticipates and avoids committing it.

  6. 6.

    Flexibility: generating alternative courses of action when faced with a situation requiring new actions where previous ones are ineffective.

  7. 7.

    Challenge Resolution: completing the challenge, regardless of the outcome, or restarting if necessary.

  8. 8.

    Resolution Evaluation: the child assesses the outcome upon finishing the challenge.

  9. 9.

    Consistency in Resolution Evaluation: focusing on the accurate evaluation of the challenge resolution by the child.

  10. 10.

    Resolution Justification: the child explains the reasons behind their evaluation of the resolution.

  11. 11.

    Complexity Evaluation: the child’s verbal and/or non-verbal opinion on the challenge difficulty.

  12. 12.

    Challenge: the game’s difficulty presented to the child that they must solve.

  13. 13.

    Piece: refers to the game piece involved in the action.

  14. 14.

    Response to Adult Assistance: the extent to which the child considers the assistance provided by the adult immediately after it is given.

  15. 15.

    Gradual Assistance: the type of support the child receives, either by adult decision or child request.

  16. 16.

    Instruction: the rules the child must follow during each challenge.

  17. 17.

    External Behaviors: actions of the child incompatible with challenge or game progress, temporarily or permanently leading to their suspension or interruption for reasons other than those considered in the game instructions.

These criteria could be coded in at least one of the three main phases established for each challenge: A) Planning Phase: before the child starts the action with the game pieces, i.e., when the child must think and verbally express how they will solve the challenge, with possible codes being Challenge Planning and Adherence to Planning); B) Execution Phase: while the child is solving the challenge (manipulating the pieces), with possible codes being Correct Piece Placement; Mistake; Mistake Prediction and Correction; Flexibility; Challenge Resolution; C) Evaluation Phase: once the challenge is solved, when the child must answer the adult’s questions about the quality of their execution, with possible codes being Resolution Evaluation; Consistency in Resolution Evaluation; Resolution Justification; Complexity Evaluation. Additionally, several criteria could be coded in more than one of these three phases. These criteria, susceptible to coding in multiple phases, were: Challenge; Piece; Response to Adult Assistance; Gradual Assistance; Instruction; External Behaviors.

Validity of the observation instrument

Within the methodological framework of the GT, an instrument is considered valid when variability corresponding to the category facet is very high and guarantees, therefore, the observational instrument’s capacity for discrimination, which translates to a generalisability coefficient equal or near to 0, as is the case with this research work (coefficient G relative and absolute = 0.00) (Barbero et al. 2023).

The variance associated with the Observers facet has been 100% for all participants and 0% for both the Categories facet and the interaction facet Observer x Categories. The generalizability coefficients for this design structure have been zero (0.00) for all participants, ensuring the validity of the instrument, indicating that the homogeneity of the categories is optimal (Blanco-Villaseñor et al. 2014).

Intra-observer reliability

As shown in Table 1, the determination of sources of variance revealed that a significant portion of the variability (92.73%) was related to the Categories facet, with null or minimal variability in the Time facet (0%) as well as in the Time x Categories interaction (7.27%). Results from the generalizability analysis indicated high reliability in result generalization (0.98).

Table 1 Intra-observer reliability for the C/T design.
Full size table

Inter-observer reliability

As presented in Table 2, the determination of variance sources showed that almost all variability (99.97%) was related to the Categories facet, being null or virtually null in the other facet, Observers (0%), as well as in the interaction of both: Observers x Categories (0.03%). Results from the generalizability analysis indicated high reliability in result generalization (1.00).

Table 2 Inter-observer reliability for the C/O design.
Full size table

Discussion

The aim of this study was to construct and validate (through the use of GT) an observation instrument that would allow for the valid and reliable assessment of early EF exhibited by 4-5-year-old preschool children while solving a playful task. The constructed instrument, in its final version, consists of 17 criteria nested within 68 categories, enabling the evaluation of children’s EF, distinguishing at which moment or phase of the playful task (Planning Phase, Execution Phase, Evaluation Phase) each of them may occur. The main EF assessed by this observation instrument include planning, inhibition, working memory, and cognitive flexibility. Additionally, it highlights the possibility of obtaining information about other actions of interest for the utilization of EF and the resolution of the playful task, such as the graded assistance provided by the adult to the child to solve the task.

The results of the generalizability coefficients were favourable, indicating that the validity and reliability, both intra- and inter-observer, of the instrument calculated through the use of GT, were excellent. Therefore, it is confirmed that the proposed instrument meets the methodological standards required in terms of reliability, precision, and validity. Consequently, the use of this observation instrument allows for obtaining objective, valid, and reliable information about various executive components (primarily planning, inhibition, working memory, and cognitive flexibility) while preschool children (4-5 years old) play. These results, therefore, once again highlight how play and systematic observation are resources of great relevance and usefulness in early childhood education. In this regard, we consider it necessary to advocate for the use of the most appropriate resources at each educational stage, and therefore, to promote the use of play as a means of learning in early childhood. Unfortunately, early childhood education is gradually evolving towards learning goals and methods traditionally associated with the Primary school curriculum, which has negative consequences for the optimal development and learning of students (Jahreie, 2023).

The findings of this research, namely the constructed observation instrument, constitute a significant contribution to the field of early childhood education, as it allows teachers to assess the development and learning of their students with the sufficient quality standards required, thereby overcoming the limitations that, as they themselves admit, often afflict the assessments they conduct on their students (Cuetos et al. 2017). Having valid and reliable instruments is a first step towards being able to detect the needs of each child, and from there, to design educational strategies that adequately address those needs and are thus effective. Identifying children’s needs and supporting them in enhancing their abilities is a characteristic that defines quality early childhood education programs (Darling-Hammond et al. 2020). In this regard, consequently, this observation instrument contributes to improving the quality of early childhood education, and therefore, contributes to the achievement of Sustainable Development Goal 4: Quality Education. Specifically, it aids in addressing goal 4.2, which focuses on ensuring quality early education services and tools for proper development during the preschool stage.

Despite the contributions and implications of this study that have already been discussed, an additional point of significance is the incorporation of GT. This theory constitutes a powerful and useful tool for assessment research, yet it is scarcely employed within the educational assessment domain. However, it must be acknowledged that a notable limitation of this study is its small sample size. In the future, it would be interesting to increase the number of participants. However, it must not be forgotten that systematic observation is an intensive methodology, not an extensive one. The interest lies in the exhaustive description of the natural behavior of a small number of participants, not in how representative it is of a larger universe (Anguera et al. 2018; Belza et al. 2019; Portell et al. 2015). Consequently, it involves working with a small number of participants but collecting a large amount of data with high accuracy. This high-precision data is of great relevance for being able to respond to children’s needs and contribute to the improvement of their development.