Abstract
To meet the comprehensive requirements of drivers for Human-Computer Interaction (HCI) during operation and to enhance situation awareness (SA), this paper proposes a design and evaluation method for flight simulator HCI interfaces grounded in SA. Firstly, the widely adopted Endsley’s SA model, which comprehensively covers processes relevant to individual drivers was selected as the foundational framework. The study concentrates on flight, cruise, and landing scenarios within the simulator, analyzing and restructuring the information architecture of the HCI interface. Secondly, a self-assessment scale was developed based on the three-level model of information processing for SA to measure drivers’ SA cognitive levels. The core scenarios of the simulation training were analyzed to establish a design-making process. Thirdly, invite participants were invited to engage in simulated driving sessions on the flight simulator, where their SA cognitive levels were assessed. An analysis of detail features was performed on samples with high SA cognitive levels. Subsequently, drivers’ requirements for the control layout features of the HCI interface design were gathered through surveys. The design detail features of the HCI interface associated with high SA cognitive levels were identified and synthesized into four distinct dimensions to guide the detailed design process. Finally, participants engaged in simulated driving using the finalized interface design, and their SA cognitive levels were reassessed. The results demonstrated that the HCI interface design derived from this method significantly improved SA cognitive levels compared to the original design.
Similar content being viewed by others
Introduction
Flight simulators, as a product of the earlier application of system simulation technology, promote the development and upgrading of the simulator industry1. It integrates functional systems such as the simulated cockpit HCI interface, visual system, motion system, control loading system, audio system, computer and network system, control console, and software system2. The HCI interface directly reflects the operational performance of the flight simulator and the driver’s behavioral touchpoints, and its design directly affects the driver’s operating experience. In the current era of intelligent interaction, many flight simulator HCI interface designs remain outdated, which has a certain hindering effect on the cognitive development of drivers’ SA3.
Situation Awareness (SA) describes the relationship among the various factors in the “human-object-environment” and human behavior perception, comprehension, and prediction4. In the scenario of flight simulators, the level of SA is directly related to the design of the HCI interface in the scenario and the driver’s cognition of the situation. The research on SA aims to improve the friendly experience of HCI interfaces in flight simulators. Zhou et al5. analyzed the effect of visual display factors on the driver’s identification performance, and provided a reference for the development of cockpit display interfaces with a high level of SA. Zhang and Zhao6mapped the three-level characteristics of SA to the design and development strategy and constructed a car navigation interface for the driver physiological loading conditions of traffic congestion scenarios. Shu et al7. and Jiang8analyzed the error-prone situations and eye-hand coordination characteristics during flight mission execution and replaced local physical components with touch screen devices to achieve the dual reduction of spatial pressure and driver’s cognitive workload. Ye et al9. focused on the attention allocation strategy in SA theory and verified the optimized design model of the HCI interface layout through quantitative methods such as visual field physiological measurement and weight assignment. Chen10analyzed the driver’s cognitive behavior through interaction scenarios and cognitive resources, and conducted interaction design improvements from a multi-sensory level. Zhu11established a multidimensional cognitive model of the driver based on the cognitive characteristics of interaction behavior in the cockpit during civil aircraft landing scenarios. Liu12optimized the interactive behavior of hydraulic operation HCI interface based on the perceptual characteristics of human eye cone cells. Nguyen et al13. provided a comprehensive overview of existing theoretical models for establishing or enhancing SA, identifying their limitations and suggesting future directions for SA evaluation research. Current advancements in SA studies have addressed the earlier shortcoming of models being confined to theoretical frameworks by integrating experimental data, thereby forging stronger connections with real-world applications. However, current SA research predominantly emphasizes the technical methods and physiological metrics associated with SA models, while insufficient attention has been given to the psychological aspects of “awareness”. This lack of focus on user psychology has resulted in a weaker link between the product and the user experience.
In summary, research on HCI interface design under the SA theory starts from the perspective of users’ physiology and psychology, aiming to improve the friendly interaction between users and interfaces by analyzing users’ perception awareness and interaction behavior touchpoints. However, HCI is complex. In the process of HCI, it is not limited to the input and output between the driver and the digital equipment. Behind an interaction is a series of inputs and outputs triggered by a particular event, as well as the different needs and motivations shown by drivers in different stages of cognition, rather than simply measuring their psychological states and judgment abilities. Moreover, this range of influences can be reflected and mapped into situation factors.
Flight simulators achieve the same effect as real aircraft simulation training in simulated scenarios. During training, the HCI interface serves as an important channel for transmitting guidance information and control commands14, and the information it outputs can directly influence the driver to make the right decision and improve the speed of decision-making. The driver’s perceptual processing in the flight scenario includes the acquisition of SA level and the interaction of the HCI interface. SA determines information filtering and decision-making, while input information from the HCI interface undertakes the operational decision-making after achieving SA. The impact of the HCI interface on SA cognitive level indicators is reflected in the driver’s perceptual processing ability eventually15, as shown in Fig. 1.
SA of drivers interacting with HCI interfaces during simulated flight training.
This study integrates SA theory into the optimization of HCI interface design for flight simulators, employing it as a theoretical foundation for refining visual displays and control operations. SA is also utilized as a metric to evaluate the ergonomic optimization of the HCI interface, addressing issues related to low SA cognitive levels in interface design and substantially enhancing the SA cognitive levels of drivers. Furthermore, the SA self-assessment scale introduced in this paper provides an accurate tool for assessing drivers’ SA cognitive levels, thereby enabling effective validation of the design framework. While this research is specifically applied to flight simulators, it also offers valuable insights for the design of other HCI interfaces that demand high SA, such as driver assistance systems and industrial control systems.
Situation awareness theory
In the context of design research, “situation” refers to the combination of subjective consciousness and objective existence, introducing psychological elements into the consideration of user experience in interface design. “Subjective consciousness” represents the emotions triggered by space and the experience of self-awareness in the follow-up feedback, and “objective existence” represents the interwoven factors in the environment and the mutual relationships of them. Both exist in the interactive system of “human-object-environment”16. “Situation” represents the conditions and environments that people need to produce their behaviors. In psychological definition, “situation” represents the objective working environment, which is special feedback when individuals form physiological and psychological sensations in the environment17. A situation can be briefly divided into the external and internal situation. The former is also known as the environmental situation, which is the influence of objective existence on an individual’s psychology. And the latter is also known as the user situation, which refers to the dynamic psychological state and task goals of individuals when interacting with objects.
Situation awareness model
Since the 1980s, research on SA has developed rapidly. It has been widely used in many fields such as aviation, air traffic control, medicine, and advanced production systems to improve the level of information perception of operators.
Numerous models have been developed for the study of SA, with prominent ones including Endsley’s three-level SA model and the Situation Awareness Global Assessment Technique (SAGAT) model18, the Agent Teaming Situation Awareness (ATSA) model19, the Contextual Situation Awareness (CSA) model20, the Team SA model21, the Organizational Situation Awareness Rating (OSCAR) model22, and the Adaptive Control of Thought-Rational (ACT-R) model23. The ATSA, CSA, Team SA, and OSCAR models primarily address the dynamics of SA within teams and organizations, the ACT-R model focuses on the situational cognition related to front-end information processing and distribution, and Endsley’s models concentrate on individual SA, encompassing both cognition and prediction aspects. This paper emphasizes a comprehensive analysis of individual drivers’ SA, thus basing Endsley’s research on the three-level model and the SAGAT model.
In numerous studies, Endsley’s definition of SA has been widely accepted24. She thought that SA was the perception and comprehension of the constituent elements in the environment at a specific time and space. The definition emphasized the identity of the resultant product of SA. In addition, Endsley proposed a model of SA information processing, as shown in Fig. 2.
Endsley’s model of SA information processing.
According to Endsley’s SA information processing model, SA is a cyclical interaction among environmental conditions, situation perception, decision-making, and actions. Environmental conditions influence the driver’s situation perception, allowing to detect and interpret these conditions, make predictions, and proceed with decision-making to take appropriate actions. These actions, in turn, alter the environmental conditions. Additionally, factors within the task or system—such as system capability, interface design, and complexity—provide feedback to influence the driver’s situation perception, decision-making, and actions. Individual factors, such as the driver’s objectives, also affect situation perception and decision-making. Information processing mechanisms, long-term memory storage, and the degree of automation—shaped by the drivers’ abilities, experience, and training—impact both decision-making and actions while also aligning with the driver’s goals and objectives. It suggests that SA is related to dynamic and constantly evolving situations, and emphasizes the changing nature of perceptual information. The components of perceptual information include individual experience and abilities, environmental conditions and factors, and task goals and expectations, all of which are perceived in the form of information inputs that subsequently influence the driver’s judgment and decision-making. To address the dynamic changes in the input of perceptual information and the related influencing factors in flight simulator scenarios, three levels are extracted from Endsley’s information processing model, namely, perception (SA1) - comprehension (SA2) - prediction (SA3), as shown in Fig. 3.
Three-level model and its performance of SA information processing.
SA1 is the “perception” layer, where drivers achieve intuitional perception of the current situation elements through sensory channels such as vision, hearing, and touch. This stage is the driver’s basic perception of the external environment, which is a prerequisite for the generation of SA.
SA2 is the “comprehension” layer, which is based on SA1 and involves cognitive processing of the perceived situation information to form a personal understanding of the perceived information components. This layer emphasizes the driver’s ability results to analyze, interpret, and store perceived information, and filter out useful information for the target.
SA3 is the “prediction” layer, which is based on perception and comprehension, and relies on personal experience and behavioral regulation to predict future states and assist in behavioral decision-making.
Factor analysis of SA cognitive level
The cognitive level of SA refers to a driver’s abilities to perceive, comprehend, and predict changes in the environment and the progression of events during flight operations. Through the analysis and refinement of the SA information processing model, we summarize a three-level model of driver response information perception in flight simulator scenarios. At the same time, combining the division of internal situation, external situation, and the situation constraints under specified conditions, the factors of SA cognitive level are divided into three categories: driver, task and environment, as shown in Fig. 4.
Conceptual model of SA cognitive level factors of flight simulator drivers.
Driver situation
Driver situation refers to driver subjective agency and is the main inherent attribute of the situation, including mainly driver’s psychology, physiology, and needs. The aim of studying the driver situation is to establish the mapping relationship between drivers and situation factors, clarify the evaluation indicators of situation factors, measure the rationality of situation factors by the driver’s perceived comfort level, and enable better information output and behavioral decision-making.
Task situation
Task situation refers to the individual cognitive status and changes in situation factors under constraints. The main factors of task situation include the form of the task, difficulty level, market effects, and so on. Task situation can be divided into necessary and non-necessary tasks based on the contribution degree to achieving the goal. Necessary tasks are those closely connected to the flight process, which often require more time and effort to handle. Non-necessary tasks refer to those that are relatively less closely related to the flight process. Analyzing and optimizing necessary tasks can improve drivers’ processing efficiency, while analyzing and optimizing non-necessary tasks can enhance the usability of the product, making it easier for drivers to focus on processing necessary tasks.
Environmental situation
Environmental situation refers to external environmental factors that are independent of drivers but are interconnected and influence each other. It mainly includes the natural and social environment, which are the sources of SA formation. The natural environment mainly encompasses factors such as temperature, humidity, altitude, terrain, noise, and geographical location25,26,27,28,29. The social environment mainly refers to communication between the driver and society, such as the driver’s social relationships, life status, cultural beliefs, and other factors. Environmental situations are often strongly variable, complex, dynamic, and random due to the complexity of their components.
As illustrated in Fig. 4, factors related to environmental and task situations ultimately impact the driver situation, which is manifested through the driver situation. Accordingly, this paper evaluates SA cognitive levels by developing a self-assessment scale that measures the environmental and task situations as perceived that responded to by the driver.
Analysis of HCI interfaces in flight simulators
The HCI interface of flight simulator is the main product of the interaction between the driver and the simulator, and it is the port for the driver to receive flight information and output information to feedback the operation. SA can explain the driver’s level of perception, comprehension, and prediction of the cockpit interactive environment and pre-flight environment. The cognitive processing process involved the level changes of its theoretical system can also help to analyze the deep essence of driver operating behavior.
Information architecture of HCI interfaces in flight simulators
The concept of the “HCI interface” only arises when both the driver and the interface enter into an interactive information transmission state and work together to serve the same task goal. The HCI interfaces in a flight simulator mainly consist of visual display interfaces and operational control interfaces. When executing a flight task, the visual interface is the channel for the driver to obtain information, and the operational interface is the object of the driver’s decision-making. Therefore, the reasonable design and expression of the HCI interface in terms of display and control are proportional to the ease of use of HCI.
The flight simulator HCI interfaces contain various information, organize the design of the information, information environment, information space, and information system architecture through a structured and organized architecture format, and display the relationship between the information displayed on the HCI interface and the driver’s operational logic based on a clear composition of information. From the objects involved in the interaction process of HCI interfaces, it is inferred that the information components involved in the information architecture are the driver, the HCI interface, and the flight simulator system. The driver completes the flight task through situation perception, situation comprehension, and situation prediction, while the HCI interfaces provide information to the driver through visual and manipulative interaction. Selecting the menu information architecture30 as the information architecture type and arranging the information horizontally according to the above information classification conclusion, the information components and organizational structure of the flight simulator HCI interfaces is obtained, as shown in Fig. 5.
Information components and organizational structure of flight simulator HCI interfaces.
Analysis framework of SA in flight simulator HCI interfaces
This study explores the formation mechanism of SA and its relationship with HCI interfaces through the three-level model of SA information processing: perception, comprehension, and prediction. The perception stage (SA1) is essentially a process of information acquisition and attention allocation of sensory channels. The flight information provided by the HCI interface is the part that needs to be specifically perceived and understood. After reaching the perception state, enter the later stages of SA, that is, comprehension (SA2) and prediction (SA3). The driver’s operational decisions in the prediction (SA3) stage are reflected in the actions and commands displayed on the HCI interface. The three-level analysis framework of SA information processing for flight simulator HCI interfaces is illustrated in Fig. 6.
Three-level analysis framework of SA information processing for flight simulator HCI interfaces.
Evaluation indicators of SA cognitive levels in flight simulator HCI interfaces
Endsley’s SAGAT model offers a quantitative method for evaluating SA cognitive levels. This method involves pausing tasks during their execution and asking participants questions about the current situation to assess their SA. However, the interruption of tasks can disrupt drivers’ SA. In this study, instead of task interruptions, we assess SA cognitive levels through a self-assessment completed by participants after the experiment.
To establish indicators for evaluating SA cognitive levels in flight simulator HCI interfaces, this study developed a self-assessment scale focused on the psychological dimensions of drivers, aiming to measure psychological stress during flight tasks. There are various scales for evaluating psychological stress, including the Beck Depression Inventory31, Beck Anxiety Inventory32, Hamilton Depression Rating Scale33, Hamilton Anxiety Rating Scale34, Minnesota Multiphasic Personality Inventory35, Brief Symptom Inventory36, and the Symptom Checklist−90 (SCL−90)37. The Beck Depression Inventory, Beck Anxiety Inventory, Hamilton Depression Rating Scale, and Hamilton Anxiety Rating Scale focus on single dimensions of depression and anxiety. The Minnesota Multiphasic Personality Inventory includes 550 items, making it time-consuming. The Brief Symptom Inventory, a simplified version of the SCL−90, covers fewer dimensions. The SCL−90 comprises 90 items that evaluate nine dimensions: somatization, obsessive-compulsive symptoms, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, and psychoticism, which is able to comprehensively assess an individual’s psychological health status and can be completed by individual self-assessment.
Based on these considerations, the self-assessment scale for evaluating SA cognitive levels in flight simulator HCI interfaces was designed by adapting the SCL−90 to align with Endsley’s SAGAT model. Initially, the SCL−90 items were generalized and filtered to produce 24 generalized questions. These questions were then categorized according to the three-level model of SA information processing (perception, comprehension, and prediction). Finally, the categorized questions were adapted to the context of SA in flight simulator HCI interfaces, resulting in the final self-assessment scale, as shown in Table 1.
Drawing from the scoring criteria of the SCL−90 Symptom Checklist, the self-assessment scale for SA cognitive levels in flight simulators developed in this study also utilizes a 5-point scale for evaluation. Higher total scores suggest greater psychological stress during simulated flight training and a corresponding decline in SA cognitive levels.
Flight simulator HCI interface design decision-making process
Establishment of design decision-making model
Activity-driven design situation generation refers to any information that can be used to describe entity characteristics. The so-called entity can refer to people, places, or objects related to user-program interaction (including users and programs themselves), that is, information in the design scenario of the flight simulator. Therefore, starting from the perspectives of user research and interaction research38, the design situation is divided based on the three-level model of SA information processing, that is, carry out complete decomposition, which can not only fully record situation information but also facilitate subsequent processing and analysis of situation information. When the driver is performing HCI interaction tasks, the performance of interaction behavior depends on individual SA level and workload. The driver must have sufficient SA ability to conduct an in-depth analysis of information display mechanisms and gain a clear understanding of the current information mechanism.
Given the increasing importance of research on HCI interfaces in flight simulators, to ensure the efficient operation of flight simulators, it is necessary to study the interaction characteristics of the control system, and deeply explore the impact mechanism of the development of flight simulators on the driver’s cognitive characteristics, workload, attention allocation, and SA. Based on SA theory, this article explores the visual design methods of driver interface information and provides design strategy guidance for future flight simulator HCI interfaces. The SA model of the HCI interface design process is shown in Fig. 7. Based on SA theory, this study examines the environmental, task, and driver situation in the HCI interface design of flight simulators from the perspectives of user research and interaction research. The environmental situation encompasses the interrelations among SA perception, decision-making, and interaction processes. In this situation, the HCI scenarios, mental load, and function allocation each affect SA perception. Task situation focuses on interaction requirements and features through research on physical, touch, gesture, and voice interactions. It integrates factors such as workload, information prompt complexity, and the degree of product intelligence to inform interaction control design. The driver situation investigates drivers’ cognitive levels, behavioral abilities, and information processing through usability factors like ease to learning, effectiveness, usability, and ease of use. Both task and driver situations significantly influence SA perception, decision-making, and interaction within the environmental situation.
Design process model of SA cognitive level factors for flight simulator HCI interfaces.
Transformation of decision-making process based on SA
This article extracts and analyzes different scenarios of flight simulators, focusing on the optimization of the visual display and operational control of the HCI interfaces in flight simulators. Based on the cognitive foundation of drivers, we analyze the take-off scenario from three influencing factors: driver situation, task situation, and environmental situation, and the purpose is to find out the design points of HCI interface functions based on SA in typical scenarios more deeply and accurately. By combining with the driver’s behavior prediction, we analyze the flight scenario in stages and dimensions, predict which decision actions the driver will need the help of the HCI interface, and analyze the timing, form, and interaction between visual display and control interface of information.
Situation classification extraction
The perception, comprehension, and prediction behavior based on interactive environments and future development changes generated by users are the essential connotations of SA. When the flying environment changes, the driver obtains information from the visual display interface, makes behavioral decisions based on the current flight mission, and causes changes in the internal and external information of the flight simulator through the operational control interface. Information extraction is carried out from different flight scenarios and three types of situational influencing factors, and is presented in the HCI interface of flight simulator and the relationship between various aspects in the design.
Flight scenarios include takeoff, cruise, and landing, each of which corresponds to a series of situations. Core situations under different flight driving statuses are extracted to analyze the behaviors of users in specific scenarios. Taking the takeoff scenario as an example, the core situation extraction of the operation process is shown in Table 2.
Core situation analysis
To identify opportunities for designing the HCI interface of the flight simulator, SA analysis is conducted on the core situations of the three flight scenarios. An analytical framework for the three-level model of SA information processing in simulated flight scenarios using flight simulators is shown in Fig. 8.
Analysis framework for the three-level model of SA information processing in simulated flight scenarios.
Takeoff scenario (J1): First, perform R1, where the driver conducts a pre-flight routine check of the cockpit and starts the engine, completing the checks (SA1, SA2). The simulated aircraft then lifts off and hovers above the tarmac (SA2), with the driver adjusting the pedals to maintain a level position during takeoff. Next, during R2, as the simulated aircraft transitions from vertical to forward motion, the airframe shakes and the nose lifts (SA1). The driver slightly pushes the cyclic forward (SA3) to ensure continued forward movement. As speed increases, the rotor blades become more efficient, requiring the driver to anticipate and correct for this effect (SA3).
Cruise scenario (J2): After takeoff, the driver slightly releases forward cyclic pressure, allowing the simulated aircraft to climb and gain airspeed (SA1). In the cruise scenario, the driver needs to maintain control of the aircraft’s direction using the cyclic control with the right hand and the tail rotor pedals with the feet. The display screen must be continuously updated with heading, airspeed, altitude, and GPS information (SA1, SA2).
Landing scenario (J3): First, perform R4, where the driver monitors airspeed and observes when the simulated aircraft is approximately 0.2 km from the landing zone (SA1). As the simulated aircraft decelerates to 40 knots (SA3) and begins to hover (SA2), the driver adjusts the pedals to maintain level during the descent. During this process, the driver must check the descent rate (SA1) to ensure it does not exceed 300 feet per minute. As the simulated aircraft approaches the edge of the landing zone (SA1), the speed is gradually reduced to 30 knots, then 20 knots (SA3), requiring the driver to decide whether to raise the nose to reduce airspeed (SA1, SA2). Upon reaching the landing zone, the driver observes the approach of the simulated aircraft’s nose to the touchdown point (SA1, SA2), reduces the collective (SA3), then gently pulls back on the cyclic to reduce momentum, and adjusts the collective as needed to maintain a minimal descent rate. Finally, perform R5, where the driver ensures the parking brake is engaged (SA1, SA2), then reduces all power (SA3).
In summary, the pedals are mainly used to correct the aircraft’s attitude, and most maneuvers only require periodic and collective control. Therefore, the relevant components of periodic and collective control should be distributed in the main operating area. In the visual area, instruments such as airspeed indicator, altimeter, vertical speed indicator, attitude indicator, heading indicator, and GPS display, which are frequently observed by drivers, should be distributed in the main visual area. The lighting system, navigation system, communication system, information system, and auxiliary driving system of the flight simulator, which are frequently used systems, should be further optimized in design.
Using the three-level model of SA information processing and task nodes analysis, real-time speed information is represented by a hollow circle to make the speed information clearer visually and shorten the perception and comprehension time. Using a large-area map display makes it more clearly for the driver to judge changes in position visually and achieve complete expression of GPS information. Different SA levels of information can be distinguished based on different light colors, such as white light indicating that the aircraft is not in a horizontal state during takeoff, which requires the driver to perceive and operate, and blue light indicating that the aircraft has reached a horizontal state.
Application examples
Taking a certain type of flight simulator based on the Bell 407 as a prototype as an example, the flight simulator has not undergone industrial design-enabled systematic product design transformation. The HCI interface product of the flight simulator has a simple appearance, dense, and difficult-to-identify information display, and lacks consideration for task efficiency and driver experience, as shown in Fig. 9.
A prototype flight simulator at the research center based on the Bell 407 model.
Flight simulator HCI interface control layout design detail orientation
Design detail orientation guided by objectives ensures that the design solution maximally meets the needs of drivers. By centering on the driver as the core subject, and applying findings from SA research, focused selection and optimization are carried out for the intensive sections of design detail in the design process. Starting from the control layout of the HCI interface, the key design elements are filtered and positioned accordingly39, as shown in Table 3.
Evaluation of SA cognitive levels in the control layout of the existing flight simulator HCI interfaces
Since Endsley’s SA model has the limitation of being static and fails to fully consider the dynamics and interactivity of SA, this paper uses driver simulation experiments to compensate for this deficiency40. The flight simulator prototype used in this study is the Bell 407. To assess SA during the task, it is essential to control for consistent cognitive levels among participants. Since an individual’s cognitive level may change after completing a task simulation, this experiment recruited 30 participants with the same age, academic background, and no prior flight experience to ensure data reliability while minimizing cognitive differences among participants. The group consisted of 15 males and 15 females.
For this experiment, 10 participants were selected, including 5 males and 5 females. Before the simulation task, participants received training that covered understanding the instrument panel, flight rules, and operational methods. Once participants were familiar with the procedures, the experiment commenced.
A different route from the training scenario, from Hongqiao Airport to Shanghai IFC, was chosen for the flight simulation, as shown in Fig. 10. After being shown the route map, participants began the timed flight simulation, and the experiment concluded when they reached the designated destination. Participants then completed the SA self-assessment scale of the flight simulator HCI interface control layout, with a maximum score of 120. Upon completion of all experiments, the researchers compiled each participant’s SA scores and corresponding operation times, with the results summarized in Table 4.
Simulation flight experiment path for the prototype scheme of the flight simulator’s HCI interface: from Hongqiao Airport to Shanghai IFC.
As indicated in Table 4, there is a positive correlation between the time taken and the SA scores for the initial flight along the route from Hongqiao Airport to Shanghai IFC. Participants who took longer to complete the flight generally had higher SA scores, suggesting that higher SA scores on the existing the control layout of the flight simulator HCI interface are associated with lower SA cognitive levels.
Extraction of design features for the control layout of the flight simulator HCI interfaces
Design detail requirements elicitation
The design elements for the control layout of flight simulator HCI interfaces have been summarized above. Based on this, initial design detail requirements for the control layout of flight simulator HCI interfaces were gathered through user interviews. This serves as the foundation for analyzing design detail requirements, providing a basis for the subsequent screening of design criteria.
To ensure the survey achieves optimal results, this study organized information from both online public sources and offline interviews to draft the initial questionnaire. Given the specialized and complex characteristics of flight simulators, 10 users aged 20 to 50, each with expertise in the field, were invited to refine the questionnaire content based on their professional backgrounds. The interviewees included flight trainees, experts, engineers, and designers, all of whom possessed at least basic proficiency in flight simulator operations. The objective was to enhance the comprehensiveness of the data collected and to understand the target users’ needs for the flight simulator HCI interface. User background information is detailed in Table 5. After revising the survey questionnaire based on expert feedback, the initial design detail requirements for the flight simulator were established, as shown in Table 6.
Based on this, a questionnaire survey method was adopted to conduct an in-depth investigation and analysis of drivers’ requirements for appearance design details, interactive system design architecture, and information display design details during the driving process. The survey was conducted using a combination of physical and online questionnaires, targeting helicopter operators, experts in helicopter research, and design students. The survey lasted 10 days, yielding 1,075 valid responses. The detailed demographics of the 1075 participants are shown in Fig. 11. The data from the questionnaires were analyzed, and the results are presented in Fig. 12.
Detailed demographic information of 1075 participants involved in the in-depth survey on the initial design detail requirements for the control layout of the flight simulator HCI interfaces.
Statistical results from the data analysis of 1075 in-depth survey questionnaires of the initial design detail requirements for the control layout of the flight simulator HCI interfaces.
In the process of extracting design elements, it is essential to integrate the results of both user interviews and quantitative surveys. A user survey was conducted based on the user samples presented in Fig. 11. The survey focused on gathering design detail requirements from three key areas: aesthetic design details, interactive system design architecture, and information display design details. The results of the user survey, as depicted in Fig. 12, were then categorized to complete a quantitative extraction of essential elements. In the appearance design details, four key elements were extracted: aesthetics, technology, simplicity, and futuristic. For the information display design details, four elements were identified: accuracy, conciseness, efficiency, and interoperability. In terms of the interactive system design architecture, the technical architecture of design framework that best meets the drivers’ requirements was selected. The results are illustrated in Fig. 13.
Design detail requirements for the control layout of the flight simulator HCI interface determined through a combination of qualitative user interviews and quantitative survey analysis. (Created by RAWGraphs 2.0 https://app.rawgraphs.io).
Analysis of design features in the control layout of mainstream helicopter HCI interfaces
In order to obtain design details and feature information of the control layout of the flight simulator HCI interfaces that are conducive to enhance drivers’ SA cognitive levels. Conceptual models and those not yet in production were excluded, resulting in samples of 70 helicopter types, as shown in Table 7.
10 participants were randomly selected from the remaining 20 undergraduate subjects, with an equal number of male and female participants. The information of the 70 helicopter types listed in Table 7 was loaded into the flight simulator shown in Fig. 9. Each participant was randomly assigned to fly simulation tasks using 7 different helicopter types. After completing each simulation, participants filled out the SA self-assessment scale for the control layout of the flight simulator HCI interfaces. This process resulted in SA scores for all 70 helicopter models. The experiment revealed that the HCI interfaces control layout of Z−15, AS−332, AW−139, Mi−171, and AW−159 received the lowest SA scores, ranging from 72 to 85, indicating that the HCI interfaces control layout of these types had the highest cognitive levels of SA, as shown in Fig. 14.
5 helicopter types with the lowest self-assessment scores on the SA cognitive level scale for the control layout of the HCI interfaces among 70 mainstream helicopter types.
An analysis of the HCI interface control layout features in Fig. 14 revealed that the 5 helicopter types with the highest SA scores shared certain design features. Notably, these types had larger control panel areas, which effectively drew the drivers’ attention to the instrument panels. Additionally, the middle section between the main and secondary control panels had a more balanced width, creating a visually harmonious layout. The instrument panels in these models were also simpler compared to other designs, facilitating easier information retrieval for the participants. These design detail features provide concrete references for improving the control layout design details and schemes of flight simulator HCI interfaces.
Output of design features for the HCI interface control layout
By integrating the results from user interviews and questionnaire surveys on the design detail requirements for flight simulator HCI control layouts, along with the SA cognitive level scoring experiments of 70 mainstream flight simulator HCI control layouts and the design detail features of 5 flight simulators with high SA cognitive levels, an SA-optimized design scheme for the flight simulator HCI control layout was produced. A schematic diagram of the feature information for the optimized design is output, as illustrated in Fig. 15.
Schematic diagram of the feature information lines for the SA optimized design scheme of the flight simulator HCI interface control layout.
Presentation and validation of design details for the HCI interface control layout
Control layout design details
Based on the extracted design detail requirements and feature results, the HCI interface control layout design scheme is generated. The overall interactive information display interface of the flight simulator adopts a semi-enclosed structure with a stronger aviation-driving atmosphere to enhance the immersive experience. The design uses simple and tough lines to make the appearance more tough and stable, giving people a sense of rigor and professionalism. Different color modules are used to divide the functions layout of each part, clarifying the functional attributes. The console layout of the flight simulator is designed to achieve the characteristics of aesthetics, technology, simplicity, and futuristic. What is more, the map display interface is placed in front to facilitate a more accurate viewing of landmark information during flight, as shown in Fig. 16.
Presentation of the design details for the SA optimized scheme of the flight simulator HCI interface control layout. (Created by Rhinocreos 7 https://www.rhino3d.com and Keyshot 10 https://www.keyshot.com).
Insights and principles for HCI interface control layout design details
Through further analysis of 5 HCI interface samples design details with the highest SA cognitive levels, the insights were extracted from the SA model. All 5 samples adhered to the design principles that the control area should be positioned in front of the driver’s seat, with the visual area located behind the control area. The driver’s physical space was divided into 10 angular units, with the visual area spanning 4 of these units. The main instrument and control areas should be directly in front of the driver, while the secondary control areas are evenly distributed on both sides.The spatial positioning of the HCI control layout in the flight simulator was segmented into functional areas, as shown in Fig. 17.
Spatial dimensional layout diagram for the SA optimized design scheme of the flight simulator HCI interface control layout.
The spatial layout is divided into the control area, visual area, passageway door area, leisure auxiliary space area, and auxiliary control working area. The control layout was analyzed across four dimensions: HCI interface basic control layout, HCI interface visual control layout, the operation functional area control layout, and the simulation functional area control layout.
The HCI interface basic control layout is shown in Fig. 18. It is mainly divided into the main visual surround electronic screen, the main control panel instrument area, the secondary control panel keyboard area, and the dynamic joystick control area.
Analysis of the basic control layout for the SA optimized design scheme of the flight simulator HCI interface.
By analyzing the horizontal and vertical visual ranges of the HCI interface and incorporating insights from the model, the visual control layout of the flight simulator HCI interface is shown in Fig. 19. The white part in the figure is the range of the main visual area, the green part is the range of the visual interface area, and the red part is the range of the visual control area.
Analysis of the visual control layout for the SA optimized design scheme of the flight simulator HCI interface. (Created by Adobe Photoshop 2022 https://www.adobe.com/products/photoshop.html).
Based on the above analysis, the operation functional area control layout of the flight simulator HCI interface is further refined, as shown in Fig. 20. It is divided into the main instrument area, the main control area, the auxiliary control area, and the control lever area. In addition, it also has a structural control area such as a quick control area, aircraft doors, and a comfort control area such as flight control and air conditioning multimedia.
Analysis of the operation functional area control layout for the SA optimized design scheme of the flight simulator HCI interface. (Created by Adobe Photoshop 2022 https://www.adobe.com/products/photoshop.html).
The simulation functional area control layout mainly includes the manipulative joystick operation area, button operation area, scale display area, main information dashboard, auxiliary information dashboard, map spatial position display area, and button operation area. The basic partition of the HCI interface is shown in Fig. 21.
Analysis of the simulation functional area control layout for the SA optimized design scheme of the flight simulator HCI interface. (Created by Adobe Photoshop 2022 https://www.adobe.com/products/photoshop.html).
Cognitive evaluation of mental workload
A quantitative research method using survey questionnaires was employed to assess the physiological aspects of mental workload in response to the optimized design of the flight simulator HCI interface control layout design details. The research group was the 10 interviewed users in Table 5 who possessed at least basic proficiency in flight simulator operations. The driver mental load self-perception scores for the before and after the design optimization were evaluated using a 5-point scale, with 5 being a perfect score, indicating a significant reduction in mental workload with the optimized interface design compared to the previous version. The distribution of scores across different segments is shown in Fig. 22. The results indicate that the optimized interface design not only meets the basic mental workload requirements for flight simulation HCI interfaces but also substantially reduces the mental workload of drivers.
Statistical distribution of 10 respondents’ evaluations regarding the reduction in mental workload of the optimized SA design scheme for the control layout of the flight simulator HCI interface compared to the original design.
SA cognitive level verification
After the optimized design scheme was modeled and rendered, it was integrated into the flight simulator hardware described in Fig. 9. The remaining 10 undergraduate participants were invited to take part in this experiment. The experimental procedure was consistent with the previous trials. The participants underwent training prior to the experiment, and then performed the same simulated flight training route, from Hongqiao Airport to Shanghai IFC. The operation time for each participant was recorded, and after the experiment, they were asked to complete the SA cognitive level self-assessment scale for the control layout of the flight simulator HCI interfaces. The results were compiled and are presented in Table 8.
The design of the flight simulator HCI interface control layout has been analyzed across four dimensions, demonstrating good compatibility for simulation across multiple platforms. Rooted in an analysis of existing helicopter types, this design maintains operational continuity with current flight simulators, resulting in minimal additional training costs. A comparison of Table 8 with Table 4 reveals that the SA optimized design scheme significantly reduces the driving time for drivers, thereby lowering training costs for beginners. Furthermore, it also contributes to improved operational efficiency, which enhances the SA cognitive level of drivers during task execution.
Conclusion
(1) This study develops a three-level model of SA information processing for flight simulator HCI scenarios, based on the theoretical foundation of the SA information processing model. The model was formulated to analyze the mechanisms of information perception and its transformation for drivers.
(2) Utilizing the SCL−90 Symptom Checklist as a basis, 24 generalized questions were selected and then adapted into a self-assessment scale for evaluating SA cognitive levels within the flight simulator HCI interfaces, in accordance with the three-level model of SA information processing.
(3) The study combines empirical findings on design detail requirements and design detail features to inform an optimized design solution for the SA of the flight simulator HCI interface. This approach integrates results from user interviews, survey-based design requirement assessments, SA cognitive level evaluations of 70 mainstream helicopter types HCI interface control layout, and design detail features of the top 5 types HCI interface control layout with the highest SA cognitive level. The results indicate that the optimized design significantly enhances the SA cognitive level of drivers.
(4) The SA-based design and evaluation methodology proposed in this study lays a foundational framework for SA research in flight simulator HCI interface design. Moreover, it offers valuable references for the design of high SA interfaces in driver assistance systems and industrial control systems. Although the application of SA theory in flight simulator HCI interfaces is still in its early stages, future iterations can benefit from software prototype feedback, potentially reducing drivers’ cognitive load and improving decision-making quality.
Data availability
All data generated or analyzed during this study are included in this published article.
References
Yuan, D., Yang, S. M. & Sun, C. W. Helicopter flight simulator and its characteristics. J. Syst. Simul. 21 (09), 2571–2573 (2009).
Wiegmann, D. A. & Shappell, S. A. Human error analysis of commercial aviation accidents: application of the human factors analysis and classification system (HFACS). Aviat. Space Environ. Med. 72 (11), 1006–1016 (2001).
Zhang, X., Sun, Y. C., Zhang, Y. J. & Su, S. Y. Multi-agent modelling and situational awareness analysis of human-computer interaction in the aircraft cockpit: a case study. Simul. Model. Pract. Th. 111, 102355 (2021).
Ofte, H. J. & Katsikas, S. Understanding situation awareness in SOCs, a systematic literature review. Comput. Secur. 126, 103069 (2023).
Zhou, Y. W., Zhuang, D. M., Wu, X. & Wanyan, X. R. Ergonomic design and analysis of character encoding for display interface. J. Beijing Univ. Aeronaut. Astronaut. 39 (06), 761–765 (2013).
Zhang, C. & Zhao, J. H. Research on car navigation interface design based on situation awareness. Packag Eng. 37 (02), 48–56 (2016).
Shu, X. L., Wang, L. J., He, X. L. & Dong, W. J. Ergonomics discussion on the application of touch screen equipment in the cockpit of civil aircraft. Adv. Aeronaut. Eng. 7 (01), 112–119 (2016).
Jiang, H. The human-computer interaction of the touch screen interface in the cockpit of the aircraft. Sci. Tech. Innov. Her. 16 (05), 21–23 (2019).
Ye, K. W., Bao, H. & Wei, S. D. Optimization of aircraft cockpit human-computer interface layout based on visual attention distribution. J. Nanjing Univ. Aeronaut. Astronaut. 50 (03), 416–421 (2018).
Chen, H. J. Aircraft Cockpit Multi-channel Human-computer Interaction Design Research. Dissertation, Southeast University (2016).
Zhu, H. Q. Risk Assessment Method for Cockpit Human-computer Interaction During the Landing Phase of Civil Aircraft. Dissertation., Nanjing University of Aeronautics and Astronautics (2019).
Liu, Y. M. & Huang, H. Research on optimized design of human-computer interaction interface based on visual perception. Mech. Des. Manuf. Eng. 50 (03), 5–8 (2021).
Nguyen, T. et al. A review of situation awareness assessment approaches in aviation environments. IEEE Syst. J. 13 (3), 3590–3603 (2019).
Han, G. X., Zhang, S. G. & Li, Z. G. The influence of civil aircraft approaching driving behavior on plateau wind field on the out-of-control trend. Acta Aeronaut. Astronaut. Sin. 36 (06), 1775–1787 (2015).
Jin, H. B., Zhu, G. L., Mu, X. M. & Liu, Y. W. Research progress in situation awareness in the field of aviation safety. J. Chongqing Jiaotong Univ. Nat. Sci. 38 (06), 103–108 (2019).
Carrella, A., Brennan, M. J. & Waters, T. P. Optimization of a quasi-zero-stiffness isolator. J. Mech. Sci. Tech. 21 (6), 946–949 (2007).
Virgin, L. N. & Davis, R. B. Vibration isolation using buckled struts. J. Sound Vib. 260, 965–973 (2003).
Dishman, D., Fallacaro, M. D., Damico, N. & Wright, M. C. Adaptation and validation of the situation awareness global assessment technique for nurse anesthesia graduate students. Clin. Simul. Nurs. 43, 35–43 (2020).
Gao, Q., Xu, W., Shen, M. & Gao, Z. Agent teaming situation awareness (ATSA): a situation awareness framework for human-AI teaming. arXiv Preprint arXiv : 2308. 16785 (2023).
Munir, A., Aved, A. & Blasch, E. Situational awareness: techniques, challenges, and prospects. AI. 3 (1), 55–77 (2022).
Guastello, S. J. et al. Team Situation Awareness, Cohesion, and autonomic synchrony 2: Group-level effects and their combined influence on Team Performance. Nonlin Dynam Psychol. 27 (4), 419 (2023).
Carmody-Bubb, M. Situational awareness and situational Assessment in Cognition and Decision Making in Complex Adaptive Systems: The Human Factor in Organizational Performance 155–164 (Springer International Publishing, Cham, (2023).
Chen, H. et al. Developing an improved ACT-R model for pilot situation awareness measurement. IEEE Access. 9, 122113–122124 (2021).
Walshe, N. et al. Situation awareness and the mitigation of risk associated with patient deterioration: a meta-narrative review of theories and models and their relevance to nursing practice. Int. J. Nurs. Stud. 124, 104086 (2021).
Adams, M. J., Tenney, Y. J. & Pew, R. W. Situation awareness and the cognitive management of complex systems. Hum. Factors. 37 (1), 85–104 (1995).
Zhu, G. M. Research and Application of Helicopter Engineering Simulator. Helicopter Tech. 2, 36–40 (2006).
Salmon, P. M. et al. Measuring situation awareness in complex systems: comparison of measures study. Int. J. Ind. Ergon. 39 (3), 490–500 (2009).
Chang, Z. Q. & Gu, H. B. Design and implementation of the instructor desk for civil helicopter simulators. Digit. Tech. Appl. 11, 167–170 (2017).
Galotti, K. M. Cognitive Psychology in and out of the Laboratory (Sage, 2017).
Wickens, C. D. et al. Attention-situation awareness (A-SA) model of pilot error in Human Performnce in Modeling Aviation (eds. Foyle, D. C. & Hooey, B. L.) 213–239 (CRC Press, (2008).
Tung, V. S. et al. Diagnostic value in screening severe depression of the Hamilton Depression Rating Scale, Hamilton anxiety rating scale, Beck Depression Inventory Scale, and Zung’s self-rating anxiety scale among patients with recurrent depression disorder. Acta Informatica Med. 31 (4), 249 (2023).
Icten, S. et al. Anxiety, depression, and sleep disorders among healthcare workers during the COVID-19 pandemic. North. Clin. Istanb. 9 (4), 295 (2022).
Yang, K. B. et al. Influence of Minnesota Multiphasic personality traits on relapse to drinking in male Alcohol Dependent patients. Chin. J. Nerv. Ment Dis. 46 (10), 603–607 (2020).
Geng, Y. et al. Factor structure and measurement invariance of the brief symptom inventory-18 among Chinese adults. Front. Psychol. 13, 882815 (2022).
Hieronymus, F., Lisinski, A., Eriksson, E. & Østergaard, S. D. Do side effects of antidepressants impact efficacy estimates based on the Hamilton Depression Rating Scale? A pooled patient-level analysis. Transl Psychiat. 11 (1), 249 (2021).
Demirkol, M. E. et al. The relationship among anxiety sensitivity, psychache, and suicidality in patients with generalized anxiety disorder. J. Nerv. Ment Dis. 210 (10), 760–766 (2022).
Chen, H. et al. Association between psychological symptoms and illegal driving behaviors in a sample of Chinese private car drivers. Front. Psychol. 13, 984860 (2022).
Zhang, X. F., Hu, Z. Y., Wei, G. & Pei, C. H. Construction method of complex product disassembly model based on connection element. Chin. J. Mech. Eng. Chin. Ed. 50 (09), 122–130 (2014).
Fan, M. K. Research on the Influence of Aircraft Cockpit Display Interface on Pilot Workload. Dissertation, Nanjing University of Aeronautics and Astronautics (2018).
Stanton, N. A., Chambers, P. R. & Piggott, J. Situational awareness and safety. Saf. sci. 39 (3), 189–204 (2001).
Acknowledgements
This work was supported by the National Social Science Foundation of China Art Project under Grant No. 21BG125 and the Funding Programme for Cultivating Innovative Abilities of Graduate Students in Hebei Province under Grant No. CXZZSS2024042.
Author information
Authors and Affiliations
Contributions
Z.S. proposed the method and contributed to the analysis. G.C. conceived and designed the study. W.T. made an investigation, accumulated resources, wrote the original draft, and organized the revision. Y.Y. and T.L. made an investigation, accumulated resources, and wrote the original draft. L.X. contributed to the review and editing. D.H. contributed to the figures and table. All authors reviewed the manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Methods
All methods were carried out in accordance with relevant guidelines and regulations. All experimental protocols were approved by the Ethics Committee of the First Hospital of Qinhuangdao under Grant No. 2024G0007. We confirmed that informed consent was obtained from all subjects and/or their legal guardians.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Shen, Z., Chen, G., Tu, W. et al. Human-computer interaction interface design of flight simulator based on situation awareness. Sci Rep 14, 27842 (2024). https://doi.org/10.1038/s41598-024-78043-9
Received:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1038/s41598-024-78043-9
