Table 1 The comparison between the presented work and related work underlines the trend in the research area of mental state classification to utilize multimodal data, preferably labeled using scientifically validated questionnaires.

Publication	Hasan³¹	Hinss³²	Wang³³	Hosseini³⁴	Coşkun³⁵	Kang³⁶	Presented	Zaman³⁷
Classification of	Cognitive Load	Cognitive Load	Mental State	Stress	Stress	Mental State	Mental State	Stress
Signals used	Facial Thermal and Visual Cameras, Visual Ceiling Camera	EEG	EEG	EDA, HR, ST, ACC, IBI, BVP	BVP, EDA, ST, Facial Expressions, ACC	ACC, HR, ST, EDA, IBI, and phone sensors	ACC, HR, ST, EDA, EEG, PPG, and Log Data	Facial Thermal and Visual Cameras, Visual Ceiling Camera, EDA, HR, Breathing Rate
Controlled	No	Yes	Yes	No	Yes	No	Yes	Yes
Uncontrolled	Yes	No	No	Yes	No	Yes	Yes	No
Self-Labeled	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes
Expert-Labeled	Yes	No	No	No	Yes	No	Not yet	No
Questionnaires	Standard	Standard	Standard	Non standard	Non standard	Standard	Standard	Standard
Given Tasks	No	Yes	Yes	No	Yes	No	Yes	Yes
Free Tasks	Yes	No	No	Yes	No	Yes	Yes	No
Multi-Modality	Yes	No	No	No	Yes	Yes	Yes	Yes
# Wearables	0	0	0	1	1	1 + phone	2	2
Synchronization	No	No	No	No	Time	Time	Time, Tapping, and Acceleration	No
# Participants	10	29	60	15	25	77	24	63

The lack of advanced synchronization tools for physiological time series can be seen. The biggest limitation, the lack of studies considering both controlled and uncontrolled environments for the same study participants, is demonstrated in rows Controlled and Uncontrolled. The number of wearable devices is another important factor, as actually wearable multi-modality allows for advanced algorithms in real-world use cases. The abbreviations in the Signals-row are: Blood Volume Pulse (BVP), Electrodermal Activity (EDA), Skin Temperature (ST), Acceleration (ACC), Heart Rate (HR), Inter-Beat Interval (IBI), Blood Volume Pulse (BVP), and Photoplethysmogram (PPG). Overall, the number of participants per study is small. The entries in the table are sorted by # Wearables and # Participants in ascending order.

Search