Emotional AI and the harvesting of non-conscious data

Artificial intelligence (AI) is rapidly transforming daily life from social interactions to workplace practices to consumption activities to education and child development to wellness and self-enhancement routines. One of the fastest-growing areas of AI development is emotional AI, i.e., algorithms that are trained to sense, read and evaluate human emotion. Its origins trace back more than two decades ago to the pioneering work of Rosalind Picard in affective computing. Picard defined this technology as machines that can respond to human emotions through computation derived from psycho-physical information. Although still nascent, the emotional AI industry is increasingly lucrative with a current worth of 24 billion dollars and expected to double by 2024 (Crawford, 2021a).

Constantly advancing in sophistication and complexity, emotion recognition technologies are now deployed in in-cabin personalization systems (McStay and Urquhart, 2022), educational settings (Manolev et al., 2019), chatbots (Mantello and Ho, 2023b; Mantello et al. 2023), smart toys (McStay and Rosner, 2021), voice-based assistants (Urquhart et al., 2022), video conferencing (Abe and Iwata, 2022), personal communications (Grammarly, 2020), advertisements (Realeyes, 2022), companion robots (White and Galbraith, 2019), as well as public and private security (Cabitza et al. 2022; Kostka et al., 2021). While other AI applications rely on data gathered from a person’s corporeal exterior, emotional AI extracts information from the far more private and highly subjective domain of a person. For example, emotional AI algorithms, biosensors, and actuators can harvest non-conscious data gleaned from someone’s heartbeat, respiration rate, blood pressure, voice tone, word choice, body temperature, skin perspiration levels, head and eye movement, and gait. Newer affect-sensing tools are moving toward multimodal fusion and transfer learning with computing techniques from texts, audio, visual signals, video analyses of various social situations, statistical machine learning, and knowledge-based approaches. All of these innovative developments are brought together to achieve higher levels of accuracy and flexibility (Ho et al., 2021). Although the premise of emotion recognition technologies adoption is about enhancement of daily existence and creative problem solving, as a far more intrusive exemplar of surveillance capitalism, emotional AI is problematized with a myriad of legal, ethical, scientific, and cultural issues. In Fig. 1, we identify five major issues that may affect behavioral determinants in the adoption of emotional AI.

Fig. 1
figure 1

Five tensions impinging on acceptance of emotional AI and non-conscious data harvesting.

Full size image

First, affect tools monitor, evaluate, and collect psycho-physical data from a person’s subjective state often without their conscious awareness or approval. This opens opportunities for egregious or malicious misuse of the technology (Mantello et al. 2023, Mantello and Ho 2023b). For example, in domestic settings, emotional AI toys can be vulnerable to hacking and thus, children who play with them may be targeted in the future by malevolent actors (McStay and Rosner, 2021). In public spaces, security-purposed emotional AI devices may wrongly identify someone as “suspicious” simply because their affective state does not comply with the normative configurations of the technology’s algorithm (Cabitza et al. 2022; Wright, 2021). As affect tools become a de-facto layer of automated management systems, they may inadvertently create an environment where individuals are forced to maintain a façade of always being positive or happy (Moore and Woodcock, 2021). Consequently, employees may be unfairly reprimanded or unfairly assessed in their performance because of their lack of “attitudinal conformity” (Mantello et al., 2021). Empathic surveillance in the workplace may also fuel higher levels of stress, mistrust, and animosity between subordinates and superiors (Bondanini et al., 2020; Brougham and Haar, 2018; Mantello and Ho, 2023a). Similarly, as the Internet of Vehicles (IOV) era unfolds, emotion-sensing devices in commercial fleets and private vehicles may lead drivers to be unfairly targeted by health or car insurance companies to pay higher premiums (Howard and Borenstein, 2018).

Second, similar to the opaque character of surveillance capitalism, emotional AI will be more difficult to regulate as it becomes an increasingly proprietary layer of future smart cities, airplanes, vehicles, and personal devices. This is due, in part, to the competitive nature of the technology industry and its well-known aversion to and fear of state-supervised algorithmic auditing. But it is also a result of an absence of international consensus on the values and ethics that should be embedded in AI technology as well as divergent cultural understanding of what constitutes privacy or for that matter, privacy violation (Mantello and Ho, 2023a; Miyashita, 2021). This means that algorithmic transparency and collective standards for non-conscious biometric data collection will not occur for some time.

Third, is the generic nature of the technology. While affect-sensing tools are being marketed worldwide, the industry itself is predominantly based in the West and thus, on Western-centric ideas. Understandings of happiness, disgust, sadness, fear, or anger left up to a 30-something, occidental computer engineer working out of Silicon Valley may not accurately reflect the same emotions of someone from mainland China, Nigeria, or Thailand. Yet the current global market in emotion-sensing technology relies on algorithms that are rarely tweaked for gender, racial, cultural, or ethnic differences (Buolamwini and Gebru, 2018; Mitchell, 2019; Wright, 2021). An increasing number of studies demonstrate potential biases in affect-recognition algorithms can inadvertently lead to prejudice or false positive profiling (Cabitza et al., 2022; Lee et al., 2019; Schelenz, 2022).

A prime illustration of cultural conflict arising over culturally divergent perceptions of technological adoption can be found in current labor disputes over AI adoption in the transnational workplace (Ishibushi and Matsakis, 2021). At Amazon Japan, employees are increasingly at odds with what they claim are the company’s culturally insensitive automated management practices (Du, 2022; Misawa and Sawaji, 2022). While Amazon’s Orwellian surveillance practices normally comply with international and local labor laws (Biswas, 2021; Greene, 2022), union representatives in Japan argue AI monitoring of employees breaches traditional norms of trust and respect between employer and employee, a feature cherished in Japanese society (Masafumi Ito, 2021, personal communication). Similar cultural tensions have arisen at IBM Japan over, the company’s failure to disclose information on its AI-driven performance evaluation and wage assessment systems (Business and Human Rights Resource Center, 2021). Umer (2021) also notes that Uber Eats workers in Japan have unionized over what they say is the excessive control the company’s app puts on its drivers. However, while current Japanese labor laws fail to support these platform workers with adequate accident insurance and better working conditions, the author notes that collective bargaining efforts are also stymied by weak participation of workers to join the union due to the cultural stigma associated with suing an employer.

These cases highlight just a few examples where local culture may account for differences in the way an individual and their respective society accept or reject new technology. Consequently, understanding the behavioral determinants in user acceptance of affect recognition devices is not as straightforward as suggested by previous 20th-century models, especially, in a transnational, transcultural context (Ho et al., 2023). Thus, our study suggests the adoption of affect tools is subject to a myriad of cultural, social, and political circumstances, and such theoretical insights should inform how we devise our statistical models.

Fourth, existing ethical frameworks for the implementation of AI solutions have been called too rigid. Critics argue that different stakeholders may not necessarily have the same rationale, expectations, or goals for adoption. Thus, the development of practical strategies for implementation and tangible metrics for measuring the technology’s effectiveness are vital. Further compounding this problem, is the lack of foresight or simply reluctance of many private and commercial institutions to have full-time staff skilled in data analysis and data management. Bean (2022) argues that wide-scale adoption of AI in both the private and public sector have failed so far in this regard.

Finally, the validity of the science behind the emotional AI industry has been called into question. Given that the science community itself cannot agree on the nature of human emotion, whether it is biologically hardwired into the physical makeup of a person or socially and culturally contingent (Barrett, 2017), a growing number of critics argue how can emotions be made computable (Heaven, 2022). Unfortunately, the majority of emotion recognition companies still rely on the famed but now discredited theory of the “universality of emotions” by Paul Eckman. The famed American sociologist suggested that all cultures and people in the world share six basic expressions of emotions. Critics argue that emotional AI is more akin to “scientism” than science. And thus, the uncritical acceptance of the technology is eerily similar to what took place in the late 19th century with the pseudo-science of phrenology. But instead of insight into human behavior achieved through measurement and numbers of a person’s corporeal exterior—insight about a person is now achieved by making a person’s subjective state computable.

Regardless of this ongoing controversy, as affect recognition systems embed themselves in our work and leisure life, they will have a profound impact on us and our society. As such, the technology prompts numerous questions about its ability to peer into our interior, intimate, and highly subjective domains to make judgments about us through algorithmic and biometric means. Thus, the purpose of this study is to better understand what individuals think about a new era in human-machine relations, in which, intelligent machines not only feel but also feed off emotion as statistical fodder to reshape human behavior.

To gauge behavioral determinants that govern an individual’s acceptance, our analysis brings together the Technological Acceptance Model (TAM) proposed by Davis (1989) with the more recent Mindsponge model of information filtering process (Vuong and Napier, 2015) as well as allows for statistical affordances of the multi-level Bayesian analysis techniques (McElreath, 2020; Vuong et al., 2020; Yao et al., 2018). Besides fusing these two models, we employ a wide range of standard demographic variables such as region, religiosity, income, political environment in the home country, as well as acceptance contingency when the technology is authored by state versus private/commercial actors. Importantly, we highlight the ambient and automated nature of the technology, how its adoption may not necessarily be equivalent to not only an individual’s acceptance of its hidden activities but also how user consent may be dependent on context and situation.

Research questions

Our study attempts to answer the following research questions:

RQ1) Would the extension of Davis’ TAM through incorporating core personal values and environmental factors yield more reliable indicators for technological acceptance for affect recognition technology?

RQ2) Are there changes in the statistical correlations between the acceptance of non-conscious emotional data harvesting and various behavioral factors when the collector of data is the government versus the private sector?

RQ3) How do perceived utility and perceived familiarity with AI technologies predict the attitude toward non-conscious harvesting of emotional data?

RQ4) How do different uses of social media platforms, a prime example of computing technologies that feel and feed off human emotions, associate with different attitudes toward the non-conscious harvesting of emotional data?

The next section provides a literature review on technological acceptance and outlines the rationale for the mindsponge-based extension of TAM.

Methodological review of studies on technological acceptance behaviors

Since the mid-20th century, numerous theoretical models have been devised to better understand factors determining a person’s adoption of technology. Some of these frameworks include the Innovation Diffusion Theory (Rogers, 1963), Theory of Reasoned Action (Ajzen and Fishbein, 1977), Theory of Planned Behavior (Ajzen, 1991), Technological Acceptance Model (Davis, 1989), Unified Theory of Acceptance and Use of Technology (Venkatesh and Davis, 2000) and most recently, the mindsponge theory (Vuong and Napier, 2015). Each of these models has sought to identify key factors which shape user intention or actual use of a given technology. However, none of these models can account for the novel and invasive way current and future affect-recognition devices endeavor to nudge, manipulate, and reshape a person’s subjective state.

In the original TAM model (1989), Davis hypothesized that the level of acceptance of new technology is determined by two factors: perceived ease of use and perceived utility. In 2000, Davis extended his original framework to include subjective norms such as how an individual may be influenced by their perception of how others will view the technology’s performance. For him, the greater the individual’s familiarity or closeness with those they think should or should not adopt a given technology will determine the degree to which this influence occurs (p.187). In other words, the model presumes a measurement of conformity due to social influence. It is important to note that both the original (1989) and extended TAM (2000) have enjoyed a high level of citation and empirical support. For example, Lew et al. (2020) found the extended model accounted for 61% of the variance in the behavioral intention (BI) to adopt mobile wallet technology. Another meta-analysis of digital technology adoption in education showed TAM models can account for up to 44% of the variance in the BI (Scherer et al., 2019).

However, the rise of smart technologies, notably for our purposes, emotion-recognition tools, highlights serious limitations of TAM. For example, TAM and its later versions do not account for cross-cultural variances (differences in core values, mindsets, etc.) in the way people form acceptance perceptions based on ease of use, utility, and social influence (Taherdoost, 2018). Indeed, emerging literature testing the TAM and TAM2 models in various countries shows cultural values do in fact influence how people form the perceptions that are pertinent to the TAM (Dutot et al., 2019; Muk and Chung, 2015). A good case in point is the opposing public opinion between China and the EU on social credit systems. While new EU AI regulations forbid social credit systems, studies have found that a significant portion of the Chinese population holds a higher level of trust in the government’s treatment and storing of citizen data (Roberts et al., 2021). Concurringly, Gruzd et al. (2020) show Indian subjects to be significantly more accepting toward social media screening for job applicants than their US counterparts. Another limitation of TAM lies in its reliance on a subject-object relationship in which the user physically adopts and consciously interacts with the technology. However, such a linear and tactile dynamic is not a prerequisite for emerging AI technologies that often operate in the background of personal devices or embed themselves in more complex automated systems. Indeed, affect recognition tools are an exemplar of the ubiquitous and stealth character of newer iterations of intelligent machines purposed to augment human endeavors (See Fig. 1).

Moreover, whereas TAM considers cost-and-benefit evaluation, i.e., the variables of perceived usefulness and ease of use of technology, to be key factors in deciding the level of acceptance, the mindsponge framework envisions the human mind as a filtering mechanism for fresh inputs such as new values, new ideas, or in this case, new technologies. In the mindsponge model, ease of use as well as utility act as trust evaluators of the filtering process, e.g., higher perceived usefulness or higher ease of use help increase the trust in technology. However, these evaluators are not the determining factors in measuring acceptance as the traditional TAM suggests. Whether the mind of a user rejects or accepts an input is also contingent on auxiliary factors such as an ability to creatively adapt new inputs to not only their specific circumstances/needs but also how their core values and external settings such as culture and society reinforce or diminish the uses of such inputs. Originally, the mindsponge model was applied in explaining the process of adopting new values as well as innovations in business settings (Vuong, 2016; Vuong and Napier, 2014, 2015). In recent years, more efforts have been made to apply the mindsponge model to the adoption of new technologies in various fields: education (Vuong et al., 2021; Vuong, 2022a, 2022b), entrepreneurial finance (Vuong et al., 2021), vaccine production (Vuong et al., 2022), etc. However, there has not been any attempt to integrate the TAM and the mindsponge model of information processing to understand attitudes toward the harvesting of non-conscious data by AI systems. Thus, this is an area where this study hopes to contribute.

Methods and material

Overview of research design and methodology

Our methodology consists of three steps. First, we identify variables relevant to the study of the acceptance of non-conscious emotional data harvesting based on the original TAM. These variables are perceived utilities, perceived familiarity with the technologies, attitude toward AI systems. Second, we incorporate various factors from the mindsponge model of information filtering to inform/nuance/expand TAM. These factors are personal core values (i.e., level of religiosity); environmental factors of culture (i.e., regions of home country) and politics (i.e., political regime of the home country); and income level. Third, relying on Bayesian multi-level analysis, we build statistical models in which each of these personal, political, and environmental factors form the varying intercepts in Bayesian network models (Ho et al., 2022; La and Vuong, 2019). Figure 2 provides a summary of how each aforementioned element (theoretical models, variables, statistical modeling choices) fit together into the empirical strategy of this study. The left side of Fig. 2 presents the above three steps, while the right side of Fig. 2 presents the process and standard of how to evaluate the models and how to report the research results.

Fig. 2: Visualization of the empirical strategy.
figure 2

This diagram presents how elements of the TAM and minsponge model are coordinated to create the empirical strategy to help investigate the research questions.

Full size image

These processes follow the principles laid out in Gelman and Shalizi’s (2013) paper on the philosophy and practice of Bayesian statistics. To briefly summarize the model evaluation and results reporting processes, models are evaluated based on plausibility distribution using Bayesian concepts of weights, convergence diagnostics for computational efficiency (McElreath, 2020), Pareto-Smoothed Importance Sampling (PSIS-LOO) tests for goodness-of-fit (Yao et al., 2018). The best models are then selected for results reporting. A comparison of results across models is also provided to ensure a high level of reliability. The fact all models with sizable plausibility distribution converge on the same set of results indicates a high level of reliability. The process of data collection and data treatment to turn raw data into variables used for subsequent analyses are detailed in the next section, Data collection and treatment.

Taking advantage of the partial pooling techniques of Bayesian statistical analysis, the cultural and environmental factors are used as varying intercepts in our model. This approach is called Bayesian multi-level or hierarchical Bayesian regression, and it has many advantages suited for this study. First, the Bayesian statistical approach also allows us to directly compare the plausibility of our models via various indicators of weights, thus offering a way to measure the plausibility of the TAM models versus the extended TAM models (Fig. 3).

Fig. 3: Visualizations of the best-performing models.
figure 3

Visualizations of the models with the most weights produced by the Bayesvl package.

Full size image

Second, since many more studies from the field of science and technology studies and social sciences rely on data from online surveys, Bayesian multi-level modeling has an advantage over traditional frequentist statistics. Online surveys often imply that non-random and limited data can render local analyses carried out in traditional statistical methods impossible or unreliable. As Spiegelhalter (2019) suggests, traditional frequentist statistics struggle with situations where data are non-random and limited such that local analyses cannot be performed reliably.

Instead, the Bayesian multi-level regression or hierarchical modeling is the perfect response to this problem. Here, “the basic idea is to break down all possible respondents into small “cells”, each comprising of a highly homogeneous group of people-say living in the same area, with the same age, gender, …” (p.329). Accordingly, our study breaks down the respondents into different regions, religiosity levels, and political regimes. Such approaches are driven by insights from the information filtering mechanism of the mindsponge model (Vuong, 2016; Vuong and Napier, 2015), which postulates the mind filters new inputs with its core values (i.e., the variable of religiosity) and with consideration to the external cultural and ideological environment (i.e., the variables of regions and political regime).

Data collection and treatment

The survey was conducted over several months, starting in late 2020 and ending in early 2021. It was distributed via a Google form in more than 10 online classes consisting of students from first year to master’s level. The final sample results in a dataset of 1015 international and domestic students from APU. The study site is the largest international higher educational institution in Japan with more than 5700 students from has students from nearly 100 countries from 8 regions of the world, as of 2022. Since the study sets out to understand the subtle effects of cross-cultural differences in attitude toward emotional AI applications, the study site’s diverse student pool provides a good opportunity for the actualization of the research design.

The survey was conducted according to the codes and practices established in the Codes of Conducts for Scientists, issued by the Science Council of Japan on January 25, 2013, and the study site’s Research Code of Ethics. Before answering the survey questions, the participants were explained the study’s purpose (i.e., to understand statistically various social perceptions of emotional AI applications), its voluntary nature (i.e., the respondents were free to choose to participate in the survey and could leave anytime), the anonymization procedure, and that informed consent was implied once the respondents chose to start answering the survey questions.

Table 1 presents the flow of questions that the respondents were presented with, describes the survey questions, and also how each variable in this study was formulated. Prior to answering the questions, the respondents were asked to provide their socio-demographic, regional, and religious characteristics, given in the form of multiple-choice questions (Part I). Then, the respondents were asked various questions on their level of familiarity and level of engagement with social media, a prime example of emotional AI algorithms (Part II). Given the research questions, for independent (predictor/explanatory) variables, we asked the respondents to provide their perceived utility of emotional AI applications, attitude toward this technology (i.e., on balance, how they are optimistic about its values for society), their familiarity with various examples of AI technologies, describe their uses of social media. Regarding the outcome (dependent/predicted) variables, we asked the respondents how they are worried about non-conscious data gathering by public and private sector actors (Part III).

Table 1 Survey questions, variable treatment procedure, and key descriptive statistics (The red color signifies the male section, and the black color signifies the female section).
Full size table

Our rationale for including usages of social media is as follows. Given the acceptance of affect-recognition technology is not necessarily equivalent to the approval of its hidden activities, coupled with the fact that a large segment of the public is still unfamiliar with the technology, we use various social media behavioral factors as control variables in our model. Our rationale is based on an increasing body of evidence that shows social media platforms harvest non-conscious behavioral data by exploiting the emotionality of users’ online communication (Woolley and Howard, 2018). Evidence of this can be found in an evolving canon of empirical work ranging from the use of micro-targeted news/ads (Bakir, 2020), to digital outrage (Brady et al., 2021; Crockett, 2017), and affective polarization (Cinelli et al., 2021; Rathje et al., 2021), extremist chatbots (Bimber and Gil de Zúñiga, 2020, Mantello and Ho, 2023a, 2023b), i.e., the mining of emotional and behavioral data in social media for the purposes of job screening, often without the subjects’ knowledge (Gruzd et al., 2020). Thus, the use of emotional AI and non-conscious data collection practices in social media platforms prompt us to treat various social media behaviors (specifically, the frequency of social media use, the tendency for public messaging, the tendency for diversifying information sources, and the tendency to engage in SM debates) as control variables.

Model construction

Multiple regression models are built to investigate the research questions. Two classes of models are constructed corresponding to two types of actors involved in non-conscious data harvesting: the private and the public sectors. In each of these classes, four models are multi-level regression model (Model 2,3, 4, 5), which uses regions, religiosity political regime, and income as varying intercepts, respectively (See Table 2).

Table 2 Model construction of all models.
Full size table

Results

Model evaluation and comparison

After running the MCMC analyses for all 10 models (4 chains, 5000 iterations, 2000 warm-ups), the basic two standard diagnostic tests returned good results. All Rhat’s values equal one (1) and all the effective sample sizes (n_eff) are above 1000. The detailed results and visualizations (of the autocorrelation coefficient, the Gelman Shrink Factor, and the Markov chains) of the diagnostic tests are presented in the Supplementary file. All models were also checked with the LOO test, and we found all had good Pareto k estimates (k < 0.5), with very few having OK Pareto k estimates (k < 0.7). This suggests the models fit with the data well (See Table 3 for details).

Table 3 Weight comparison.
Full size table

The Bayesian plausibility of each model is measured by the three types of weight (listed on the top row of Table 3). In the case of the perception toward non-conscious emotional data harvesting by the private sectors, the best model is the one with regions as the varying intercept, Model 2_Private. In the case of the public sector, the model with regions as the varying intercept, Model 2_Public, massively outperforms all other models. Model 3_Public and Model 3_Private, which have religiosity as the varying intercept, are the second-best models, with one category of Bayesian weight, namely, the Bayesian stacking, taking up around 33 and 36% of the weights, respectively.

It is striking that the best-performing models are the two models that use cultural characteristics, regions, and religiosity, as the varying intercepts. And this is true for both the public and private cases. Models that use the home country’s political regime and income level as varying intercepts take up negligible Bayesian weights of less than 1% in all cases. These findings tell us that segmenting an international sample of respondents by region and religiosity would produce better estimates than segmenting the sample by home country politics or income level. This finding lends credence to the literature that shows cultural factors underlie, in other words, are the antecedents of perceptions of risks and rewards of a technology (Alsaleh et al., 2019; Dutot et al., 2019; Vu and Lim, 2021), which is a prediction made explicitly in the mindsponge model of information filtering (Q.-H. Vuong, 2022a, 2022b; Vuong and Napier, 2015). Now, we will compare the effects of the explanatory variables across models. Consistency of explanatory variables across models indicates reliable effects (See Fig. 2 for understanding the standard of results reporting).

Private sectors

In Model 2_Private (i.e., the model with regions as the varying intercept), the best model in the class that explains the attitude toward non-conscious data harvesting of the private sector, the variables whose posterior distribution is exclusively positive include b_PostingSM_Private, b_AngrySM_Private, b_Attitude_Private, b_Familiarity_Private, b_PerUtility_Private (See Fig. 4).

Fig. 4: .
figure 4

The posterior distributions of variables in Model 2_Private.

Full size image

Across all other models, however, there are some inconsistencies. For example, for the Model 3_Private (i.e., the model with religiosity as the varying intercept and the second-best model), the variable Attitude and the variable DiversifySM are slightly more ambiguous, with their 89% HDPI being −0.0184–0.101, and −0.0555–0.0453, respectively. Similar results are found in Model 3_Private and Model 1_Private as well.

Thus, based on the model weights and the comparison of posterior distributions across all models, we can reasonably be sure to assert that the positive effects on the attitude toward non-conscious data harvesting of the private sector by the variables of b_PostingSM_Private, b_AngrySM_Private, b_Familiarity_Private, b_PerUtility_Private. The results suggest that people who frequently make public posts on social media, engage less in heated debates on social media platforms, rate themselves as more familiar with AI technologies, and perceive higher utility in AI applications will likely hold a more accepting view of non-conscious emotional data collection by the private sectors.

Public sectors

In Model 2_Public (i.e., the model with the region as the varying intercept), the best model in the class that explains the attitude toward non-conscious data harvesting of the public sector, the variables whose posterior distribution is exclusively positive include b_PostingSM_Public, b_AngrySM_Public, b_Attitude_Public, b_Familiarity_Public, b_PerUtility_Public. In contrast, the posterior distribution of b_SocialMedia_Public, i.e., how much time a person spends on SM, is almost exclusively negative (89% HDPI −0.144–0.0181). Meanwhile, whether a person diversifies their followers or news feed does not have clear effects on the attitude toward non-conscious emotional data harvesting of the public sector (b_DiversifySM_Public’s 89% HDPI −0.0221–0.0799) (See Fig. 5).

Fig. 5: .
figure 5

Posterior distributions of variables in Model 2_Public.

Full size image

Given that Model 3_Public (i.e., the model with religiosity as the varying intercept), in two cases, takes up a reasonable amount of weight (Bayesian stacking and Pseudo-BMA with Bayesian bootstrap), it is worth closely examining the outcome of Model 3_Public. We find that the results are similar to Model 2_Public. As for the three less fitted models, Model 1_Public, Model 4_Public, and Model 5_Public, the results are quite consistent with the above.

Thus, overall, we can ascertain the effects of the studied variables as follows. First, regarding social media use, people who frequently engage in public posting and less frequently engage in heated debates are more likely to hold an accepting view of the government’s harvesting of non-conscious emotional data (heart rates, skin conductance, eyes gaze, etc.). On the other hand, we found that people who spend more time on social media are more likely to express a worried view of the government’s harvesting of non-conscious emotional data. This result diverges from those of the private sector, whereby the effect of how much time spent on social media is ambiguous.

Second, for variables that belong to TAM, self-rated familiarity with AI technologies, perceived utility, and optimistic attitude toward the future of AI applications are all positive predictors of the attitude toward the public sector’s engagement in non-conscious dataveillance. Conversely, we find that for data harvesting by private sector actors, the attitude toward AI applications is more ambiguous. Concurrent with Zuboff’s (2019) writings on surveillance capitalism, our study indicates that when it comes to non-conscious data collection, people are generally willing to sacrifice privacy for personalized benefits. This disinterest in privacy is more pronounced when people perceive they have greater autonomy and mastery over the new technology, i.e., they are less likely to worry about the collection and analysis of their non-conscious emotional data. Such findings concur with existing literature. For example, a study by Mohallick et al. (2018) looked into privacy concerns of the European population over the AI-powered recommender system and found that negative attitudes toward these systems decreased when a person felt an increased sense of control and ownership over their data. And yet, McStay (2020) found surprisingly weak consensus among industry spokespersons, citizens, and NGOs on the importance of control-oriented opt-in consent for the collection and analysis of emotional data. No doubt the lack of consensus reflects the divergent interests of the stakeholders involved. Whereas control-oriented opt-in consent may allow users better chance to safeguard privacy and negotiate the terms of their interactions with emotional AI, it diminishes the industry’s ability to maximize their data harvesting and monetization efforts.

Discussion

Explanatory values of mindsponge-based extended TAM

Answering RQ1, this article extends and nuances TAM by incorporating the mindsponge framework. The extended TAM offers great explanatory values compared to its older counterpart. The evidence of which comes in the form of model weight comparison, whereby, models based on the traditional TAM (Model 1_Private and Model 1_Public) are distributed with extremely less weight compared to models that consider core beliefs (i.e., religiosity in Model 3) or cultural factors (i.e., regions in Model 2) as varying intercepts. Although, by the goodness-of-fit test based on the PSIS-LOO procedure, all models exhibit a good fit with the data, the plausibility distribution indicates clear evidence in favor of accounting for cross-cultural factors in estimating effects of predictor variables on attitude toward non-conscious data harvesting.

Our results also show the complexity of interaction between context and stakeholders in determining attitudes toward non-conscious emotional data harvesting. For example, when the collector of emotional data is from the private sector, how much time a person spends on social media platforms has an ambiguous effect, meanwhile, when the collector is from the public sector, the same variable exerts a clear negative effect on the attitude toward the practice of non-conscious data harvesting (See Table 4).

Table 4 Differences among the determinants of attitude toward the public vs. private sector’s engagement with non-conscious data harvesting.
Full size table

Differences in acceptance regarding the public versus private actors

Based on a close examination of the analysis results across all eight models, Table 4 summarizes similar and diverging results between the attitude toward the public and private sector use of affect tools and non-conscious emotional data harvesting. This answers RQ2 on whether there are changes in the statistical correlations between the acceptance of non-conscious emotional data harvesting and various behavioral factors when the collector of data is the government versus the private sector.

With regards to converging results, the variables of familiarity with and perceived utility of AI technologies, as predicted by both TAM and the mindsponge model, are found to be positive correlates of attitude toward non-conscious emotional data harvesting in both sectors. In terms of differences, the most outstanding result centers on the dwelling time of a person on social media.

While engagement time with social media appears to negatively correlate with attitude toward the public sector’s usage in terms of nonconscious dataveillance, it has no clear relationship with the attitude toward the private sector. This insight suggests that there is less trust in the public sector’s handling of non-conscious data compared to the private sector, especially when the subject spends more time on social media. This result aligns with the results from a study of approximately 6000 American subjects, where Hidalgo et al., (2021) found people tend to judge AI more negatively when it is used by the government. Thus, according to the Bayesian mindsponge way of thinking, the result suggests acceptance toward non-conscious data harvesting is filtered through a deep bias against the government actor. It also reflects the relative novelty of emotional AI technologies in the public consciousness. One of the predictions from the Bayesian mindsponge framework is there will be a process of Bayesian updating of beliefs (McCann, 2020; Nguyen et al., 2022). Thus, it is expected in the future, core values/beliefs in the handling of non-conscious data by the government and private sector will become more balanced and nuanced. Such findings provide a promising window of opportunity for further empirical and theoretical studies.

Privacy and autonomy: self-efficacy in the age of emotional AI

For RQ3 and RQ4, for both corporate and state harvesting of non-conscious emotional data, our analysis discovers four behavioral predictors of a non-worrying attitude: (i) using social media for public messaging purposes (b_PostingSM), (ii) low participation in heated debates of social media (b_AngrySM), (iii) higher self-rated familiarity with AI technologies (b_Familiarity), and (iv) higher perceived utility of AI technologies (b_PerUtility) (See Fig. 6). Interestingly, all of the variables that positively correlate with acceptance of non-conscious data harvesting concern a person’s assertion of agency and self-efficacy over their social media behavior and smart technology interactions.

Fig. 6: .
figure 6

Visualization of the posterior distribution of the variables originated in the TAM.

Full size image

The first two variables are indicators of a person’s sense of autonomy and control in social media platforms. Our findings show that those who feel they have a clear purpose when engaging with social media (i.e., mainly using social media for public messaging) and exhibit emotional control (i.e., the low frequency of participating in heated online debates) are less inclined to feel threatened by the practice of non-conscious data harvesting. More importantly, our results also indicate a stronger sense of digital self-efficacy over these platforms and smart technologies correlates with less worry over non-conscious dataveillance. The higher sense of familiarity with various aspects of AI technologies (including emotional AI, AI in general, smart cities, and coding) also heightens the sense of mastery over the new technologies.

Interestingly, our findings present an intriguing alignment with recently revisited writings related to Albert Bandura’s theory of self-efficacy (Alharbi and Drew, 2019; Latikka et al., 2019; Moreira-Fontán et al., 2019). Over two decades ago, the eminent psychologist’s research focused on the efficacy beliefs of individuals regarding their usage of robots, digital technology, online learning, etc. According to Bandura (1997), the self-efficacy of technology users is formed with four pillars: vicarious experiences, mastery experiences, physiological and affective feedback, and verbal persuasion. Arguably, all of his categories can apply to affect-recognition devices. For example, in terms of vicarious experience, many people are already directly or indirectly, consciously or unconsciously, experiencing or interacting with the nascent technology. Some of these vicarious experiences include interactions with mental health chatbots (Ho et al., 2022), email tone detection software (Koltovskaia, 2020), in-vehicle concierge systems (McStay and Urquhart, 2022)) and mood-based music apps (Freeman et al., 2022). In terms of mastery, Lu’s (2020) study of content curation shows that perceived user controllability, the perceived ability of self-control, positively predicts active engagement in a user’s curating behavior. In other words, intensification of self-efficacy occurs for users who feel they have gained mastery/control or tamed their smart technology, such as an understanding of Spotify, Instagram, or Facebook’s algorithm to make it recommend the contents they want.

In terms of user efficacy-related physiological and affective feedback, emotional AI is now considered a de facto tool of the wellness industry. Self-quantification and algorithmically induced mindfulness and self-quantification are prime features of affect tools (Ho and Mantello, 2023). For example, the Japanese voice analytics company, Empath, see their technology as a way for call centers to optimize workplace productivity by providing supervisors with a panoptic window into the subjective state of each member of their customer service team. On the other hand, Moodbeam, an emotion-sensing bracelet by the 55 Group, offers companies a neoliberal alternative to more management-intensive and costly worker wellness programs. As the company’s public statement suggests, a worker simply needs to wear the bracelet and it will automatically share data of their subjective state to both managers and co-workers. This neoliberal approach to mindfulness is premised on the assumption that “sharing is caring” (Mantello et al., 2021).

Finally, Bandura’s source of efficacy through verbal persuasion helps to better understand the current bandwagon appeal of emotional AI (Crawford, 2021b; Mantello and Ho, 2022). Despite still unproven claims, the current vogue and proliferation of emotional AI devices are not surprising. A good case in point is the Russian company ELSYS, a global manufacturer and distributor of emotion-recognition software that allegedly detects suspicious behavior by analyzing the ‘micro-vibrations’ of an individual’s head and body movements from video images. In his article, “Suspect AI” (Wright, 2021), the Alan Turing Institute scholar James Wright interrogates the scientific credibility of ELSYS’s products. Although concluding there is no discernible science to ELSYS’s technology, he notes an extensive list of high-profile private and government contracts. Indeed, on a recent visit to the ELSYS’s website, we observed a client list that included airports, government facilities, and retail outlets in Russia and Japan, hotels and casinos in Korea, China, and Philippines, airports in Australia, Dubai, Israel, and Germany, as well as police departments in Canada and USA (www.ELSYS.com).

In sum, the results of this study have corroborated and enriched insights from the recently revisited Albert Bandura’s theory of self-efficacy as it is applied to the study of human interaction with smart technologies. Thus, our analysis of emotional AI shows the potential for further theoretical insights and explanatory power from the extension of TAM with elements from the mindsponge model of information processing. This novel combination can indeed be further refined in future studies that examine the subtlety of human interactions with the emerging smart technological infrastructure.

Limitations and future research directions

Overall, with a diverse sample of 1015 post-millennial respondents from 48 countries and 8 regions, this study is the first to extend the TAM model with cultural insights afforded by the mindsponge model of information filtering. However, the results must be cautiously generalized due to the following reasons. First, the survey sample only covers international and domestic students (ages 18–27) at an international university in Japan. Second, the data collection was performed through Zoom-based remote lectures. Third, even though the survey was distributed to a large, diverse group of students ranging in their first to final year, due to the lack of randomization, some ethnic populations are over-represented. However, the multi-level Bayesian analysis with partial pooling can correct for the smaller size of other ethnic groups (Gelman, 2010; Kim et al., 2018; Vuong et al., 2020). Future studies can further modify the mindsponge-based extension of the TAM and include factors such as how much awareness a person has on the extent of non-conscious data collection.

Conclusion

This study upgrades the Technological Acceptance Model with insights afforded by the mindsponge model of information filtering and statistical modeling utilizing the Bayesian approach. We find that the data fitting results of mindsponge-based models are distributed more Bayesian weights, i.e., plausibility in the language of the Bayesian statistical paradigm. Critically, this study finds respondents who feel more familiar with, perceive more utilities in AI technologies, and who rate themselves as more restrained from heated arguments when engaging with social media, feel less threatened by the practice of non-conscious data harvesting by both the government and the private sector. Importantly, our findings indicate that cultural factors play an important role in determining positive or negative attitudes toward non-conscious data harvesting. Finally, concurrent with the existing literature on big data surveillance, this study found an indicator of the lack of trust toward the government’s engagement in non-conscious dataveillance as opposed to commercial actors’ usage of the technology. The results carry important implications for the governance of the emotional AI industry. As such, our findings offer a fertile platform for further exploration of the complex intersection between psychology, culture, and emotion-recognition technologies. It also provides vital insights for policymakers wishing to ensure design and regulation of the technology serve the best interests of society.