Introduction

The COVID-19 pandemic, declared by the World Health Organisation (WHO) in March 2020, has posed unparalleled challenges leading to a period of uncertainty alongside accelerated scientific advancements related to the disease, its treatment, and the lack of comprehensive information on the best available evidence for healthcare professionals (Goossens et al., 2021). In 2020, these professionals were confronted with the imperative to update their knowledge and training to guarantee the best quality and safety of their interventions. However, researchers and healthcare professionals had to overcome the lack of previous knowledge on how to manage these novel healthcare demands. Thus, as newly developed knowledge was made available, dissemination was fundamental to guarantee its effective application in patient treatment worldwide, prompting the introduction of digital lifelong learning courses designed with open and asynchronous properties and tailored to large audiences, such as Massive Open Online Courses (MOOCs), considered the ‘forefront of education’ (Li et al., 2022).

The COVID-19 pandemic, coupled with excessive workloads and confinement restrictions has drastically transformed healthcare professionals’ access to lifelong learning. As a result, digital technologies emerged as effective tools for knowledge dissemination, updating and upskilling. During the pandemic, healthcare professionals’ time constraints and the imperative to access evidence-based and certified education shifted their attention and interest to e-learning courses, such as MOOCs.

With the pandemic outbreak, the number of registered users and courses on these international platforms has grown exponentially, from 91 million users before the pandemic to 178 million (Shah, 2021). In parallel, the body of available evidence was also increasing before the pandemic (Liu et al., 2021), likely predicting the forthcoming events in the field of education and training amidst the pandemic.

In the healthcare area, the World Health Organization (WHO) made the ‘COVID-19 Vaccination’ MOOC available in the OpenWHO platform directed to healthcare professionals (Goldin et al., 2021). However, despite this initiative, there was a lack of advanced upskilling courses related to the diagnosis and treatment of the disease.

In this context, a group of tertiary education teachers and healthcare professionals in nursing, medicine, and allied health areas collectively developed new educational and simulation contents, addressing the need for specialised training and evidence-based curriculum for the diagnosis and treatment of the COVID-19 disease. This endeavour culminated in the creation of a single MOOC, presented in Portuguese, directed at clinical diagnosis and treatment.

This study aimed (i) to assess the intention of healthcare professionals to use a MOOC in lifelong learning during a pandemic outbreak by analysing the perceived ease of use, usefulness, and the intention to use this massive course in continuing training; (ii) to assess knowledge retention by comparing knowledge before and after the course; and (iii) to analyse the characteristics of participants who did not request a certificate.

Literature review

The evidence about the integration of Massive Open Online Courses (MOOCs) into educational and training processes in a pandemic context has primarily focused on five major dimensions related to (1) the motivations for digital transformation, particularly concerning the use of MOOCs, (2) the perceived benefits of integrating MOOCs into education and training, (3) the challenges and implementation strategies, (4) efficacy and perceived perception of massive learning, and (5) envisioning future developments of this technology and their impact on education and training.

Regarding the first dimension, literature is dedicated to adapting learning methodologies to the outbreak context and the need for digital resources for students and trainees’ reskilling and upskilling (Almufarreh and Arshad, 2023; Despujol et al., 2022). This dimension focuses on the transition to online learning, reflecting on adaptation and digital transformation in organisations and its relevance for academic continuity through MOOCs and other online learning courses (Arima et al., 2021; Mejia et al., 2020).

Regarding the second dimension, the literature emphasises academic, professional, and personal gains, reinforcing the role of MOOCs in promoting flexibility, internationalization, and ensuring academic continuity (Arima et al., 2021; Almufarreh and Arshad, 2023; Despujol et al., 2022).

When examining the challenges and implementation strategies, the third dimension, evidence shows the existence of technological barriers to MOOC implementation due to inadequate infrastructure and limited internet access in various global contexts, posing a significant challenge to digital learning in several regions (Amit et al., 2022; Mejia et al., 2020).

Regarding the fourth dimension, online learning efficacy and perception are addressed mainly at the level of perceived perceptions. Overall, students/trainees and teachers/trainers reported positive perceptions of MOOCs’ efficacy, expressing satisfaction with their learning experiences (Arima et al., 2021; Ceballos and Mexía, 2021; Jivet and Saunders-Smits, 2021).

Thus, beyond offering flexibility, MOOCs were perceived as significant contributors to career development and continuing education, particularly within the pandemic outbreak context (Almufarreh and Arshad, 2023).

In the context of online education, the fifth dimension emphasises the integration of MOOCs into the regular learning process, suggesting that institutions start integrating MOOCs and online learning strategies into their academic curricula, leveraging lessons learned during the pandemic (Despujol et al., 2022).

Lastly, evidence highlights MOOCs’ role in continuing professional development, emphasizing the importance of trainers and teachers’ continuous training in digital and pedagogical skills to maximize the benefits of MOOCs and online learning (Mejia et al., 2020).

However, the literature on the perceived ease of use, usefulness, and the intention to use this type of course in the COVID-19 context remains less explored. Nevertheless, some studies focus on specific topics, evidencing two different dimensions: adoption and continuity and pedagogical and instructional quality.

Regarding adoption and continuity, the literature highlights the perceived usefulness of the contents addressed, ease of access and platform usage (Padilha et al., 2021; Daneji et al., 2018; Gómez Gómez and Munuera Gómez, 2021), the institutional reputation of the creators (Tella et al., 2021), and satisfaction with previous experiences and confirmation of expectations (Daneji et al., 2019) as key drivers for students and trainees’ engagement and completion of MOOCs.

Regarding the pedagogical and instructional quality, the curriculum and the programme layout are relevant for retention, perceived utility, motivation, and trainee experience, adding to other add-ons and tools for an immersive learning experience, along with intuitive and responsive design for the seamless digital experience of users (Chan et al., 2017; Liu et al., 2015).

Methodology

A descriptive study was carried out to assess the MOOC for perceived ease of use, usefulness, intention to use and effectiveness in knowledge retention in lifelong learning.

Study design

In the initial phase, a collaborative and interprofessional updating course aimed at healthcare professionals was developed on the diagnosis and treatment of COVID-19. This process involved healthcare professionals from different areas and higher education teachers from an HEI, who jointly contributed to the design and development of the MOOC throughout all the development and assessment phases (Fig. 1).

Fig. 1: Study design.
figure 1

The study was conducted in two phases: (1) Development of the COVID-19 MOOC, including interprofessional collaboration, design and development, and implementation; (2) MOOC assessment, focused on user acceptance (ease of use, usefulness, intention to use) and knowledge retention before and after course completion.

Full size image

After the drawing of a catalogue of technical and pedagogical tools to be embedded in the e-learning platform based on educational needs and the thematic state of the art, the developed MOOC was designed based on a modular structure using video, text, and a virtual patient simulator, directed to the development of clinical decision-making skills.

This MOOC was structured into ten learning modules (Table 1). Each module fulfilled specific objectives materialized in lessons supported by text, multimedia documents, and a digital immersive simulation scenario for clinical reasoning training using virtual patients.

Table 1 Learning modules of the Ecare-COVID-19 MOOC.
Full size table

The MOOC offered 39 lessons with ~5 h of video content, each lasting an average of 7.5 min. Participants had an estimated workload of 35 h to complete this MOOC. Participants who completed the MOOC and achieved a minimum score of 70% on the final assessment were eligible to issue a certificate of completion.

Following a one-month period of beta-testing for usability, functionality, and performance assessment, the updating and validation of readiness the COVID-19 MOOC was made available between March and October 2021. It was hosted by the Portuguese Foundation for Science and Technology (FCT) on an OpenEDX open-source learning management system platform, supported by high-performance storage technology (CEPH), in full open-source. The second phase of the study aimed to assess the perceived ease of use, usefulness, and the intention to use the MOOC for lifelong education and the participants’ level of knowledge.

At the end of the course, participants were surveyed about their perception of ease of use, usefulness, and the intention to use this massive course in continuing training. Additionally, before starting and after completing the course, participants completed a questionnaire to assess their level of knowledge.

Sampling process

A non-probabilistic convenience sample of Portuguese healthcare professionals was used, with those voluntarily enrolled in the MOOC, made available in the OpenEDX open-source learning management system platform.

Of the 2629 participants enrolled in the MOOC between March and October 2021, 66.6% were female, and participants were aged, on average, 36.7 years (SD ± 11.5).

Data collection

In the data collection process, two questionnaires were applied: one to assess the perceived ease of use, usefulness, and the intention to use a MOOC, and another with true-false questions to assess the initial and final level of knowledge.

The assessment questionnaire used for data collection was based on the ‘Davis Technology Acceptance Model’ (Davis, 1989; Venkatesh and Davis, 1996) and the ‘Determinants of Perceived Ease of Use’ of Venkatesh (2000).

The ‘Davis Technology Acceptance Model’ offers a framework for evaluating the use of information systems based on three main variables: perceived ease of use, usefulness, and the intention to use the technology. Additionally, the quality and satisfaction with the MOOC experience were also assessed.

The ‘Determinants of Perceived Ease of Use’ focuses on the perceived ease of use and behaviour towards technology, proposing a theoretical model based on self-efficacy, intrinsic motivation, and emotional dimensions to understand the individual’s perceptions of ease of use of a system.

Based on these premises and dimensions, the questionnaire used in this study has been previously validated by other published studies (Padilha et al., 2012; 2018; 2020; 2021). The questionnaire was organised into two sections: the first section addressed the sociodemographic characterization, and the second section included 12 items (Table 2). Responses were rated on a five-point Likert scale, ranging from 1 (worst possible opinion) to 5 (best possible opinion).

Table 2 Questionnaire items.
Full size table

The questionnaire assessing the perceived ease of use, usefulness, and intention to use this type of resource in lifelong training reached a Cronbach’s Alpha coefficient value of 0.96 (n = 12 items) and item-total correlations ranging between 0.54 and 0.81.

A questionnaire with a set of 31 true or false questions was delivered to assess the level of knowledge. These questions were identified by the developers of MOOC content and were based on the best available evidence.

Data was collected by automated processes within the platform and exported in CSV format for subsequent normalization and redundancies elimination. The collected data encompassed sociodemographic characteristics, issuance of certificates, completeness, and initial and final questionnaire results.

Data analysis

Content validity was established by the research team. Following data collection, the second part of the questionnaire was tested for reliability by analysing internal consistency using exploratory factorial analysis and Cronbach’s Alpha. Student’s t-test was utilised for inferential analysis of the variables under study. Data analysis was performed by IBM SPSS Statistics – v.27 (IBM, 2020). Results are reported following the APA standards, indicating effect size measurements of the Cohen’s d (0.2 low; 0.5 medium, and 0.8 high) and considering a significance level of P < 0.05.

Ethical considerations

Authorization was obtained from the Ethics Committee of the Nursing School of Porto with the reference ADHOC_907/2021.

Results

In this study, of the 2629 participants enrolled in the MOOC, 32.2% (n = 847) completed the course and issued a certificate. Of these, 97.9% (n = 830) completed the questionnaire assessing the dimensions ‘Ease of use’, ‘Usefulness’, and the ‘Intention to use’. Regarding the level of knowledge, 93.9% (n = 795) completed the initial and final assessment.

The sample included 847 certified individuals and 1782 (67.8%) who did not completed the course nor issued a certificate upon completion of the course.

Among non-certified and non-completers, a detailed characterisation by age, gender, profession, and academic qualifications was not feasible since 1527 participants did not answer these questions, corresponding to 85.7% of the participants without certification and those who did not complete the course. However, no statistically significant differences were found in the socio-demographic characterisation between certified and non-certified respondents who completed the questionnaire.

The average age of respondents was 36.9 years (SD ± 11.5), most were female (63.5%). Regarding academic background, 77.8% had stated their educational qualifications, with 60.4% (n = 502) holding at least a higher education degree (Bachelor's/Licentiate's, Master's, or Doctorate Degrees). Additionally, 18.1% of individuals interested in the MOOC subject were operative and technical health professionals.

Concerning the respondents’ professional characterisation, 64.7% had disclosed their occupation. Among the overall sample, 34.1% were nursing professionals, 19.8% were physicians and other health professionals with higher education degrees, and 3.8% were health students.

Table 3 shows the sociodemographic and professional characterization of the participants.

Table 3 Sociodemographic and professional characterisation of the participants who issued a certificate.
Full size table

Evaluation of the perceived ease of use, usefulness, and the intention to use this resource in lifelong training

Table 4 provides descriptive statistics on the assessment of the perceived ease of use, usefulness, and the intention to use this type of resource in lifelong training.

Table 4 Evaluation of the perceived ease of use, usefulness, and the intention to use this resource in lifelong training.
Full size table

The exploratory factor analysis (KMO = 0.96, Bartlett’s sphericity test = 0), using the principal components extraction method with Varimax rotation and Kaiser criteria for the extraction of the components that explain the maximum variance, allowed identifying a factor explaining 69.4% of data variance, which aggregates all 12 items.

In the evaluation of the perceived ease of use, usefulness, and the intention to use this type of resource in lifelong training in the future and satisfaction with the course, global average values of 4.4 (SD ± 0.7) were observed, consistent with the score attained by participants ineligible for a certificate (4.3 SD ± 0.75). Inferential analysis of the data did not reveal any other statistically significant relationships between the variables under analysis for participants who completed the course and were issued a certificate.

In addition to assessing the perceived ease of use, usefulness, the intention to use, and satisfaction, other dimensions were assessed related to the learning experience (‘Content quality’, ‘Adequacy to the needs’, ‘Learning’ and ‘Overall assessment’) and the e-learning relevance and impact (‘E-learning relevance’, ‘E-learning professional updating’, ‘E-learning vs face-to-face’, and ‘Recommends the course to another person’). In these two groups, the overall values were also high, 4.3 (SD ± 0.7) for the learning experience evaluation category and 4.5 (SD ± 0.7) for the e-learning relevance and impact group.

Knowledge retention

Of the certified participants, 93.9% (n = 795) completed an initial knowledge assessment questionnaire, reaching an average score of 50% (SD ± 0.2; Max. = 84%; Min. = 0%). These participants obtained an average score of 88.8% (SD ± 0.9; Max. = 100%; Min. = 70) in all the evaluation tests of the course. The analysis of the average score of the initial and final assessment, per participant, revealed statistically significant differences (t795 = 58.5; P < 0.001; d = 0.19), confirming the effectiveness of the programme for knowledge retention.

Analysis of the participants who did not issue a certificate

Of the 67.8% (n = 1782) participants that did not issue a certificate nor completed the course, 2.8% (n = 50) obtained an average assessment of 80.40% (SD = ± 8.5%; Max. = 100%; Min. = 70%), which ultimately would allow them to issue a certificate of completion.

Among these non-certified participants, 70% were women, 78% held a Master’s or Bachelor’s degree, and had a mean age of 36 years (SD = ± 11.15). Additionally, 54% of those participants were nurses, 20% worked in other non-related health professions, 10% were senior health technicians, and 2% were physicians. The remaining 14% of the non-certified participants did not specify their profession.

This group of participants scored overall mean values of 4.5 (SD ± 0.72) on the evaluation of the perceived ease of use, usefulness, and the intention to use this type of resource in lifelong training in the future, indicating satisfaction with the programme.

Regarding the participants who did not meet the criteria to issue the MOOC certificate (65% of the sample), 10.4% (n = 185) reached an assessment between 50 and 69% and a mean of 61% (SD = ± 5.8%; Max. = 69%; Min. = 50%).

No statistically significant differences in age, training, and profession were found among non-certified participants compared to the participants eligible to issue the certificate of completion of the programme. In this group of participants, the evaluation of the dimensions perceived ease of use, usefulness, the intention to use, and satisfaction with the programme reached global average values of 4.3 (SD = ± 0.72).

Of the participants who did not issue a certificate, 86.9% (n = 1550) obtained a classification below 50%. Table 5 shows the distribution of assessments per module and the number of participants who completed each module of the training programme.

Table 5 Distribution of evaluations per module and the number of participants who completed modules of the programme despite having a final mark of less than 50%.
Full size table

Discussion

This study encompasses two major areas of analysis. The first focuses on formal aspects, namely certification, dropout rates, completion rates, and their relationship with sociodemographic characterisation. The second addresses knowledge retention and related trainees’ perceptions.

Albeit the scattered evidence on completion rates, data produced within the pandemic context suggests a decrease in certification rates (Yee et al., 2022). The study presents a completion rate of 32.2%, surpassing the rates typically reported in the literature ~12% (Roy et al., 2022). This highlights the importance of integrating practical and interdisciplinary content into MOOC programmes, e.g., collaborative, to improve completion rates, as stated by Guest et al. (2021).

Notably, the high completion rate observed in this study can be attributed to the increased need perceived by healthcare professionals rather than by students, as indicated by the low representation of nursing and medical scholars compared to the overall professions stated in the sample. This suggests an organic completion rate rather than a biased one by integrating MOOCs into formal learning embedded within academic curricula (Cha and So, 2020).

Given that the results did not show statistically significant differences between the variables and the demographic and professional profiles of the participants, these study findings suggest other explanations for the use of MOOCs in the lifelong training of healthcare professionals.

The participants who did not issue a certificate despite meeting the criteria may have been primarily interested in acquiring information applicable to clinical practice rather than obtaining a certificate. These participants reported significant satisfaction levels with the programme and high overall mean values in the evaluation of the perceived ease of use, usefulness, and the intention to use MOOCs in the future. Milligan and Littlejohn (2016) describe scenarios in which participants may attribute less emphasis on certification but remain engaged and complete their training based on personal indicators of success, demonstrating intrinsic motivation and the ability to self-regulate learning.

The diverse professional backgrounds and qualifications of this group of participants (nurses, physicians, and senior technicians) may explain this behavioural pattern. According to Zeng et al. (2015), this behaviour is presented in three of the four typified motivations for participating in a MOOC: meeting current needs, preparing for the future, and satisfying curiosity, all within the scope of a personal pursuit of knowledge.

Notably, participants with a greater capacity for self-regulation adopt a strategic approach and decide not to attend modules they consider less interesting because completing the course is not crucial for their success (Milligan and Littlejohn, 2016).

This evidence supported the existence of two major groups of participants in MOOCs: certificate achievers, those who issued a certificate, and explorers, those who did not issue a certificate, albeit meeting the criteria, as stated by Bowen et al. (2021).

In this sense, a self-directed learning behaviour emerges, aligning with Li's (2019) assertion that individuals with high qualifications, clear objectives, and a recognition of effective learning outcomes tend to report higher satisfaction levels with their learning journey. This finding is also corroborated by Alamri (2022), suggesting that MOOCs should adopt a learning-oriented structure and allow the assessment of acquired skills and competencies.

Participants who did not meet the conditions to issue a certificate illustrate the two groups of less productive participants, namely ‘Only registered’ and ‘Only viewed’ (Ho et al., 2014), accounting for 65% of all users.

These less prolific participants seem to struggle with meaningful self-regulation of their learning process. Despite regular engagement with the course material, they may find it too challenging and lack the ability or readiness to change their approach (Milligan and Littlejohn, 2016). These difficulties can be related to the complex network of internal factors (personal-psychological) and contextual factors (environmental) that interact in self-regulated learning (Littlejohn et al., 2015; Pintrich, 2020; Hendriks et al., 2020). Evidence suggests completion rates averaging below 10% (Huang et al., 2023).

The second dimension of analysis, related to knowledge retention, e.g., the effectiveness of the MOOC for upskilling or reskilling, is addressed by comparing the average completion score with the initial average score. This suggests a high level of active learners, as evidenced by a higher completion rate and improved course grades (Tseng et al., 2016) compared to the diagnostic assessment.

When assessing the perceived ease of use, usefulness, and the intention to use the MOOC for lifelong education in the future, the findings revealed high overall values (M = 4.4; SD ± 0.7) and similar values in satisfaction (M = 4.5; SD ± 0.7), in line with satisfaction levels evidenced in recent studies (Amit et al., 2022; Despujol et al., 2022; Gómez Gómez and Munuera Gómez, 2021; Martín-Valero et al., 2021; Yilmaz et al., 2021).

In this study, ‘Perceived ease of use’ scored higher, which is a crucial characteristic in the acceptance process of information technology (Davies, 1989), such as MOOC (Cabral et al., 2023). ‘Ease of use’ supports trainees, retention, and engagement with the content, ultimately influencing perceived usefulness (Kim and Song, 2022; Padilha et al., 2024).

Similarly, perceived usefulness is bidirected since it relates to improving both professional and knowledge retention. In sum, usefulness in MOOC context refers to the utility of these programmes to gaining or upgrading skills (Mohan et al., 2020), leading to greater satisfaction with the learning experience (Despujol et al., 2022).

The intention to use the knowledge addressed in the future also scored high. Evidence has shown the motivations for future intention to use (Kim and Song, 2022) with significant relation to the perceived need for reskilling, habitual engagement with technology, hedonic motivations, experimental value, and interest in digital self-learning (Mohan et al., 2020).

The context and the contents have encouraged trainees to reuse those contents in the future. The context of individual distancing and the challenges faced by health professionals in managing high workloads during this period prompted the use of digital technologies for training.

On the other hand, the content focused on the reskilling of professional abilities to combat the new disease, and the integration of an immersive simulator introduced elements of fun and amusement, typically of hedonic stimuli, along with the experimental value.

The dimension ‘Satisfaction’ scored the highest. Considering its critical role in trainees’ engagement, retention, and future use (Almufarreh and Arshad, 2023), it brings together a set of previous dimensions, driven by the ease of use of the platform, usefulness for reskilling, and intention to use in the future.

The dimensions addressed in the ‘learning experience evaluation’ and ‘e-learning relevance and impact’ groups reflected the overall quality and effectiveness perceived by trainees, further reinforcing the perceived satisfaction with their learning experience.

Limitations

The main limitations of this study were (1) the impossibility of making a prospective assessment of knowledge retention and (2) the participants’ inability to access the certificate by module, which would provide further data to assess the compelling interest in the modular contents provided.

Conclusion

This study demonstrated the intention of healthcare professionals to use MOOCs in lifelong learning since they are easy to use and useful in keeping professionals updated on advances in medical areas and thus ensure the safety and quality of healthcare delivery.

This study brings significant contributions to the state of the art in two major dimensions: new contributions and the reinforcement of knowledge already acquired.

Regarding new contributions, the paper shows the effectiveness of this MOOC for the promotion of knowledge retention amid a pandemic, demonstrating significant differences between the scores of knowledge before and after the completion of the course’s modules (50% to 88.8%).

Additionally, despite the typically high dropout rates influenced by the difficulty of self-regulation of learning, this MOOC exhibited a notably higher completion rate compared to those reported in the literature. This highlights the importance of implementing a learning experience tailored to reskilling based on practical, collaborative, and dynamic content.

This study contributes to the existing literature by demonstrating that MOOCs developed collaboratively by higher education institutions and leading healthcare professionals in the field, designed with a modular, practical, and interactive approach, and incorporating immersive learning experiences such as virtual patient simulators, can enhance completion rates, learner satisfaction and knowledge retention.

Moreover, this study contributes to supporting existing knowledge since it unveiled significant levels of perceived usefulness and ease, reinforcing the importance of these courses for the continuous update of health professionals.

Furthermore, the findings underscore the high levels of satisfaction with the perceived ease of use, usefulness, and intention to use content; therefore, suggesting the crucial contribution of these dimensions to learners’ retention and engagement.

In sum, the findings demonstrate that MOOCs are useful in the lifelong training of health professionals and are easy to use. They also highlight the professionals’ intention to pursue this training in lifelong learning and its effectiveness for knowledge retention, irrespective of whether participants completed the entire course or only attended it partially. This may be explained by the fact that not all participants sought to obtain all the content available in the course, but only the topics of the lessons that were more relevant to them professionally.

In future similar programmes, it would be interesting to evaluate the opportunity and effectiveness of offering participants modules perceived as interesting. This approach could facilitate the creation of modular certificates since the flexibility of MOOCs may be particularly advantageous for participants with conscientious learning practices.

In the development of MOOCs for lifelong training of healthcare professionals, some recommendations based on the current findings are highlighted: courses should include a (1) modular structure using both (2) practical content and interactive resources, and (3) digital immersive simulators for development of clinical decision-making based on digital patients.

Additionally, (4) contents must be presented by leading professionals with different clinical backgrounds, (5) certified by institutions valued by participants, and (6) enabling professional and academic accreditation, namely in a micro-credential format.

The acceptance and success of the learning experience should be supported by the (7) easiness of use of the technological tool and by a (8) personalised experience considering the different participants´ motivations and interests, supported by (9) guided self-learning tools for supporting participant’s self-regulation during the course.