Introduction

‘E-learning’ is an educational distance-based approach that adopts electronic devices to lecture1. E-learning can be provided asynchronously, synchronously, hybridly, or in a mixture1,2,3. In asynchronous classes, students schedule their learning path and timing to access pre-recorded lectures, readings, assignments, etc., with no simultaneous educator-student presence4. In synchronous classes, the teacher and students are connected simultaneously5. In hybrid courses, students are divided into two groups, one in the classroom and one remotely connected by video conferencing, attending the same lecture simultaneously6,7.

E-learning has become increasingly important during the COVID-19 pandemic, as all schools at various levels were forced to close8,9,10. While the pandemic restrictions have significantly reduced, universities have substantially invested in digital technology to facilitate online learning within their organisations. Online education has several advantages: low costs, high flexibility, and allowing students to attend classes remotely11,12. Considering these benefits, examining what aspects of online learning can be retained is crucial. So far, different studies have investigated e-learning’s efficacy in Physiotherapy students’ knowledge learning13,14,15.

However, little is known regarding teaching practical skills via e-learning16,17. Practical skills are fundamental for physiotherapists and are usually taught face-to-face through clinical placements, laboratories, and seminars18,19. Among the different practical skills, chest physiotherapy and airway clearance techniques are paramount as part of the physiotherapists’ background20,21. In the last few years, chest physiotherapy has become increasingly important - even more so during the COVID-19 pandemic - thanks to the new knowledge in the field and the increased demands in the job market22. Therefore, adequate teaching of knowledge and practical skills in chest physiotherapy and airway clearance techniques is necessary23. The terms chest physiotherapy and airway clearance techniques are used similarly and interchangeably. Both terms define the treatments people with cardiorespiratory disease must perform if pulmonary re-expansion or airway clearance is to be achieved. Techniques include breathing exercises (such as autogenous drainage or deep breathing), the use of positive pressure instruments (such as the Acapella, PEP mask - Positive Expiratory Pressure) or other devices such as the in-exsufflator cough machine, or expiratory flow acceleration systems24,25.

Notwithstanding, little effort has been put into teaching strategies to teach chest physiotherapy, above all in e-learning. Only one study compared the effect of online (synchronous) versus classroom lectures on some manual cardiorespiratory physiotherapy skills (i.e., oxygen transport and manual hyperinflation manoeuvres), finding little to no differences between the two26. So far, no other studies have evaluated other e-learning methods, such as hybrid learning in chest physiotherapy. Moreover, students’ satisfaction with a course is fundamental for a thriving learning environment27. Therefore, this study evaluated physiotherapy students’ academic performance and satisfaction after following a hybrid classroom to teach airway clearance techniques during the COVID-19 pandemic.

Materials and methods

Study design

We conducted a retrospective case-control study during the academic years 2020/2021 and 2021/2022 at the Bachelor of Science (BSc) in ‘Physiotherapy’ at the University of Verona (Verona, Italy). We compared physiotherapy students’ academic performance at theoretical knowledge and practice tests after following a hybrid ‘Critical area and airway clearance techniques’ seminar designed following the restrictions imposed by the COVID-19 pandemic and their level of satisfaction with this seminar. Students attended the seminar, either face-to-face or remotely, in a hybrid learning mode. The study procedure followed the Declaration of Helsinki28. Ethics approval during the COVID-19 pandemic for this type of study was not required according to the ‘Ethics and data protection’ regulations of the European advisory body and European Commission29,30,31,32. Informed consent was obtained from all students at the beginning of the BSc program. Students understand that some of their learning data may be anonymised and used for research. This study is reported following the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) guideline33.

Study context

The physiotherapy BSc programmes in Italy last three years, resulting in 180 university credits (ECTs). Several physiotherapy disciplines, including chest physiotherapy, are taught during these years. In addition to the usual face-to-face lectures, the degree courses include practical activities such as laboratories, seminars, exercises and training with expert lecturers and tutors. At the University of Verona, a chest physiotherapy class is taught during the 3rd year. The present seminar on ‘Critical area and airway clearance techniques’ follows the frontal lectures on chest physiotherapy. This course is taught by an experienced physiotherapist with a Master’s degree in ‘Cardiopulmonary Rehabilitation, with 22 years of clinical experience and nine years of lecturing experience. Because of the COVID-19 pandemic, classrooms had limited seating capacity in Italy, allowing only half of the students to attend class lectures34,35. Therefore, some students participated in the lecture face-to-face, and others participated remotely simultaneously (hybrid).

Study population

To be considered eligible for this study and partake in the ‘Critical area and airway clearance techniques’ course, students had to be in their last year of the BSc of ‘Physiotherapy’ at the University of Verona (Verona, Italy), attend chest physiotherapy classes, passing its examination and be in the academic years 2020-21 and 2021-22. This seminar was mandatory. Hence, all students (41 students) were required to attend. The classes were conducted in a hybrid format due to the restrictions imposed by the COVID-19 pandemic, with students attending either face-to-face or remotely. The lecturer used Excel’s ‘RAND’ function to randomly divide the students into two equal groups: one attended remotely (the case group) and the other in person (the control group). This randomisation, achieved through an electronic random number generation system, was implemented to objectively determine attendance modes due to COVID-19 restrictions on the number of people allowed in a room. It was not part of a formal study design.

Critical area and airway clearance techniques course

The ‘Critical Area and Airway Clearance Technique’ course lasted six hours, all performed in one day. Three breaks (two of 15 min and one of 60 min) were planned during the day. During breaks, students attending face-to-face had to leave the classroom to avoid using the devices (i.e. Positive Expiratory Pressure such as Acapella) as those remotely could not do so. An expert in chest physiotherapy explains how to use techniques and devices for airway clearance. A camcorder was present in the class and pointed towards the lecturer. Online students were invited to keep their cameras on. The lecturer used a digital PowerPoint presentation to send to the students after the course. All students from both groups could interact anytime with the lecturer. For the practical part, due to the safety regulations given by the COVID-19 pandemic, only the teacher showed the students the manoeuvres to be performed. Students at home could see the manoeuvres from the camera. Students in the classroom could see them live. Each technique or device was shown once and for all its parts without interruption. Then, the same techniques were shown again, separated into steps and repeated without interruption. When the lecturer showed the techniques, an educational tutor played the role of the patient. After each demonstration of technique or device use, the lecturer showed a video to all students. The devices adopted and shown were 1 PEP mask, 1 Ez-PAP, and 1 Acapella. A Syllabus about the seminar explaining the learning objectives can be consulted in Supplementary Material 1. As a result of COVID-19 pandemic restrictions, people were not allowed to breathe into the same devices unsterilised between each person, and a distance of at least one meter was required. So, even face-to-face students did not try the techniques.

Data Collection

The lecture delivered the contents of the hybrid course, with some students remotely and others face-to-face (Time 0 – T0). At the end of the seminar (Time 1 – T1), all students had to fill out an online multiple-choice quiz, where only one answer was correct, to assess their theoretical knowledge (see ‘Outcome measure section’ for the technical characteristics of the quiz). The online multiple-choice quiz was available on the ‘Moodle platform’ that students and lecturers could access through the University in Verona (https://moodle.org/). Students could leave the classroom or disconnect from the meeting once the lecturer had received all the quiz results. The next day ((Time 2 – T2), 16 h after the end of the seminar, all students came to the university to perform the practical test. An expert educational tutor in chest physiotherapy evaluated practical skills. The tutor worked in a respiratory physiotherapy service and was trained by the lecturer on which devices the students saw in the lecturer and which techniques for bronchial clearance they studied. The evaluator did not know to which groups each student belonged (remote or face-to-face). Practice skills were assessed using a checklist (see ‘Outcome measure section’ for the technical characteristics of the checklist). The following day (Time 3 – T3), 24 h after T2, the seminar lecturer sent an online questionnaire on student satisfaction with remote and face-to-face teaching satisfaction (see ‘Outcome measure section’ for the technical characteristics of the questionnaire). Students had to return it within 48 h. The lesson and assessment schedule are reported in Fig. 1.

Figure 1
figure 1

Graphical representation of students’ evaluation timing divided into the groups of online and in-classroom students. Legend: T0, (Time 0) before the seminar, when the lecturer randomised the students; T1, (Time 1) at the end of the seminar; T2, (Time 2) the next day, 16 hours after the end of the seminar; T3, (Time 3) 24 hours after T2.

Full size image

Outcomes

The researchers extracted the data from Moodle on 1st October 2022. Anonymisation before data extraction was performed.

Primary outcomes

The primary outcome of this study was the students’ academic performance at the theoretical knowledge and practice assessments measured by a multiple-choice quiz (Supplementary Material 2) and a checklist (Supplementary Material 3), respectively.

Secondary outcome

The secondary outcome of this study was students’ satisfaction with the hybrid lecture.

Outcome measurements

Multiple-choice quiz (theoretical knowledge)

The multiple-choice quiz is a widely used assessment method in higher education36,37. It is considered a valid and standard tool for assessing students’ performance. It is divided into two parts: a question and different possible answers, one of which is correct36,37. For this study, the quiz included ten questions on airway clearance techniques or related devices, with a choice of four possible answers and only one correct. Question topics were taken from the lecture slides. The lecturer created the questions (and the slides) by consulting the primary reference literature on the course topic38,39,40,41. A score was associated with each question. Each correct answer was valued at 2 points. Each wrong answer is valued at 0 points. The multiple-choice quiz was scored on a scale of 0 to 20. The quiz score was combined with the score on a separate practice skills test (see the next paragraph), which was scored on a scale of 0 to 10 to determine the maximum overall score for the course evaluation according to the Italian University evaluation system42. Once the multiple-choice quiz was ready, the lecturer sent it to five Italian chest physiotherapists (two men and three women) for content, readability and feasibility review. They were all members of the Association of Rehabilitators of Respiratory Insufficiency (Associazione dei Riabilitatori dell’Insufficienza Respiratoria (Associazione dei Riabilitatori dell’Insufficienza Respiratoria - ArIR) and they all had a Master’s degree in ‘Chest physiotherapy’. These physiotherapists were all lecturers in chest physiotherapy. The experts approved the test, and no further changes were made.

Checklist (practice skills)

A checklist was used to assess students’ practice skills. Checklists are a primary tool for evaluating practical medical education skills37. From the literature, no checklist was found for practice skills in airway clearance techniques. Therefore, a new checklist was created ad hoc for this course based on other studies that adopted a checklist in physiotherapy education43,44. The checklist included five items. The checklist involved scoring with a standardised 3-point Likert scale where 2 means ‘correct action’, 1 ‘partially correct action’ and 0 ‘incorrect action’. The total score of the checklist ranged from 0 to 10. Then, this checklist was sent to three physiotherapists to see if it could be fair and adequate to evaluate students. They were three out of the five physiotherapists specialised in chest physiotherapy who assessed the multiple-choice quiz. The experts approved the checklist, and no further changes were made. The items on the checklist were explained to the students before the examiner assessed the practice skills.

Satisfaction questionnaire

After the course, the lecturer sent a satisfaction questionnaire on the Moodle platform. The students in both groups self-reported anonymously to what extent they were satisfied with this course with a 5-point Likert scale (1 ‘very dissatisfied’, 2 ‘dissatisfied’, 3 ‘neutral’, 4 ‘satisfied’, 5 ‘very satisfied’). Those who reported 1 and 2 were classified as dissatisfied with the course. Those who reported 4 and 5 were considered satisfied. Those who reported 3 were considered neither satisfied nor dissatisfied.

Statistical analysis

Descriptive statistics were computed to represent the characteristics of students (age, gender they identified with) using mean and standard deviation (SD) for continuous variables and absolute and percentage frequencies for categorical ones. The analysis was performed using Stata 17 (StataCorp. 2021). Data are available from the corresponding author upon reasonable request.

Multiple-choice quiz and practical skills checklist

Data did not follow a normal distribution after visually inspecting q-q plots and histograms. Therefore, a quartile regression test (stata function ‘qreg’) was used to see the median differences between the two groups and 95% confidence intervals (CIs). We used bootstrapping (500 replications) to compute 95% CI, calculating the Cohen’s d effect size of this intervention. We calculated the median differences, the effect sizes and their 95% CIs for both the multiple-choice quiz (20 points), the checklist (10 points) and the overall results of both (30 points).

Satisfaction questionnaire

The closed-ended question was analysed. Absolute and percentage frequencies were calculated on the total population and between the two groups.

Results

A total of 41 students from 2020/2021 to 2021/2022 academic years were included, 22 (54%) men and 19 (46%) women, with a mean age of 23 ± 2.1 years (Table 1).

Table 1 Descriptive characteristics of the sample.
Full size table

Primary outcomes – students’ performance

The scores for the multiple-choice quiz and the checklist are reported in Table 2. The total score presented respectively a median of 19 [Q1-Q3; 15-21.5] in the remote group and 24.5 [22.25–26.75] in the face-to-face one, with a median difference of 6 [95%CI 2.89;9.10] in favour of face-to-face lecture and an effect size of 1.48 [95%CI 0.72;2.23]. However, different results were found between the multiple-choice quiz and the checklist. At the multiple-choice quiz, the remote group presented a median of 14 [12–16] and the face-to-face group of 16 [14.5–18.], with a median difference of 2 [95% CI -0.24; 4.24] in favour of the face-to-face one and an effect size of 0.61 [95%CI -0.07;1.29]. At the checklist, the remote group presented a median of 5 [3–7] and the face-to-face of 8.5 [8–10], with a median difference of 4 [95%CI 2.31;5.68] in favour of the face-to-face group, and effect size of 2.05 [95%CI 1.35;2.76].

Table 2 Scores at the tests.
Full size table

Secondary outcome – satisfaction questionnaire

One student did not complete this part, so 40 returned their satisfaction score (98%). Among the 19 face-to-face students who answered the question, 18 were satisfied, and only one was dissatisfied. Among the 21 remote students, 14 were satisfied, three were dissatisfied, and four were dissatisfied. In general, 32 students were satisfied with this hybrid class (80%).

Discussion

This study evaluated physiotherapy students’ academic performance and satisfaction after following a hybrid classroom to teach airway clearance techniques. Once we considered the total score on the test (both multiple-choice quiz and checklist), the academic performance of those who attended this seminar face-to-face was better than that of those remotely, even if both groups averagely scored above the fail mark. However, different considerations were made once the scores were split between theoretical knowledge and practice skill assessments.

Regarding knowledge, we could not affirm face-to-face learning was better than remote learning. Drawing comparisons with previous evidence is daunting as past studies focused on e-learning (synchronous or asynchronous) versus face-to-face separately and not on hybrid methods. One study reported online learning worse than face-to-face classes45. Other studies said that remote e-learning was higher or at least similar to face-to-face learning in terms of students’ learning11,13,14,43,44,45. Jones et al. (2010) conducted a comparative study to assess the effectiveness of e-learning versus traditional face-to-face classes26. Their study focussed on evaluating the knowledge of cardiopulmonary rehabilitation topics among students from Hong Kong and Canada, with both groups performing similarly regarding manual hyperinflation in mechanically ventilated patients26. However, a notable difference was observed in their knowledge of oxygen transport. The face-to-face students from Hong Kong achieved significantly higher scores in this area than their e-learning counterparts26. Taking our and others’ results, we might argue that hybrid methods, frontal lectures and synchronous/asynchronous e-learning courses might lead to similar results when learning theoretical concepts.

Then, considering the practical skills learning, we observed that students in the face-to-face group scored better than their remote counterparts. Also, in this case, we do not have specific studies on hybrid methods but on e-learning against face-to-face. A few studies reported greater effects of hands-on learning in favour of online teaching46,47. In another study by Moore et al., they found no difference in learning practical skills between online or face-to-face modality, considering the two approaches similar48. Therefore, the three modalities (hybrid, synchronous/asynchronous e-learning, and face-to-face) may affect students differently when teaching practical skills. Specifically, in our study, no group could practice airway clearance techniques before the evaluation. Consequently, we anticipated that remote teaching would be at least non-inferior to face-to-face instruction. However, our results suggest that face-to-face students performed better than those in other learning modalities. The only study evaluating chest physiotherapy supports that face-to-face students in Hong Kong did better because they were more used to class lectures26. This might have occurred even in our students, who were more used to traditional face-to-face lessons. Other studies that assessed manual skills in physiotherapy have not studied the hybrid mode, and remote lectures were conducted asynchronously, with more time available to the students before the evaluation than our study47.

Finally, when it came to students’ satisfaction, our questionnaire showed that students were satisfied with this hybrid teaching mode with 80% of the responses. The remaining 20% were equally divided between those who were unsatisfied and those who were neutral. Other studies confirm this result on mixed-mode lecture satisfaction49. Specifically, for physiotherapy students, no evidence investigated students’ satisfaction with hybrid methods. Other studies on e-learning showed that most physiotherapy students appreciated a blended teaching model compared to traditional face-to-face classes1,50,51. Conversely, no differences were found among those who attended only online or in-person classes. The reason behind the appreciation of mixed courses is that they can preserve the pros of both methods while reducing their cost52. Specifically, socialisation is more present during face-to-face courses, while during online classes, students perceived a greater sense of independence and time self-management6.

This study suggested that our cohort appreciated hybrid lessons. Hybrid classes could be a helpful alternative for theoretical lectures that do not involve acquiring manual skills, as students can choose to attend in person. Hybrid classes benefit workers or economically disadvantaged students by reducing travel and accommodation costs near campus. Therefore, hybrid learning can be considered an additional educational approach, combining the advantages of online learning and face-to-face components. This conclusion does not extend to teaching hands-on activities. The literature on this topic needs to be improved, particularly in chest physiotherapy.

This study presents several limitations. Firstly, a small number of people were evaluated due to our convenience sample. Future studies should adopt other study designs (e.g., randomised controlled trials) and adequate sample power. Also, the design of this study (observational) is another limit. Moreover, we have only short-term follow-up and cannot test students’ information retention in the long run. Finally, the tools used for students’ evaluation and final scores were constructed and defined ad hoc for this study without psychometric testing since validated outcome measures are yet to be carried out in this specific educational field (i.e., chest physiotherapy). However, these tools were created based on the literature, and specialists in chest physiotherapy were involved37. Finally, a limitation of this study is that no students practised the taught techniques and used the devices due to COVID-19 pandemic restrictions. All face-to-face and remote students only observed the manoeuvres performed by the lecturer.

Conclusions

Our study found that the hybrid seminar on ‘Critical Area and Airway Clearance Techniques’ appeared to disadvantage students who learned from home. This issue was more pronounced in the acquisition of practical skills than in theoretical knowledge. Despite the differences between the groups, the mean scores indicate that this method allowed students to acquire the necessary knowledge during the COVID-19 pandemic. This outcome would not have been possible otherwise due to the restrictions induced by the pandemic. Further studies are needed to understand the real potential of this teaching method.