Struggle in the bubble – a prospective study on the effect of remote learning and distance education on confidence in practical surgical skills acquired during COVID-19 | BMC Medical Education

Demographics

A total of 232 out of 244 medical students completed both the baseline and follow-up questionnaire-based surveys, resulting in a response rate of 95{e4f787673fbda589a16c4acddca5ba6fa1cbf0bc0eb53f36e5f8309f6ee846cf}. Demographic data was comparable between the two cohorts (Table 1).

Improvement in self-confidence for unit 1

First, it was evaluated whether the respective teaching methods in both cohorts resulted in an improvement in the self-confidence of students regarding their surgical skills. While analyzing unit 1 (sterile working), we found that both the COV-19 (Fig. 2A) and postCOV-19 (Fig. 2B) cohorts showed significant improvement in post-course confidence compared to pre-course confidence. This result was observed for all five subcategories of unit 1 (Table 2).

Self-assessment comparing pre- and post-course confidence of COV-19 and postCOV-19. Spider web graphs displaying the difference between pre- (full line) and post- (dotted line) course self-assessment. Unit 1 (sterile working): A (COV-19) + B (postCOV-19); unit 2 (knot tying and skin suturing): C (COV-19) + D (postCOV-19); unit 3 (history and physical): E (COV-19) + F (postCOV-19). COV-19 = cohort of summer semester 2021 (full COVID-19 restrictions), postCOV-19 = cohort of winter semester 2021/2022 (reduced COVID-19 restrictions)

Improvement in self-confidence for unit 2

While analyzing unit 2 (knot tying and skin suturing), we observed that both the COV-19 (Fig. 2C) and postCOV-19 (Fig. 2D) cohorts exhibited significant improvement in post-course confidence compared to pre-course confidence. This result was similar for all five subcategories of unit 2 (Table 3).

Improvement in self-confidence for unit 3

Upon analyzing unit 3 (history and physical), we identified that both, the COV-19 (Fig. 2E) and postCOV-19 (Fig. 2F) cohorts, revealed significant improvement in post-course confidence compared to pre-course confidence. This result was observed for all three subcategories of unit 3 (Table 4).

Having established that both the traditional interactive face-to-face hands-on courses and the newly developed interactive remote learning courses were able to significantly improve the confidence of medical students regarding basic surgical skills, it was necessary to determine the course that resulted in a higher difference between the pre- and post-course confidence and the subgroup of students that would benefit the most from a particular teaching method. Subgroup analysis was performed based on sex (male/female), age group (19–22 years/23–29 years/≥30 years), and prior surgical experience (with and without prior surgical experience) for evaluating the difference between the pre- and post-course self-assessment (Δ self-assessment).

Subgroup analysis

Sex

The cohorts were first stratified based on the sex (male or female) of the participants, and the subgroup that benefited the most from a particular learning method was determined. For unit 1, the mean Δ self-assessment in the COV-19 cohort was significantly higher in male students (1.96) than in female students (1.44) (p = 0.0003). However, in the postCOV-19 cohort, the mean Δ self-assessment was significantly higher in female students (1.57) compared to male students (1.29) (p = 0.0372) (Fig. 3A).

Subgroup analysis comparing pre- and post-course self-assessment (Δ self-assessment). A subgroup (sex: male vs. female) analysis for differences in Δ self-assessment, B) subgroup (age: 19–22 years vs. 23–29 years vs. ≥ 30 years) analysis for differences in Δ self-assessment, C) subgroup (prior surgical experience: with vs. without surgical experience) analysis for differences in Δ self-assessment, D) analysis for differences in Δ self-assessment comparing COV-19 vs. postCOV-19. Data are presented as mean and compared using Student’s t-test or ANOVA. A p-value less than 0.05 was considered statistically significant. Significance is indicated by the following symbols: * p < 0.05, ** p < 0.01, *** p < 0.001, **** p < 0.00001, ns = not significant. COV-19 = cohort of summer semester 2021 (full COVID-19 restrictions), postCOV-19 = cohort of winter semester 2021/2022 (reduced COVID-19 restrictions)

For unit 2, the mean Δ self-assessment in the COV-19 cohort was significantly higher in male students (2.59) compared to female students (2.16) (p < 0.0001), whereas no significant difference between males (1.92) and females (2.01) was observed in the mean Δ self-assessment in the postCOV-19 cohort (p = 0.0813) (Fig. 3A).

Nonetheless, for unit 3, we found that the mean Δ self-assessment was comparable between the female and male groups in both cohorts (Fig. 3A).

Age

The two cohorts were stratified based on age, which resulted in three subgroups: 19–22, 23–29, and ≥ 30 years. For unit 1, we found that the mean Δ self-assessment in the COV-19 cohort was the highest for the participants in the age group of 23–29 years (mean Δ self-assessment = 19–22 years: 1.51; 23–29 years: 1.82; ≥30 years: 1.42). Furthermore, the mean Δ self-assessment was significantly higher in students of ages 23–29 years compared to those in the age group of 19–22 years (p = 0.0234). However, no significant differences in the mean Δ self-assessment were observed between the subgroups 19–22 years and ≥ 30 years (p = 0.8443), as well as the subgroups 23–29 years and ≥ 30 years (p = 0.0761).

By contrast, the mean Δ self-assessment of unit 1 did not vary significantly between different age groups in the postCOV-19 (mean Δ self-assessment = 19–22 years: 1.58; 23–29 years: 1.33; ≥30 years: 1.23) cohort (Fig. 3B).

Considering unit 2, we determined that the youngest (19–22 years) subgroup exhibited the maximum improvement in self-assessment for the COV-19 and post-COV19 cohorts. In the COV-19 cohort, the mean Δ self-assessment was significantly higher in the subgroup with participants aged 19–22 years compared to the subgroup with participants aged 23–29 years (p = 0.0017). However, there was no significant difference between the subgroups with participants aged 19–22 years and ≥ 30 years (p = 0.4096), as well as the subgroups with participants aged 23–29 years and ≥ 30 years (p = 0.5073).

In the postCOV-19 cohort, the mean Δ self-assessment was significantly higher in the subgroup with participants aged 19–22 years compared to the subgroups with participants aged 23–29 years (p = 0.0020) and ≥ 30 years (p = 0.0017). In contrast, there was no significant difference observed between the mean Δ self-assessment of the subgroups with participants aged 23–29 years and ≥ 30 years (p = 0.2499) (Fig. 3B).

Upon analyzing unit 3, the mean Δ self-assessment in the COV-19 cohort was significantly higher in the youngest students (19–22 years) compared to the subgroup with participants aged 23–29 years (p = 0.0061) in COV-19. However, there was no significant difference in the mean Δ self-assessment between the participants aged 19–22 years and ≥ 30 years (p = 0.0934) and 23–29 years and ≥ 30 years (p = 0.9923).

Nonetheless, for unit 3, the mean Δ self-assessment was significantly higher in the subgroup with participants aged ≥30 years compared to subgroups with participants aged 19–22 years (p = 0.0224) and 23–29 years (p = 0.0181) in the postCOV-19 cohort (mean Δ self-assessment = 19–22 years: 1.73; 23–29 years: 1.68; ≥30 years: 2.35). However, no significant difference was noted in the mean Δ self-assessment of subgroups with students aged 19–22 years and 23–29 years (p = 0.9332) in the postCOV-19 cohort (Fig. 3B).

Prior surgical experience

Lastly, the two cohorts were stratified based on prior surgical experience. Students without prior surgical experience showed a significantly higher improvement in their self-assessment of post-course confidence compared to pre-course confidence. This result was found for unit 1 and 2 in the COV-19 (unit 1 = mean Δ self-assessment with surgical experience: 0.58; without surgical experience: 1.74; p < 0.0001; unit 2 = mean Δ self-assessment with surgical experience: 1.65; without surgical experience: 2.14; p < 0.0001) and postCOV-19 cohorts (unit 1 = mean Δ self-assessment with surgical experience: 0.77; without surgical experience: 1.57; p < 0.0001; unit 2 = mean Δ self-assessment with surgical experience: 1.15; without surgical experience: 2.10; p < 0.0001).

However, for unit 3, we observed that the mean Δ self-assessment did not vary significantly between students with and without prior surgical experience in the COV-19 cohort (mean Δ self-assessment with surgical experience: 1.21; without surgical experience: 1.09; p = 0.2242) but was significantly higher for students without surgical experience in the postCOV-19 cohort (mean Δ self-assessment with surgical experience: 1.19; without surgical experience: 1.89; p < 0.0001) (Fig. 3C).

To summarize, the mean Δ self-assessment was the highest in the young (19–22 years) male students without surgical experience in the COV-19 cohort and young (19–22 years) and elderly (≥30 years) female students without surgical experience in the postCOV-19 cohort.

Finally, we compared the mean Δ self-assessment of both cohorts using each unit. Both, the COV-19 (Δ self-assessment: 1.58) and postCOV-19 (Δ self-assessment: 1.46) cohorts showed comparable (p = 0.1485) results for unit 1. For unit 2, the mean Δ self-assessment was significantly (p < 0.0001) higher in the COV-19 cohort (Δ self-assessment: 2.26) compared to the postCOV-19 (Δ self-assessment: 1.98). In contrast, for unit 3, the Δ self-assessment was significantly (p < 0.0001) higher in the postCOV-19 cohort (Δ self-assessment: 1.76) compared to the COV-19 cohort (Δ self-assessment: 1.1) (Fig. 3D).