Effectiveness of Simulation-based Clinical Skills Training for Medical Students in Respiratory Medicine:
A Pilot Study

By Russell Seth Martins¹, Saniya Raghib Sabzwari², Meesha Iqbal³

Affiliations

1. Department of Medical College, The Aga Khan University Hospital, Karachi, Pakistan
2. Department of Family Medicine, The Aga Khan University Hospital, Karachi, Pakistan
3. Independent Researcher, Ammon ID, USA

doi: 10.29271/jcpsp.2021.12.1468

ABSTRACT
Objective: To assess the effectiveness of high-fidelity simulation-based medical education (HF-SBME) in teaching and learning respiratory clinical examination in medical students.
Study Design: Quasi-experimental pilot study.
Place and Duration of Study: The Aga Khan University, Karachi, from November 2018 to January 2020.  
Methodology: This study was conducted amongst third year medical students at the University. Students were assigned to intervention (IG) or control groups (CG). The IG underwent training for the respiratory clinical examination on a high-fidelity simulator mannequin, while the CG received the conventional practice session on standardised patients. Students were assessed on their respiratory clinical examination skills in five domains, and each domain was scored between 1-3 points (poor=1, fair=2, good=3) for a maximum composite score of 15. Feedback on use of SBME was also obtained from students.
Results: There were no statistically significant differences in demographics for the CG (n=41) and IG (n=40). Composite score for control and intervention groups was not significantly different (CG: 12.9 ± 1.89 vs. IG: 12.0 ± 2.35; p=0.067). However, a greater percentage of CG students were rated good in all five domains, with the difference being statistically significant for ability to correlate findings with clinical history (CG: 87.8% vs. IG: 67.5%; p=0.028).
Conclusion: Although medical students perceived HF-SBME as a beneficial teaching modality, it did not translate into improved performance. More research is required to determine the utility of HF-SBME in a developing country, like Pakistan.

Key Words: Simulation, High-fidelity, Medical education, Developing country, Clinical skills.

INTRODUCTION

Simulation-based medical education (SBME) is well established in education systems of developed countries. High-fidelity simulators have been successfully used to teach and assess clinical and basic skills in the respiratory and cardiovascular systems.^1-3

In developing countries, the lack of advanced facilities, expensive equipment and experienced operators represent significant barriers to the assimilation of SBME as a routine part of medical education.⁴

Pakistan, a developing country, relies primarily on didactic lectures, simulated (standardised) patients and small-group clinical sessions for clinical skills education in medical school.⁵ However, students report dissatisfaction with current methods of teaching clinical skills at the patient’s bedside.⁶ There have been limited studies exploring the effectiveness of SBME. One study used an intermediate-fidelity simulator to teach normal vaginal delivery; and reported significantly higher scores using SBME as compared to traditional methods.⁷

In Pakistan, there is mainly use of low- or intermediate-fidelity simulators; and therefore, a lack of research on the effectiveness of high-fidelity simulators for SBME. The Center for Innovation in Medical Education (CIME) at the Aga Khan University (AKU) is the first simulation centre in South-Asia to be accredited by the American-based Society for Simulation in Healthcare.

This study was designed to determine the effectiveness of high-fidelity SBME in teaching/learning respiratory clinical examination to third-year medical students of AKU, Karachi, and also to assess the attitude of medical students towards SBME.

METHODOLOGY

A quasi-experimental pilot study was conducted among third-year medical students at the Aga Khan University (AKU). This study was a pilot project within this clerkship to offer respiratory clinical skills learning on a high-fidelity mannequin. It was conducted from November 2018 to January 2020 with approval from the Ethical Review Committee of the University.

The sample size was calculated for comparing mean performance scores between two groups (ratio of intervention group to control group 1:1). With an assumed mean difference of 2.5 points and standard deviation of 3, and at 95% power and 95% CI, the sample size was calculated to be 38 each for the control and intervention arm. The sample size was inflated to include minimum 40 participants in each arm.

The study participants were assigned in a 1:1 allocation ratio to either the control or the intervention group, based on the pre-assigned clinical rotation groups to evaluate the effectiveness of SBME. The clinical rotation groups are randomly formed by the curriculum office and not made on the basis of any socio-demographic or academic criteria. Thus, using existing clinical rotation groupings ensured random allocation to control and intervention groups while permitting logistical convenience.

The control group received the usual medical education, i.e. an interactive discussion session elaborating the theoretical basis and steps of clinical examination, common pathologic presentations of the respiratory system, correlation of clinical findings, and hands-on practice on standardised patients.

The students of the intervention group received the same training by the same faculty members as the control group and underwent an additional training session on a high-fidelity simulator mannequin to practice techniques of respiratory system clinical examination. This was a 30-minute session that included auscultation of normal and pathological findings (normal vesicular breathing; added lung sounds, wheezing, rhonchi and crepitations) that was followed by a discussion of clinical relevance to underlying disease. This simulation session was conducted by a senior instructor, an assistant professor, and associate professor of family medicine at AKU.

All the participants of the study were individually assessed by formative examination at the end of the family medicine clerkship. The students were given a brief clinical scenario and were required to perform the relevant clinical examination. Examiners included an associate professor of family medicine, an assistant professor of family medicine and a senior instructor at AKU. The examiners objectively assessed the students (via a checklist) in the domains of (a) organisation, (b) correctness of technique, (c) ability to identify abnormal findings, (d) ability to correlate findings with clinical history, and (e) a global rating. This global rating was an overall assessment of the students’ performance of the clinical examination. To eliminate bias, the same examiners assessed both the control and intervention groups, keeping examination conditions uniform as well, i.e. exam sequence and time allocation. Each domain was scored on a 3-point Likert scale (1=poor, 2=fair, 3=good). Equal weightage was given to all domains and a composite score was built by adding the individual scores of each domain (maximum score of 15).

Examiners also recorded socio-demographic characteristics of the students, such as age, gender, and high-school (A-Levels/Other) on the scoring sheet. The students of the intervention group completed a self-administered anonymous feedback form to assess attitude towards SBME (Adapted and modified from Joseph et al., India, 2016).⁸

Family medicine clerkship scores were then obtained for both groups. The clerkship score is a sum of student scores obtained during ongoing observation during the clerkship by faculty members and the performance in the end-of-rotation summative OSCE.

The performance in each domain was first compared by applying the Pearson Chi-square test. For each domain, the categories poor and fair were grouped into a single category. The mean composite scores of the formative examination (/15) of both groups were compared. Then family medicine clerkship scores of the intervention and control group were compared for significant differences by applying the independent sample t-test. A p-value of less than 0.05 was considered statistically significant. All analysis were performed using SPSS version 22.

RESULTS

A total of 81 students, 41 in control and 40 in the intervention group, were included in this study. Mean student age was 20.0 years (SD: 0.7), and 48 (59.3%) of students were males. Most students had studied in A-Levels (n=68; 84%). No statistical difference across socio-demographic characteristics of control and intervention groups was seen.

On comparison of mean composite score, there was no difference between the control and intervention group (control group: 12.9 ± 1.89 vs. Intervention Group: 12.0 ± 2.35; p = 0.067).

The percentage of students performing poorly across domains was low (A: 1.2%; B: 11.1%; C: 14.8%; D: 16.0%; and E: 3.7%), the categories into poor/fair were merged. A greater percentage of students in the control group were rated good in each of the five domains. The difference in percentages was statistically significant for domain D, i.e. the ability to correlate findings with clinical history (control: 87.8% vs. Intervention: 67.5%; p = 0.028, Figure 1).

Figure 1: Percentage of students rated good in each domain.

Figure 2: Students attitudes towards SBME.

On comparison of the family medicine clerkship score, the control group obtained a significantly higher mean clerkship score (79.0% ± 4.15%) as compared to the intervention group (72.0% ± 5.93%) in the family medicine rotation (p <0.001).

Students’ attitude towards SBME was largely positive, with majority strongly agreeing/ agreeing that SBME was beneficial especially in increasing student confidence in clinical skills and providing a safe environment (Figure 2).

DISCUSSION

Although a regular feature in the developed world, SBME is only recently gaining interest in Pakistan, a developing country. Surprisingly, these results showed that the control group performed better than the intervention group in a demographically well-balanced cohort.

Although the findings of this study may seem counter-intuitive, they need not necessarily be taken at face value. The higher end-of-clerkship scores obtained by the control group indicate that this group may worked harder than the intervention group, as they perceived themselves as disadvantaged by missing out on the simulation session using the high-fidelity simulator mannequin.

This is not the first study to show less-than-favorable results for the use of high-fidelity simulation. Although several studies have shown the advantage high-fidelity simulation over low-fidelity simulation,^9-11 increasing evidence shows that both are essentially comparable in terms of knowledge acquisition and skill.^12-17 Indeed, medical students training with high-fidelity simulation may sometimes have significantly poorer performance than those training with low-fidelity simulation.¹⁸ One disadvantage of high-fidelity simulation is misplaced overconfidence in self-ability and skill.^18,19 The overconfidence that accompanies training on high-fidelity simulators is something of a hazard, with possible negative impact on patient healthcare that overconfidence in abilities may have.^18,20

Nevertheless, attitudes towards SBME amongst the students in the intervention group were generally positive and similar to those seen in the study by Joseph et al.⁸ Most student-perceived benefits of SBME lay in its capacity for improving clinical skills and increasing patient safety and comfort. The chief drawback perceived was deterioration of soft skills, such as empathy and communication skills. Around 57.5% students believed that SBME could reduce students’ communication skills, similar to a study by Joseph et al.⁸ Moreover, 45% felt that the use of SBME could reduce students’ empathy towards patients, also reported by Joseph et al.⁸

In a developing country, like Pakistan, cost is an important factor when considering incorporation of SBME into medical curricula. Therefore, judicious planning is required prior to acquisition and use of such technology.

There are a few limitations of this study. Being a single centre study, the findings of our study may not be generalisable to other medical institutions across Pakistan. Moreover, this study used only a respiratory system simulation mannequin, and thus findings may not necessarily be generalisable to other simulations. Lastly, student clinical examination assessment was subjective to the examiner, and more objective methods of assessment could have been employed. There is a need for multi-centre randomised control trials conducted with a wider variety of simulation exposures to provide deeper insight.

CONCLUSION

Though medical students demonstrate positive attitudes towards simulation-based education, high-fidelity simulation may not necessarily bring about significant increases in performance. Thus, keeping in mind the costs involved, the utility of high-fidelity simulation in a developing country like Pakistan is questionable.

ETHICAL APPROVAL:
Ethical approval was received from the Ethics Review Committee of the Aga Khan University, Pakistan (Reference Number: 2019-1856-5300).

PATIENTS’ CONSENT:
Informed consent was obtained by means of a consent form administered to participants prior to their inclusion in the study. It explained the nature of the study, the extent of their involvement in the study, risks and benefits, and use of their academic performance data and survey responses for research purposes and publication.

CONFLICT OF INTEREST:
The authors report no conflict of interest.

AUTHORS’ CONTRIBUTION:
SRS: Conceptualized and designed the study, executed its methodology, and coordinated study proceedings.
RSM, MI: Performed data synthesis and analysis.
RSM, SRS: Wrote the manuscript.
All authors critically reviewed the manuscript and approved the final version.

REFERENCES

1. Euliano TY. Teaching respiratory physiology: Clinical correlation with a human patient simulator. J Clin Monit Comput 2000; 16(5-6): 465-70. doi: 10.1023/a:1011401011887.
2. Ohrn M, Vanmeurs W, Good M, editors. Laboratory Classes-Replacing Animals With a Patient Simulator. Anesthesiology; 1995: Lippincott Williams & Wilkins Two Commerce Sq, 2001 Market St, Philadelphia.
3. Euliano TY. Small group teaching: Clinical correlation with a human patient simulator. Adv Physiol Edu 2001; 25(1):36-43. doi: 10.1152/ advances.2001.25.1.36.
4. Kim H. Experience of simulation-based training in a developing country. Simul Health C 2017; 12(3):202. doi: 10.1097/SIH. 0000000000000203.
5. Iqbal M, Khizar B, Zaidi Z. Revising an objective structured clinical examination in a resource-limited Pakistani Medical School. Educ Health (Abingdon) 2009; 22(1):209.
6. Tariq M, Motiwala A, Ali SU, Riaz M, Awan S, Akhter J. The learners' perspective on internal medicine ward rounds: A cross-sectional study. BMC Med Educ 2010; 10(1):53. doi: 10.1186/1472- 6920-10-53.
7. Shah N, Baig L, Hussain R, Aly SM. Simulation based medical education; teaching normal delivery on intermediate fidelity simulator to medical students. J Pak Med Assoc 2017; 67(10): 1476-81. Epub 2017/09/29.
8. Joseph N, Nelliyanil M, Jindal S, Utkarsha U, Abraham A, Alok Y, et al. Perception of simulation‑based learning among medical students in South India. Ann Med Health Sci Res 2015; 5(4):247-52. doi: 10.4103/2141-9248.160186.
9. Grady JL, Kehrer RG, Trusty CE, Entin EB, Entin EE, Brunye TT. Learning nursing procedures: The influence of simulator fidelity and student gender on teaching effectiveness. J Nurs Edu 2008; 47(9): 403-8. doi: 10.3928/01484834- 20080901-09.
10. Brydges R, Carnahan H, Rose D, Rose L, Dubrowski A. Coordinating progressive levels of simulation fidelity to maximise educational benefit. Acad Med 2010; 85(5): 806-12. doi: 10.1097/ACM.0b013e 3181d7aabd.
11. Conlon LW, Rodgers DL, Shofer FS, Lipschik GY. Impact of levels of simulation fidelity on training of interns in ACLS. Hosp Prac 2014; 42(4):135-41. doi: 10.3810/hp.2014. 10.1150.
12. Cheng A, Lockey A, Bhanji F, Lin Y, Hunt EA, Lang E. The use of high-fidelity manikins for advanced life support training — A systematic review and meta-analysis. Resuscitation 2015; 93:142-9. doi: 10.1016/j.resuscitation. 2015.04.004.
13. Finan E, Bismilla Z, Whyte H, Leblanc V, McNamara P. High-fidelity simulator technology may not be superior to traditional low-fidelity equipment for neonatal resuscitation training. J Perinatol 2012; 32(4):287-92. doi: 10.1038/ jp.2011.96.
14. Nimbalkar A, Patel D, Kungwani A, Phatak A, Vasa R, Nimbalkar S. Randomised control trial of high fidelity vs low fidelity simulation for training undergraduate students in neonatal resuscitation. BMC Res Notes 2015; 8(1):636. doi: 10.1186/s13104-015-1623-9.
15. Diederich E, Rigler S, Mahnken J, Dong L, Williamson T, Sharpe M. The effect of model fidelity on learning outcomes of a simulation-based education program for central venous catheter insertion. Chest 2012; 142(4_MeetingAbstracts):535A-A.
16. Curran V, Fleet L, White S, Bessell C, Deshpandey A, Drover A, et al. A randomised controlled study of manikin simulator fidelity on neonatal resuscitation program learning outcomes. Adv Health Sci Educ Theory Pract 2015; 20(1):205-18. doi: 10.1007/s10459-014- 9522-8.
17. Matsumoto ED, Hamstra SJ, Radomski SB, Cusimano MD. The effect of bench model fidelity on endourological skills: A randomised controlled study. J Urol 2002; 167(3):1243-7.
18. Massoth C, Röder H, Ohlenburg H, Hessler M, Zarbock A, Pöpping DM, et al. High-fidelity is not superior to low-fidelity simulation but leads to overconfidence in medical students. BMC Med Educ 2019; 19(1):29. doi: 10.1186/ s12909-019-1464-7.
19. Wenk M, Waurick R, Schotes D, Wenk M, Gerdes C, Van Aken HK, et al. Simulation-based medical education is no better than problem-based discussions and induces misjudgment in self-assessment. Adv Health Sci Educ Theory Pract 2009; 14(2):159-71. doi: 10.1007/s10459- 008-9098-2.
20. Saposnik G, Redelmeier D, Ruff CC, Tobler PN. Cognitive biases associated with medical decisions: A systematic review. BMC Med Inform Decis Mak 2016; 16(1):138. doi: 10.1186/s12911- 016-0377-1.