Introduction  

Since the beginning of 2020, Lebanon has been assailed by an unprecedented economic and financial crisis, which was aggravated by COVID-19 lockdowns and the aftermath of the massive explosion at the Port of Beirut on August 4, 2020 [1]. The economic crisis has had a huge impact on all aspects of people’s lives, especially education. The difficulties of providing online teaching amid frequent power outages, Internet cuts, and tight budgets and disruptions caused by the COVID-19 pandemic were common problems and challenges for educators and students; the shortcomings of these problems still prevail [2].

At the Lebanese International University School of Pharmacy, a transitional plan of learning was implemented, whereby the pedagogical process shifted from regular classrooms to Google Classroom to match these new challenges. Educators and students alike had to adapt to a new normal, where the teaching and the learning processes where deviated from what once was the standard operating procedure to course delivery online via virtual platforms. Such challenges of course delivery are always more pronounced to meet the needed objectives with practical courses [3]. Among such courses, the Parenteral Dosage Forms course, which is offered to senior pharmacy students during their fifth year, is an example. This course introduces the students to all aspects of parenteral products and develops the knowledge and skills necessary to prepare sterile products safely and effectively under aseptic techniques following updated practice standards. Moreover, it aims to identify the behaviors and skills needed for compounding sterile products and review safety procedures and equipment employed in compounding cytotoxic drugs. This course is important for the establishment of some competencies needed by pharmacy graduates. According to the literature, an improved aseptic technique, as evidenced by a reduced rate of microbial contamination, was demonstrated by pharmacy students upon completion of a 1 semester credit hour course offered in the fall semester of the second-professional year [4]. In our curriculum, the assessment of Parenteral Dosage Forms course was done based on four exams; two theoretical exams (a midterm and a final), a project presentation, and a virtual Objective Structured Practical Examination (OSPE) to test the students’ practical skills of implementing the learned parenteral related techniques. Previously, fifth-year pharmacy students positively evaluated the OSPE as a method of assessing skills and reflecting on real situations in the pharmacy [5]. The FDA rejected buprenorphine/samidorphan for the treatment of major depressive disorder because of inadequate proof of efficacy [6].

Several experiences of electronic examination and evaluation were introduced after the COVID-19 pandemic [7, 8]. A pilot study in India showed that the newly developed e-OSPE system was a promising alternative to traditional OSPE, especially in the COVID-19 era [9].

This study aimed to evaluate the online teaching method for the Parenteral Dosage Forms course with an online e-OSPE to assess students’ practical skills in parenteral dosage forms, and also to evaluate students’ feedback.

Materials and Methods        

Study design

During the Parenteral Dosage Forms course in the spring semester of the academic year 2021-2022, the Parenteral Dosage Forms course at the Lebanese International University's School of Pharmacy adopted a virtual e-learning method using the Google Classroom platform. Accordingly, each section of the course had a dedicated Google Classroom created by the instructor, and all registered students were invited to join. The platform served as a hub for sharing course materials, assignments, announcements, and updates. Google Classroom was chosen as the method of delivery as it was available to all students through their university accounts using Google Workspace.

Teaching methods

The delivery of the material was agreed to be both synchronous (live meetings) and asynchronous (recorded lectures. The materials were uploaded on Google Classroom in advance for students to review, allowing them to pose questions and concerns to be addressed during the live sessions. The course materials were adapted to be more friendly for online delivery, incorporating videos, pictures, illustrative tools, and explanatory methods.

live online sessions with the students were conducted via the Google Meet application, which was recorded and shared with the students on Google Classroom for their reference, especially for those who could not attend the live streaming due to bandwidth or power supply limitations.

The practical demonstration of the aseptic compounding of a suspension for injection under a virtual vertical laminar flow hood (LFH) following USP 797 guidelines [10], was also done in a live online session that was also recorded and made available to students on Google Classroom. Then, a virtual OSPE had been adopted to evaluate objectively and reliably the practical skills of the students.

Assessment methods

The course assessment included online exams, a project presentation, and the virtual OSPE. The project presentation required groups of 3-4 students to share a novel topic related to a parenteral field not covered in the course. The OSPE aimed to objectively evaluate students' practical skills and was divided into three stations. The objective of the virtual OSPE was shared with the students ahead of time, and a complete live session was done to demonstrate the detailed procedure of performing the aseptic techniques of reconstituting a sterile powder for injection under a controlled clean environment within a virtual vertical laminar flow hood. The virtual OSPE was divided into 3 stations. In each station, a specific checklist was used for student evaluation and linked to the relative intended learning outcome (ILO). The School of Pharmacy follows a set of competencies divided into six domains, whereby students, upon completing the BPharm program, are expected to acquire:(1) Foundational knowledge; (2) Pharmaceutical care; (3) Essentials for practice and care; (4) Approach to practice and care; (5) Professionalism; and (6) Personal and professional development. These domains were developed by the School of Pharmacy and aligned with the national standards of pharmacy education set by the Ministry of Education and Higher Education (MEHE) and developed by the Lebanese Order of Pharmacists (OPL). Moreover, these competencies and program learning outcomes are selected based on the Accreditation Council for Pharmacy Education (ACPE) standards, the Center for the Advancement of Pharmaceutical Education (CAPE) outcomes, and the International Pharmaceutical Federation (FIP) framework [11-14].

With respect to the Parenteral Dosage Form course, the OSPE evaluates the ILOs from Pharmaceutical Care Domain as shown in Table 1.

During the virtual OSPE, students were asked to turn on their video cameras and microphones and to be prepared with all the supplies the student needs to complete the exam.

The OSPE stations were as follows

First station

In the first station, each student was asked 3 questions from a checklist (Table 2) by the instructor to evaluate his/her understanding of the terminology and the preparatory procedure for compounding a sterile product by the aseptic technique. This station accounted for 40% of the total grade.

Second station

In this station, the students were asked to reconstitute a lyophilized powder for injection according to aseptic preparation guidelines. The compounding procedure was started upon the instructor’s advice, during which the instructor evaluated the major aseptic steps and graded them accordingly based on another checklist (Table 2). The second station accounted for 30% of the total grade.

Third station

In the third station, students were asked to aseptically withdraw the reconstituted solution or suspension from a vial to prepare the final dose that is intended to be administered to the patient, and the instructor observed and graded the procedure based on a third checklist (Table 2). This station accounted for 30% of the total grade.

By the end of the semester, the aim was to evaluate the remote model of the Parenteral Dosage Forms course from students’ perspective, accordingly an evaluation tool for the course and the remote learning model was done by an end-of-course evaluation survey including 50 questions that were divided into 7 sections. This survey was anonymous and distributed electronically using Google Forms. The survey was based on the course objectives and learning outcomes and adopted to evaluate students’ perceptions and opinions regarding both the theoretical and practical parts using a Likert Scale of 1–5, (1 = strongly disagree and 5 = strongly agree). First, students’ background information was collected; then, feedback on online learning was evaluated. The different sections addressed students’ experiences, perceptions towards the virtual lectures, the practical session, development of skills, meeting the ILOs, satisfaction, and attitudes towards remote learning in the parenteral dosage form course. Three open-ended questions were added after the questionnaire to capture students’ additional comments. Categorical data based on Likert Scale was analyzed descriptively using frequency and percentage. Students’ feedback, which described their views and experiences of online learning was included.

Data analysis

The collected students' feedback, including their views and experiences of online learning, was analyzed and incorporated into the study.

Results and Discussion

Students' performance in OSPE

One hundred and fifty-five students participated in this virtual OSPE. The resulting grades were divided into two cohorts, Beqaa and Beirut, which are the two main campuses where the School of Pharmacy operates. For each cohort, the Average, Standard Deviation (SD), and percentage of achievement for each station were calculated as shown in Table 3.

The results presented in Table 3 indicate that, when considering stations 2 and 3 together, the students achieved higher average scores and percentages of achievement in practical competencies. Specifically, for the Beirut cohort, the cumulative average score, and percentage of achievement for stations 2 and 3 were 54.88±3.45 and 91.46%, respectively. For the Beqaa cohort, these values were 59.37±1.94 and 98.95%, respectively. These results demonstrate that students performed better in practical skills compared to the results of the first station, (35.26±3.39, 88.16%) for the Beirut cohort and (30.48±3.22, 76.18%) for the Beqaa cohort, which were more correlated to theoretical knowledge of aseptic procedures.

These findings showed that students may recognize the real demonstration as more perceptible to their practical skills and this is aligned with the results of previous studies, for instance, a study referenced as [5] reported that 75% of students agreed that OSPE allows them to assess their skills and identify areas where they need to supplement their knowledge. Similarly, the results obtained by [15] in the study, that 74% of students acknowledged that the exam helped them identify areas for improvement in their knowledge and skills related to anatomy. Other articles in the literature also support the notion that OSPE, due to its form, is considered a good, valuable tool for assessing students’ skills [15, 16], Overall, these results suggest that students perceive practical demonstrations as more indicative of their practical skills. This emphasizes the importance of incorporating practical assessments, such as OSPE, in evaluating students' competence in various subjects.

Students feedback

A total of 110 students responded to the questionnaire. Distributed between Beirut and Beqaa campuses by a ratio of 83% and 17%, respectively. Among the respondents, male students constituted 29.09% while 70.91% were females with a mean age ± SD of 24.3 ± 2.7. The gender distribution is parallel with the enrollment at the School and with the gender distribution of pharmacists in Lebanon [17].

The results of the questionnaire showed that 92.27% of the students believed that they had met the foundational knowledge of the course, and 89.77% agreed that they had acquired the pharmaceutical care knowledge and essentials of practice. In general, 98.86% of the students expressed satisfaction with their practical skills after completing the course, and 86.36% described their remote learning experience as a good one. These findings indicate that the majority of the students reported a positive experience and overall satisfaction with the remote learning model for the Parenteral Dosage Forms course. Similar results have been observed in several other studies in medical and non-medical fields supporting the positive experiences of the remote learning models [18-20]. For example, the results of [7] a study revealed that students had generally positive perceptions about online learning during the pandemic and the majority of the students wanted to continue with some online learning post-pandemic, as students appreciated the flexibility of the remote learning and considered asking their questions remotely was less intimidating than speaking in class [7].

Concerning the assessment of practical skills, and as demonstrated in Figure 1, 47% of the students described their practical skills after taking this course as “Excellent”, 29% described them as “Very good”, 22.5% described them as “Good”, 0.68% described them as “Fair”, though only 0.45% described their practical skills after taking this course as “poor”. The OSPE experience was also evaluated and as shown in Figure 2, the results revealed that 93% of the students described the practical exam guidelines as clear, the goals were achievable and the requirements were clear and accessible. In addition, 83% of the students considered the remote performance of the practical exam as fair and successful. However, 60% of the students expressed a preference for conducting a real practical exam on campus. It is worth noting that for practical courses, the demonstration of practical skills and their evaluation were the most challenging aspects of online learning methods. The students’ responses in our study showed that although students were highly satisfied with the management of the online practice examination and they disclosed that the goals and the guidelines of the OSPE were very clear and achievable, still a relatively fair percentage which is 60% preferred the real practical examination in the school. This preference could be attributed to various reasons, but it is possible that students were more accustomed to direct on-site examinations for practical skills. This topic could be further evaluated by comparing the results and feedback of on-site OSPEs versus online OSPEs. A study by [9] conducted such a comparison and concluded that the newly developed e-OSPE system showed promise as an alternative to traditional OSPEs, especially in the COVID-19 era.

Regarding the presentation activity, 88% of the students agreed that the online presentations had developed their skills in delivering virtual presentations and finding creative methods and tools to engage the audience. Moreover, students described the method of managing the presentations (such as topic approval, sharing schedules, delivering presentations, etc.) as organized and clear. Currently, and after the era of COVID, we believed that students should experience performing virtual presentations as virtual conferences and academic meetings are increasing even post-COVID era [21]. 

According to the feedback received from the students, 24.5% of them found the remote delivery of the course to be clear, simple, and interesting. This indicates that a significant portion of the students had a positive experience with the online format. Additionally, 41% of the students particularly enjoyed the experience of the practical exam, suggesting that they found it valuable and engaging even in a remote setting.

However, it is important to acknowledge that 11% of the students expressed their dissatisfaction with the course being delivered remotely. Their concerns and feedback should be carefully considered to understand the reasons behind their dissatisfaction and explore potential areas for improvement.

When asked about potential changes or additions to make the course more useful, valuable, and interesting, 32% of the students mentioned that they would have liked to have more practice incorporated into the course. This feedback highlights the importance of providing ample opportunities for hands-on learning, even in a remote setting. Additionally, 5% of the students expressed a preference for conducting the course on campus, indicating their desire for in-person learning experiences.

Interestingly, one student mentioned that this course could always be conducted online, suggesting a positive perspective on the remote delivery format. It is worth considering this viewpoint when evaluating the feasibility and effectiveness of online learning models in the future.

Furthermore, 7% of the students indicated their preference for conducting the practical exam on campus. This aligns with the earlier finding that some students expressed a preference for in-person assessments.

Balancing the advantages of remote assessments with the student's preferences and learning needs should be considered in future implementations. The comments received from the students serve as valuable feedback for improving the content, design, and students’ satisfaction with remote learning of the Parenteral Dosage Forms course. Incorporating more hands-on practice, exploring hybrid models that combine online and on-campus elements, and addressing concerns related to remote delivery can contribute to enhancing the overall learning experience. About [22] a study evaluated the effectiveness of hybrid learning in pharmacy education, highlighting the benefits of combining online and in-person learning experiences, in which it concluded that hybrid learning was associated with better academic performance and achievement than didactic teaching in pharmacy education.

The novelty and flexibility of virtual methods present ample opportunities for improvement and innovation. Sharing experiences and best practices with other institutions and educators can contribute to the collective knowledge and advancement of virtual learning in the field of pharmacy education. Collaborative efforts, research studies, and continuous evaluation of virtual learning strategies can lead to further enhancements in the future.

1. Abouzeid M, Halwani DA, Mokdad AH, Habib RR. A generation at risk: the impacts of Lebanon's escalating humanitarian crisis on children. Front Public Health. 2021;9:704678. doi:10.3389/fpubh.2021.704678
2. Mhanna M, El Zouki CJ, Chahine A, Obeid S, Hallit S. Dissociative experiences among Lebanese university students: Association with mental health issues, the economic crisis, the COVID-19 pandemic, and the Beirut port explosion. PLoS One. 2022;17(11):e0277883. doi:10.1371/journal.pone.0277883
3. Halat DH, Cherfan M, Mourad N, Rahal M. Highlights from a model for remote delivery of pharmacy laboratory courses: Design, implementation and student feedback: Innovation in learning assessment. Pharm Educ. 2020;20(2):49-51. doi:10.46542/pe.2020.202.4951
4. Isanhart CM, McCall KL, Kretschmer D, Grimes BA. Parenterals laboratory course to reduce microbial contamination rates in media fill tests performed by pharmacy students. Am J Pharm Educ. 2008;72(2):27. doi:10.5688/aj720227
5. Dymek J, Kowalski TM, Skowron A, Nowakowski MI, Golda AN. Pharmacy student’s feedback about objective structured practical examination (OSPE) after the pharmaceutical care course in Poland. Acta Pol Pharm Drug Res. 2021;78(4):583-8. doi:10.32383/appdr/141299
6. Syed IB, Nannapaneni M. Estimation of Olanzapine and Samidorphan in bulk and Pharmaceutical Dosage Form Using RP-HPL. Int J Pharm Res Allied Sci. 2022;11(4):87-94. doi:10.51847/ZqD8WfwA6Y
7. Zheng M, Bender D, Lyon C. Online learning during COVID-19 produced equivalent or better student course performance as compared with pre-pandemic: empirical evidence from a school-wide comparative study. BMC Med Educ. 2021;21:1-1. doi:10.1186/s12909-021-02909-z
8. Naciri A, Radid M, Kharbach A, Chemsi G. E-learning in health professions education during the COVID-19 pandemic: a systematic review. J Educ Eval Health Prof. 2021;18:1-11. doi:10.3352/jeehp.2021.18.27
9. Liebig J, Giessen U, Liebig J, Giessen U, Liebig J, Giessen U. Evaluation of e-OSPE as compared to traditional OSPE: a pilot study. J Biochem Mol Biol Educ MS. 2022;0:4-11.
10. Hung JC. Commentary: USP General Chapter< 797> Pharmaceutical Compounding–Sterile Preparations. J Nucl Med. 2004;45(6):20N-.
11. Sharif SI, Sharif RS. Choosing pharmacy as a major: motivations and influences. Pharm Educ. 2014;14(1):116-20.
12. Shaikh H, Elhissi AM, Vohra S, Pearson BJ, Taylor KM. Motivations of MPharm students in the United Kingdom to study pharmacy at a time of change. Pharm Educ. 2013;13(1):187-91.
13. Davey A, Evans AM, Stupans I. Pharmacy: Factors that influence the choice of career and study options. Pharm Educ. 2006;6(1):216. doi:10.1080/15602210500523381
14. Nakagomi K, Hayashi Y, Komiyama T. Survey of attitudes toward career choice among pharmacy students at a private university. Pharm Educ. 2016;16(1):146-57.
15. Branch C. An assessment of students’ performance and satisfaction with an OSCE early in an undergraduate pharmacy curriculum. Curr Pharm Teach Learn. 2014;6(1):22-31. doi:10.1016/j.cptl.2013.09.006
16. Al-Haqan A, Al-Taweel D, Koshy S, Alghanem S. Evolving to Objective Structured Clinical Exams (OSCE): Transitional experience in an undergraduate pharmacy program in Kuwait. Saudi Pharm J. 2021;29(1):104-13. doi:10.1016/j.jsps.2020.12.013
17. Hallit S, Sacre H, Hajj A, Sili G, Zeenny RM, Salameh P. Projecting the future size of the Lebanese pharmacy workforce: forecasts until the year 2050. Int J Pharm Pract. 2019;27(6):582-8. doi:10.1111/ijpp.12564
18. Rad FA, Otaki F, Baqain Z, Zary N, Al-Halabi M. Rapid transition to distance learning due to COVID-19: Perceptions of postgraduate dental learners and instructors. Plos One. 2021;16(2):e0246584. doi:10.1371/journal.pone.0253683
19. Riley E, Capps N, Ward N, McCormack L, Staley J. Maintaining Academic Performance and Student Satisfaction during the Remote Transition of a Nursing Obstetrics Course to Online Instruction. Online Learn. 2021;25(1):220-9. doi:10.24059/olj.v25i1.2474
20. Amir LR, Tanti I, Maharani DA, Wimardhani YS, Julia V, Sulijaya B, et al. Student perspective of classroom and distance learning during COVID-19 pandemic in the undergraduate dental study program Universitas Indonesia. BMC Med Educ. 2020;20(1):1-8. doi:10.1186/s12909-020-02312-0
21. Schreiner LJ, Badawi R, Baldock C. The post-COVID future of research conferences should be virtual. Phys Eng Sci Med. 2022;45(2):413-7. doi:10.1007/s13246-022-01138-y
22. Balakrishnan A, Puthean S, Satheesh G, MK U, Rashid M, Nair S, et al. Effectiveness of blended learning in pharmacy education: A systematic review and meta-analysis. PloS One.2021;16(6):e0252461. doi:10.1371/journal.pone.0252461