VOL. 22, No. 3, 149-156
The potential impact of on-line learning in health care is significant. By providing access to educational material from an internet-connected computer anytime and anywhere, healthcare workers (HCWs), whose workload demands are often changing and somewhat unpredictable, have increased ability to self-educate. (Carr et al, 2005, Fordis et al, 2005; Tilleczek et al, 2005; Desai et al, 2000) For example, the growing recognition of the importance of infection control education among HCWs in the prevention of infectious diseases has reinforced the need for accessible and timely content delivery in this particular area. (Shaw, 2003; Zoutman et al, 2003) Attractive as its potential may be, it is imperative that the on-line learners easily obtain access to appropriate learning programs and are satisfied with this type of learning environment in order for online learning to be a useful educational strategy for these adult learners.
This report outlines the introductory and promotional strategies as well as the initial results and adult learner satisfaction surveys of an on-line infection control course during the first five months of the course's introduction. The objectives of this phase of the project were to: a) evaluate the ability of the module to be accessed by healthcare workers; b) to assess the ability of participants to learn from the on-line module and; c) to assess the satisfaction of adult learners working in healthcare with this form of content delivery.
Analysis of these surveys was co-sponsored by the Canadian Institutes of Health Research, Vancouver Coastal Health Authority, the British Columbia Centre for Disease Control, Children's and Women's Health Centre of British Columbia, and the Occupational Health and Safety Agency of BC and was approved by the University of British Columbia Behavioural Research Ethics Board.
Vancouver Coastal Health (VCH) provides a full range of health care services ranging from hospital treatment to community-based residential, home health, mental health and public health services. The 21,723 staff members, 11,087 of whom are full time equivalents, work in 475 buildings, including 13 acute care facilities, two diagnostic/treatment centers, long term, rehabilitative, and community care.
Healthcare workers require and are provided with a variety of educational programs and services for continuing upgrading and maintenance of competencies including: 1) new staff orientations; 2) in-services provided by supervisors, clinical educators and Infection Control and Worksafe and Wellness staff; 3) self-serve materials on the intranet; 4) and a variety of courses, offered through Learning and Development, available via traditional classroom instruction or available online. Infection control is a priority in all healthcare facilities, for all HCWs, and continuous reminders and education are required to assure adherence to good infection control practice. The original on-line infection control content for the module examined in this study had been alpha-tested by regional infection control experts and piloted to key groups of HCWs prior to implementation. The module consists of basic infection control content, video clips describing appropriate donning and doffing of personal protective equipment, safe sharps management and waste disposal, isolation categories, and information on antibiotic resistant organisms. Learning is guided through these educational "Lessons" and is self-directed. The modular format allows learners to enter and exit at any point in the course, and repeat sections as desired.
Upon completion of the pilot testing, the module was made available to all staff via the organizational Learning Management System (LMS), which enabled collection of user accessibility data. Data related to user accessibility included the staff member name; their department and facility; the day of the week and time of day the module was accessed; gender and age; number of years worked at the facility; job experience; how staff found out about the module; their level of computer skills; and the reason they completed the module Ð whether they completed it voluntarily or whether it was mandated by their supervisor. User identities were password protected by the participants who generated their user profiles and only Learning and Development senior staff could access the demographic data, consistent with their role as the health authority educators.
For the purposes of evaluation of learning, on-line quizzes were incorporated into the module; a) a comprehensive 25 question pre-quiz, taken before the module; and b) a post-quiz taken upon module completion (consisting of the same questions as the pre-quiz). Paired t-tests were used to compare the aggregate mean of the pre-test to the aggregate mean of the post-test scores.
Learners were also presented with a survey (built into the system as part of the course to ensure a high response rate) regarding their satisfaction with the module. The user satisfaction survey recorded learner feedback regarding the module's ease of use, interest level, and effectiveness of the content. Learners were asked specific questions with regard to each of these topic areas in which they provided a score on a Likert-scale of 1 to 5, where 1= "strongly disagree" and 5 = "strongly agree". Results of the survey were collected automatically by the LMS database, and analyzed as aggregate data.
User Accessibility
From March 2005 to July 2005, 280 study participants completed the entire module consisting of the pre-test, lessons, and post-test. The majority of the participants were females (91.4%) and the most common occupation was nursing (42.5%). Other occupations are summarized in Figure 1. The majority of participants were between 21-30 years (50.7%), followed by 31-40 years (23.2%), 41-50 years (13.9%), greater than 50 years (11.4%), and less than 21 years (0.7%). The average number of years of work experience was 6.9 years (SD = 9.25), with 40.7% having less than one year's experience, and 9.6% having greater than 20 years of experience.
Most participants (63.5%) were recruited to take the module by their supervisors or as part of their hospital orientation, while the remaining 36.5% indicated that they took the module voluntarily. Enrollment peaked at four months after the course was introduced, with June having the most participants (33.9%) and March the least (5.7%) (Figure 2).
The module was primarily accessed from Monday to Friday (95.3%), compared to 4.7% on Saturday and Sunday. Wednesday was the most popular individual day, with the module accessed 101 times (36.1%) on that day over the five-month period. Most users (92.1%) logged in between 0800 and 1800 hrs. The most popular time slots for access were 1200 to 1400 hrs (34.6%) and 1400 to 1600 hrs (31.1%).
Learning
Average time to take the entire course was 45 minutes; this included the pre and post-test, the lessons, and completion of the user satisfaction survey. When the components of the course were analyzed, the average time taken to complete the pre-test was 8.7 minutes (SD = 4.95; range 0.91-42.1 minutes). This was a significant difference (p < 0.01) compared to the average time to complete the post-test which was 5.2 minutes (SD = 2.56; range 1.9-28.8 minutes). The time to complete the post-test decreased in all age groups except for those < 21 years old.
Figure 1. Distribution of Participants by Occupation.
ICP = infection control practitioner
OT/PT = occupational therapy/physiotherapy
There was a significant difference between the average pre-test score which was 18.20 (SD = 6.25; range 11-25) and post-test score, which was 21.84 (SD = 5.04; range 8-25; p < 0.01). There was no significant difference in pre or post-test scores when participants were stratified by age or years of experience. The data was also grouped by those who voluntarily took the module and those who took it based on supervisor request; again, no significant difference was found in either pre-test scores (p = 0.79) or post-test scores (p = 0.41).
Figure 2. Number of Participants by Month.
User Satisfaction
Eighty-two learners completed the user satisfaction survey (Table 1). The overall score for user satisfaction was 4.22 out of a possible 5.0 (SD = 0.15). Thirteen participants specifically commented as to the usefulness of the module and its presentation excellence. An additional four learners remarked on the interactive element, specifically the matching exercises and the ability to "explore" a lesson in depth. The ability to learn independently at one's own pace with the option to review content areas was also mentioned by the learners.
Table 1. Results from User Satisfaction Survey
| Category | Question | Score* |
| Ease of Use | Easy to begin, exit, re-enter and move through the course | 4.1 |
| Easy to identify where you are in the course | 4.2 |
|
| I found it more convenient to complete the course through the computer than attending a classroom workshop | 4.3 |
|
| Holds User Interest | Captures and holds attention | 4 |
| Stimulates involvement/motivating | 3.9 |
|
| Text is easy to read | 4.3 |
|
| Use of color is pleasing | 4.3 |
|
| Use of picture/video is acceptable/not distracting | 4.2 |
|
| Content Delivery | Content is clearly and logically presented | 4.3 |
| Information is relevant and useful | 4.4 |
|
| Instructional level is appropriate | 4.3 |
|
| Course objectives is adequately covered | 4.3 |
*out of a possible 5
Negative comments included the lack of audio capability on some of the organizational computers, which limited the presentation quality of the video demonstrations; the lack of feedback when users answered a question incorrectly in the post-test; and the need to supplement written content with video demonstrations. Interestingly, some learners remarked that it was difficult to navigate between module pages despite the fact that there was a course index button located at the top corner of each module page.
The majority of participants were adult females (nursing staff); this is not unexpected, as the module was originally aimed at nurses. Most of the participants were under 30 years of age; well below the average age of a Canadian nurse of 45 years. (British Columbia Ministry of Heath, 2000) This result suggests that younger nurses may be more comfortable using computers and that ensuring access by more mature healthcare workers may require more well-planned support (Kramarae, 2001). This is also implied by the comments from mature users regarding the difficulty navigating the course despite clear navigational buttons on every page
The day and time the module was accessed is also of relevance. The predominance of Wednesdays can be attributed to the fact that the majority of the mandated sessions were held on that day. It was noted that 2.5% of all VCH health care workers (280/11,087) successfully completed the web-based infection control basic learning module in the first few months of its introduction and that there was a three to one ratio of staff completing the module on weekdays as opposed to weekends. This ratio difference may be due to the fact that the mandated sessions occurred only on weekdays but it should be noted also that the ratio is in proportion to the number of staff working the weekend shifts. Regardless, weekend staff are often not available for regular weekday orientation and in-service courses, and it may be desirable to increase their rates of enrolment by means of incentive campaigns.
Neither age nor years of experience affected test results. This was gratifying as the module was intended to be generic and capture the interest of all users. Adults are self-directed, visual learners, responding best to practical experiences where learning activities are relevant to the work experience. (Belfiore, 1996; Caffarella 1994; Moore, 1991) Online learning and digitized clinical scenarios fulfilled this adult learning profile. Similarly, there was no significant difference in test scores between the mandated and voluntary participants, suggesting that mandating course completion does not impede learning nor engagement in the course. This has important implications for employers when considering policies that would mandate certain learning topics for staff.
It was disappointing that only a few physicians completed the web-based module, but not unexpected as Jay & Anderson's (1982) review of 23 hospitals revealed that only 20% of physicians actually used computer information systems. In recognition of this, the local Medical Advisory Council has voted to make the completion of the e-learning module an annual requirement for admission privileges. The education team is currently completing the requirements of the Royal College of Physicians and Surgeons with respect to maintenance of competence credits for physicians as an incentive for them to complete the on-line learning module. Similarly, confirmation of module completion is now a required element for final training evaluation of medical residents.
The infection control on-line module is an initial success with users able to log-on with minimal difficulty, learn at their own pace, with demonstrable learning through improvement in post-test scores. The module scored well on user satisfaction. Targeting the more mature workforce and permanent weekend staff will be two of the strategies to improve participation of these groups as the module implementation progresses. This study shows that on-line learning is an accepted and satisfactory method for delivering topics of importance to adults in the work force, in this case, in healthcare.
Belfiore, M.E. (1996). Understanding curriculum development in the workplace: A resource for educators. ABC Canada. Chapter 2: Summary, Principles.
British Columbia Ministry of Health. (2000). Appendix 6, Provincial Data from the BC Nursing Recruitment and Retention Survey. In Recruitment and retention of Registered Nurses and Registered Psychiatric Nurses in British Columbia. Author.
Caffarella, R.S. (1994). Planning programs for adult learners: A practical guide for educators, trainers, and staff developers (2nd ed.). Hoboken, NJ: John Wiley and Sons. pp. 24-25.
Carr, H., Hinson P. (2005). Education and Training. In APIC text of infection control and epidemiology (2nd ed.). Washington, DC: APIC. pp. 11-1Ð11-17.
Desai N., Philpott-Howard J., Wade J., & Casewell, M. (2000). Infection control training: Evaluation of a computer-assisted learning package. Journal of Hospital Infection, 44:193-199.
Fordis, M., King J., Ballantyne, C., Jones P., Schneider K., Spann, S., Greenberg S., & Greisinger, A. (2005). Comparison of the instructional efficacy of Internet-based CME with live interactive CME workshops: A randomized controlled trial. JAMA, 294(9): 1043-1051.
Jay, S.J., & Anderson, J.G. (1982) Computerized hospital information systems: Their future role in medicine. Journal of the Royal Society of Medicine, 75(5), 303-305.
Kramarae, C. (2001). The third shift: Women learning online. American Association of University Women Educational Foundation. Washington, DC. ISBN-1-879922-29-0.
Moore, P. (1991). Clinical teaching in the health sciences. BCIT Part-time Studies School of Health Sciences Course Manual. EDUC 610, pp. 2.8-2.15
Shaw, K. (2006). The 2003 SARS outbreak and its impact on infection control practices. Public Health, 120(1): 8-14.
Tilleczek, K., Pong, R., & Caty, S. (2005). Innovations and issues in the delivery of continuing education to nurse practitioners in rural and northern communities. Canadian Journal of Nursing Research, 37(1): 146-162.
Zoutman, D.E., Ford, B.D., Bryce E., Gourdeau, M., Hebert, G, Henderson E., & Paton, S. et al. (2003, August, 31). The state of infection surveillance and control in Canadian acute care hospitals. American Journal of Infection Control, (5): 266-72.
Dr. Elizabeth Ann Bryce is Regional Medical Director of Infection Control at Vancouver Coastal Health, and has been involved in acute care hospital infection control and adult education for the past 13 years. Dr. Bryce currently co-holds three grants related to educating health care professionals in infection control. She was the previous director for Standards and Guidelines for the National Infection Control Association (Community Hospital Infection Control Association) and is a co-chair of the Canadaian Nosocomial Infection Surveillance Program. E-mail: elizabeth.bryce@vch.ca
Peter Choi, Regional e-Learning Advisor for Vancouver Coastal Health Authority, formerly at Children's and Women's Health Centre of BC has led in the design and development of healthcare on-line learning programs for the past 10 years at C&W and VCH. E-mail: peter.choi@vch.ca
Margaret Landstrom is Director of Learning and Development and Children's and Women's Health Centre of BC. She was previously involved in instruction, research, and administration at the University of British Columbia and the University of Windsor. She is a founding member and third president of CADE/ACED. E-mail: mlandstrom@cw.bc.ca
Mr. Justin LoChang is a senior researcher at the University of British Columbia, and the Manager of Technology Enabled Knowledge Translation at the Institute of Health Promotion Research. He has an extensive background in human factors, usability, and evaluation, and has been working on healthcare related research projects for the past six years.