The Effect of High-Fidelity Simulation on Nursing Students' Knowledge ...

NCSBN research brief Volume 40 | June 2009

The Effect of High-Fidelity Simulation on Nursing Students’ Knowledge and Performance: A Pilot Study

Report of Findings from

The Effect of High-Fidelity Simulation on Nursing Students’ Knowledge and Performance: A Pilot Study Principal Investigators Frank D. Hicks, PhD, RN, Rush University College of Nursing, Chicago, IL Lola Coke, PhD, RN, Rush University College of Nursing, Chicago, IL Suling Li, PhD, RN, National Council of State Boards of Nursing, Chicago, IL

National Council of State Boards of Nursing, Inc. (NCSBN®)

ii

Report of Findings from The Effect of High-Fidelity Simulation on Nursing Students’ Knowledge and Performance: A Pilot Study

Mission Statement The National Council of State Boards of Nursing, composed of member boards, provides leadership to advance regulatory excellence for public protection. Copyright ©2009 National Council of State Boards of Nursing, Inc. (NCSBN®) All rights reserved. NCSBN®, NCLEX®, NCLEX-RN®, NCLEX-PN® and TERCAP® are registered trademarks of NCSBN and this document may not be used, reproduced or disseminated to any third party without written permission from NCSBN. Permission is granted to boards of nursing to use or reproduce all or parts of this document for licensure related purposes only. Nonprofit education programs have permission to use or reproduce all or parts of this document for educational purposes only. Use or reproduction of this document for commercial or for-profit use is strictly prohibited. Any authorized reproduction of this document shall display the notice: “Copyright by the National Council of State Boards of Nursing, Inc. All rights reserved.” Or, if a portion of the document is reproduced or incorporated in other materials, such written materials shall include the following credit: “Portions copyrighted by the National Council of State Boards of Nursing, Inc. All rights reserved.” Address inquiries in writing to NCSBN Permissions, 111 E. Wacker Drive, Suite 2900, Chicago, IL 60601-4277. Suggested Citation: National Council of State Boards of Nursing. (2009). Report of Findings from The Effect of High-Fidelity Simulation on Nursing Students’ Knowledge and Performance: A Pilot Study. (Research Brief Vol. 40). Chicago: Author. Printed in the United States of America ISBN# 978-0-9822456-5-1

National Council of State Boards of Nursing, Inc. (NCSBN) | 2009

TABLE OF CONTENTS

iii

table of contents List of Tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 I. Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Advantages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Disadvantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Review of the Literature. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 II. Study Goals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 III. Research Questions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 IV. Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Design and Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Setting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Simulation only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Simulation and clinical . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Clinical only. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Outcome Measures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Knowledge Acquisition and Retention. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Clinical Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Faculty Review of Videotaped Student Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Self-confidence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Protection of Human Research Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 V. Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Demographics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Knowledge Acquisition and Retention. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Clinical Performance Assessed with Standardized Patients. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Self-confidence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Course Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 VI. Limitations of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 VII. Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 VIII. Avenues of Future Research. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 IX. References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Appendix A: Chest Pain Scenario. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Appendix B: Shortness of Breath Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Appendix C: Level of Consciousness Scenario. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Appendix D: Self-confidence Scale. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27


iv

LIST OF TABLES

LIST OF TABLES Table 1. Demographics of Participants. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Table 2. Written Exam Scores Before and After Simulation/Clinical Experiences. . . . . . . . . . . . . . . . 11 Table 3. Clinical Performance on Three Patient Care Scenarios Portrayed by Standardized Patients: Tape Review Checklist Rating. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Table 4. Total Time to Complete Three Patient Care Scenarios Portrayed by Standardized Patients. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Table 5. Dimensions of Perceived Confidence Before and After Simulation/Clinical Experiences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Table 6. Perceived Confidence Before and After Simulation/Clinical Experiences. . . . . . . . . . . . . . . 14 Table 7. Perceived Clinical Experience. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Table 8. Perceived Simulation Experience (n=37) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Table 9. Student Learning Based on Self-Evaluation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17


ACKNOWLEDGMENTS

v

Acknowledgments This study would not have been possible without the support provided by the Rush University College of Nursing (RUCON) and Rush University Simulation Laboratory (RUSL). The authors acknowledge the contributions of the involved faculty, staff and students who expended time and energy to help implement the study protocol. Specifically, the authors acknowledge the following individuals for their valuable contributions to the study: Marcia Bosek, PhD, RN, former associate professor, RUCON, Chris MacNeal, simulation coordinator, RUCON and Lynn Richter, MSN, RN, APRN-BC, assistant professor, RUCON. The authors also gratefully acknowledge Nancy Spector, PhD, RN, NCSBN, director, innovations, Kevin Kenward, PhD, NCSBN, director, research, Maryann Alexander, PhD, RN, NCSBN, chief officer, nursing regulation, and Mary E. Doherty, JD, BSN, RN, NCSBN, associate, nursing regulation for their support and valuable feedback to the project, and Richard Smiley, MS, NCSBN, statistician, research, for his help with the statistical analysis.


INTRODUCTION

Introduction Simulation, the art and science of recreating a clinical scenario in an artificial setting, has been an important aspect of nursing program curriculums for decades (Gomez and Gomez, 1987). As an adjunct to clinical experience, simulation has allowed deliberate practice in a controlled environment. Students are able to practice a procedure prior to performance on a live patient. The value of this is unquestionable. Recently, however, high-fidelity simulation, with the increased level of sophistication and realism it brings to the laboratory setting, has elicited the possibility of simulation being used as a substitute for actual clinical experience. This study explores that concept. High-fidelity simulation refers to structured student learning experiences with the use of a technologically advanced computerized mannequin, the Human Patient Simulator (HPS). HPS is anatomically precise and reproduces physiologic responses. Students are administered sequential decision-making events within an environment that mimics a clinical setting. Instructors can control the mannequin’s responses and the HPS can respond to interventions provided by the student (Gilley, 1990; Graedler, 1992; Lasater, 2007). Gaba (2004) describes simulation “as a strategy – not a technology, to mirror, anticipate, or amplify real situations with guided experiences in a fully interactive way.”

1

as a teaching method beyond the acquisition of psychomotor skills. How well simulation assists students in acquiring and integrating knowledge, skills and critical thinking, and how it fares when compared to traditional clinical/real patient encounters, are a few of the questions being asked by both educators and regulators alike. It is unknown as to what degree simulation learning is transferable to the clinical area or if simulation learning is as effective as actual clinical experience in developing professional judgment. There is also little evidence demonstrating how well it assists in building confidence. Because of these questions and the lack of evidence to answer them, it is unknown as to what degree highfidelity simulation would be an appropriate and suitable substitute for real clinical experiences now required by state regulations for nursing programs. As the first step in addressing these questions, it is important to determine if high-fidelity simulation provides an effective learning strategy equivalent to clinical experiences.

The subject of simulation and its uses in prelicensure education is of prime significance to regulators. From a regulatory perspective, simulation offers advantages. It harbors patients from unnecessary risk and/or discomfort, thus providing an element of public protection. The increased opportunities to encounter infrequent and atypical clinical problems may better prepare new graduates for transition into the workforce. It may also offer an alternative for nursing programs dealing with a limited number of clinical sites or clinical sites that have inadequate learning opportunities. A goal of regulation is to make certain that the future workforce has safe and competent nurses. Thus, regulators are also interested in ensuring that nursing students are provided with optimal education experiences. Despite its obvious benefits, there is a paucity of evidence regarding the efficacy of high-fidelity simulation National Council of State Boards of Nursing, Inc. (NCSBN) | 2009

2

BACKGROUND

BACKGROUND Documented throughout the literature are the advantages and disadvantages of high-fidelity simulation. The following list summarizes the pros and cons of this methodology.

Advantages No direct risk to patients. Simulation offers opportunities to practice rare and critical events in a safe and controlled environment, affording unlimited practice without risk to patients (Gilley, 1990; Graedler, 1992; Maran and Glavin, 2003, Decker, et al., 2008). The potential to increase the speed of acquisition of clinical skills to a defined level of competence by allowing the opportunity for repetitive practice at the learner’s own pace (Maran and Glavin, 2003). Team training. Scenario themes can focus on understanding team collaboration and communication (Ellis, et al., 2008; Lasater, 2007). Can allow a standardized curriculum to be developed as the same exact scenarios can be presented to all learners. Simulation reduces training variability and increases standardization. With simulation, faculty can guarantee the same experience for every student (Maran and Glavin, 2003). Reflective learning by facilitated debriefing of scenarios and video feedback (Lederman, 1992; Gilley, 1990; Hertel and Millis, 2002; Gaba, 2000). Potential to decrease the number and effect of errors through crisis resource management (CRM) training (Ziv, et al., 2003).

Disadvantages Equipment fidelity. Fidelity is the degree to which the simulator itself replicates reality. A major limitation of simulation is the fidelity; no matter how high the fidelity is, it is not real. It is often impossible to imitate actual physiological signs or symptoms. For example, it is impossible to display crackles

in the lung, when such a symptom would be important to the scenario. In addition to the need for high equipment fidelity, simulation requires psychological fidelity. This reflects the degree to which the trainee perceives the simulation to be a believable representation of the reality it is duplicating. Students may not take it seriously, since mistakes or errors have no real consequences on patient safety. Along with equipment and psychological fidelity, simulation requires environmental fidelity – that is, the realism of the environment in which the simulation takes place. For example, emotional stress does not exist in mannequins and standardized patients are not really sick. Cost. A study by McIntosh, et al. (2007) calculated the set-up cost of a simulation center was $876,485 (renovation of existing facility, equipment). Fixed costs per year totaled $361,425. Variable costs totaled $311 per course hour. The economic benefits of increasing the number of billable teaching hours per week are significant until about 21 hours (equivalent of three full or six and a half day courses) per week (averaged over 52 weeks/year) when they started to taper off. Lack of faculty time and training in simulation instruction. Effective use of technology in education depends on faculty readiness to operate the tools for maximum educational impact. It is important to recognize the need for faculty training in the specific demands which simulation-based teaching imposes. Access to the simulator is limited and dependent on the availability of instructors and operators. Simulation also limits the number of students that can be taught at any one time. Negative transfer. Negative transfer occurs if the student learns something incorrectly due to imperfect simulation. This most commonly occurs because the instructor fails to make


BACKGROUND

clear to the students the differences between the training device and the real situation. These differences are usually due to a lack of physical or equipment fidelity. For example, the different feel of intubating a mannequin or the artificial acceleration of tasks might imprint incorrect clinical practices or procedures (Bond, et al., 2007). Certain assessments are not possible, such as reflexes for example. Swelling and redness does not appear, and the psychologic effects of an illness and the emotional response of the patient cannot be assessed using the simulator (Lasater, 2007).

Review of the Literature An examination of the literature related to simulation reveals that despite a growing body of literature, there is a substantial need for more evidence. Most studies lack rigor, have small sample sizes, provide little statistical analysis and conclusions are drawn based on student perception. Studies focusing on student reaction to the HPS report positive responses to the simulation experience. In a review of the literature conducted by Ravert (2002), 75% of the studies reviewed concluded that students pursuing a degree in one of the health professions favored or highly favored simulation as a teaching/learning tool. In a qualitative study by Mikkelson, Reime and Harris (2007), students (N=21) who experienced the simulationbased training stated they had greater awareness of the complexity of a health care condition and it raised their awareness to aspects they had not thought of. Two studies specifically examined the impressions of undergraduate nursing students. Bearnson and Wilker (2005) used an HPS to demonstrate medication side effects to students. Students reacted favorably to the experience and reported increased knowledge of medication side effects and an understanding of the differences in patient responses; it also increased their ability to safely administer medications and their self-confidence in medication administration skills. No tests were administered to establish whether the students’ perception actually

3

correlated with increased knowledge. No control group was used to compare responses. Statistical analysis beyond calculating the means of nominal data collected from a questionnaire with Likert scales was not performed. In addition, the report does not describe the number of students that participated in the study. Lasater (2008) also examined high-fidelity simulation experiences on nursing students (N=8) and its effect on the development of clinical judgment. Students enrolled in a Nursing Care of the Adult course were given weekly simulation experiences as part of the course. After the completion of the course, eight of the 15 students participated in a focus group and discussed their experience with the HPS. Several themes emerged, including simulation integrates learning; it increased the breadth of experiences the students were exposed to; and the scenarios forced them to anticipate potential problems. The investigator concluded that this last theme, anticipation of potential problems, was an indication that simulation fostered clinical judgment. While the experience may have done so, the small sample size and the lack of outcome data measuring performance makes it difficult to draw a definite correlation between the HPS and clinical judgment. Several recent studies have attempted to determine how simulation compares with traditional clinical experience. The lack of consistency in the results, however, makes drawing conclusions about simulation difficult. Alinar, Hunt, Gordon and Harwood (2006) compared clinical performance of students in traditional clinical settings with those that received clinical experience (control) and clinical plus simulation experience (experimental). Using a pretest/posttest design, undergraduate nursing students (N=99) were randomly assigned to one of these two groups. Students were pre- and posttested using the Objective Structured Clinical Examination (OSCE) method. Students in both groups improved their clinical performance; however, the experimental group improved their performance on the OSCE 14 to 18 percentage points (95% CI 12.52-15.85) compared to seven to 18 points (95% CI 5.33-9.05) in the control group. The 7.0 percentage point difference


4

BACKGROUND

between the means (95% CI 4.5-9.5) was statistically significant (p< 0.001). Radhakrishnan, Roche and Cunningham (2007) conducted the first study to test nursing student performance with complex two-patient assignment simulations. Undergraduate nursing students (N=12) were randomized into a traditional clinical experience (control) or a traditional clinical experience plus simulation (intervention). Results indicated that the intervention group had statistically significant higher scores in two specific areas of clinical performance: safety (p