The Effects of Simulation-Based Learning Using Virtual Reality in Nursing Student:A Meta-Analysis

Run-Jie ZHANG, Shu-Jin YUE , Yun-Yue TANG, Zhi-Qi CHEN,Zi-Pan WANG, Jing WANG, Yu LI

School of Nursing, Beijing university of Chinese Medicine, Beijing 102488, China

Abstract

Keywords: Virtual reality; Simulation-based learning; Nursing students; Meta-analysis

Introduction

Responding to a dynamic health care environment for nursing students is a formidable challenge based on current national initiatives in patient safety and injury reduction[1]. Nursing students are faced with the diversity and complexity of challenges, such as the increase in the number of students, the shortage of clinical practice, and the transformation of medical models. This requires nursing students to master enough nursing skills. However, current teaching methods can not be satisfied with the cultivation of these skills[2]. Therefore, nursing educators shoulder the responsibility of reforming nursing curriculum to meet the needs of students, including the ability to negotiate and resolve conflicts, oral and written communication,technical skills, critical thinking and interdisciplinary cooperation[3].

The World Health Organization (WHO) has published the standards for nursing education. According to these standards, e-learning and simulation methods are recommended for use in nursing school programs[4]. With the development of science and technology, computerbased virtual reality teaching is widely used in many fields, and it is also a new teaching strategy applied to narrow the gap between theory and evidence-based clinical practice. Virtual reality (VR) is a computergenerated, real-time and dynamic three-dimensional realistic image, including sight, sound, touch, motion and other sensory perception[5]. Moreover, Gaddis (1997)defines VR as a computer-generated simulation of a three dimensional environment that the user is able to view and manipulate or interact with[6]. Users can experience the real environment in VR in person. VR environments can be divided into immersive and non-immersive ones.Immersive VR is a kind of VR with strong interactivity and high cost of peripheral equipment, and nonimmersive VR is usually called desktop VR. It enters the virtual world in the form of window, displays on the computer monitor and interacts with the mouse[7]. Virtual environments can provide a way to overcome clinical training barriers while also allowing people to focus on the present. This technology is a powerful medium for training, and offers different benefits that supplement the use of mannequin-based simulators or standardized patients[8].

Recently, VR is more and more received in nursing education field. However, there is limited research on the use of VR technology as an educational method for nursing students to acquire and transfer knowledge and skills[9]. There is evidence that VR is effective, but when nursing educators are considering the application of VR, they are also faced with the problem of inconsistent information found in previous studies. There is a lack of systematic review or meta-analysis on the effect of VR application on the acquisition of nursing knowledge and the improvement of nursing skills in nursing education.Therefore, this study aims to find the best evidence.

Methods

Data sources

Studies were identified through 8 databases including the Cochrane library, PubMed, EMBASE, Web of Science,Proquest, ScienceDirect, Springer and OVID. This limited threshold in time from January 2008 to December 2018.The main search strategy combined terms and keywords including “virtual reality”, “human patient simulator”,“nursing students”, “nursing education”, “simulation training”, and “virtual simulation teaching”. Searches were limited to articles in English.

Study selection

Studies were selected for inclusion if they: (1) Include nursing students, (2) are randomized control trials (RCTs)and non-RCTs, (3) use virtual reality related to nursing,(4) the primary outcome measure (i.e., knowledge and performance) and the secondary outcome (i.e., confidence).The exclusion criteria were: the essential information was not complete to extract the data and also could not acquire primary data from authors. The studies were determined by electronic retrieval, then downloaded to a reference management database, and then eliminated duplication.The two researchers independently reviewed the studies to determine whether inclusion criteria were met: (1)eliminate obviously irrelevant studies through reading the titles and abstracts; (2) read the full text, excluding those that did not meet the inclusion criteria; (3) contact the author to get the missing data; (4) determine the final inclusion of the studies.

Data extraction

Data were extracted separately by 2 researchers. The following main information of data extraction included:(1) authors and years of publication, (2) study design, (3)sample size and content, (4) means of intervention, and(5) measurement and outcomes.

Risk of bias assessment

The risk of bias in included studies were independently evaluated by 2 researchers and cross-checked. In case of disagreement, a third party was consulted and discussed for judgment, and the missing data were supplemented by contacting the author as far as possible. According to the Cochrane Handbook for Systematic Reviews (Version 5.1.0)[10]to evaluate randomized controlled trial studies,that included random sequence generation, allocation concealment, blinding of participants and personnel,blinding of outcome assessment, incomplete outcome data, selective reporting and other sources of bias. The quality of each trial was categorized into low/unclear/high risk of bias. Meanwhile, risk of bias in non-randomized studies of Interventions (ROBINS-I) was used for non-RCTs, that included bias due to confoundingbias in selection of participants into the study, bias in classification of interventions, bias due to deviations from intended interventions, bias due to missing data, bias in measurement of outcomes and bias in selection of the reported result[11].The quality of each study was classified as low/moderate/serious/critical risk/no information.

Data analysis

RevMan 5.3 software was used for statistical analysis.If all studies used the same scale to evaluate the same results, mean differences (MDs) were used. Standardized mean difference (SMD) was used if all studies assessed the same outcome by different scales. MDs or SMDs with a continuous outcome 95% confidence interval(CI) were selected to estimate the combined effect size. A heterogeneity test was conducted using I2value for each analysis. In our included studies heterogeneity was found to be high (I2＞ 50%), so random effects model were adopted. When I2less than 50%, fixed effect model is adopted. Subgroup analysis was performed when there were some detectable heterogeneity resources. A P＜0.05 was considered to be statistically significant. Publication bias was assessed by funnel plots when more than 10 studies were included[12].

Results

Literature screening results

According to databases searched, a total of 5,480 studies were found. Following by NoteExpress removal of duplicates, 3,076 publications were remaining. Screening of titles and abstracts resulted in identification of 60 studies having potential relevance. Finally, on the basis of inclusion and exclusion criteria, 46 studies were nonrelevant interventions, 2 studies had insufficient data, 1 study were non-nursing student studies, and 1 study had a one-group design. A total of eligible 10 studies were included in the current meta-analysis. And the literature search and screening process was shown in Figure 1.

Overall description of the studies

The characteristics of the 10 studies included in the review were summarized in Table 1.

5 (50%) studies[13,14,16,19,21]used a pre-post design, and 5 studies[15,17,18,20,22]compared only post-test. 5 (50%)studies[13,15,17-19]used an RCT design, and 5 studies[14,16,20-22]used a non-RCT design. 8 (80%) studies[13,14,16,18-22]used computer-based VR, 1 study[15]used computer-based VR incorporating regular course, and 1 study[17]used computer-based VR incorporating simulation. The sample size across the 10 included studies varied between 20 and 118, with a total of 630 participants. The grades of nursing students were reported as follows: freshman (1 study[17]),sophomore (2 studies[14,15]), junior (1 study[13]), senior (3 studies[18,19,21]), and unclear (3 studies[16,20,22]). The subject courses of computer-based virtual regular learning were diverse, with adult nursing (k=2), fundamental nursing(k=3), emergency nursing (k=3), maternity nursing(k=1) and pediatric nursing (k=1). Usual lecture teaching control groups were most common (50%). The outcome measurements found in this study were categorized according to 3 domains of learning: cognitive outcomes,skillful outcomes and affective outcomes.

In case of the cognitive domain outcomes, knowledge was the most measured indicator and referred to 7 studies[13-16,18,21,22]. The majority of knowledge evaluation were the investigator-designed scale. In term of skillful domain, performance skills were assessed by instructor or investigator. There are 6 studies[15-17,19-21]that reported the outcome. With respect to the affective domain,Measures mentioned self-confidence and satisfaction were respective 3 studies[13,16,17].

Quality of studies

Of the 5 RCTs studies[13,15,17-19], 2 studies[18,19]reported using a random number table and 3 studies[13,15,17]did not describe details on random sequence. In the case of allocation concealment, only 1[19]of the studies provided adequate details. 3[13,17,19]of these studies used a masked approach to participants to reduce performance bias and others did not report it. As for blinding of outcomes assessment, none of the studies provided adequate details. Thus, detection bias of all studies were judged to be “unclear”. 4 studies[13,17-19]have not incomplete outcome data needed, however, there is a study[14]with high risk. All studies showed a low risk with respect to reporting bias. Regarding other bias, all studies provided an intervention scenario, nevertheless, it was not clear whether or not they evaluated the intervention procedure.(Figure 2)

Figure 1 Literature search and screening process

Assessment of 5 non-RCT studies for confounding bias, selection bias, classification of interventions bias,intended interventions bias, missing data bias, measurement of outcomes bias and reporting bias was performed using ROBINS. For the confounding factors of baseline and intervention, 2 studies[16,22]described the details,2 studies[20,21]did not mention, and 1 study[14]existed confounding bias. In selection of participants into the study bias is the same as confounding bias. For bias in classification of interventions, deviations from intended interventions were small, so 4 studies[14,16,21,22]were defined as low risk, only 1 study[20]remains unclear. Regarding missing data bias, only 1 study[22]did not fall off the subject, and the other 4 studies[14,16,20,21]all occurred. As for measurement of outcomes bias and reporting bias, there are low risks. (Table 2)

Effects of cognitive domain

On the knowledge side, the integration of 7 studies[13-16,18,21,22]showed some heterogeneity (I2=50%) and significant difference (95%CI 0.35[0.09, 0.62], P=0.009) (Figure 3). Due to the heterogeneity, subgroup analyses were conducted based on study design and control group variables. In knowledge acquisition subgroup analyses,2 studies involving 125 nursing students still had no significant effects when control group used model or simulation (95% CI -0.06 [-0.50, 0.39], P=0.81),indicating some heterogeneity between study estimates(I2=37%). Moreover, 5 studies[14,15,18,21,22], conductedtraditional method in the control group, showed a large amount of significant effects on knowledge (95% CI 0.50[0.29, 0.72], P＜0.00001), and statistical homogeneity was observed between study estimates (I2=0%). In general, the knowledge was statistically significant (P=0.009) (Figure 4).

Table 1Descript ive sum m ary of 10studies

Figure 2 Risk of bias assessment of RCT

Effects of skillful domain

A combined analysis across 5 studies[16,17,19-21]yielded strong heterogeneity and the difference was not statistically significant on performance skill (I2=97%,95% CI 1.05 [-0.54, 2.63], P=0.19) (Figure 5). Because of the existence of heterogeneity, subgroup analysis was used according to performance on the model after the experience. The 3 studies[16,19,21]found that the heterogeneity was significantly reduced, and there was statistically significant difference in terms of performance of post (I2=16%, 95% CI 1.45 [1.34, 1.56], P＜0.00001)(Figure 6). At the same time, a subgroup analysis was performed on the performance of experiment.2 studies[17,20]found some heterogeneity (I2=35%),manifesting no significant effects (P=0.48) (Figure 7).

Effects of affective domain

Emotional areas include self-confidence and satisfaction,combining 3 studies findings[13,16,17](201 subjects) had very homogeneity, then a fixed model was selected to get a statistically significant difference (I2=0%, 95% CI -0.43[-0.71,-0.15], P=0.003) (Figure 8).

Discussion

Although simulation-based learning using virtual reality is becoming increasingly prevalent in nursing and medical education, few comprehensive systematic reviews or meta-analyses of its effectiveness have been reported. Then according to Bloom[23], we conducted a meta-analysis of the effects of simulation-based learning using virtual reality. The aim of this meta-analysis was to examine whether virtual reality is effective in three domains of knowledge, performance, and affection.

The results of our meta-analyses suggested that simulation-based learning using VR appeared to have beneficial effects on the cognitive (P=0.009) and skill

performance domains of learning (P=0.19). However, in terms of affection, it is seemingly beneficial to the control group. In the simulation experiment, each student’s caring decision was evaluated, and personalized feedback was given after the scenario, which was helpful to reflect and translate knowledge into clinical nursing practice[24].However, learning can be deepened with guided and thoughtful reflection rather than mere simulation experiences and customized feedback[25].

Table 2 Risk of bias assessment of non-RCT

Figure 3 Knowledgescores of 2 groups

Figure 4 The knowledge scores were divided into the subgroup based on the traditional teaching methods and model teaching methods adopted by the control group

Through a comprehensive analysis of knowledge, there existed some heterogeneity, but it showed a significant difference. Then in the way of the control group was carried out an analysis, which was divided into traditional lecture teaching and model training. In general, the VR group was superior to the traditional teaching. The subgroup analysis was adopted, but the model training and virtual learning produced similar results. This actually proves that VR can achieve the same effect as model training. However, many problems have been found in model training. This did not seem to consider the degree of proficiency of each training material on model[26].While the benefits of simulations are well known, the educational benefits of high-fidelity simulators are limited,because of their cost of purchase and maintenance, the demands on long-term teams of professional teachers, and the small number of students that can be accommodated at one time[27].

Figure 5 Performance scores of 2 groups

Figure 6 Subgroup based on post-performance scores

Figure 7 Subgroup based on performance scores on the model after the scenario

Figure 8 Affective domain scores of 2 groups

Now the teaching method is mainly based on traditional teaching, and the model teaching is more and more common. Compared with direct teaching and active practice, students prefer the combination of theory and practice. But now that VR technology applied by nursing teaching is just emerging, and some researchers argue that this estimation might not have considered the fact that sufficient time to fully achieve educational goals was not allocated to the simulation process[28]. Students may not be familiar to virtual education and have no previous experiences, which may have affected their experiences and scores on the assessments. Participants need to know more about information on the technological issues. It is believed that this problem can be improved by more simulated task time[29]. This kind of repetition is essential if learners are to keep learning for the long term[30]. VR technology, if properly used in nursing education, will be an alternative to the replacement of model.

There is a great deal of heterogeneity in the skill performance. The sub-group variables are analyzed separately for the performance of the virtual practice process and the performance of the practice on the mannequin after the experience.The experimental group on the virtual task performed better than the control group on the non-virtual task, and the results were significant. In addition, Walsh advocates computer-based simulation as an opportunity for students to critically analyze patient situations and practice skills in a risk-free setting[31]. This indicates that virtual operations are likely to be beneficial to practice. However,there was no difference between the scores of the control group and mannequin practice after the VR training. This may have something to do with experimental group students not practicing with models beforehand.

The affective field showed the opposite result. This result is not conducive to the development of VR group, but the reasons analyzed made clear that the VR technology is just budding, students are used to traditional examinations and unfamiliar to VR, moreover, the lack of preparation may contribute to tension without confidence.Furthermore, it is similar to the subgroup analysis of knowledge, the 3 studies control groups used the mannequin model and simulation. It is well known that exposure to the familiarity helps to increase confidence.

Limitations

There are several limitations in this review. Firstly,although the initial idea was to consider only high-quality evidence from randomized controlled studies, even though this would reduce the number of selected articles, after the first stage of evaluation, we decided to relax the inclusion criteria, so this study included non-RCT, which may cause greater heterogeneity. Secondly, since some gray literatures are not available, there may be some research overlooked. Finally, due to the small sample size, further researches are required to confirm the effect of VR.

Conclusion

Although this study has certain limitations, it can be concluded that the use of VR as an emerging means of simulation learning may have a beneficial impact on knowledge acquisition and clinical skills acquisition. The results of this study show that the use of VR education may have a negative impact on self-confidence and satisfaction,but because there are few literatures, the amount of samples that can be integrated is small, so the results need further research. Future research should clearly state the intended outcome and the size of the expected educational effect, in order to allow a proper definition of the experimental setting, with special reference to the sample size of the study and control groups needed to obtain possibly significant findings and to a clear identification and control of confounding effects as differences among students in pre-test knowledge and attitudes to Internetbased learning[32]. Virtual teaching strategies provide a cost-effective and accessible solution that allows students to repeatedly interact in VR environments and has the potential to become a popular teaching method. The form of VR technology adopted, students use the knowledge and skills learned to complete virtual situational tasks, build their own new knowledge and skills system and improve the comprehensive nursing abilities.

Declaration

The authors of this article declare they have no conflict of interest.