Your message has been successfully sent to your colleague.
Margalit, Adam MD; Suresh, Krishna V. BS; Marrache, Majd MD; Lentz, Jonathon M. DO; Lee, Rushyuan MD; Tis, John MD; Varghese, Ranjit MD; Hayashi, Brooke DO; Jain, Amit MD; Laporte, Dawn MD
From the Department of Orthopaedic Surgery, Johns Hopkins University School of Medicine, Baltimore, MD.
Correspondence to Dr. Laporte: [email protected]
This project was supported by a grant from OrthoPediatrics.
Dr. Laporte or an immediate family member serves as a member on the advisory board for Osso VR Training platforms. None of the following authors or any immediate family member has received anything of value from or has stock or stock options held in a commercial company or institution related directly or indirectly to the subject of this article: Dr. Margalit, Suresh, Dr. Marrache, Dr. Lentz, Dr. Lee, Dr. Tis, Dr. Varghese, Dr. Hayashi, and Dr. Jain.
This is an open access article distributed under the Creative Commons Attribution License 4.0 (CCBY) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
The purpose of this study was to compare outcomes between orthopaedic trainees using various preoperative training platforms (physical simulation [PS], virtual reality [VR], and reading/videos) in a slipped capital femoral epiphysis model.
Participants were randomly assigned to one of the three groups: (1) reading/video control group (n = 7), (2) VR group (n = 7), or (3) PS group (n = 7). Participants in the VR group completed a VR slipped capital femoral epiphysis module while participants in the PS group practiced the placement of a screw in the physical module before evaluation of percutaneous screw placement in the PS model. Outcomes evaluated included overall surgical time, amount of fluoroscopy, Global Rating Scale score, radiographic screw position, physical screw accuracy, presence of breeching of the articular surface or femoral neck, and overall platform rating (0 to 10).
No difference was observed in surgical time, Global Rating Scale score, radiographic or physical accuracy of screw position, or articular surface breaching between the groups. Subjectively, there was a difference in utility of platform rating between the groups (PS: 10 ± 0, VR: 7 ± 2, and control: 6 ± 1, P = 0.001).
Training with VR was subjectively rated higher in value compared with reading/video methods and had similar performance outcomes compared with training with PS.
Surgical skills simulation allows for the practice of procedures in a risk-free environment and has been shown to improve patient safety and optimize clinical outcomes. Although not widely adopted, both traditional virtual reality (VR) surgical platforms (plastic models with arthroscopic haptic and tactile feedback with interactive surgical instruments) and more modern VR systems (simulators with head-mounted display and generic hand controllers) may have some benefit in orthopaedic training. Previous literature reports that VR simulator platforms may improve resident skill level in the operating room or on cadaver models for knee arthroscopy or total hip arthroplasty.1-6 Other studies have demonstrated improvements in procedural accuracy and rate of successful completion of tibial intramedullary nailing among medical students trained with VR headsets compared with a technique guide review.7-9 However, there is a paucity of literature that has directly compared VR as a surgical training platform in pediatric orthopaedic procedures with other educational platforms in novice orthopaedic trainees, with limited information on the transferability of acquired skills from simulation to the operating room.
Slipped capital femoral epiphysis (SCFE) screw fixation is a routine pediatric orthopaedic procedure which demands advanced three-dimensional skills. The utility of VR as a surgical learning platform to train novice orthopaedic trainees on how to perform in situ screw fixation has not yet been evaluated. The primary purpose of this study was to compare screw placement accuracy and surgical technique among orthopaedic trainees trained with either VR, isolated text, video, and literature review or physical simulation (PS) training in a SCFE sawbones model.
Twenty-one orthopaedic trainees, consisting of 9 junior orthopaedic residents and 12 senior medical students, were recruited from a single institution to participate in this study. A customized phantom limb model (Sawbones) encasing a SCFE radiopaque Sawbones femur was used to simulate a pediatric patient (Figure 1). Trainees were randomly assigned to one of the three educational platforms. After completing their respective educational platforms, each trainee was allowed one trial to perform in situ pin fixation on the phantom limb with conventional C-Arm (Cios Fusion; Siemens Sector Healthcare). Outcomes evaluated included overall surgical time, number of radiographs taken, Global Rating Scale (GRS) score, radiographic screw position, physical screw accuracy, presence of breaching of the articular surface or femoral neck, life-like rating of the physical module (0 to 10), and overall platform rating (0 to 10).
For each trial, a reusable phantom limb model of a left pediatric thigh was loaded with a grade III (severe) SCFE Sawbones model and placed on a radiolucent table. Bone models were replaced with a new sample between every trial. Before beginning a trial, a felt sticker was used to cover previous incisions sites on the reusable model. Surgical instruments included a scalpel, 2.8-mm Kirschner wire with power, a 4.5-mm drill bit with power, and a 7.0-mm cannulated screw (OrthoPediatrics). Operating room setup and instrument tray are shown in Figure 2. Trainees were given the opportunity to visualize AP and lateral preoperative radiographs of the SCFE femur to plan guide pin trajectory and start point using preoperative planning software (Surgimap; Nemaris) immediately before the trial (Figure 3).
Trainees were assigned to one of the three educational platforms: (1) group 1 (control group): standardized reading and video material10-13 on how to perform in situ screw fixation (Appendix I, https://links.lww.com/JG9/A202 ), (2) group 2 (PS group): PS training on the SCFE Sawbones model, or (3) group 3 (VR group): VR-based SCFE simulation training module using the Osso VR platform (Figure 4). All groups had access to the standardized reading and video material. Trainees in group 1 were not allowed to use any resources or training material outside the ones that were provided. PS training consisted of a timed 15-minute session in which participants were allowed to practice guide pin placement on the SCFE Sawbones model. During this time, participants were allowed to ask any questions regarding SCFE screw fixation, interact with the physical model and surgical instruments, and practice the procedure. Participants in the VR group were trained on the Osso VR platform and asked to complete a standardized SCFE fixation module (Osso VR). This module allows users to freely manipulate the C-arm virtually from the AP to lateral positions during the procedure. The program also requires users to not only obtain images but also interpret them to manually adjust the position of the guide wire on the screen using physical hand controllers. Hand controllers did not provide any tactile feedback to users, with exception to vibration when using an instrument. Participants were asked to repeat the module until they achieved three “A” grades on pin placement accuracy, as graded within the VR module. Participants were not timed during completion of the VR practice modules.
Trainees were given the option to either maintain the C-arm in a single position and manually maneuver the limb to obtain an AP and frog-leg lateral view or position the C-arm while keeping the phantom model static and obtaining an AP and lateral view. An imaging technology was available for all C-arm trials to assist in intraoperative positioning of the C-arm. Total number of AP and lateral images were recorded for each trial.
To gauge physical accuracy, the center of the femoral head was marked preoperatively to indicate optimal screw trajectory. After placement of the screw, a 2.8-mm Kirschner wire was used to breech the cortex of the femoral head to assess postoperative screw trajectory. Screw placement accuracy was measured as the shortest distance from the preoperative marked site on the femoral head to actual postoperative trajectory (Figure 5).
Radiographic accuracy of pin placement within the epiphysis was graded on both AP and lateral radiographs using an A-C system as previously described by Pring et al.14 As described by Pring et al, “grade A” accuracy of pinning implied that the pin was placed within the central 50% of the physeal width, with the screw tip 5 to 10 mm across the physis, at least 5 mm from the subchondral bone, and at an angle of 70° to 90° to the capital physis. “Grade B” accuracy of pinning was reported when the pin was placed outside the “A” area, but not in the “C” area. This was considered to be an area containing the central 75% minus the central 50% of the physeal width, with the screw tip less than 5 mm across the physis or less than 5 mm from the subchondral bone, and at an angle of 50° to 90° to the physis. “Grade C” accuracy of pinning described screws placed outside the central 75% of the physeal width, less than 2.5 mm of screw tip across the physis, less than 2.5 mm from the subchondral bone, or at an angle less than 50° to the physis (Figure 6). Two board-certified orthopaedic surgeons (A.J. and B.H.) graded screw trajectory independently using the A-C grading system template.
In addition, both providers independently graded blinded recorded videos of participants on skill level using GRSs. GRSs have been developed as generic tools that assess performance across several skill domains (including respect for tissue, time, and motion, instrument handling/knowledge, flow of operation, knowledge of specific procedure, and communication skills) to generate an overall performance rating (range: 6 to 30 in our study). GRSs have been used in multiple surgical specialties in both simulated and clinical settings.15
Rates of screw articular breaching and femoral neck in-out-in screw placement were recorded for each group. Articular breeching was defined as screw penetration through the articular surface of the femoral head. In-out-in was defined as screw penetration outside the femoral neck and re-entry into bone at a more proximal location within the femoral neck or head. Examples of screw breach and in-out-in screw placement are shown in Figure 7.
We used standard analysis of variance testing to compare continuous variables and chi-square test to compare categorical variables. Statistical significance was defined as a P value of 0.05 or less. Statistical analyses were done with Stata, version 15 (StataCorp LP).
We recruited 21 junior orthopaedic residents and medical students from a single institution to participate in this study. Participants were equally divided into each of the three study platforms, with each group consisting of four medical students and three junior residents. All study participants completed the standardized reading and video study material before participation and verified that no additional learning tools were used.
No significant differences were observed in mean (±SD) physical accuracy of the screw placement between all three study platforms (PS: 10.5 ± 7.9 mm, control: 9.4 ± 7.7 mm, VR: 13.3 ± 10.2 mm, P = 0.69).
No significant differences were observed in mean (±SD) total surgical time between all three study platforms (PS: 14.2 ± 5.2 minutes, control: 24.4 ± 16.3 minutes, VR: 28.5 ± 10.2 minutes, P = 0.085). Compared with the control and PS groups, participants in the VR group took significantly greater numbers of AP, frog-leg/lateral, and total radiographs (Table 1). No differences were observed in the number of AP (P = 0.31), frog-leg/lateral (P = 0.83), and total radiographs (P = 0.62) between the PS or control groups.
Across study platforms, no significant differences were observed in the number of participants who breached the articular surface of femoral head (PS: 5/7, control: 3/7, VR: 3/7, P = 0.47). No differences were also observed in the number of screws that breeched the femoral neck before entering the femoral head (PS: 0/7, control: 2/7, VR: 4/7, P = 0.061).
When assessed by two independent reviewers (B.H. and A.J.), there were no significant differences in AP (Table 2) or lateral (Table 3) radiographic accuracy of screw placement across all three study platforms, as evaluated by the ABC grading system using templates overlayed onto the radiographs (AP: P = 0.413, lateral: P = 0.118, Figure 8). Inter-rater reliability was rated to be “perfect” (kappa = 1), with no disagreements.
Across all three study platforms, no significant differences were observed in GRS scores (reviewer 1: P = 0.11, reviewer 2: P = 0.06; Table 4). Paired t-test analysis for each participant determined no significant differences in GRS scores between the two reviewers for a given participant (reviewer 1: 15.3, reviewer 2: 15.24, P = 0.74).
When participants were asked to evaluate how life-like (Likert scale: 0 to 10, 0 = not life-like, 10 = extremely life-like) their study platform was, no significant differences were observed in mean (±SD) between the three groups (PS: 8 ± 2, control: 7.7 ± 1.4, VR: 7 ± 3.2, P = 0.71).
Participants ranked PS as the most useful study platform compared with the control group and VR group (PS: 9.7 ± 0.5, control: 6.0 ± 1.0, VR: 7.3 ± 2.3, P = 0.0006).
Participants in the PS group reported that the study platform was “extremely useful,” “felt like an operating room,” and “was an extremely enjoyable experience.” Participants in the control group reported that the “videos were excellent and useful” but reported that “having a physical model would have been more helpful.” Participants in the VR group believed that “it was helpful in understanding concepts of surgery” and “useful for the portion of the physical simulation when inserting the Kirschner wire.”
We conducted a prospective study comparing the educational utility of VR to other learning methods in improving the surgical performance of orthopaedic trainees for in situ screw fixation of high-grade SCFE. We assessed objective parameters, including screw accuracy, breaching of the femoral head or neck, surgical time, radiographic accuracy, surgical technique, and collected subjective evaluations of each educational platform. Our results indicate that no significant differences were observed in most objective parameters. Subjectively, participants preferred PS, followed by VR, and then conventional book/video material. Overall, our results may suggest that although practicing with a physical SCFE model may be the most realistic method to train novice surgeons, VR training modules may yield similar radiographic accuracy and surgical technique performance while offering a more convenient to practice surgical skills.
VR simulation has been previously described to improve resident skill level in the operating room or on cadaver models for knee arthroscopy, shoulder arthroscopy, and total hip arthroplasty.1-6 For example, a randomized controlled trial by Banaszek et al1 found that residents who trained with a knee arthroscopy VR simulator that provided haptic and tactile feedback had superior GRS scores in a cadaver model compared with residents who trained with a bench-top Sawbones model. More modern VR simulators with head-mounted displays have also demonstrated improvements in surgical performance, even among novice trainees. For example, a randomized trial by Blumstein et al demonstrated that compared with passive methods of learning (technique guide review), VR headsets with hand controllers provided improved procedural in intramedullary nailing among first-year and second-year medical students. However, educational benefits of modern VR surgical training platforms and the transferability of skills to a physical model have never been evaluated in pediatric orthopaedic procedures.
Unlike the performance benefits previously reported in intramedullary nailing procedures among novices, our results indicated that no notable differences were observed in most objective parameters (screw accuracy, radiographic grade, surgical technique, and breaching) between a modern VR headset and practice with a physical model. These results are likely secondary to (1) the relative technical difficulty of in situ SCFE fixation and (2) the lack of notable prior experience with SCFE fixation in our cohort of participants. SCFE fixation with in situ screws or pins has been previously associated with high rates of complications, even when done by experienced providers. For example, in a cohort of 202 pediatric patients with SCFE treated with screw fixation by trained orthopaedic surgeons, Riley et al16 reported a complication rate of 26%, with joint penetration being the most common complication. Therefore, the inherent difficulty of performing this procedure may have masked any short-term gains in surgical performance from VR training in an extremely novice cohort. Moreover, the learning curve for successfully performing a SCFE procedure likely requires more than one trial. Although the minimum number of procedures to gain proficiency in SCFE fixation has not been previously quantified, the recommended number of supervised practice procedures for pediatric orthopaedic fellows for other procedures, such as supracondylar humerus pin fixation, has been reported to be approximately 15.17 Therefore, in our novice cohort, increased exposure to the basics of SCFE fixation and multiple trials may have resulted in greater differences among various educational platforms.
Although our results indicated that VR training offered similar performance as a phantom limb or a didactic teaching model, the mobility of modern VR platforms may offer extra convenience benefits. The novel coronavirus pandemic in 2020 forced orthopaedic residency curriculums to develop and adopt more remote-based learning modalities.18 Often times, these include more didactic sessions and video/text-based learning modules to decrease resident exposure to the virus.18 VR training modules may offer excellent supplementary content to didactic learning because it enables residents to directly visualize important surgical concepts and repeat procedures multiple times from the comfort and safety of their home while avoiding skill decay. However, despite the convenience of this system, program directors must consider the cost of the Osso VR platform. The hardware alone needed to run the Osso VR module is approximately 300 dollars for a single unit; residency leadership may consider bulk ordering the hardware and software implants from Osso for resident access as part of institutional educational funds. Although training with cadaver models may offer a more cost-effective method of instruction, because these specimens can be bulk ordered and used for multiple procedures across many residents, access to these models during times of social distancing and mandatory remote learning may preclude residents from participating in such learning opportunities. In this scenario specifically, surgical education with VR technology may be particularly useful and have a justified upfront cost.
Although the portability of the VR headset system offers notable convenience and advantages during a pandemic, participants offered some suggestions for VR training improvement to better suit their needs. First, the controllers with the VR system are not modular. As such, the same hand controller must represent a variety of different surgical instruments, decreasing the realism of the virtual simulation and potentially limiting gains in actual surgical technique. This could be rectified with addition of secondary controllers or instruments in the same VR kit. Second, we did not use the available test mode in the VR module, which would allow the user to critically think about pin placement rather than following outlined trajectories in the training module. Users may have gained further improvements in performance if both training and testing modules were completed before surgical simulation. Finally, the VR headset module for SCFE fixation does not have any modality to artificially palpate anatomic landmarks on the extremity to mark the incision site or feel differences in resistance while placing the guide wire. Although tactile components of anatomy are primarily limited by the portability of the system, ability to identify the incision site should be tested in the VR modules. These missing attributes in VR learning are key to mastering SCFE fixation and highlight the benefits of using a physical or cadaver model. Moreover, these limitations may explain why the VR group required greater radiograph images during the evaluation process.
There are limitations to our study. First, our study had a small sample size, reducing the external validity of our results. Owing to the low sample size, outliers in the data set may have more notable effects on the mean. For example, one participant in the VR group took a total 185 radiographs during surgical simulation, compared with the next highest participant in the group with 132 radiographs. Notably, although there were no statistically significant differences in outcomes between all three platforms, our findings seem to trend toward favoring PS training over VR. Perhaps with a greater study sample size, these differences may be further pronounced. Moreover, individuals in the PS group were allowed to train and subsequently evaluated on the same phantom limb, which likely introduced some bias. Finally, using a sample of participants with limited orthopaedic experience may result in diminished gains in improvement with any educational platform. Participants may require prolonged exposure to the physical model and fundamentals of SCFE fixation to realize benefits from any particular surgical training methodology.
VR training holds promise for the future of orthopaedic training, and no statistically significant differences were observed in in situ screw placement accuracy, radiographic grades, and surgical technique compared with practice with a physical model in novice orthopaedic trainees. However, those trained with physical models performed nominally better across most of the outcomes and subjectively evaluated this method of training as the most life-like and useful. Future work may focus on improving current VR technology to make controllers more modular and realistic while simultaneously expanding training modules to be more detailed-oriented and rigorous (https://links.lww.com/JG9/A203 and https://links.lww.com/JG9/A204 ).
The authors thank Dr. Paul D. Sponseller, MD, for his constructive feedback regarding study design and implementation.
Your message has been successfully sent to your colleague.
Your message has been successfully sent to your colleague.
This website uses cookies. By continuing to use this website you are giving consent to cookies being used. For information on cookies and how you can disable them visit our Privacy and Cookie Policy.