A Randomized Controlled Trial of a Robotic Simulation Curriculum to Teach Robotic Suturing
BACKGROUND AND RATIONALE
A number of pressures make it challenging to learn robotic suturing in the operating room
including budgetary constraints necessitating near maximum operating room efficiency,
concerns about patient safety, and decreased trainee working hours. Robotic surgery is a
novel surgical technique which has significantly decreased surgical morbidity compared to
Robotic simulation is a technique that enables skill acquisition in robotic surgery without
the constraints of the operating room. The Mimic® dv-Trainer TM is a novel virtual reality
robotic surgery simulator which first became available in 2007. Further refinements and
developments led to incorporation of the Mimic software into the da Vinci® Skills Simulator.
This simulator has been shown to have face validity (appears realistic to experienced
surgeons) and construct validity (surgeons with more experience in robotic surgery score
higher than less experienced surgeons when performing the simulation exercises). The
impact of training using the da Vinci® Skills Simulator on performance using the actual
surgical robot (da Vinci® Surgical System) has started to be studied. One study assessed
the impact of training using the da Vinci® Skills Simulator on performance of three animal
tissue based tasks using the da Vinci® Si HD TM Surgical System (Hung, A.J., et al.,
Concurrent and predictive validation of a novel robotic surgery simulator: a prospective,
randomized study. Journal of Urology, 2012. 187(2): p. 630-7). One of three tasks assessed
was suturing closed a hole in an injured animal bladder. This study showed statistically
significant improvement in the animal tissue exercises for the subgroup of participants who
started with scores in the lower 50th percent for those exercises.
Our study will differ from this study in several ways. First, we will be focusing on a more
select group of simulation exercises focusing on robotic suturing (six exercises rather than
seventeen). Second, we will be developing a proficiency curriculum rather than one based on
time spent training. Third, our primary outcome will be performance of suturing, using the
actual surgical robot (da Vinci® Surgical System), of a vaginal cuff model made out of
inanimate materials rather than an animal tissue model. Due to logistic reasons, the
suturing of the inanimate model will be performed with the da Vinci® S Surgical System at
the March 23rd testing date (pretest for group 1) and with the da Vinci® Si Surgical System
at the April 27 testing date (pretest for group 2 and post-test 1 for group 1) and at the
June 1 testing date (post-test for group 2 and post-test 2 for group 1 controls who then
cross over to training in May). Our study will be open to trainees and surgeons from
obstetrics and gynecology, general surgery, urology, and cardiac surgery rather than just
SAMPLE SIZE CALCULATIONS
Our sample size calculations suggest the need for 38 participants, 19 in each group. This
was calculated using PASS 11. The plan is to compare the following: (a) control group
scores at final evaluation minus control group scores at initial evaluation versus (b)
training group scores at final evaluation minus training group scores at initial evaluation.
Thus, we assume two independent groups (because the differences in the control group should
be independent of the differences in the training group). For our sample size calculation,
we select the Mann-Whitney test and assume a mean difference between the improvement in
scores in the training group versus the improvement of scores in the control group of 7 (in
the context of a maximum possible score of 70 on any possible evaluation). We will also
assume a standard deviation of the differences of 7, an alpha of 0.05, and a power 0.8.
The non-parametric Mann-Whitney test (equivalent to the Wilcoxon Rank Sum Test) for
comparing the ranks of independent groups forms the basis for the estimates.
If we look at a sample size calculation for paired means (for example training group at
initial assessment versus post-training), we calculate the need for a sample size of 16 in
the training group (32 total with controls). The same assumptions for difference in means,
standard deviation, alpha and beta error are made as above. The Wilcoxon (signed rank) test
for comparing ranks of two paired groups forms the basis of the estimate.
RANDOMIZATION AND ALLOCATION CONCEALMENT
Randomization was achieved with an online random number generator (www.randomization.com) to
create a random sequence of train versus control for 40 participants using randomly permuted
blocks of 4 or 6 with a ratio of participants of 1:1 train:control in each block. The
randomization sequence was generated by an associate not otherwise involved with the study.
The group assignment (train or control) was written by this associate on an index card and
then a piece of carbon paper was stapled onto the front of the index card and it was placed
in an opaque, sequentially numbered, sealed envelope for opening by the participants in the
order of arrival on the pre-test day. Participants write their name on the front of the
envelope before opening so that this is transmitted by the carbon paper to the index card.
Two participants initially gave consent to participate on the condition that they would
train in a given 5 week period but then changed their mind on the day of the pre-test and
agreed to be randomized.
DESIGN AND DESCRIPTION OF METHODOLOGY
After obtaining informed consent, participants will undergo baseline testing on two domains:
(1) virtual reality robotic simulation using a da Vinci® Skills Simulator and (2) robotic
suturing of an inanimate model using the da Vinci® Surgical System. For testing using the
da Vinci® Skills Simulator, participants will perform the following task: Suture sponge 1.
Scoring will be done automatically by the built-in algorithm of the da Vinci® Skills
Simulator. Baseline scores for the other da Vinci® Skills Simulator tasks will be obtained
during the first training session. It is due to time constraints on the testing days that
we limit assessment with the da Vinci® Skills Simulator to the Suture Sponge 1 task alone.
For baseline physical (as opposed to virtual reality) robotic simulation testing,
participants will suture a vaginal cuff model using the actual surgical robot, the da Vinci®
Surgical System. Due to logistic reasons, the da Vinci® S Surgical System will be used for
the testing on March 23 (pre-test for group 1) and the da Vinci® Si Surgical System will be
used for the testing on April 27 (post-test 1 for group 1 and pre-test for group 2) and June
1 (post-test 2 for group 1 controls who do cross-over training in the period April 27 to May
31 and post-test for group 2). The vaginal cuff model is built from inexpensive inanimate
materials based on a prototype from the Journal of Robotic Surgery[Finan MA, C.M., Rocconi,
A novel method for training residents in robotic hysterectomy. Journal of Robotic Surgery,
2010. 4(1): p. 33-39.]. Their performance of this task will be recorded using the built-in
video camera of the da Vinci® Surgical System. Participants will be given a ten minute time
limit to try to suture as many figure of 8 knots in the model of the vaginal cuff as
possible. They are instructed to place a double throw and then three single throws for each
knot. At study conclusion, one expert robotic surgeon and one gynecologic oncology fellow,
blinded to participant identity and group randomization status, will grade the performance
of this task. In order to ensure blinding, the videorecordings will be arranged in random
order by a member of the study team (who is not involved in grading performance) in such a
way that no content is lost but so that the sequence of participants suturing and whether
this was a pre- or post-intervention recording is randomly arranged using a random number
generator. The original study codes will also be removed from this random sequence of videos
and a new study code will be attached to each video. A master sheet will be kept to
re-assign the original codes but will not be available to the evaluators until after grading
is completed. To score the videotaped performance, grading will performed using the GOALS
score for laparoscopy [Vassiliou, M.C., et al., A global assessment tool for evaluation of
intraoperative laparoscopic skills. American Journal of Surgery, 2005. 190(1): p. 107-13.]
plus two additional metrics described by AJ Hung [Hung, A.J., et al., Concurrent and
predictive validation of a novel robotic surgery simulator: a prospective, randomized study.
Journal of Urology, 2012. 187(2): p. 630-7.] specifically for robotics: 1) precision and 2)
instrument and camera awareness.
Following initial testing, participants will be randomized to two groups: 1)participation in
the da Vinci® Skills Simulator-based robotic suturing simulation proficiency curriculum in
addition to usual clinical/residency practice or 2) usual clinical/residency practice alone.
The robotic suturing simulation curriculum is a "proficiency" curriculum in that it is
focused on the goal of achieving proficiency targets for five of the six tasks included in
the curriculum. Those five tasks are : Camera targeting 1, Camera targeting 2, Suture
sponge 1, Suture sponge 2, and Suture sponge 3. For these five exercises, we will use the
latest version of the MScore TM assessment. MScore TM is part of the software accompanying
the da Vinci® Skills Simulator which provides scoring of task performance. We will ask
participants in the training group to aim to complete these exercises to the target score
(overall score green checkmark, equivalent to a "completed score") on two nonconsecutive
attempts. For the "Suturing skills (Symbionix TM): Horizontal suturing defect" task, the
scoring is different and adjusts to the participant's prior attempts, indicating whether
improvement has been made. Thus, we have elected to ask participants to aim to complete
this task ten times, rather than aiming for a particular score. The curriculum is
designated "proficiency-based" (with the exception of the Horizontal suturing defect task)
in that the goal is to achieve the target scores rather than to complete a set amount of
time training or a set number of repetitions. However, to provide some structure, we will
recommend at least five hours total of training sessions over the five week study period.
We will also facilitate the organization and scheduling of additional training time during
the study period, if desired by individual participants in the training group. In terms of
simulator access, participants in the training group will be using a da Vinci® Skills
Simulator attached to the mentoring console of a da Vinci Si robot. This da Vinci® Si robot
is used for actual surgery during regular working hours. In general this is Monday-Friday
during the regular, elective surgery schedule. Thus, the da Vinci® Skills Simulator will,
in general (unless elective robotic cases are cancelled or not scheduled that day), be
available to participants only after the elective robotic surgeries of the day are completed
on week days. On Saturday and Sunday, the da Vinci® Skills Simulator will generally be
available to participants all day.
The control group will carry on with regular clinical work or residency training. After
the intervention period, we will then re-evaluate all participants in the same da Vinci®
Skills Simulator task (Suture Sponge 1) and in the task of suturing the vaginal cuff model
using the da Vinci® S Surgical System. Testing of the task of suturing the vaginal cuff
model using the da Vinci® S Surgical System will be graded in the same manner as the
pretesting: one expert robotic surgeon and one gynecologic oncology fellow (blinded to
participant group and to whether the task was performed pre- or post-intervention) will
review the videotaped recordings.
Following this, the control group that started March 23rd will be allowed to crossover and
perform the proficiency-based curriculum. Finally, this group (originally controls, then
crossed over to training) will be tested one last time in the simulation task "Suture Sponge
1" using the da Vinci® Skills Simulator and in the task of robotic suturing of the vaginal
cuff model using the da Vinci® Surgical System. The robotic suturing of the vaginal cuff
model will again be assessed by review of videotapes by blinded experts. In order to
organize the training sessions participants will be given the investigator's contact
information (DK). Access to the da Vinci® Skills Simulator will be organized by a proctor
familiar with the da Vinci® Skills Simulator (DK), who will be present to facilitate access
to the simulator (given that it is located in the operating room) and to supervise.
There are two separate dates on which participants can enter the study and be randomized.
They are: March 23 or April 27, 2013. Participants entering the study March 23 will have
pretesting that day and be randomized to training March 24 to April 26 or April 28 - May 31,
2013. All participants entering the study March 23 will be asked to participate in a
posttest on April 27, 2013. Those participants randomized to training April 29 to May 31
(that is, the initial control group participants which elect to crossover to training in the
next period) will be asked to participate in a second post-test on June 1, 2013.
Participants who enter the study on April 27, 2013 will have a pretest that day and then be
randomized to training April 29 to May 31 or control with optional training June 2 to June
28. All participants entering the study April 27 will be asked to participate in a
post-test on June 1, 2013. Those participants entering the study on April 27 and randomized
to the control group April 28 to May 31 will have the option to crossover and complete
training June 2-28. However, at present, no post-test is planned for after this training
period (June 2 - 28) due to logistic reasons.
DATA ANALYSIS PLAN
There are three time periods for each of the two entry points (March 23 and April 27) in
this study: time 0 = initial testing; time 1 = post-intervention period; time 2 =
post-cross-over intervention period. To clarify, time 1 will occur at the time of
retesting following the initial study period of training or no training. Time 2 will occur
after the control group crosses over, completes training, and is then retested. A subgroup
analysis of the participants starting with pretest scores in the lower 50% is planned.
Independent group comparisons will also be conducted using the Wilcoxon rank sum test.
Independent group comparisons include (a) the differences from time 0 to time 1 for training
group versus (b) the differences from time 0 to time 1 for the control group. Paired group
comparisons will be conducted with the Wilcoxon signed rank test. Paired group comparisons
include: training group at time 0 versus at time 1 and control group with cross-over
training at time 2 versus at time 1 and at time 1 versus time 0.
Allocation: Randomized, Intervention Model: Crossover Assignment, Masking: Single Blind (Outcomes Assessor)
Ability to suture an inanimate model with the da Vinci Surgical System
Participants will suture an inanimate model of the vaginal cuff using the da Vinci Surgical System. The performance of this task will be videotaped using the da Vinci Surgical System's built in camera. Surgical video will be scored by one expert robotic surgeon and one gynecologic oncology fellow, blinded to intervention group, participant identity, participant experience level, and pre- versus post-intervention status. Surgical video will be scored using the Global Operative Assessment of Laparoscopic Skills (GOALS) in addition to two additional metrics developed by AJ Hung et al specifically for robotics: 1) instrument and camera awareness and 2) precision. The GOALS score contains 5 domains. The 5 domains of the GOALS score and the two additional domains developed by AJ Hung et al are each scored on a 5 point Likert scale providing a maximum possible score of 35 per rater and 70 overall.
Daniel J Kiely, MDCM FRCSC
Canada: Ethics Review Committee