Background. The equality of subjective- and objective-assessment methods in laparoscopic surgery are unknown. The aim of this study was to compare a subjective assessment method to an objective assessment method to evaluate laparoscopic skill. Methods. A prospective observational cohort study was conducted. Seventy-two residents completed a basic laparoscopic suturing task on a box trainer at two consecutive assessment points. Laparoscopic skill was rated subjectively using the Objective Structured Assessment of Technical Skills (OSATS) list and objectively using the TrEndo, an augmented-reality simulator. Results. TrEndo scores between the two assessment points correlated. OSATS scores did not correlate between the two assessment points. There was a correlation between TrEndo and OSATS scores at the first assessment point, but not at the second assessment point. Overall, OSATS scores correlated with TrEndo scores. There was a greater spread within OSATS scores compared to TrEndo scores. Conclusion. OSATS scores correlated with TrEndo scores. The TrEndo may be more responsive at rating individual’s laparoscopic skill, as demonstrated by a smaller overall spread in TrEndo scores. The additional value of objective assessment methods over conventional assessment methods as provided by laparoscopic simulators should be investigated. 1. Introduction Simulation based training in laparoscopic or minimally invasive surgery (MIS) is gaining recognition. To enhance patient safety the initial learning curve in MIS may be moved outside the operating room (OR). Several validation studies demonstrated simulator based training to improve laparoscopic psychomotor skills and OR performance [1–3]. Simulation-based training is also cost-effective and easily implementable in a surgical training curriculum [4, 5]. Three classes of MIS simulators are currently available: computerized virtual reality trainers (VR), traditional box trainers (BT) and a combination of the former two; augmented reality trainers (AR) [6, 7]. All simulators aim for a maximal realistic setting. However, laparoscopic basic skills training remains unrealistic as VR and AR simulators are computer based. Furthermore, VR and AR simulators do not provide tactile (haptic) feedback. Box trainers do preserve a realistic setting including haptic feedback and augmented depth perception [8] but would lack objective assessment methods and a low fidelity to actual laparoscopic procedures [9]. Training in MIS should include objective assessment. Various MIS assessment methods have been developed and typically
References
[1]
G. M. Fried, L. S. Feldman, M. C. Vassiliou et al., “Proving the value of simulation in laparoscopic surgery,” Annals of Surgery, vol. 240, no. 3, pp. 518–528, 2004.
[2]
D. J. Scott, P. C. Bergen, R. V. Rege et al., “Laparoscopic training on bench models: better and more cost effective than operating room experience?” Journal of the American College of Surgeons, vol. 191, no. 3, pp. 272–283, 2000.
[3]
N. E. Seymour, A. G. Gallagher, S. A. Roman et al., “Virtual reality training improves operating room performance results of a randomized, double-blinded study,” Annals of Surgery, vol. 236, no. 4, pp. 458–464, 2002.
[4]
M. T. Goova, L. A. Hollett, S. T. Tesfay et al., “Implementation, construct validity, and benefit of a proficiency-based knot-tying and suturing curriculum,” Journal of Surgical Education, vol. 65, no. 4, pp. 309–315, 2008.
[5]
D. J. Scott, E. M. Ritter, S. T. Tesfay, E. A. Pimentel, A. Nagji, and G. M. Fried, “Certification pass rate of 100% for fundamentals of laparoscopic surgery skills after proficiency-based training,” Surgical Endoscopy and Other Interventional Techniques, vol. 22, no. 8, pp. 1887–1893, 2008.
[6]
S. M. B. I. Botden and J. J. Jakimowicz, “What is going on in augmented reality simulation in laparoscopic surgery?” Surgical Endoscopy and Other Interventional Techniques, vol. 23, no. 8, pp. 1693–1700, 2009.
[7]
E. J. B. Furnée, P. J. Van Empel, B. M. Delavary, D. L. Van Der Peet, M. A. Cuesta, and W. J. H. J. Meijerink, “Evaluation of a technical skills training program in surgical residents,” Journal of Laparoendoscopic and Advanced Surgical Techniques, vol. 19, no. 5, pp. 615–621, 2009.
[8]
P. J. Van Empel, W. M. Van Der Veer, L. B. Van Rijssen et al., “Mapping the maze of minimally invasive surgery simulators,” Journal of Laparoendoscopic and Advanced Surgical Techniques, vol. 22, no. 1, pp. 51–60, 2012.
[9]
R. Aggarwal, K. Moorthy, and A. Darzi, “Laparoscopic skills training and assessment,” British Journal of Surgery, vol. 91, no. 12, pp. 1549–1558, 2004.
[10]
S. Cotin, N. Stylopoulos, M. Ottensmeyer, P. Neumann, D. Rattner, and S. Dawson, “Metrics for laparoscopic skills trainers: the weakest link!,” in Medical Image Computing and Computer-Assisted Intervention, vol. 2488 of Lecture Notes in Computer Science, pp. 35–43, 2002.
[11]
K. Moorthy, Y. Munz, A. Dosis, F. Bello, and A. Darzi, “Motion analysis in the training and assessment of minimally invasive surgery,” Minimally Invasive Therapy and Allied Technologies, vol. 12, no. 3-4, pp. 137–142, 2003.
[12]
R. K. Reznick, “Teaching and testing technical skills,” The American Journal of Surgery, vol. 165, no. 3, pp. 358–361, 1993.
[13]
D. Dath, G. Regehr, D. Birch et al., “Toward reliable operative assessment: the reliability and feasibility of videotaped assessment of laparoscopic technical skills,” Surgical Endoscopy and Other Interventional Techniques, vol. 18, no. 12, pp. 1800–1804, 2004.
[14]
J. A. Martin, G. Regehr, R. Reznick et al., “Objective structured assessment of technical skill (OSATS) for surgical residents,” British Journal of Surgery, vol. 84, no. 2, pp. 273–278, 1997.
[15]
B. A. Goff, G. M. Lentz, D. Lee, D. Fenner, J. Morris, and L. S. Mandel, “Development of a bench station objective structured assessment of technical skills,” Obstetrics & Gynecology, vol. 98, no. 3, pp. 412–416, 2001.
[16]
P. D. Van Hove, G. J. M. Tuijthof, E. G. G. Verdaasdonk, L. P. S. Stassen, and J. Dankelman, “Objective assessment of technical surgical skills,” British Journal of Surgery, vol. 97, no. 7, pp. 972–987, 2010.
[17]
C. B. M. Bijen, J. M. Bri？t, G. H. de Bock, H. J. G. Arts, J. A. Bergsma-Kadijk, and M. J. E. Mourits, “Total laparoscopic hysterectomy versus abdominal hysterectomy in the treatment of patients with early stage endometrial cancer: a randomized multi center study,” BMC Cancer, vol. 9, article 23, 2009.
[18]
M. K. Chmarra, S. Klein, J. C. F. De Winter, F. W. Jansen, and J. Dankelman, “Objective classification of residents based on their psychomotor laparoscopic skills,” Surgical Endoscopy and Other Interventional Techniques, vol. 24, no. 5, pp. 1031–1039, 2010.
[19]
P. J. van Empel, L. B. van Rijssen, J. P. Commandeur et al., “Validation of a new box trainer-related tracking device: the TrEndo,” Surgical Endoscopy, vol. 26, no. 8, pp. 2346–2352, 2012.
[20]
R. Aggarwal, J. Hance, S. Undre et al., “Training junior operative residents in laparoscopic suturing skills is feasible and efficacious,” Surgery, vol. 139, no. 6, pp. 729–734, 2006.
[21]
V. Datta, A. Chang, S. Mackay, and A. Darzi, “The relationship between motion analysis and surgical technical assessments,” American Journal of Surgery, vol. 184, no. 1, pp. 70–73, 2002.
[22]
R. Aggarwal, T. Grantcharov, K. Moorthy et al., “An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room,” Annals of Surgery, vol. 245, no. 6, pp. 992–999, 2007.
[23]
S. G. C. Kroeze, E. K. Mayer, S. Chopra, R. Aggarwal, A. Darzi, and A. Patel, “Assessment of laparoscopic suturing skills of urology residents: a pan-European study,” European Urology, vol. 56, no. 5, pp. 865–873, 2009.
[24]
W. Kolkman, R. Wolterbeek, and F. W. Jansen, “Gynecological laparoscopy in residency training program: Dutch perspectives,” Surgical Endoscopy and Other Interventional Techniques, vol. 19, no. 11, pp. 1498–1502, 2005.
[25]
L. P. Mandel, G. M. Lentz, and B. A. Goff, “Teaching and evaluating surgical skills,” Obstetrics and Gynecology, vol. 95, no. 5, pp. 783–785, 2000.
