Table 1 Overview of included studies, categorised in order of machine learning aim

From: Systematic review of machine learning applications using nonoptical motion tracking in surgery

Index	Author	Year	Sensor	Video	Field	Task	Subjects	Trials	Machine Learning Model	Performance Metric (%)	Cross-Validation
Skill Assessment
1	Ahmidi, N⁷².	2015	EM	CO	Open	CM	14	86	(Stroke-based) SVM	MA: 74.24-90.91	LOTO, LOUO
									Descriptive Curve Coding + SVM	MA: 81.03-91.66
									HMM + SVM	MA: 23.06-70.93
2	Albasri, S¹².	2021	DK (J)	CO	Robotic	BB	10	150	Procrustes DTW + kNN	MA: 88.9-100	LOSO
2	Albasri, S¹².	2021	I	No	Open	CS	4	120	Procrustes DTW + kNN	MA: 80-100	LOSO, LOTO
3	Allen, B⁷⁰.	2010	EM	No	Lap.	BB	30	696	SVM	MA: 90-93.7	Hold out
4	Baghdadi, A⁵⁰.	2020	DK + M	No	Robotic	BB	30	1440	LASSO + RF	MA: 63	k-fold
									LASSO + kNN	MA: 63
									LASSO + LR	MA: 70
									LASSO + RF + kNN + LR	MA: 78
5	Bissonnette, V⁴⁶.	2019	DK	No	Open	CS	41	41	SVM	MA: 97.6	LOO, k-fold
									kNN	MA: 92.7
									LDA	MA: 87.8
									Naive Bayes	MA: 86.9
									Decision tree	MA: 70.7
6	Brown, J.D⁸⁵.	2017	I + M	CO	Robotic	BB	38	110	SVM + Elastic Net Regression + Regression Trees + kNN	MA: 63.3-73.3	LOO
6	Brown, J.D⁸⁵.	2017	I + M	CO	Robotic	BB	38	110	RF	MA: 51.7-75	LOO
7	Brown, K.C³².	2020	DK	CO	Robotic	CM	-	100-131	LR	MA: 76.32-98.27	k-fold
8	Chen, A.B³⁹.	2021	DK	CO	Robotic	CM	17	68	RF	MA: 71.6-76.9	-
									AdaBoost	MA: 69.9-80.1
									Gradient Boosting	MA: 67.2-78.4
9	Fard, M.J⁵³.	2018	DK (J)	CO	Robotic	BB	8	80	kNN	MA: 71.9-89.7	LOSO, LOUO
									LR	MA: 70.2-89.9
									SVM	MA: 75.4-79.8
10	Horeman, T⁹².	2012	M	No	Lap.	BB	31	93	PCA + LDA	MA: 78-84	LOO
11	Hung, A.J³⁸.	2018	DK	No	Robotic	CM	9	78	RF	MA: 87.2	Stratified k-fold
									SVM	MA: 83.3
									LR	MA: 82.1
12	Hung, A.J¹⁰.	2019	DK	No	Robotic	CM	8	100	MLP (DeepSurv)	-	k-fold
13	Hung, A.J⁶⁸.	2022	DK	CO	Robotic	BB	22	226	NoiseRank + LSTM	-	-
14	Jiang, J⁷³.	2017	EM	CO	Robotic	BB	10	10	DTW	-	-
15	Jog, A⁶⁷.	2011	DK	No	Robotic	BB	17	41	Decision tree + SVM	MA: 67.5-87.5	k-fold
16	Kelly, J.D⁴⁰.	2020	DK	CO	Lap.	BB	117	454	Bi-LSTM	MA: 73.33-96.88	Hold out
17	Khan, A⁸⁶.	2020	I	CO	Open	BB	15	50	SVM	-	LOTO, LOUO, k-fold
18	Laverde, R⁸⁸.	2018	I	No	Lap.	BB	7	207	ANN	-	k-fold
19	Li, K⁵¹.	2020	DK (J)	No	Robotic	BB	-	96	kMC + DNN	ME: 9.18-9.47	-
20	Lin, Z⁸⁹.	2011	I	No	Lap.	BB	16	48	PCA + LDA	MA: 93.75	LOO
21	Lin, Z⁸⁷.	2013	I	No	Lap.	BB	16	96	PCA + LDA	MA: 94	LOO
22	Lyman, W.B⁵².	2021	DK	No	Robotic	CS	2	25	Kernel Regularised Linear Squares Multivariate prediction + Multivariate Linear Regression	MA: 89.3	-
23	Megali, G⁴⁸.	2006	DK	No	Lap.	BB	6	24	HMM	-	Hold out
24	Oquendo, Y.A⁷¹.	2018	EM + M	CO	Lap.	BB	32	63	Regularised Least Squares + Regression Trees	MA: 38-88	LOUO
25	Sbernini, L⁹⁰.	2018	I + M	No	Open	BB	18	360	LDA	ME:5.86-8.06	LOO
									SVM	ME: 0.89-2.05
									MLP	ME: 0.57-0.61
26	Sewell, C⁶⁹.	2008	DK	CO	Open	CS	15	30	HMM	MA: 87.5	LOO
									Naive Bayes	-
									LR	MA: 50-100
27	Soangra, R¹³.	2022	I + EMG	No	Open + Lap. + Robotic	BB	26	234	RF	MA: 40-60	Hold out
									Naive Bayes	MA: 28-47
									SVM	MA: 35-57
28	Uemura, M⁴¹.	2018	EM	No	Lap.	BB	67	67	Chaotic NN	MA: 79	Hold out
29	Wang, Z.H⁴³.	2018	DK (J)	CO	Robotic	BB	8	40	CNN	MA: 84.9-95.4	LOSO, Hold out
30	Watson, R.A⁹¹.	2014	I	No	Other	CS	24	48	SVM	MA: 83	-
31	Xu, J⁹³.	2023	M	No	Open	BB	13	20	LSTM	MA: 76.67-78.86	LOUO
									Bi-LSTM	MA: 80.51-84.92
									GRU	MA: 75.46-77.57
									Convolutional LSTM DNN	MA: 93.65-96.19
									Transformer network	MA: 86.68-90.67
									TCN	MA: 88.95-97.45
32	Zhang, D²⁰.	2020	DK	Yes	Robotic	BB	8	66	CNN	MA: 84.72-97.92	LOSO
32	Zhang, D²⁰.	2020	DK (J)	CO	Robotic	BB	8	103	CNN	MA: 80.80-99.17	LOSO
Feature Detection
33	Ahmidi, N²¹.	2017	DK (J)	CO	Robotic	BB	8	101	LDA + GMM-HMM	MA: 64.12-92.56	LOSO, LOUO
									K-Singular Value Decomposition + Sparse-HMM	MA: 62.48-83.54
									Markov semi-Markov CRF	MA: 44.68-81.99
									Skip Chain CRF	MA: 74.77-85.18
									Linear Dynamical System	MA: 47.96-84.61
			DK (J)	Yes	Robotic	BB	8	101	Markov semi-Markov CRF	MA: 65.87-85.1	LOSO, LOUO
			DK (J)	Yes	Robotic	BB	8	101	Skip Chain CRF	MA: 81.60-85.04	LOSO, LOUO
34	van Amsterdam, B⁶³.	2019	DK (J)	CO	Robotic	BB	8	40	GMM	MA: 59-85	Experimental Validation
35	van Amsterdam, B⁴⁵.	2020	DK (J)	CO	Robotic	BB	8	39	Bi-LSTM	MA: 85.1-89.2	LOUO
36	van Amsterdam, B²².	2022	DK (J)	Yes	Robotic	BB	8	39	CNN + Concatenation TCN	MA: 82.3	LOUO
									CNN + Ensemble TCN	MA: 82.6
									CNN + Multimodal Attention TCN	MA: 83.4
			DK	Yes	Robotic	CM	8	45	CNN + Concatenation TCN	MA: 79.3	Hold out
									CNN + Ensemble TCN	MA: 78.1
									CNN + Multimodal Attention TCN	MA: 80.9
37	Despinoy, F⁶¹.	2016	DK	CO	Robotic	BB	3	12	kNN	MA: 78.4-97.4	LOO
37	Despinoy, F⁶¹.	2016	DK	CO	Robotic	BB	3	12	SVM	MA: 77.5-96.2	LOO
38	DiPietro, R¹⁴.	2019	DK	CO	Robotic	BB	15	39	RNN	ME: 17.9	LOUO
									LSTM	ME: 15.3
									GRU	ME: 15.2
									MIST RNN	ME: 15.3
			DK (J)	CO	Robotic	BB	8	39	RNN	ME: 11.6	LOUO
									LSTM	ME: 8.7
									GRU	ME: 8.6
									MIST RNN	ME: 9.7
39	Fard, M.J⁶⁴.	2016	DK (J)	CO	Robotic	BB	8	-	PCA + DTW + Soft-Boundary Unsupervised Gesture Segmentation	MA: 64-73.8	Experimental Validation
40	Gao, Y²³.	2016	DK (J)	CO	Robotic	BB	8	39	DTW + Autoencoder	MA: 68-84	-
40	Gao, Y²³.	2016	DK	CO	Robotic	BB	15	55	DTW + Autoencoder	MA: 59-74	-
41	Goldbraikh, A⁸¹.	2022	EM	CO	Open	BB	24	96	MS-TCN + +	MA: 82.4-94.69	k-fold
									LSTM	MA: 79.94-94.18
									GRU	MA: 82.21-95.04
42	Goldbraikh, A²⁴.	2024	EM	CO	Open	BB	25	11	Bi-LSTM MS-TCRN	MA: 83-84.2	k-fold
			EM	CO	Open	BB	25	11	Bi-GRU MS-TCRN	MA: 83.1-84.3	k-fold
			EM	CO	Open	CM	52	255	Bi-LSTM MS-TCRN	MA: 77.8-80.5	LOUO
			EM	CO	Open	CM	52	255	Bi-GRU MS-TCRN	MA: 77.4-79.2	LOUO
			DK (J)	CO	Robotic	BB	8	39	Bi-LSTM MS-TCRN	MA: 84.2-84.8	LOUO
			DK (J)	CO	Robotic	BB	8	39	Bi-GRU MS-TCRN	MA: 85.0-86.4	LOUO
43	Itzkovich, D²⁵.	2019	DK (J)	CO	Robotic	BB	8	39	LSTM	MA: 67-72	LOUO
43	Itzkovich, D²⁵.	2019	DK	CO	Robotic	BB	2	14	LSTM	MA: 55-71	LOUO
44	Itzkovich, D²⁶.	2022	DK (J)	CO	Robotic	BB	8	75	LSTM	MA: 46-64	Hold out
			DK	CO	Robotic	BB	2	15	LSTM	MA: 8-52	Hold out
			DK	CO	Robotic	CM	6	-	LSTM	MA: 13-68	Hold out
45	Lea, C⁶⁵.	2016	DK (J)	CO	Robotic	BB	8	39	Latent Convolutional Skip Chain CRF	MA: 81.69-83.45	LOUO
46	Lin, H.C⁵⁴.	2006	DK	No	Robotic	BB	2	27	LDA + Bayes Classifier	MA: 92.21-95.26	k-fold
47	Long, Y²⁷.	2021	DK (J)	Yes	Robotic	BB	8	75	CNN + TCN-LSTM + Graph NN	MA: 87.9-88.1	LOUO
47	Long, Y²⁷.	2021	DK	Yes	Robotic	BB	-	36	CNN + TCN-LSTM + Graph NN	MA: 87.3-91.0	k-fold
48	Loukas, C⁷⁵.	2013	EM	CO	Lap.	CS	21	21	Gaussian mixture MAR	-	-
49	Meißner, C⁸⁴.	2014	I + EM	CO	Other	CS	2	24	HMM	MA: 81-99	LOO
50	Murali, A⁶⁶.	2016	DK (J)	Yes	Robotic	BB	8	67	PCA + CNN + GMM + Transition state clustering	-	-
51	Peng, W⁶².	2019	DK	CO	Robotic	BB	12	360	DTW + Continuous HMM	MA: 94.73-97.48	Experimental Validation
52	Qin, Y²⁸.	2020	DK (J)	Yes	Robotic	BB	8	39	CNN-TCN + LSTM-TCN	MA: 86.3	LOUO
52	Qin, Y²⁸.	2020	DK	Yes	Robotic	CM	5	10	CNN-TCN + LSTM-TCN	MA: 82.7	LOUO
53	Zheng, Y⁷⁴.	2022	EM	CO	Lap.	BB	29	29	LSTM	MA: 68.18-75.86	LOUO
54	Zia, A³⁷.	2019	DK	Yes	Robotic	CM	-	100	CNN-LSTM + LSTM	-	Hold out
Skill Assessment and Feature Detection
55	Anh, N.X⁵⁵.	2020	DK (J)	No	Robotic	BB	8	40	CNN + SVM	MA: 92.75-96.84	LOSO
									LSTM + SVM	MA: 89.75-95.09
									CNN-LSTM + SVM	MA: 90.98-96.39
									Autoencoder + SVM	MA: 80.63-83.46
56	Baghdadi, A³⁶.	2023	M	No	Open	CM	13	50	CNN + DNN-LSTM	MA FD: 82-95	k-fold
56	Baghdadi, A³⁶.	2023	M	No	Open	CM	13	50	KNN + XGBOOST + DNN-LSTM	MA SA: 71	k-fold
57	Ershad, M⁷⁶.	2019	EM	CO	Robotic	BB	14	84	PCA + SVM	MA: 71.03-98.5	k-fold
58	Forestier, G¹⁵.	2018	DK (J)	CO	Robotic	BB	8	101	SAX-VSM	MA FD: 75.29-93.69	LOSO, LOUO
			DK (J)	CO	Robotic	BB	8	101	SAX-VSM	MA SA: 61.11-96.3	LOSO, LOUO
			DK	No	Robotic	BB	3	30	SAX-VSM	MA FD: 100	LOO
			DK	No	Robotic	BB	3	30	SAX-VSM	MA SA: 83.33	LOO
			DK	CO	Robotic	CS	6	27	SAX-VSM	MA SA: 85.19	LOO
59	King, R.C¹⁶.	2009	I + M	No	Lap.	BB	5	25	HMM	MA FD: 56-100	-
59	King, R.C¹⁶.	2009	I + M	No	Lap.	CM	7	28	PCA + HMM + GMM Clustering	-	-
60	Loukas, C⁷⁷.	2011	EM	CO	Lap.	BB	22	44	MAR + PCA + SVM	MA: 86-96	-
60	Loukas, C⁷⁷.	2011	EM	CO	Lap.	BB	22	44	HMM	MA: 65-87	-
61	Loukas, C⁷⁸.	2013	EM	CO	Lap.	CS	22	22	MAR	-	-
62	Nguyen, X.A¹⁷.	2019	I	CO	Open	BB	15	75	SVM	MA: 71.3-81.7	LOSO
									CNN-LSTM + SVM	MA: 88.1-95.4
									CNN-LSTM + SENet + SVM	MA: 90.3-96.7
									CNN-LSTM + SENet + Restart + SVM	MA: 92.1-98.2
			DK (J)	No	Robotic	BB	8	101	CNN-LSTM + SVM	MA: 91.5-97.3	LOSO
									CNN-LSTM + SENet + SVM	MA: 94.7-98.3
									CNN-LSTM + SENet + Restart + SVM	MA: 94.8-98.4
63	Reiley, C.E⁶⁰.	2010	DK	CO	Robotic	BB	11	20	DTW + GMM/GMR + HMM	-	-
64	Rosen, J³³.	2001	M	CO	Lap.	CM	10	10	kMC + HMM	MA: 87.5	-
65	Topalli, D⁴⁹.	2019	DK	No	Other	BB	28	1260	kNN + AdaBoost M1	MA: 85.71	k-fold
									kNN + Jrip	MA: 64.28-78.57
									kNN + kNN	MA: 57.14-75
									kNN + Locally Weighted Learning	MA: 67.86-82.14
									kNN + LR	MA: 75-82.14
									kNN + SVM	MA: 64.28-82.14
66	Wang, Z⁴⁴.	2018b	DK (J)	CO	Robotic	BB	8	120	GRU-CNN	MA FD: 100	LOSO
66	Wang, Z⁴⁴.	2018b	DK (J)	CO	Robotic	BB	8	120	GRU-CNN	MA SA: 96	LOSO
67	Zia, A¹⁸.	2018	I	CO	Open	BB	41	103	ApEn + Cross ApEn + Nearest Neighbour	MA: 78.7-86.8	k-fold, LOO
67	Zia, A¹⁸.	2018	I	Yes	Open	BB	41	103	kMC + ApEn + Cross ApEn + Nearest Neighbour	MA: 93.2-94	k-fold, LOO
Tool Tracking
68	Korte, C⁴⁷.	2021	DK	No	Open	CS	5	60	LSTM-RNN	-	Experimental validation
69	Lee, E.J¹⁹.	2019	EM	Yes	Lap.	BB	-	1500	Random walk + Deep CNN	-	Hold out
69	Lee, E.J¹⁹.	2019	EM	Yes	Lap.	CM	-	100	Random walk + Deep CNN	-	-
70	Liu, J³⁴.	2023	DK	Yes	Robotic	CM	-	950	CNN	-	LOO
71	Pachtrachai, K³⁰.	2021	DK	Yes	Robotic	BB	-	8502	CNN + LSTM	-	Experimental validation
71	Pachtrachai, K³⁰.	2021	DK	Yes	Robotic	CM	-	15002	CNN + LSTM	-	Experimental validation
72	Qin, Y²⁹.	2020	DK (J)	Yes	Robotic	BB	8	39	CNN-LSTM + LSTM Encoder + LSTM Decoder	ME: 4.72-10.14	LOUO
72	Qin, Y²⁹.	2020	DK	Yes	Robotic	CM	5	40	CNN-LSTM + LSTM Encoder + LSTM Decoder	ME: 1.1-2.43	LOUO
73	Rocha, C.D³¹.	2019	DK	Yes	Robotic	BB	-	910	GMM + CNN	MA: 99	Experimental validation
			DK	Yes	Robotic	BB	-	2737	GMM + CNN	MA: 98.2	Experimental validation
			DK	Yes	Robotic	CM	-	481	GMM + CNN	MA: 97	Experimental validation
74	Shu, X⁵⁶.	2021	DK	No	Robotic	NCS	-	1524	MLP	ME: <1.5	Hold out
74	Shu, X⁵⁶.	2021	DK	No	Robotic	NCS	-	1524	LSTM	ME: <1.5	Hold out
75	Sun, Z⁸³.	2018	EM	No	Other	NCS	-	150	ANN	-	Experimental validation
76	Wang, Z⁸².	2022	EM	No	Lap.	BB	4	80	LSTM	ME: 11.43-15.11	Hold out
77	Xu, W⁷⁹.	2017	EM	No	Robotic	NCS	-	20000	GMR	MA: 87.39-95	Hold out
									kNN	MA: 90.5-95.9
									Extreme machine learning	MA: 98.2
78	Zhao, H⁵⁹.	2018	DK (J)	Yes	Robotic	BB	8	67	PCA + DTW + Transition State Clustering Dense Convolutional Encoder-Decoder Network	MA: 60.1-70.6	LOO
Undesirable Motion Filtration
79	Sang, H⁵⁷.	2016	I + DK	No	Other	NCS	-		Zero Phase Adaptive Fuzzy Kalman Filter	-	Experimental validation
80	Tatinati, S⁹⁵.	2015	I	Yes	Other	NCS	3	6	Moving Window Least Squares - SVM	MA: 71	Experimental validation
81	Tatinati, S⁹⁴.	2017	I	Yes	Other	NCS	3	9	Moving Window Least Squares - SVM	MA: 74	Experimental validation
									Multidimensional Robust Extreme Learning Machine	MA: 78
									Online sequential Multidimensional Robust Extreme Learning Machine	MA: 81
Other
82	Sabique, P.V³⁵.	2023	M + DK	Yes	Robotic	BB	-	-	PCA + Generalised Discriminant Analysis + RNN-LSTM	-	Experimental validation
									PCA + Generalised Discriminant Analysis + CNN-LSTM	-
									PCA + GDA + Encoder network	-
83	Song, W⁸⁰.	2006	M + EM	Yes	Open	BB	-	120	Fuzzy NN	-	-
84	Su, H⁵⁸.	2019	M + DK	No	Robotic	NCS	-	73776	ANN	-	Experimental validation

An overview of the methodologies and technologies employed across different studies.
Sensor: DK device kinematics, (J) JHU-ISI Gesture and Skill Assessment Working Set dataset, I inertial, EM electromagnetic, M mechanical, EMG electromyography. Video: CO context only. Field: Lap. Laparoscopic. Task: BB basic bench-top, CS clinical simulation, NCS non-clinical simulation, CM clinical model. Machine Learning Model: SVM support vector machine, HMM hidden Markov model, DTW dynamic time warping, kNN k-nearest neighbours, LASSO least absolute shrinkage and selection operator, RF random forest, LR logarithmic regression, LDA linear discriminant analysis, PCA principal component analysis, MLP multilayer perceptron, LSTM long short-term memory, Bi- bidirectional, ANN artificial neural network, kMC k-means clustering, DNN deep neural network, NN neural network, CNN convolutional neural network, GRU gated recurrent unit, TCN temporal convolutional network, GMM Gaussian mixture model, CRF conditional random field, MIST mixed history, RNN recurrent neural network, MS multi-stage, TCRN temporal convolutional recurrent network, SAX-VSM symbolic aggregate approximation vector space model, MAR multivariate autoregressive, SENet squeeze-and-excitation network, GMR Gaussian mixture regression, ApEn approximate entropy. Performance Metric: MA mean accuracy, ME mean error. Cross Validation: LOTO leave one trial out, LOUO leave one user out, LOSO leave one super-trial out, LOO leave one out.

Back to article page

Table 1 Overview of included studies, categorised in order of machine learning aim

Search

Quick links