Abstract
Multi-object tracking faces two persistent challenges: managing detector confidence and preventing track loss under prolonged occlusions. We introduce Sentinel, an uncertainty-aware tracker that diagnoses per-track uncertainty online and proactively optimizes its tracking strategy. Sentinel consists of two components. Confidence Aware Association (CAA) dynamically reweighs the association-cost terms according to the current track state, enabling the effective use of low-confidence detections while suppressing identity switches. Survival Boosting Mechanism (SBM) preserves tracks at risk of disappearance by exploiting weak detection signals to bridge long occlusions, thereby reducing fragmentation and maintaining identity continuity. Evaluations on MOT17, MOT20, and DanceTrack demonstrate that Sentinel achieves strong performance in Higher Order Tracking Accuracy (HOTA), Identification F1-score (IDF1), and Association Accuracy (AssA), demonstrating its strength in identity preservation and association quality. While this design introduces modest computational overhead and may increase false positives when exploiting low-confidence detections, Sentinel improves robustness in realistic, crowded environments by moving beyond passive reliance on detector outputs to uncertainty-driven, per-track optimization.
Similar content being viewed by others
Data availability
The data supporting the findings of this study are publicly available. The datasets used in this study include MOT17 and MOT20 from the MOT Challenge repository, and the DanceTrack dataset. They can be accessed at the following URLs respectively: https://motchallenge.net/data/MOT17/, https://motchallenge.net/data/MOT20/, and https://github.com/DanceTrack/DanceTrack. The MOT Challenge datasets are provided under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 license (https://creativecommons.org/licenses/by-nc-sa/3.0/), while the DanceTrack dataset is available for non-commercial research purposes. All usage of the data in this study complies with the terms of their respective licenses.
References
Fengwei, Y. & et al. Poi: Multiple object tracking with high performance detection and appearance feature. In European conference on computer vision. 36–42, (2016). https://doi.org/10.1007/978-3-319-48881-3_3
Bochinski, E., Eiselein, V. & Sikora, T. High-speed tracking-by-detection without using image information. In 14th IEEE international conference on advanced video and signal based surveillance. 1–6,. 1–6, (2017). https://doi.org/10.1109/AVSS.2017.8078516 (2017).
Yunhao, D. Strongsort: Make deepsort great again. IEEE Trans. Multimedia. 25, 8725–8737. https://doi.org/10.1109/TMM.2023.3240881 (2023).
Nguyen, P., Quach, K. G., Kitani, K. & Luu, K. Type-to-Track: Retrieve Any Object via Prompt-based Tracking. Proceedings of the 37th International Conference on Neural Information Processing Systems. 3205–3219. (2024).
Liu, Y., Li, Y., Xu, D., Yang, Q. & Tao, W. Adaptive Kalman Filter with power transformation for online multi-object tracking. Multimedia Syst. 29 (3), 1231–1244. https://doi.org/10.1007/s00530-023-01052-7 (2023).
Zhang, Y. & et al. Bytetrack: Multi-object tracking by associating every detection box. European conference on computer vision. 1–21. (2022). https://doi.org/10.1007/978-3-031-20047-2_1
Pang, Z. & et al. SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking. European Conference on Computer Vision. 680–696, (2023). https://doi.org/10.1007/978-3-031-25056-9_43
Chiu, H. et al. &. Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Driving. In IEEE International Conference on Robotics and Automation. 14227–14233,. 14227–14233, (2021). https://doi.org/10.1109/ICRA48506.2021.9561754 (2021).
Stadler, D. & et al. Improving Multiple Pedestrian Tracking by Track Management and Occlusion Handling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10958–10967, (2021). https://doi.org/10.1109/CVPR46437.2021.01081
Aharon, N. & et al. BoT-SORT: Robust Associations Multi-Pedestrian Tracking. arXiv preprint arXiv :220614651 (2022).
Miah, M., Bilodeau, G. A., and Saunier, N. Learning data association for multi-object tracking using only coordinates.Pattern Recognition 160, (2025).
Luiten, J. et al. Hota: A higher order metric for evaluating multi-object tracking. Int. J. Comput. Vis. 129, 548–578. https://doi.org/10.1007/s11263-020-01375-2 (2021).
Milan, A., Leal-Taixé, L., Reid, I., Roth, S. & Schindler, K. MOT16: A benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831 (2016).
Sun, P. et al. Multi-object tracking in uniform appearance and diverse motion, In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 20993–21002. (2022). https://doi.org/10.1109/CVPR52688.2022.02032
Bewley, A., Ge, Z., Ott, L., Ramos, F. & Upcroft, B. Simple online and realtime tracking. In IEEE international conference on image processing. 3464–3468,. 3464–3468, (2016). https://doi.org/10.1109/ICIP.2016.7533003 (2016).
Kalman, R. E. A new approach to linear filtering and prediction problems. J. Basic Eng. 82(1), 35–45. https://doi.org/10.1115/1.3662552 (1960).
Wojke, N., Bewley, A. & Paulus, D. Simple online and realtime tracking with a deep association metric. IEEE Int. Conf. Image Process. https://doi.org/10.1109/ICIP.2017.8296962 (2017).
Liu, H. et al. Improved DeepSORT algorithm based on multi-feature fusion. Appl. Syst. Innov. https://doi.org/10.3390/asi5030055 (2022).
Wang, Z., Zheng, L., Liu, Y., Li, Y. & Wang, S. Towards real-time multi-object tracking. In European conference on computer vision. 107–122, (2020). https://doi.org/10.1007/978-3-030-58621-8_7
Zhang, Y., Wang, C., Wang, X., Zeng, W. & Liu, W. FairMOT: On the fairness of detection and re-identification in multiple object tracking. Int. J. Comput. Vis. https://doi.org/10.1007/s11263-021-01513-4 (2021).
Zhou, X., Koltun, V. & Krähenbühl, P. Tracking objects as points. Eur. Conf. Comput. Vis. 474 (490). https://doi.org/10.1007/978-3-030-58548-8_28 (2020).
Pang, J. & et al. Quasi-dense similarity learning for multiple object tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 164–173, (2021). https://doi.org/10.1109/CVPR46437.2021.00023
Wu, J. & et al. Track to detect and segment: An online multi-object tracker. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 12352–12361, (2021). https://doi.org/10.1109/CVPR46437.2021.01217
Liu, Z. et al. Multi-object tracking by performing scene decomposition based on pseudo-depth. arXiv preprint arXiv:2306.05238 (2023).
Guo, X., Yang, H., Wei, M. & Zhang, Y. Few-shot object detection via class encoding and multi-target decoding. IET Cyber-Systems Rob. 5 (2), 218–226. https://doi.org/10.1049/csy2.12088 (2023).
Pang, B., Xu, Y., Chen, J. & Li, L. P. B. M. O. T. Pose-aware Association Boosted Online 3D Multi-Object Tracking. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. 14170–14177,. 14170–14177, (2025). https://doi.org/10.1109/IROS60139.2025.11247375 (2025).
Wang, G., Wang, Y., Zhang, H., Gu, R. & Hwang, J. N. Exploit the Connectivity: Multi-Object Tracking with TrackletNet. arXiv preprint arXiv:1811.07258 (2018).
Pang, B. et al. Adopting Tubes to Track Multi-Object in a One-Step Training Model. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6307–6317, (2020). https://doi.org/10.1109/CVPR42600.2020.00634
Kefu, Y. I. et al. UCMCTrack: Multi-Object Tracking with Uniform Camera Motion Compensation. Proc. AAAI Conf. Artif. Intell. 38(7), 6702–6710. https://doi.org/10.1609/aaai.v38i7.28493 (2024).
Guo, X., Zheng, Y. & Wang, D. PMTrack: Multi-object Tracking with Motion-Aware. Asian Conf. Comput. Vis. 3091-3106 https://doi.org/10.1007/978-981-96-0960-4_26 (2024).
Sinishaw, M. L. & Liu, S. JDECMC: Improving JDE-based multi-object tracking with camera motion compensation. Displays https://doi.org/10.1016/j.displa.2024.102682 (2024).
Zeng, K. Noise-control multi-object tracking. Complex. Intell. Syst. 9(4), 4331–4347. https://doi.org/10.1007/s40747-022-00946-9 (2023).
Memon, S. A. et al. Tracking multiple autonomous ground vehicles using motion capture system operating in a wireless network. IEEE Access 12, 60592–60604. https://doi.org/10.1109/ACCESS.2024.3394536 (2024).
Memon, S. A. et al. Tracking multiple unmanned aerial vehicles through occlusion in low-altitude airspace. Drones 7(4), 241. https://doi.org/10.3390/drones7040241 (2023).
Seidenschwarz, J., Braso, G., Elezi, I. & Leal-Taixe, L. Simple Cues Lead to a Strong Multi-Object Tracker. Conf. Comput. Vis. pattern Recognit. 13813-13823 https://doi.org/10.1109/CVPR52729.2023.01327 (2023).
Wang, Y. H. & et al. Similarity learning for occlusion-aware multiple object tracking. In Proceedings of the AAAI Conference on Artificial Intelligence. 38(6), 5740–5748, (2024). https://doi.org/10.1609/aaai.v38i6.28386
Xiao, C. et al. Motiontrack: Learning motion predictor for multiple object tracking. Neural. Netw. https://doi.org/10.1016/j.neunet.2024.106539 (2024).
Ge, Z. et al. & Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430 (2021).
He, L. et al. Fastreid: a pytorch toolbox for real-world person re-identification. arXiv preprint arXiv:.02631 (2020). (2020). (2006).
Yang, H. S., Park, S. W., Jung, S. H. & Sim, C. B. EnhanceCenter for improving point based tracking and rich feature representation. Sci. Rep. https://doi.org/10.1038/s41598-025-88924-2 (2025).
Vaquero, L., Xu, Y., Alameda-Pineda, X., Brea, V. M. & Mucientes, M. Lost and found: Overcoming detector failures in online multi-object tracking. In European Conference on Computer Vision. 448–466, (2024). https://doi.org/10.48550/arXiv.2407.10151
Stadler, D. & Beyerer, J. Modelling ambiguous assignments for multi-person tracking in crowds. In Proceedings of the IEEE/CVF winter conference on applications of computer vision. 133–142, (2022). https://doi.org/10.1109/WACVW54805.2022.00019
Wang, S. et al. Extendable multiple nodes recurrent tracking framework with RTU++. IEEE Trans. Image Process. 31, 5257–5271. https://doi.org/10.1109/TIP.2022.3192706 (2022).
Zhang, S., Zhu, Y., Sun, Y., Liu, W. & Huang, Z. RAP-SORT: Advanced multi-object tracking for complex scenarios. Displays https://doi.org/10.1016/j.displa.2026.103361 (2026).
Zhang, X., Zhao, H., He, S. & Li, Y. REAL-SORT: Relation-aware for real-time multiple object tracking. Knowl. Based Syst. https://doi.org/10.1016/j.knosys.2026.115373 (2026).
Jung, H., Kang, S., Kim, T., Kim, H. & ConfTrack Kalman Filter-Based Multi-Person Tracking by Utilizing Confidence Score of Detection Box. In Proceedings of the IEEE/CVF Winter Conference on Computer Vision and Pattern Recognition. (2024). https://doi.org/10.1109/WACV57701.2024.00645
Liu, Z., Wang, X., Wang, C., Liu, W. & Bai, X. Sparsetrack: Multi-object tracking by performing scene decomposition based on pseudo-depth. IEEE Trans. Circuits Syst. Video Technol. https://doi.org/10.1109/TCSVT.2024.3524670 (2025).
Stadler, D. & Beyerer, J. Past information aggregation for multi-person tracking. In 2023 IEEE International Conference on Image Processing. 321–325, (2023)., October https://doi.org/10.1109/ICIP49359.2023.10223159 (2023).
Yi, K., Wu, H., Hao, W. & Hu, R. EscapeTrack: Multi-object tracking with estimated camera parameters. Signal. Process. https://doi.org/10.1016/j.sigpro.2025.109958 (2025).
Chen, B., Wei, Z., Lei, W., Wang, C. & GMMotion Neighborhood Information Matters for Online Multi-Pedestrian Tracking. In PRICAI 2024: Trends in Artificial Intelligence. (2024). https://doi.org/10.1007/978-981-96-0122-6_8
Gao, Y., Xu, H., Li, J., Wang, N. & Gao, X. Multi-Scene Generalized Trajectory Global Graph Solver with Composite Nodes for Multiple Object Tracking. Proceedings of the AAAI Conference on Artificial Intelligence, 38(3), 1842–1850, (2024). https://doi.org/10.1609/aaai.v38i3.27953
Zhang, Y. et al. Handling Heavy Occlusion in Dense Crowd Tracking by Focusing on the Heads. In AI 2023: Advances in Artificial Intelligence. 79–90, (2023). https://doi.org/10.1007/978-981-99-8388-9_7
Stadler, D. & Beyerer, J. An Improved Association Pipeline for Multi-Person Tracking. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 3170–3179, (2023). https://doi.org/10.1109/CVPRW59228.2023.00319
Sun, P. et al. & Global Tracking Transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8761–8770 (2022).
Ren, H. et al. Focus on Details: Online Multi-Object Tracking with Diverse Fine-Grained Representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11289–11298, (2023). https://doi.org/10.1109/CVPR52729.2023.01086
Fang, Y. M. O. T. & FCG++ Enhanced Representation of Spatio-temporal Motion and Appearance Features. arXiv preprint arXiv:2411.10028 (2024).
Qin, Z. et al. Motiontrack: Learning robust short-term and long-term motions for multi-object tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 17939–17948, (2023). https://doi.org/10.1109/CVPR52729.2023.01720
Yang, F., Odashima, S., Masui, S. & Jiang, S. Hard to Track Objects with Irregular Motions and Similar Appearances? Make It Easier by Buffering the Matching Space. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 4799–4808. (2023). https://doi.org/10.1109/WACV56688.2023.00478
Larsen, M., Rolfsjord, S., Gusland, D., Ahlberg, J. & Mathiassen, K. BASE: Probably a Better Approach to Visual Multi-Object Tracking. In Proceedings of the 19th International Joint Conference on Computer Vision. 110–121 (2024).
Shim, K., Hwang, J., Ko, K. & Kim, C. A confidence-aware matching strategy for generalized multi-object tracking. In IEEE international conference on image processing. 4042–4048,. 4042–4048, (2024). https://doi.org/10.1109/ICIP51287.2024.10647729 (2024).
Acknowledgements
This paper was supported by Sunchon National University Glocal University Project Fund in 2025. (Grant number: 2025-G024)
Author information
Authors and Affiliations
Contributions
H.S.Y. conceived Sentinel and designed CAA and SBM. S.W.P. and C.B.S. implemented the tracking pipeline and conducted experiments on the MOT17, MOT20, and DanceTrack datasets. S.H.J. supervised the project and contributed to manuscript revision. All authors have reviewed and approved the final manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Informed Consent
This study utilizes three publicly available datasets: MOT17 and MOT20 from the MOT Challenge benchmark, and the DanceTrack dataset. The MOT Challenge datasets consist of fully anonymized data, for which no additional informed consent is required. The DanceTrack dataset was collected with prior informed consent from all participants. All datasets were used in accordance with their terms and conditions, which permit their use for research purposes and in academic publications.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Yang, HS., Park, SW., Sim, CB. et al. Sentinel for confidence-aware multi-object tracking. Sci Rep (2026). https://doi.org/10.1038/s41598-026-43938-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-026-43938-2
