Abstract
This paper introduces a novel channel estimation method for Orthogonal Time Frequency Space (OTFS) systems affected by nonlinear distortion from High-Power Amplifiers (HPA). The method integrates a Bidirectional Gated Recurrent Unit (Bi-GRU) with a dynamic gating mechanism driven by the Input Back-Off (IBO) parameter of the HPA, combined with a multi-head attention network. The dynamic gating mechanism adaptively adjusts the update gate of the Gated Recurrent Unit (GRU) based on real-time IBO values, optimizing the trade-off between historical memory and current input during training. The multi-head attention module further captures long-range dependencies in the channel response. Theoretical analysis indicates that the proposed IBO-driven dynamically gated Bi-GRU achieves a computational complexity reduction of 20–46.7% compared to a Bi-GRU architecture. Simulation results demonstrate the superior performance of the proposed method across both bit error rate (BER) and normalized mean square error (NMSE) metrics under high mobility and nonlinear distortion. It achieves up to 22.6 quantified in decibels (dB) lower NMSE and, at a signal-to-noise ratio (SNR) of 30 dB, a 15.2 dB reduction in logarithmic BER compared to conventional methods, along with a 3–4 dB improvement over deep learning baselines at the same SNR. It also provides over 7 dB peak-to-average power ratio (PAPR) reduction over traditional methods, confirming strong robustness and accuracy in challenging communication scenarios.
Data availability
The data supporting the findings of this study are available from the corresponding author upon reasonable request and with appropriate permission.
References
Fan, P., Zhao, J. & Chih-Lin, I. 5G high mobility wireless communications: Challenges and solutions. China Commun. 13 (2), 1–13 (2016).
Wen, M., Basar, E., Li, Q., Zheng, B. & Zhang, M. Multiple-mode orthogonal frequency division multiplexing with index modulation. IEEE Trans. Commun. 65 (9), 3892–3906 (2017).
Wei, Z., Li, S., Yuan, W., Schober, R. & Caire, G. Orthogonal time frequency space modulation—Part I: Fundamentals and challenges ahead. IEEE Commun. Lett. 27 (1), 4–8 (2022).
Hadani, R. et al. Orthogonal time frequency space modulation. In 2017 IEEE Wireless Communications and Networking Conference (WCNC), 1–6 (2017).
Hadani, R. & Monk, A. OTFS: A new generation of modulation addressing the challenges of 5G. arxiv preprint arxiv:1802.02623 (2018).
Raviteja, P., Phan, K. T. & Hong, Y. Embedded pilot-aided channel estimation for OTFS in delay–Doppler channels. IEEE Trans. Veh. Technol. 68 (5), 4906–4917 (2019).
Hashimoto, N., Osawa, N., Yamazaki, K. & Ibi, S. Channel estimation and equalization for CP-OFDM-based OTFS in fractional Doppler channels. In 2021 IEEE International Conference on Communications Workshops (ICC Workshops), 1–7 (2021).
Zhang, H., Huang, X. & Zhang, J. A. Low-overhead OTFS transmission with frequency or time domain channel estimation. IEEE Trans. Veh. Technol. 73 (1), 799–811 (2023).
Aghda, O. A., Omidi, M. J. & Saeedi-Sourck, H. Low-PAPR joint channel estimation and data detection in ZP-OTFS system. IEEE Trans. Veh. Technol. (2024).
Kumar, A., Gaur, N., Chakravarty, S. & Nanthaamornphong, A. Reducing the PAPR of OTFS modulation using hybrid PAPR algorithms. Wirel. Pers. Commun. 133 (4), 2503–2523 (2023).
Hu, J. et al. DNN and LS based channel estimation in OTFS system. In 2023 IEEE 23rd International Conference on Communication Technology (ICCT), 106–110 (2023).
dos Reis, A. F., Chang, B. S., Medjahdi, Y., Brante, G. & Bader, F. LSTM-based time-frequency domain channel estimation for OTFS modulation. IEEE Trans. Veh. Technol. 73 (10), 15049–15060 (2024).
Surabhi, G. D. & Chockalingam, A. Low-complexity linear equalization for OTFS modulation. IEEE Commun. Lett. 24 (2), 330–334 (2019).
Parkvall, S., Dahlman, E., Furuskar, A., Frenne, M. & NR. The new 5G radio access technology. IEEE Commun. Stand. Mag. 1 (4), 24–30 (2018).
Colantonio, P., Giannini, F. & Limiti, E. High Efficiency RF and Microwave Solid State Power Amplifiers (Wiley, 2009).
Shaiek, H., Zayani, R., Medjahdi, Y. & Roviras, D. Analytical analysis of SER for beyond 5G post-OFDM waveforms in presence of high power amplifiers. IEEE Access. 7, 29441–29452 (2019).
Hou, J., Liu, H., Zhang, Y., Wang, W. & Wang, J. GRU-based deep learning channel estimation scheme for the IEEE 802.11 p standard. IEEE Wirel. Commun. Lett. 12 (5), 764–768 (2022).
Mohsin, M. A. et al. Continual learning for wireless channel prediction. arXiv preprint arXiv:2506.22471 (2025).
Cheng, L., Pandey, A., Xu, B., Delbruck, T. & Liu, S. C. Dynamic gated recurrent neural network for compute-efficient speech enhancement. arxiv preprint arxiv:2408.12425 (2024).
Fang, Y., Yang, S., Zhao, B. & Huang, C. Cyberbullying detection in social networks using bi-gru with self-attention mechanism. Information 12 (4), 171 (2021).
Funding
This research was supported by the National Key R&D Program of China (2025YFE0115800), the Guangxi Science and Technology Program (AA24263010, AA24263006, AA24206043, and AD25069103), the National Natural Science Foundation of China (U23A20280, 62471153, and U25A20397), the Nanning Scientific Research and Technology Development Program (20231029 and 20231011), the Guangxi Zhuang Autonomous Region Major Talent Project, the Project of Guangxi Higher Education Institutions Engineering Research Center for BeiDou Positioning Services and Border-Coastal Defense Security Applications, the project of Guangxi Key Laboratory of Precision Navigation Technology and Application (No. DH202312), and the Innovation Project of Guangxi Graduate Education (YCSW2025355).
Author information
Authors and Affiliations
Contributions
J.H. wrote the paper and did experiment, Z.W. and Y.J. provided supervision, X.D. also provided supervision, All authors have reviewed the manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Hou, J., Wei, Z., Ji, Y. et al. OTFS channel estimation method based on IBO-dynamic gated Bi-GRU. Sci Rep (2026). https://doi.org/10.1038/s41598-026-44747-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-026-44747-3
