Signal detection in optical orthogonal time space modulation for efficient VLC application
-
Anjali Shrivastav
and
Aziz Nanthaamornphong
Abstract
This paper presents a detailed analysis of the bit error rate (BER) performance of a proposed maximum likelihood (ML) detection scheme for optical orthogonal time space modulation (OTSM) systems using 64-QPSK, considering practical optical impairments and diverse channel conditions. The evaluation covers scenarios with 5 % and 10 % channel estimation errors, as well as Rayleigh and Rician fading environments. Simulation results confirm that the proposed machine learning (ML) detector consistently outperforms conventional methods – including OTSM, zero-forcing equalization (ZFE), minimum mean square error (MMSE), and conventional ML – by delivering substantial SNR gains. For instance, under 10 % and 5 % estimation errors, the target BER of 10−3 is achieved at 12.2 dB and 10.8 dB, respectively, providing up to 6 dB improvement over baselines. In Rayleigh fading, the same BER is attained at 9.6 dB with a gain of 7.7 dB, while in Rician fading, the detector achieves optimal performance at only 6 dB, outperforming others by as much as 9.5 dB. These results underscore the robustness of the proposed ML approach against estimation inaccuracies and fading, making it well-suited for low-power, high-reliability applications in 6G, Internet of things (IoT), vehicular networks, and satellite communications.
-
Research ethics: Not applicable.
-
Informed consent: Not applicable.
-
Author contributions: The authors have accepted responsibility for the entire content of this manuscript and approved its submission.
-
Use of Large Language Models, AI and Machine Learning Tools: Not applicable.
-
Conflict of interest: The authors state no conflict of interest.
-
Research funding: Not applicable.
-
Data availability: Not applicable.
References
1. Raviteja, P, Phan, KT, Hong, Y, Viterbo, E. Interference cancellation and iterative detection for orthogonal time frequency space modulation. IEEE Trans Wireless Commun 2018;17:6501–15. https://doi.org/10.1109/twc.2018.2860011.Search in Google Scholar
2. Hadani, R, Rakib, S, Tsatsanis, M, Monk, A, Goldsmith, AJ, Molisch, AF, et al.. Orthogonal time frequency space modulation. In: Proc. IEEE Wireless Commun. Netw. Conf. (WCNC). San Francisco: IEEE Publisher; 2017.10.1109/WCNC.2017.7925924Search in Google Scholar
3. Ebihara, T, Leus, G. Doppler-resilient orthogonal signal-division multiplexing for underwater acoustic communication. IEEE J Ocean Eng 2016;41:408–27.10.1109/JOE.2015.2454411Search in Google Scholar
4. Kumar, A, Nanthaamornphong, A. Lowering the PAPR of the optical OTFS-based 6G radio with a hybrid PTS-PSO genetic approach. Discov Appl Sci 2024;6:574. https://doi.org/10.1007/s42452-024-06292-4.Search in Google Scholar
5. Kumar, A, Gaur, N, Aly, AA, Nanthaamornphong, A. PAPR reduction of OTFS using an automatic amplitude reduction neural network with vendermonde matrix-based PTS and SLM algorithms. J Wireless Com Network 2024;2024:84. https://doi.org/10.1186/s13638-024-02414-z. Search in Google Scholar
6. Hao, L, Wang, D, Tao, Y, Cheng, W, Li, J, Liu, Z, et al.. The extended SLM combined autoencoder of the PAPR reduction scheme in DCO-OFDM systems. Appl. Sci. 2019;9:852. https://doi.org/10.3390/app9050852.Search in Google Scholar
7. Kumar, A, Nanthaamornphong, A, Gaur, N. Optimizing PAPR performance for 6G OTFS waveform using adaptive genetic PTS method for Rician fading channels. EURASIP J Appl Signal Process 2025;2025:22. https://doi.org/10.1186/s13634-025-01234-7.Search in Google Scholar
8. Thaj, T, Viterbo, E, Hong, Y. Orthogonal time sequency multiplexing modulation: analysis and low-complexity receiver design. IEEE Trans Wireless Commun 2021;20:7842–55. https://doi.org/10.1109/twc.2021.3088479.Search in Google Scholar
9. Vanin, K, Ryu, H -G. Design and evaluation of OTSM(orthogonal time sequency multiplexing) communication system in Rayleigh fading channel. In: 2024 international conference on artificial intelligence in information and communication (ICAIIC). Osaka, Japan: IEEE Publisher; 2024:1–4 pp.10.1109/ICAIIC60209.2024.10463447Search in Google Scholar
10. Nalluru, S, Neelam, SG. Low complexity detection of IRS-aided MIMO-OTSM under perfect phase optimization. IEEE Commun Lett 2025;29:393–7. https://doi.org/10.1109/lcomm.2024.3520700.Search in Google Scholar
11. Al Ahsan, R, Ghannouchi, FM, Fapojuwo, AO. Analysis of adapted tone reservation PAPR reduction techniques in OTSM system. In: 2023 33rd international telecommunication networks and applications conference. Melbourne, Australia: IEEE Publisher; 2023:162–8 pp.10.1109/ITNAC59571.2023.10368554Search in Google Scholar
12. Sun, Y, Cheng, Y, Liu, T, Huang, Q, Guo, J, Jin, W, et al.. Research on signal detection of OFDM systems based on the LSTM network optimized by the improved Chameleon Swarm algorithm. Mathematics 2023;11:1989. https://doi.org/10.3390/math11091989.Search in Google Scholar
13. Kumar, A, Albreem, MA, Gupta, M, Alsharif, MH, Kim, S. Future 5G network based smart hospitals: hybrid detection technique for latency improvement. IEEE Access 2020;8:153240–9. https://doi.org/10.1109/ACCESS.2020.3017625.Search in Google Scholar
14. Gopal, A, Reddy, DK, Chintagunta, S. Symbol interferometry and Companding transform for PAPR reduction of OTFS signal. ETRI J 2024;46:595–603. https://doi.org/10.4218/etrij.2023-0142.Search in Google Scholar
15. Cinemre, I, Aydin, V, Hacioglu, G. Papr reduction through Gaussian pre-coding in DCO-OFDM systems. Opt Quant Electron 2024;56:958. https://doi.org/10.1007/s11082-024-06911-9.Search in Google Scholar
16. Kumar, A, Nanthaamornphong, A, Alsharif, MH, Masud, M. SIC based RL for massive MIMO NOMA signal detection for different modulation schemes under diverse channel conditions. Sci Rep 2025;15:24641. https://doi.org/10.1038/s41598-025-06492-x.Search in Google Scholar PubMed PubMed Central
17. Kumar, A, Gaur, N, Gupta, M, Nanthaamornphong, A. Implementation of the deep learning method for signal detection in massive-MIMO-NOMA systems. Heliyon 2024;10. https://doi.org/10.1016/j.heliyon.2024.e25374.Search in Google Scholar PubMed PubMed Central
18. Kumar, A, Gaur, N, Nanthaamornphong, A. Signal detection of M-MIMO-orthogonal time frequency space modulation using hybrid algorithms: ZFE + MMSE and ZFE + MF. Results Eng 2024;24:103311. https://doi.org/10.1016/j.rineng.2024.103311.Search in Google Scholar
© 2025 Walter de Gruyter GmbH, Berlin/Boston
