DL-NOMA-CancelNet: A Deep Learning Framework for Interference Cancellation Using CNN and BiLSTM
DOI:
https://doi.org/10.24237/djes.2026.19111Keywords:
Wireless Communication, 6G, NOMA, CNN, BiLSTMAbstract
The rapid growth of user density and heterogeneous service requirements in 5G and emerging 6G networks has intensified multi-user interference, particularly in power-domain non-orthogonal multiple access (PD-NOMA) systems. Conventional successive interference cancellation (SIC) techniques suffer from error propagation and performance degradation under imperfect channel conditions, limiting their reliability in practical deployments. This paper proposes a deep learning–based interference cancellation framework that combines Convolutional Neural Networks (CNNs) and Bidirectional Long Short-Term Memory (BiLSTM) networks for end-to-end signal detection in downlink NOMA systems. The proposed model exploits CNN layers to extract spatial interference features from complex-valued received signals, while BiLSTM layers capture bidirectional temporal dependencies caused by channel fading and symbol correlation. Unlike traditional SIC-based receivers, DL-NOMA-CancelNet directly reconstructs user signals without explicit channel estimation or manual interference subtraction. Simulation results obtained using MATLAB under Rayleigh fading and AWGN conditions demonstrate that our framework significantly outperforms conventional Least Squares (LS), Minimum Mean Square Error (MMSE), and single-network deep learning detectors. Specifically, this method achieves up to an 8–10× reduction in bit error rate (BER) across a wide signal-to-noise ratio (SNR) range, while maintaining low inference latency and moderate model complexity. These results confirm that our work provides a robust and scalable solution for interference mitigation in future 5G/6G downlink NOMA systems, particularly for ultra-reliable low-latency communications (URLLC) and massive machine-type communications (mMTC).
Downloads
References
[1] J. Ghosh, I.-H. Ra, S. Singh, Huseyin Haci, K. A. Al-Utaibi, and S. M. Sait, “On the Comparison of Optimal NOMA and OMA in a Paradigm Shift of Emerging Technologies,” IEEE Access, vol. 10, pp. 11616–11632, Jan. 2022, doi: https://doi.org/10.1109/access.2022.3146349.
[2] F. C. Ogenyi, D. B. Mohammed, B. O. Sadiq, Enerst Edozie, and Kelechi John Ukagwu, “Exploring Power Allocation and User Fairness Optimization in NOMA for 5G Networks: A Comprehensive Narrative Review,” Wireless Personal Communications, Sep. 2025, doi: https://doi.org/10.1007/s11277-025-11822-3.
.[3] S. M. R. Islam, N. Avazov, O. A. Dobre, and K. Kwak, “Power-Domain Non-Orthogonal Multiple Access (NOMA) in 5G Systems: Potentials and Challenges,” IEEE Communications Surveys & Tutorials, vol. 19, no. 2, pp. 721–742, 2017, doi: https://doi.org/10.1109/comst.2016.2621116.
[4] Z. Ding et al., “Application of Non-Orthogonal Multiple Access in LTE and 5G Networks,” IEEE Communications Magazine, vol. 55, no. 2, pp. 185–191, Feb. 2017,doi: https://doi.org/10.1109/mcom.2017.1500657cm.
[5]E. Hossain and Y. Al-Eryani, “Energy–Efficient Communication Protocols For Massive Machine-Type Communications (MMTC),” National Journal Of Antennas And Propagation, vol. 7, no. 1, Jan. 2025, doi: https://doi.org/10.31838/njap/07.01.10.
.[6] J. Choi, “NOMA-Based Random Access With Multichannel ALOHA,” IEEE Journal on Selected Areas in Communications, vol. 35, no. 12, pp. 2736–2743, Dec. 2017, doi: https://doi.org/10.1109/jsac.2017.2766778.
[7] M. Ghous, A. K. Hassan, Z. H. Abbas, G. Abbas, A. Hussien, and T. Baker, “Cooperative Power-Domain NOMA Systems: An Overview,” Sensors, vol. 22, no. 24, p. 9652, Dec. 2022, doi: https://doi.org/10.3390/s22249652.
.[8] S. Arunprasath, A. Suresh, and Jayden Khakurel, “Interference Techniques Based on Deep Learning in Wireless Networks,” pp. 161–182, Oct. 2023, doi: https://doi.org/10.1002/9781119827603.ch8.
[9] T. O’Shea and J. Hoydis, “An Introduction to Deep Learning for the Physical Layer,” IEEE Transactions on Cognitive Communications and Networking, vol. 3, no. 4, pp. 563–575, Dec. 2017, doi: https://doi.org/10.1109/tccn.2017.2758370.
[10] H. Ye, G. Y. Li, and B.-H. Juang, “Power of Deep Learning for Channel Estimation and Signal Detection in OFDM Systems,” IEEE Wireless Communications Letters, vol. 7, no. 1, pp. 114–117, Feb. 2018, doi: https://doi.org/10.1109/lwc.2017.2757490.
[11]M. Beck et al., “xLSTM: Extended Long Short-Term Memory,” Advances in Neural Information Processing Systems, vol. 37, pp. 107547–107603, Dec. 2024, doi: https://doi.org/10.52202/079017-3417
[12] S. Rajendran, W. Meert, D. Giustiniano, V. Lenders, and S. Pollin, “Deep Learning Models for Wireless Signal Classification With Distributed Low-Cost Spectrum Sensors,” IEEE Transactions on Cognitive Communications and Networking, vol. 4, no. 3, pp. 433–445, Sep. 2018, doi: https://doi.org/10.1109/tccn.2018.2835460.
[13] N. E. West and T. O’Shea, “Deep architectures for modulation recognition,” 2017 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN), Mar. 2017, doi: https://doi.org/10.1109/dyspan.2017.7920754.
[14] H. Huang, Y. Song, J. Yang, G. Gui, and F. Adachi, “Deep-Learning-Based Millimeter-Wave Massive MIMO for Hybrid Precoding,” IEEE Transactions on Vehicular Technology, vol. 68, no. 3, pp. 3027–3032, Mar. 2019, doi: https://doi.org/10.1109/TVT.2019.2893928.
[15] G. Van Houdt, C. Mosquera, and G. Nápoles, “A review on the long short-term memory model,” Artificial Intelligence Review, vol. 53, no. 8, May 2020, doi: https://doi.org/10.1007/s10462-020-09838-1.
[16] F. Meng, P. Chen, L. Wu, and X. Wang, “Automatic Modulation Classification: A Deep Learning Enabled Approach,” IEEE Transactions on Vehicular Technology, vol. 67, no. 11, pp. 10760–10772, Nov. 2018, doi: https://doi.org/10.1109/tvt.2018.2868698.
[17] P. LI, Z. XI, G. YU, and X. LI. "Research progress and trends of deep learning based wireless communication receiving method". Telecommunications Science. 2022 Feb 1;38(2). doi: https://doi.org/ 10.11959/j.issn.1000-0801.2022025
[18] H. Alzubaidi, Salah Alheejawi, MHD Nour Hindia, Kaharudin Dimyati, and Kamarul Ariffin Noordin, “Interference Mitigation Strategies in Beyond 5G Wireless Systems: A Review,” Electronics, vol. 14, no. 11, pp. 2237–2237, May 2025, doi: https://doi.org/10.3390/electronics14112237.
[19] S. A. H. Mohsan, Y. Li, A. V. Shvetsov, J. Varela-Aldás, S. M. Mostafa, and A. Elfikky, “A Survey of Deep Learning Based NOMA: State of the Art, Key Aspects, Open Challenges and Future Trends,” Sensors, vol. 23, no. 6, p. 2946, Jan. 2023, doi: https://doi.org/10.3390/s23062946.
[20] B. Makki, K. Chitti, A. Behravan, and M.-S. Alouini, “A Survey of NOMA: Current Status and Open Research Challenges,” IEEE Open Journal of the Communications Society, vol. 1, pp. 179–189, 2020, doi: https://doi.org/10.1109/ojcoms.2020.2969899.
[21] W. M. Salama, M. H. Aly, and E. S. Amer, “Deep learning based BER improvement for NOMA-VLC systems with perfect and imperfect successive interference cancellation,” Optical and quantum electronics, vol. 55, no. 8, Jun. 2023, doi: https://doi.org/10.1007/s11082-023-04988-2.
[22]L. L. Dong, “Transformer-Driven Resource Allocation for Enhanced Multi-Carrier NOMA Downlink,” pp. 1–6, Apr. 2024, doi: https://doi.org/10.1109/wcnc57260.2024.10570567.
[23] M. A. Abdelhamed, M. Samy, B. E. Elnaghi, and A. Magdy, “A hybrid CNN-BILSTM deep learning framework for signal detection of a massive MIMONOMA system,” Results in Engineering, vol. 27, p. 105852, Jun. 2025, doi: https://doi.org/10.1016/j.rineng.2025.105852.
[24] R. Kumar and P. Tanna, “Deep Learning-Based NOMA System for Enhancement of 5G Networks: A Review,” IEEE Transactions on Neural Networks and learning systems, pp. 1–15, Jan. 2022,doi: https://doi.org/10.1109/tnnls.2022.3200825.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Emad A. Hussien
This work is licensed under a Creative Commons Attribution 4.0 International License.
