Citation: | Emanuele De Santis, Alessandro Giuseppi, Antonio Pietrabissa, Michael Capponi, Francesco Delli Priscoli. Satellite Integration into 5G: Deep Reinforcement Learning for Network Selection. Machine Intelligence Research, vol. 19, no. 2, pp.127-137, 2022. https://doi.org/10.1007/s11633-022-1326-3 |
[1] |
3GPP TR 38.811 Study on New Radio (NR) to Support Non-terrestrial Networks, Technical Report. 3GPP, France 2017.
|
[2] |
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, M. Riedmiller. Playing Atari with deep reinforcement learning, [Online], Available: https://arxiv.org/abs/1312.5602v1, 2013.
|
[3] |
K. S. S. Anupama, S. S. Gowri, B. P. Rao. A comparative study of outranking MADM algorithms in network selection. In Proceedings of the 2nd International Conference on Computing Methodologies and Communication, IEEE, Erode, India, pp. 904−907, 2018. DOI: 10.1109/ICCMC.2018.8487931.
|
[4] |
Y. F. Zhong, H. Q. Wang, H. W. Lv. A cognitive wireless networks access selection algorithm based on MADM. Ad Hoc Networks, vol. 109, Article number 102286, 2020. DOI: 10.1016/j.adhoc.2020.102286.
|
[5] |
S. Radouche, C. Leghris, A. Adib. MADM methods based on utility function and reputation for access network selection in a multi-access mobile network environment. In Proceedings of International Conference on Wireless Networks and Mobile Communications, IEEE, Rabat, Morocco, 2017. DOI: 10.1109/WINCOM.2017.8238177.
|
[6] |
Q. Y. Song, A. Jamalipour. Network selection in an integrated wireless LAN and UMTS environment using mathematical modeling and computing techniques. IEEE Wireless Communications, vol. 12, no. 3, pp. 42–48, 2005. DOI: 10.1109/mwc.2005.1452853.
|
[7] |
T. Ding, L. Liang, M. Yang, H. Q. Wu. Multiple attribute decision making based on cross-evaluation with uncertain decision parameters. Mathematical Problems in Engineering, vol. 2016, Article number 4313247, 2016. DOI: 10.1155/2016/4313247.
|
[8] |
R. K. Goyal, S. Kaushal, A. K. Sangaiah. The utility based non-linear fuzzy AHP optimization model for network selection in heterogeneous wireless networks. Applied Soft Computing, vol. 67, pp. 800–811, 2018. DOI: 10.1016/j.asoc.2017.05.026.
|
[9] |
X. Y. Yan, P. Dong, T. Zheng, H. K. Zhang. Fuzzy and utility based network selection for heterogeneous networks in high-speed railway. Wireless Communications and Mobile Computing, vol. 2017, Article number 4967438, 2017. DOI: 10.1155/2017/4967438.
|
[10] |
M. M. R. Mou, M. Z. Chowdhury. Service aware fuzzy logic based handover decision in heterogeneous wireless networks. In Proceedings of International Conference on Electrical, Computer and Communication Engineering, IEEE, Cox′s Bazar, Bangladesh, pp. 686-691, 2017. DOI: 10.1109/ECACE.2017.7912992.
|
[11] |
A. Wilson, A. Lenaghan, R. Malyan. Optimising wireless access network selection to maintain QoS in heterogeneous wireless environments. In Proceedings of International Symposium on Wireless Personal Multimedia Communications, Aalborg, Denmark. 1236−1240, 2005.
|
[12] |
R. Trestian, O. Ormond, G. M. Muntean. Game theory-based network selection: Solutions and challenges. IEEE Communications Surveys &Tutorials, vol. 14, no. 4, pp. 1212–1231, 2012. DOI: 10.1109/surv.2012.010912.00081.
|
[13] |
J. Antoniou, A. Pitsillides. 4G converged environment: Modeling network selection as a game. In Proceedings of the 16th IST Mobile and Wireless Communications Summit, IEEE, Budapest, Hungary, 2007. DOI: 10.1109/ISTMWC.2007.4299242.
|
[14] |
T. Rahman, M. Z. Chowdhury, Y. M. Jang. Radio access network selection mechanism based on hierarchical modelling and game theory. In Proceedings of International Conference on Information and Communication Technology Convergence, IEEE, Jeju, Korea , pp. 126−131, 2016. DOI: 10.1109/ICTC.2016.7763451.
|
[15] |
L. Rajesh, K. B. Bagan, B. Ramesh. User demand wireless network selection using game theory. In Proceedings of International Conference on Nano-electronics, Circuits & Communication Systems, Jharkhand, India, pp.39−53, 2017. DOI: 10.1007/978-981-10-2999-8_4.
|
[16] |
Meenakshi, N. P. Singh. A comparative study of cooperative and non-cooperative game theory in network selection. In Proceedings of International Conference on Computational Techniques in Information and Communication Technologies, IEEE, New Delhi, India, pp. 612−617, 2016. DOI: 10.1109/ICCTICT.2016.7514652.
|
[17] |
R. S. Sutton, A. G. Barto, Reinforcement Learning: An Introduction, Cambridge, UK: MIT Press, 1998.
|
[18] |
Z. H. Zhang, X. F. Jiang, H. S. Xi. Optimal content placement and request dispatching for cloud-based video distribution services. International Journal of Automation and Computing, vol. 13, no. 6, pp. 529–540, 2016. DOI: 10.1007/s11633-016-1025-z.
|
[19] |
F. S. Lin, B. Q. Yin, J. Huang, X. M. Wu. Admission control with elastic QoS for video on demand systems. International Journal of Automation and Computing, vol. 9, no. 5, pp. 467–473, 2012. DOI: 10.1007/s11633-012-0668-7.
|
[20] |
Z. Y. Du, C. X. Wang, Y. M. Sun, G. F. Wu. Context-aware indoor VLC/RF heterogeneous network selection: Reinforcement learning with knowledge transfer. IEEE Access, vol. 6, pp. 33275–33284, 2018. DOI: 10.1109/access.2018.2844882.
|
[21] |
Y. Yang, Y. Wang, K. Y. Liu, N. Zhang, S. S. Gu, Q. Y. Zhang. Deep reinforcement learning based online network selection in CRNs with multiple primary networks. IEEE Transactions on Industrial Informatics, vol. 16, no. 12, pp. 7691–7699, 2020. DOI: 10.1109/tii.2020.2971735.
|
[22] |
D. D. Nguyen, H. X. Nguyen, L. B. White. Reinforcement learning with network-assisted feedback for heterogeneous RAT selection. IEEE Transactions on Wireless Communications, vol. 16, no. 9, pp. 6062–6076, 2017. DOI: 10.1109/twc.2017.2718526.
|
[23] |
F. Liberati, A. Giuseppi, A. Pietrabissa, V. Suraci, A. Di Giorgio, M. Trubian, D. Dietrich, P. Papadimitriou, F. Delli Priscoli. Stochastic and exact methods for service mapping in virtualized network infrastructures. International Journal of Network Management, vol. 27, no. 6, Article number e1985, 2017. DOI: 10.1002/nem.1985.
|
[24] |
X. W. Wang, J. D. Li, L. X. Wang, C. G. Yang, Z. Han. Intelligent user-centric network selection: A model-driven reinforcement learning framework. IEEE Access, vol. 7, pp. 21645–21661, 2019. DOI: 10.1109/access.2019.2898205.
|
[25] |
K. S. Shin, G. H. Hwang, O. Jo. Distributed reinforcement learning scheme for environmentally adaptive IoT network selection. Electronics Letters, vol. 56, no. 9, pp. 462–464, 2020. DOI: 10.1049/el.2019.3891.
|
[26] |
T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, D. Wierstra. Continuous control with deep reinforcement learning. In Proceedings of the 4th International Conference on Learning Representations, San Juan, Puerto Rico, 2016.
|
[27] |
Y. B. Zhou, Z. M. Fadlullah, B. M. Mao, N. Kato. A deep-learning-based radio resource assignment technique for 5G ultra dense networks. IEEE Network, vol. 32, no. 6, pp. 28–34, 2018. DOI: 10.1109/MNET.2018.1800085.
|
[28] |
B. M. Mao, F. X. Tang, Y. Kawamoto, N. Kato. Optimizing computation offloading in satellite-UAV-served 6G IoT: A deep learning approach. IEEE Network, vol. 35, no. 4, pp. 102–108, 2021. DOI: 10.1109/MNET.011.2100097.
|
[29] |
E. De Santis. Trunk96/wireless-network-simulator, [Online], Available: https://github.com/trunk96/wireless-network-simulator, 2022.
|
[30] |
F. D. Priscoli, A. Giuseppi, F. Liberati, A. Pietrabissa. Traffic steering and network selection in 5G networks based on reinforcement learning. In Proceedings of European Control Conference, IEEE, St. Petersburg, Russia, pp. 595−601, 2020. DOI: 10.23919/ECC51009.2020.9143837.
|
[31] |
5G; NR; Physical Channels and Modulation, ETSI TS 138 211 v15.2.0. 3GPP, 2018.
|
[32] |
Final report for COST Action 231, [Online], Available: http://www.lx.it.pt/cost231/final_report.htm, 2022.
|
[33] |
G. Maral, M. Bousquet, Z. L. Sun. Satellite Communications Systems: Systems, Techniques and Technology. 6th ed., Hoboken, USA: Wiley, 2020. DOI: 10.1002/9781119673811.
|