Reinforcement Learning Rebirth, Techniques, Challenges, and Resolutions

Wasswa Shafik - Computer Engineering Department, Yazd University, Yazd, Iran
Mojtaba Matinkhah - Computer Engineering Department, Yazd University, Yazd, Iran
Parisa Etemadinejad - Computer Engineering Department, Yazd University, Yazd, Iran
Mammann Sanda - Department of Physics, Yazd University, Yazd, Iran


Citation Format:



DOI: http://dx.doi.org/10.30630/joiv.4.3.376

Abstract


Reinforcement learning (RL) is a new propitious research space that is well-known nowadays on the internet of things (IoT), media and social sensing computing are addressing a broad and pertinent task through making decisions sequentially by deterministic and stochastic evolutions. The IoTs extend world connectivity to physical devices like electronic devices network by use interconnect with others over the Internet with the possibility of remotely being supervised and meticulous. In this paper, we comprehensively survey an in-depth assessment of RL techniques in IoT systems focusing on the main known RL techniques like artificial neural network (ANN), Q-learning, Markov Decision Process (MDP), Learning Automata (LA). This study examines and analyses learning technique with focusing on challenges, models performance, similarities and the differences in IoTs accomplish with most correlated proposed state of the art models. The results obtained can be used as a foundation for designing, a model implementation based on the bottlenecks currently assessed with an evaluation of the most fashionable hands-on utility of current methods for reinforcement learning.


Keywords


Internet of Things; Reinforcement Learning; Artificial neural networks; Learning Automata; Q-learning; Markov decision process.

Full Text:

PDF

References


J. Su, D. V. Vargas, and K. Sakurai, “Attacking convolutional neural network using differential evolution,†IPSJ Trans. Comput. Vis. Appl., vol. 11, no. 1, p. 1, 2019.

A. Sakata, N. Takemura, and Y. Yagi, “Gait- based age estimation using multi-stage convolutional neural network,†IPSJ Trans. Comput. Vis. Appl., vol. 11, no. 1, p. 4, 2019.

V. Gullapalli, “A stochastic reinforcement learning algorithm for learning real-valued functions,†Neural networks, vol. 3, no. 6, pp. 671–692, 1990.

W. D. Smart and L. P. Kaelbling, “Effective reinforcement learning for mobile robots,†in Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292), 2002, vol. 4, pp. 3404–3410.

J. Garcia, F. R. Ervin, and R. A. Koelling, “Learning with prolonged delay of reinforcement,†Psychonomic Science, vol. 5, no. 3, pp. 121–122, 1966.

L. K. Fellows and M. J. Farah, “Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm,†Brain, vol. 126, no. 8, pp. 1830–1837, 2003.

B. Chatterjee, D. Das, S. Maity, and S. Sen, “RF- PUF: Enhancing IoT Security through Authentication of Wireless Nodes using In-situ Machine Learning,†IEEE Internet of Things Journal, vol. 6, no. 1, pp. 388–398, 2019.

P. Sun, J. Li, M. Z. A. Bhuiyan, L. Wang, and B. Li, “Modeling and clustering attacker activities in IoT through machine learning techniques,†Information Sciences, vol. 479, pp. 456–471, 2019.

A. Singla and A. Sharma, “Physical Access System Security of IoT Devices using Machine Learning Techniques,†Available at SSRN 3356785, 2019.

P. Punithavathi, S. Geetha, M. Karuppiah, S. H. Islam, M. M. Hassan, and K.-K. R. Choo, “A lightweight machine learning-based authentication framework for smart IoT devices,†Information Sciences, vol. 484, pp. 255–268, 2019.

F. Hussain, R. Hussain, S. A. Hassan, and E. Hossain, “Machine Learning in IoT Security: Current Solutions and Future Challenges,†arXiv preprint arXiv:1904.05735, 2019.

I. Grondman, L. Busoniu, G. A. Lopes, and R. Babuska, “A survey of actor-critic reinforcement learning: Standard and natural policy gradients,†IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 6, pp. 1291–1307, 2012.

J. Qiu, Q. Wu, G. Ding, Y. Xu, and S. Feng, “A survey of machine learning for big data processing,†EURASIP Journal on Advances in Signal Processing, vol. 2016, no. 1, p. 67, 2016.

J. Schatzmann, K. Weilhammer, M. Stuttle, and S. Young, “A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies,†The knowledge engineering review, vol. 21, no. 2, pp. 97–126, 2006.

Z. Shi, J. Tu, Q. Zhang, L. Liu, and J. Wei, “A survey of swarm robotics system,†in International Conference in Swarm Intelligence, 2012, pp. 564– 572.

M. L. Valarmathi, L. Sumathi, and G. Deepika, “A survey on node discovery in Mobile Internet of Things (IoT) scenarios,†in 2016 3rd International Conference on Advanced Computing and Communication Systems (ICACCS), 2016, vol. 1, pp. 1–5.

O. B. Sezer, E. Dogdu, and A. M. Ozbayoglu, “Context-aware computing, learning, and big data in internet of things: a survey,†IEEE Internet of Things Journal, vol. 5, no. 1, pp. 1–27, 2018.

R. Ahad, and Y. Yagi, “Spatio-temporal silhouette sequence reconstruction for gait recognition against occlusion,†IPSJ Trans. Comput. Vis. Appl., vol. 11, no. 1, p. 9, 2019.

P. Goyal, H. Malik, and R. Sharma, “Application of Evolutionary Reinforcement Learning (ERL) Approach in Control Domain: A Review,†in Smart Innovations in Communication and Computational Sciences, Springer, 2019, pp. 273–288.

Z. Liu, C. Yao, H. Yu, and T. Wu, “Deep reinforcement learning with its application for lung cancer detection in medical Internet of Things,†Future Generation Computer Systems, 2019, pp 1-9.

M. Shojafar and M. Sookhak, Internet of everything, networks, applications, and computing systems (IoENACS). Taylor & Francis, 2019, pp 1-3.

M. Jamshidi, S. S. A. Poor, N. N. Qader, M. Esnaashari, and M. R. Meybodi, “A Lightweight Algorithm against Replica Node Attack in Mobile Wireless Sensor Networks using Learning Agents,†IEIE Transactions on Smart Processing & Computing, vol. 8, no. 1, pp. 58–70, 2019.

X. Lu, Y. Tsao, S. Matsuda, and C. Hori, “Speech enhancement based on deep denoising autoencoder.,†in Interspeech, 2013, pp. 436–440.

H. Leopold, H. van der Aa, J. Offenberg, and H. A. Reijers, “Using Hidden Markov Models for the accurate linguistic analysis of process model activity labels,†Information Systems, vol. 83, pp. 30–39, 2019.

T. Chen, S. Barbarossa, X. Wang, G. B. Giannakis, and Z.-L. Zhang, “Learning and Management for Internet of Things: Accounting for Adaptivity and Scalability,†Proceedings of the IEEE, vol. 107, no. 4, pp. 778–796, 2019.

S. A. M. Shihab, C. Logemann, D.-G. Thomas, and P. Wei, “Towards the Next Generation Airline Revenue Management: A Deep Reinforcement Learning Approach to Seat Inventory Control and Overbooking,†arXiv preprint arXiv:1902.06824, 2019.

R. Rocchetta, L. Bellani, M. Compare, E. Zio, and E. Patelli, “A reinforcement learning framework for optimal operation and maintenance of power grids,†Applied Energy, vol. 241, pp. 291–301, 2019.

R. Vafashoar and M. R. Meybodi, “Reinforcement learning in learning automata and cellular learning automata via multiple reinforcement signals,†Knowledge-Based Systems, vol. 169, pp. 1–27, 2019.

X. Qi, Y. Luo, G. Wu, K. Boriboonsomsin, and M. Barth, “Deep reinforcement learning enabled self- learning control for energy efficient driving,†Transportation Research Part C: Emerging Technologies, vol. 99, pp. 67–81, 2019.

F. Zantalis, G. Koulouras, S. Karabetsos, and D. Kandris, “A Review of Machine Learning and IoT in Smart Transportation,†Future Internet, vol. 11, no. 4, p. 94, 2019.

J. Muñuzuri, L. Onieva, P. Cortés, and J. Guadix, “Using IoT data and applications to improve port- based intermodal supply chains,†Computers & Industrial Engineering, 2019.

N. Jiang, Y. Deng, O. Simeone, and A. Nallanathan,"Cooperative deep reinforcement learning for multiple-group NB-IoT networks optimization,†in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 8424–8428.

M. Jamshidi, S. S. A. Poor, N. N. Qader, M. Esnaashari, and M. R. Meybodi, “A Lightweight Algorithm against Replica Node Attack in Mobile Wireless Sensor Networks using Learning Agents,†IEIE Transactions on Smart Processing & Computing, vol. 8, no. 1, pp. 58–70, 2019.

H. Leopold, H. van der Aa, J. Offenberg, and H. A. Reijers, “Using Hidden Markov Models for the accurate linguistic analysis of process model activity labels,†Information Systems, vol. 83, pp. 30–39, 2019.

T. Chen, S. Barbarossa, X. Wang, G. B. Giannakis, and Z.-L. Zhang, “Learning and Management for Internet of Things: Accounting for Adaptivity and Scalability,†Proceedings of the IEEE, vol. 107, no. 4, pp. 778–796, 2019.

L. Velasco and D. Rafique, “Fault Management Based on Machine Learning,†in Optical Fiber Communication Conference, 2019, pp. W3G–3.

P. Wei, “Towards the Next Generation Airline Revenue Management: A Deep Reinforcement Learning Approach to Seat Inventory Control and Overbooking,†arXiv preprint arXiv:1902.06824, 2019.

X. Zhao, S. Ding, Y. An, and W. Jia, “Applications of asynchronous deep reinforcement learning based on dynamic updating weights,†Applied Intelligence, vol. 49, no. 2, pp. 581–591, 2019.

Shafik, Wasswa, Mojtaba Matinkhah, and Mamman Nur Sanda. "Network Resource Management Drives Machine Learning: A Survey and Future Research Direction." Journal of Communications Technology, Electronics and Computer Science 30 (2020): 1-15.

J. Heyn, P. Gümbel, P. Bobka, F. Dietrich, and K. Dröder, “Application of artificial neural networks in force-controlled automated assembly of complex shaped deformable components,†Procedia CIRP, vol. 79, pp. 131–136, 2019.

F. Aznar, M. Pujol, and R. Rizo, “Obtaining fault tolerance avoidance behavior using deep reinforcement learning,†Neurocomputing, 2019, pp 77-91.

T. Narendra, M. S. Athulya, and P. S. Sathidevi, “Classification of Pitch Disguise Level with Artificial Neural Networks,†in 2019 International Conference on Communication and Signal Processing (ICCSP), 2019, pp. 0631–0635.

F. Cauteruccio et al., “Short-long term anomaly detection in wireless sensor networks based on machine learning and multi-parameterized edit distance,†Information Fusion, vol. 52, pp. 13–30, 2019.

S. Madjiheurem and L. Toni, “Representation Learning on Graphs: A Reinforcement Learning Application,†arXiv preprint arXiv:1901.05351, 2019.

J. A. Carvajal Soto, F. Tavakolizadeh, and D. Gyulai, “An online machine learning framework for early detection of product failures in an Industry 4.0 context,†International Journal of Computer Integrated Manufacturing, pp. 1–14, 2019.

M. M. Aburas, M. S. S. Ahamad, and N. Q. Omar, “Spatio-temporal simulation and prediction of land- use change using conventional and machine learning models: a review,†Environmental monitoring and assessment, vol. 191, no. 4, p. 205, 2019.

A. Enami, J. A. Torkestani, and A. Karimi, “Resource selection in computational grids based on learning automata,†Expert Systems with Applications, vol. 125, pp. 369–377, 2019.

A. Muñoz, J. Toutouh, and F. Jaime, “A Review of Dynamic Verification of Security and Dependability Properties,†in Artificial Intelligence and Security Challenges in Emerging Networks, IGI Global, 2019, pp. 162–187.

F. Zantalis, G. Koulouras, S. Karabetsos, and D. Kandris, “A Review of Machine Learning and IoT in Smart Transportation,†Future Internet, vol. 11, no. 4, p. 94, 2019.

R. Kashyap, “Deep Learning: An Application in Internet of Things,†in Computational Intelligence in the Internet of Things, IGI Global, 2019, pp. 130– 158.

T. Chilimbi, Y. Suzue, J. Apacible, and K. Kalyanaraman, “Project adam: Building an efficient and scalable deep learning training system,†in 11th ${$USENIX$}$ Symposium on Operating Systems Design and Implementation (${$OSDI$}$ 14), 2014, pp. 571–582.

A. M. Saghiri, M. D. Khomami, and M. R. Meybodi, Intelligent Random Walk: An Approach Based on Learning Automata. Springer, 2019.

A. Enami, J. A. Torkestani, and A. Karimi, “Resource selection in computational grids based on learning automata,†Expert Systems with Applications, vol. 125, pp. 369–377, 2019.

B. Bordel, R. Alcarria, and D. Sánchez-de-Rivera, “A Two-Phase Algorithm for Recognizing Human Activities in the Context of Industry 4.0 and Human- Driven Processes,†in World Conference on Information Systems and Technologies, 2019, pp. 175–185.

Z. Bouyahia, H. Haddad, N. Jabeur, and A. Yasar, “A two-stage road traffic congestion prediction and resource dispatching toward a self-organizing traffic control system,†Personal and Ubiquitous Computing, pp. 1–12, 2019.

A. Karami, “An anomaly-based intrusion detection system in presence of benign outliers with visualization capabilities,†Expert Systems with Applications, vol. 108, pp. 36–60, 2018.

J. Li et al., “A Traffic Prediction Enabled Double Rewarded Value Iteration Network for Route Planning,†IEEE Transactions on Vehicular Technology, 2019, vol. 68, pp 4170 – 4181.

S. Zhang, Z. Kang, Z. Zhang, C. Lin, C. Wang, and J. Li, “A Hybrid Model for Forecasting Traffic Flow: Using Layerwise Structure and Markov Transition Matrix,†IEEE Access, vol. 7, pp. 26002–26012, 2019.

A. S. Hsu, J. B. Martin, A. N. Sanborn, and T. L. Griffiths, “Identifying category representations for complex stimuli using discrete Markov chain Monte Carlo with people,†Behavior research methods, pp. 1–11, 2019.

Y. Zou, W. Zhang, W. Weng, and Z. Meng, “Multi- Vehicle Tracking via Real-Time Detection Probes and a Markov Decision Process Policy,†Sensors, vol. 19, no. 6, p. 1309, 2019.

A. Berger and F. Maly, “Speech Activity Detection for Deaf People: Evaluation on the Developed Smart Solution Prototype,†in Asian Conference on Intelligent Information and Database Systems, 2019, pp. 55–66.

N. Jain and S. Rastogi, “Speech Recognition Systems–A Comprehensive Study Of Concepts And Mechanism,†Acta Informatica Malaysia (AIM), vol. 3, no. 1, pp. 1–3, 2019.

E. Lin, Q. Chen, and X. Qi, “Deep Reinforcement Learning for Imbalanced Classification,†arXiv preprint arXiv:1901.01379, 2019.

A. Rao and M. Diamond, “Deep Learning of Markov Model Based Machines for Determination of Better Treatment Option Decisions for Infertile Women,†bioRxiv, p. 606921, 2019.

S. S. Oyewobi, G. P. Hancke, A. M. Abu-Mahfouz, and A. J. Onumanyi, “An Effective Spectrum Handoff Based on Reinforcement Learning for Target Channel Selection in the Industrial Internet of Things,†Sensors, vol. 19, no. 6, p. 1395, 2019.

E. Kayir and H. Hilal, “Q-Learning Based Failure Detection and Self-Recovery Algorithm for Multi- Robot Domains,†Elektronika ir Elektrotechnika, vol. 25, no. 1, pp. 3–7, 2019.

X. Lin, R. Gu, H. Li, and Y. Ji, “A service reconfiguration scheme for network restoration based on reinforcement learning,†in 17th International Conference on Optical Communications and Networks (ICOCN2018), 2019, vol. 11048, p. 110481K.

F.-C. Ghesu et al., “Multi-scale deep reinforcement learning for real-time 3D-landmark detection in CT scans,†IEEE transactions on pattern analysis and machine intelligence, vol. 41, no. 1, pp. 176–189, 2019.

D. Terada, and C. Guo, “Automatic collision avoidance of multiple ships based on deep Q- learning,†Applied Ocean Research, vol. 86, pp. 268– 288, 2019.

X. Lin, R. Gu, H. Li, and Y. Ji, “A service reconfiguration scheme for network restoration based on reinforcement learning,†in 17th International Conference on Optical Communications and Networks (ICOCN2018), 2019, vol. 11048, p. 110481.

Hodo, E., Bellekens, X., Hamilton, A., Dubouilh, P. L., Iorkyase, E., Tachtatzis, C., & Atkinson, R. (2016, May). Threat analysis of IoT networks using artificial neural network intrusion detection system. In 2016 International Symposium on Networks, Computers and Communications (ISNCC) (pp. 1-6). IEEE.

H. H. Pajouh, R. Javidan, R. Khayami, D. Ali, and K.-K. R. Choo, “A two-layer dimension reduction and two-tier classification model for anomaly-based intrusion detection in IoT backbone networks,†IEEE Transactions on Emerging Topics in Computing, 2016, vol. 7, pp 314 – 323.

C. Perera, A. Zaslavsky, P. Christen, and D. Georgakopoulos, "Context-aware computing for the internet of things: A survey," IEEE communications surveys & tutorials, vol. 16, no. 1, pp. 414–454, 2014.

Y. Zhu et al., “Target-driven visual navigation in indoor scenes using deep reinforcement learning,†in 2017 IEEE international conference on robotics and automation (ICRA), 2017, pp. 3357–3364.

M. Lopez-Martin, B. Carro, A. Sanchez-Esguevillas, and J. Lloret, “Conditional variational autoencoder for prediction and feature recovery applied to intrusion detection in iot,†Sensors, vol. 17, no. 9, p. 1967, 2017.

M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,†in European conference on computer vision, 2014, pp. 818–833.

Szegedy, Christian, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. "Intriguing properties of neural networks." arXiv preprint arXiv:1312.6199 (2013).

J. E. Villaverde, D. Godoy, and A. Amandi, “Learning styles’ recognition in e-learning environments with feed-forward neural networks,†Journal of Computer Assisted Learning, vol. 22, no. 3, pp. 197–206, 2006.

H. H. Pajouh, R. Javidan, R. Khayami, D. Ali, and K.-K. R. Choo, “A two-layer dimension reduction and two-tier classification model for anomaly-based intrusion detection in IoT backbone networks,†IEEE Transactions on Emerging Topics in Computing, 2016, vol.7, pp 314 – 323.

G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, “Extreme learning machine: a new learning scheme of feedforward neural networks,†Neural networks, vol. 2, pp. 985–990, 2004.

N. B. Karayiannis and G. W. Mi, “Growing radial basis neural networks: Merging supervised and unsupervised learning with network growth techniques,†IEEE Transactions on Neural networks, vol. 8, no. 6, pp. 1492–1506, 1997.

P. V. Krishna, S. Misra, D. Joshi, and M. S. Obaidat, “Learning automata based sentiment analysis for recommender system on cloud,†in 2013 International Conference on Computer, Information and Telecommunication Systems (CITS), 2013, pp. 1– 5.

D. Hakkani-Tür, G. Riccardi, and A. Gorin, “Active learning for automatic speech recognition,†in 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2002, vol. 4, pp. IV–3904.

S. Misra, P. V. Krishna, V. Saritha, H. Agarwal, and Ahuja, “Learning automata-based multi- constrained fault-tolerance approach for effective energy management in smart grid communication network,†Journal of Network and Computer Applications, vol. 44, pp. 212–219, 2014.

S.-H. Zahiri, “Learning automata-based classifier,†Pattern Recognition Letters, vol. 29, no. 1, pp. 40– 48, 2008.

B. Braune, S. Diehl, A. Kerren, and R. Wilhelm, “Animation of the generation and computation of finite automata for learning software,†in International Workshop on Implementing Automata, 1999, pp. 39–47.

K. S. Narendra and M. A. Thathachar, “Learning automata-a survey,†IEEE Transactions on systems, man, and cybernetics, no. 4, pp. 323–334, 1974.

A. K. Ghosh, C. Michael, and M. Schatz, “A real- time intrusion detection system based on learning program behavior,†in International Workshop on Recent Advances in Intrusion Detection, 2000, pp. 93–109.

C. L. Giles, C. B. Miller, D. Chen, H.-H. Chen, G.-Z. Sun, and Y.-C. Lee, “Learning and extracting finite state automata with second-order recurrent neural networks,†Neural Computation, vol. 4, no. 3, pp. 393–405, 1992.

B. B. Zarpelão, R. S. Miani, C. T. Kawakani, and S.C. de Alvarenga, “A survey of intrusion detection in Internet of Things,†Journal of Network and Computer Applications, vol. 84, pp. 25–37, 2017.

A. Abduvaliyev, A.-S. K. Pathan, J. Zhou, R. Roman, and W.-C. Wong, “On the vital areas of intrusion detection systems in wireless sensor networks,†IEEE Communications Surveys & Tutorials, vol. 15, no. 3, pp. 1223–1237, 2013.

R. Zhao, R. Yan, Z. Chen, K. Mao, P. Wang, and R. X. Gao, “Deep learning and its applications to machine health monitoring: A survey,†arXiv preprint arXiv:1612.07640, 2016.

S. Misra, P. V. Krishna, H. Agarwal, A. Saxena, and M. S. Obaidat, “A learning automata-based solution for preventing distributed denial of service in Internet of things,†in 2011 International Conference on Internet of Things and 4th International Conference on Cyber, Physical and Social Computing, 2011, pp. 114–122.

M. Weisman et al., “Machine Learning and Data Mining for IPv6 Network Defence,†in International Conference on Cyber Warfare and Security, 2018, pp. 681–XVI.

W. Jiang, C.-L. Zhao, S.-H. Li, and L. Chen, “A new learning automata-based approach for online tracking of event patterns,†Neurocomputing, vol. 137, pp. 205–211, 2014.

S. Raza, L. Wallgren, and T. Voigt, “SVELTE: Real- time intrusion detection in the Internet of Things,†Ad hoc networks, vol. 11, no. 8, pp. 2661–2674, 2013.

Shafik, Wasswa, S. Mojtaba Matinkhah, and Mohammad Ghasemzadeh. "A Fast Machine Learning for 5G Beam Selection for Unmanned Aerial Vehicle Applications." Information Systems & Telecommunication: 262, 2019.

Z. Yan, P. Zhang, and A. V. Vasilakos, “A survey on trust management for Internet of Things,†Journal of network and computer applications, vol. 42, pp. 120– 134, 2014.

M. A. Al-Garadi, A. Mohamed, A. Al-Ali, X. Du, and M. Guizani, “A survey of machine and deep learning methods for internet of things (IoT) security,†arXiv preprint arXiv:1807.11023, 2018.

K. Zaheer, M. Othman, M. H. Rehmani, and T. Perumal, “A Survey of Decision-Theoretic Models for Cognitive Internet of Things (CIoT),†IEEE Access, vol. 6, pp. 22489–22512, 2018.

L. Cao, G. Weiss, and S. Y. Philip, “A brief introduction to agent mining,†Autonomous Agents and Multi-Agent Systems, vol. 25, no. 3, pp. 419–424, 2012.

O. B. Sezer, E. Dogdu, and A. M. Ozbayoglu, “Context-aware computing, learning, and big data in internet of things: a survey,†IEEE Internet of Things Journal, vol. 5, no. 1, pp. 1–27, 2018.

C. Gomez, A. Shami, and X. Wang, “Machine Learning Aided Scheme for Load Balancing in Dense IoT Networks,†Sensors, vol. 18, no. 11, p. 3779, 2018.

F. M. Al-Turjman, “Information-centric sensor networks for cognitive IoT: an overview,†Annals of Telecommunications, vol. 72, no. 1–2, pp. 3–18, 2017.

M. A. Alsheikh, S. Lin, D. Niyato, and H.-P. Tan, “Machine learning in wireless sensor networks: Algorithms, strategies, and applications,†IEEE Communications Surveys & Tutorials, vol. 16, no. 4, pp. 1996–2018, 2014.

K. Ye, “Key Feature Recognition Algorithm of Network Intrusion Signal Based on Neural Network and Support Vector Machine,†Symmetry, vol. 11, no. 3, p. 380, 2019.

J. Abreu, L. Fred, D. Macêdo, and C. Zanchettin, A. Rezvanian, B. Moradabadi, M. Ghavipour, M. M. D. Khomami, and M. R. Meybodi, “Wavefront Cellular Learning Automata: A New Learning “Hierarchical Attentional Hybrid Neural Networks for Document Classification,†arXiv preprint arXiv:1901.06610, 2019.

K. Wang, “Network data management model based on Naïve Bayes classifier and deep neural networks modelling and data-driven techniques for systems analysis,†Journal of Intelligent Information Systems, in heterogeneous wireless networks,†Computers & Electrical Engineering, vol. 75, pp. 135–145, 2019.

P. F. Fantoni, “A neuro-fuzzy model applied to full range signal validation of PWR nuclear power plant data,†INTERNATIONAL JOURNAL OF GENERAL SYSTEM, vol. 29, no. 2, pp. 305–320, 2000.

M. Kahng, N. Thorat, D. H. P. Chau, F. B. Viégas, and M. Wattenberg, “GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation,†IEEE transactions on visualization and computer graphics, vol. 25, no. 1, pp. 310–320, 2019.

D. Popa, F. Pop, C. Serbanescu, and A. Castiglione, “Deep learning model for home automation and energy reduction in a smart home environment platform,†Neural Computing and Applications, pp. 1–21, 2019.

S. Baruah, “Botnet Detection: Analysis of Various Techniques,†International Journal of Computational Intelligence & IoT, vol. 2, no. 2, 2019, pp 7-14.

A. Mollalo, L. Mao, P. Rashidi, and G. E. Glass, “A GIS-Based Artificial Neural Network Model for Spatial Distribution of Tuberculosis across the Continental United States,†International journal of environmental research and public health, vol. 16, no. 1, p. 157, 2019.

R. V. McCarthy, M. M. McCarthy, W. Ceccucci, and L. Halawi, “Predictive Models Using Neural Networks,†in Applying Predictive Analytics, Springer, 2019, pp. 145–173.

A. Rezvanian, B. Moradabadi, M. Ghavipour, M. M. D. Khomami, and M. R. Meybodi, “Introduction to Learning Automata Models,†in Learning Automata Approach for Social Networks, Springer, 2019, pp. 1– 49.

W. Shafik, S. M. Matinkhah, and M. Ghasemzadeh, “Internet of Things-Based Energy Management, Challenges, and Solutions in Smart Cities,†J. Commun. Technol. Electron. Comput. Sci., vol. 27, pp. 1–11, 2020.

W. Shafik and S. A. Mostafavi, “Knowledge Engineering on Internet of Things through Reinforcement Learning,†Int. J. Comput. Appl., vol. 975, p. 8887.

S. M. Matinkhah, W. Shafik, and M. Ghasemzadeh, “Emerging Artificial Intelligence Application: Reinforcement Learning Issues on Current Internet of Things,†in 2019 16th international Conference in information knowledge and Technology (ikt2019), p. 2019.

S. Mostafavi and W. Shafik, “Fog Computing Architectures, Privacy and Security Solutions,†J. Commun. Technol. Electron. Comput. Sci., vol. 24, pp. 1–14, 2019.

W. Shafik, M. Matinkhah, M. Asadi, Z. Ahmadi, and Z. Hadiyan, “A Study on Internet of Things Performance Evaluation,†J. Commun. Technol. Electron. Comput. Sci., vol. 28, pp. 1–19, 2020.

S. Mostafavi and W. Shafik, “Fog Computing Architectures, Privacy and Security Solutions,†J. Commun. Technol. Electron. Comput. Sci., vol. 24, pp. 1–14, 2019.