Multipath Routing Implementation in SD-IoT Network Using OpenFlow-based Routing Metrics

— The implementation growth of the Internet of Things (IoT) may increase the complexity of the data transmission process between smart devices. The route generation process between available nodes on the network will burden the intermediary node. One of the possible solutions for resolving the problem is the integration of Software Defined Networks and IoT (SD-IoT) to provide network automation and management. The separation of networking control and data forwarding functions may provide a multipath delivery path between each node in the IoT environment. In addition, the controller can directly extract the resource usage of the intermediary devices, which can be utilized as the routing metric variable in order to maintain the resource utilization on the intermediary devices. Instead of using traditional routing, this paper aims to develop multipath routing based on Deep First Search (DFS) and Dijkstra algorithms for acquiring an efficient path using OpenFlow-based routing metrics. The traffic monitoring module delivered the metrics extraction process, which obtained the variables using Port and Aggregate Flow Statistic features. The metrics calculation aimed to provide the multipath, which was constructed based on switches resource usage. Each selected path was chosen based on the smallest cost and probability provided by the group table feature in OpenFlow. The results showed that the Dijkstra algorithm could create the multipath more swiftly than DFS with a time difference of 0.6 s. The Quality of Service (QoS) results also indicated that the proposed routing metric variables could maintain the transmission process efficiently.


I. INTRODUCTION
IoT has proliferated in recent years. IoT integrates sensors, embedded computing, and communication technologies to expand internet connectivity in sharing data, remote control, or sensing [1], [2]. However, IoT has limitations in processing the formation of the delivery path due to its complexity and limited computing resources, thus slowing down the path formation and affecting the data transmission process [3].
An alternative solution can be applied in implementing multipath routing in IoT because the routing model can be used to distribute traffic on the formed paths. Multipath routing is also more efficient than single-path routing by implementing load balancing. Routers can divide traffic well and have multiple paths in multipath routing, improving network performance, security, and reliability because multipath routing makes full use of network resources [4]. However, the application of multipath routing in IoT has a problem that requires significant computing resources [5], [6]. Given the problem of limited resource computing on IoT, it is necessary to have an integration process with Software Defined Network (SDN) architecture, commonly referred to as SD-IoT, as a potentially feasible solution to strengthen management and control capabilities on IoT networks.
SDN is an ideal new paradigm for centralized management processes on a computer network because SDN is dynamic and efficient. SDN can improve network elasticity and scalability, a fundamental characteristic in large-scale IoT applications. Effectively the control plane is separated from the data plane and implements a centralized management process on the controller node [7]. IoT can take advantage of SDN to have centralized control, network device abstraction, and flexibility [8], [9]. The SD-IoT controller can be easily programmed according to the needs of the network administrators due to the application-based configuration model [8]. This scheme may allow the implementation of multipath routing that runs automatically. Implementing multipath routing and integrating SDN with IoT requires the OpenFlow protocol so that the control plane can communicate with the data plane [10]. Through the commands defined in OpenFlow, the multipath routing algorithm can define routing metrics variables by focusing on the resource efficiency for each device.
Several studies have been conducted to investigate the application of the multipath routing model on SDN and SD-IoT architectures. Rhamdani et al. [11] applied Multipath Routing to SDN by integrating the Equal-Cost Multipath Routing (ECMP) scheme and the modified Dijkstra's Algorithm to find the shortest path. The results showed that the proposed algorithm takes as much as 1 ms in route selection. The algorithm developed by the researcher is also able to play an effective role in the transmission process that is sensitive to the Quality of Service (QoS) variables such as video streaming. Ahmed et al. [12] performed research to overcome difficulties in identifying flows that passed through the IoT network because there were devices that played the role of controllers and devices that played the role of flows using a method called Collaborative Flow-Identification Mechanism (CFIM). CFIM identifies flows that pass through the IoT network by collecting information from devices that play the flow role and sending that information to devices that play the controller role. In addition, CFIM also implements a decision-making mechanism that can make optimal decisions in identifying flows that pass through the IoT network. CFIM is expected to increase efficiency in identifying flows passing through the IoT network and can be used in applications that require high QoS, such as medical applications.
Ali et al. [13] researched how to effectively place controllers in SD-IoT networks to improve network performance and reduce delays using a controller placement method based on a genetic algorithm called Effective Controller Placement (ECP). This algorithm uses the concept of fitness function to evaluate network performance and find the most effective location to place the controller. In addition, the ECP algorithm also implements dynamic traffic control to control the number of data packets allowed to enter the network and reduce delay. This research also uses simulation to test the ECP method and shows that implementing this method can improve network performance and reduce delay compared to existing controller placement methods. Modi et al. [14] developed an algorithm based on Host-limited and link-limited flows in the Data Center Networks (DCN) management process. The results obtained by the researchers indicate that the proposed algorithm has a level of effectiveness and efficiency that is far superior to the Dijkstra and Hedera algorithms. The researcher wishes to apply Deep Learning (DL) in the network management process in the next stage.
Zhong et al. [15] also discovered that the SD-IoT controller could not meet the large-scale networks' enormous data flow requirements. The researcher studied data plane load balancing problems in a Software Defined Large-scale Network. The researcher examines the controller pool's vertical structure for the SD-IoT's control plane, including the controllers (main controllers) of the main control layer and the controllers (primary controllers) of the basic control layer. Then, the author proposes a dynamic core controller balancing algorithm based on an election mechanism and a dynamic base controller load balancing algorithm based on a balanced delay model. A similar subsequent study [16] has research on multipath routing in SDN networks which is handled and completed under the OpenFlow protocol, which uses the Ryu controller to implement SDN networks. In the simulation results, the Depth First Search (DFS) Algorithm gives superior results compared to the Breadth First Search (BFS) Algorithm in terms of Round-Trip Time (RTT), delay, and throughput. From the simulation results, the researchers validated the QoS improvement for SDN architecture in local area networks. Syaifuddin et al. [17] applied multipath routing, which was implemented on SDN. The researcher implemented a modified DFS and Dijkstra multipath routing algorithm. In the comparison results, it can be concluded that the performance of the modified DFS multipath algorithm is better than Dijkstra's multipath algorithm.
Similarly, Chen et al. [18] proposed an aultipath routing method that utilizes reinforcement learning based on network state and flow characteristics. The authors concluded that their method outperformed the current mainstream shortest path and ECMP algorithm. Aljohani et al. [19] investigated their proposed method called Multipath Resilient routing system based on Software Defined Networking (MPResiSDN) which adopted the Floyd-Warshall algorithm for finding the multipath resilient route during natural disaster scenario in smart city. The conclusion stated that the proposed approach could improve the data delivery under the experiment scenario better than spanning tree algorithm.
Based on previous research that investigated the implementation of multipath routing, it could be concluded that there was no research focused on the implementation of multipath in the SD-IoT architecture. In addition, the research that has been done does not focus on the routing model, which aims to minimize the use of resources on an intermediary device. Thus, this manuscript is directed to contribute to the implementation of multipath routing by calculating OpenFlow-based metric values. In OpenFlow rules, controllers can quickly get statistical information on resource usage in an SDN Switch device. The author focuses on taking advantage of the variables of the average number of received data, the average of transmitted data, the number of collisions, and the number of flow rules in an SDN Switch device. These variables are obtained by requesting statistical data on the Aggregate Flow and Port Statistics features. The generated path consists of the preferred route, which is considered to utilize a small amount of resource usage.

A. Research Topology and Scenario
The hardware specification that had been used to run SD-IoT network emulation in this study utilized a computer with the Ubuntu 20.04 LTS Operating System (OS), which had a specification of an Intel® Core™ i5-10400 CPU @ 2.90GHz, 8 GB memory (RAM), and 240 GB of Solid State Drive (SSD) storage space. The software needed to support the success of this research is the Mininet emulation system [20] with a network simulation model using a fat-tree topology, Ryu controller [21] to apply the DFS Multipath algorithm [16] and Dijkstra Multipath [22] algorithm, and Wireshark which used to observe the network traffic through QoS parameters [23] including throughput, delay, jitter, and packet loss. Fat-Tree topology, as depicted in Fig. 1, was used because it had several variations of paths with branching forms like a tree which was very effective for applying the multipath routing concept.
The experiment carried out in this study was based on emulation using Mininet and RYU. Mininet was used as a network topology emulator, while RYU was used as a controller. In this study, several tests were carried out using two different algorithms, namely the DFS and Dijkstra multipath, and also using three different protocols, namely Message Queuing Telemetry Transport (MQTT), Constrained Application Protocol (CoAP), and Hypertext Transport Protocol (HTTP) [24], [25]. The list of tested variables to determine the effectiveness of the algorithm development composes path discovery, path generation time, and testing based on QoS variables (bandwidth, throughput, delay, jitter, and packet loss) [23].
The path discovery scenario was carried out by sending Internet Control Message Protocol (ICMP) packets [26] from h1 as a client to h2 role as a server to find all possible delivery paths. The path generation time scenario determines the average time required for the two algorithms to find the path. Then the bandwidth testing scenario was performed by manual calculations on the Wireshark application in Kilobytes (KB). The QoS scenario, namely throughput, delay, jitter, and packet loss, was done by turning one host into a server (h2) while another host became a client (h1). With the same pattern, the values of these variables were extracted from the transmitted data using HTTP, MQTT, and CoAP packets received on the server side using Wireshark [27].

B. Openflow Matrics for Multipath Routing
The design of the system that has been used in this study is depicted in the block diagram in Fig. 2. On the left side of the block diagram is a network topology created with a mininet emulator, while on the right side is the mechanism for running the RYU controller on the SD-IoT network. In the process of determining the delivery path, the first step performed by the controller is summarizing topology information, including links and connectivity between OpenFlow switches when the network is initializing or if there are new devices connected in the topology. Next, the traffic monitoring module extracts routing metric information from Port Statistics, including the average number of transmitted packets, received packets, and collisions. In addition, other information is obtained from the Aggregate Flow statistics feature, which contains flow rules count. Then the path computation module is enabled to apply the routing algorithms (DFS and Dijkstra) in determining the three paths that will be used in the data transmission process. When a path has been found via the path computation module, the path installation module will be executed to send the OFPT_FLOW_MOD instruction to install a flow rule with a group table type based on the previously generated path on each switch. Subsequently, the data transmission on an SD-IoT network can already be executed.
This study uses a path assessment matrix as the total path weight described in equation (1).
From equation (1) above, pw(p) states the path weight, which is the value of the addition operation of several variables, including ew(e), stating the edge weight obtained from the topology module in the form of bandwidth capacity, rx_bytes(e) stating the total bytes received divided by rx_packets(e) representing the total packets received, tx_bytes(e) representing the total bytes sent divided by tx_packets(e) representing the total packets sent, flowcount(e) representing the number of flow rules in an SDN switch device, and collision(e) representing the number of collisions on a switch. The greater the average value of incoming and transmitted packets, flowcount, and collisions on an SDN switch device, the smaller the possibility of the switch being selected in the delivery path due to the large traffic load on the device. The pseudocode of the routing metric calculation formula used in this study is presented in Fig. 3

C. Openflow Group Table for Multipath Action
Multipath routing can be implemented in SD-IoT networks using the OpenFlow protocol and Fat-Tree topology because the Fat-Tree topology has many paths that can be used as data transmission scenarios [28]. In an SDN network, the controller establishes a Transport Layer Security Protocol (TLS) connection with each switch where the switch, as a data plane, installs the flow rules in the flow table and group table so that the controller can control the data transmission path [29]. A detailed example of the group table component can be seen in Table 1 to handle packet transmission with group_id=2644423182. The switch can select bucket actions based on the path weights calculated during the path calculation process. In Table 1, the switch sends packets through ports 2, 4, and 5. The probability on each port is 0.8 for port 2, 0.15 for port 4, and 0.05 for port 5. The probability value is adjusted based on the final cost for each generated delivery path formed in the path calculation process. A weight of 0.8 is the highest probability of being chosen by the switch, meaning that the path has the lowest cost among other paths. Based on the OpenFlow rules, the bucket weight variable becomes a benchmark in the selection of paths using the group type method as select. After the delivery path is found from h1 to h2, the path is installed on each switch. By default, Openflow will give the switch device the freedom to choose actions that are defined using the scheduling algorithms that are already available. However, by defining bucket_weight, the probability of path selection is based on the value specified for this variable.

A. Path Discovery Results
The results of the path discovery from h1 to h2 using the DFS and the Dijkstra multipath algorithm, which are carried out five times, are presented in Tables 2 and 3. Based on five trials using the multipath DFS algorithm, a path was chosen with the smallest cost value in the 5th experiment with paths [13,6,2,3,11,20] and a cost of 1206.88. The alternative path was path [13,6,2,8,3,11,20] with a cost value of 1440.84, and path [13,6,2,9,3,11,20] with a cost value of 1444.02. Meanwhile, from five trials using the Dijkstra multipath algorithm, there was a path with the smallest cost value in the 4th experiment with path [13,6,2,3,11,20] and a cost value of 1263.05. Other alternative paths were on the path [13,6,2,8,3,11,20] with a cost value of 1498.42 and path [13,5,1,2,3,11,20] with a cost value of 1521.09.

B. Path Generation Time Results
The results of the path generation time for the DFS and Dijkstra's multipath algorithm are presented in Tables 4 and  5. The evaluation was done to determine the time required for the path-searching algorithm that has been developed to be able to find all paths in the Fat-Tree network topology. The results of time readings from each experiment had very different values. This was due to the unstable nature of the emulation using mininet [30]. The average path generation time from the five trials on the DFS multipath algorithm was 450,597 ms, and the Dijkstra multipath algorithm was 1087,962 ms. The multipath DFS algorithm had a much different average time difference from the multipath Dijkstra algorithm. In accordance with the results in research [17], the results of the execution time testing carried out on the same three algorithms, namely the modified DFS multipath algorithm was 0.0903 ms, the multipath DFS algorithm was 0.0858 ms, and the Dijkstra multipath algorithm was 0.901 ms. The test results in these two studies showed that the multipath DFS algorithm had a faster path generation time than the Dijkstra multipath algorithm.

C. QoS Extraction Results
The bandwidth variable was extracted by statistical analysis on the network being tested by collecting statistical measurement data on Wireshark, such as the number of packets received in bytes. The results of data extraction were converted into KiloBytes (KB). Based on the results listed in Table 6, it can be seen that the results of bandwidth testing using the MQTT protocol on the DFS and Dijkstra multipath algorithms resulted in 825.372. While bandwidth testing using CoAP and HTTP protocols has the same results at 708 KB. Each protocol produced similar results for both algorithms. The other QoS experiment was the throughput variable. The emulation was carried out to know a network's ability in actual data transfer. Throughput is a calculation of the value of a packet that has been successfully received at the destination for a certain period and divided by the length of the time interval. The throughput calculation was manually processed by taking Wireshark's statistical data in bytes and period (s). The results of throughput testing using three different protocols that have been carried out are presented in Table 7. The obtained values showed the same pattern for each protocol. MQTT, CoAP, and HTTP protocols had the same throughput value in each algorithm. The third QoS test was delay and jitter, which was done by extracting the reception time between the received packets using the Wireshark application, which was then processed manually using the delay and jitter formula. The delayed test was also carried out using three different protocols, MQTT, CoAP, and HTTP, within five trials, and 100 packets were accumulated for each trial. The results of calculating the 100 packets are calculated on an average, as presented in Table 8. Jitter testing was performed to determine the delay variation between data transmission on the network that could occur due to the influence of traffic load on the network itself. Jitter testing was done by using the data from the calculation of the delay, and then all the resulting values were divided by the number of packets received. Referring to the delay and jitter values, all tested protocols produced relatively the same average values. However, there was a tendency for better delay variations in the DFS algorithm because the preprocessing (path generation time) was not too significant. Packet loss testing was calculated from the percentage of transmitted packets that were not received by the server described in Table 9. Based on the test results, both algorithms can effectively determine the path, as evidenced by the packet loss value equal to 0%.

IV. CONCLUSION
Significant results were obtained on the path discovery experiment's path generation time variable between the DFS and Dijkstra multipath algorithm. In the QoS variable, the value obtained gives the same pattern in both algorithms because the generated path for sending data on the network is approximately the same as the overall multipath routing algorithms. From the results of the path discovery using two different algorithms, the DFS multipath algorithm can find delivery paths faster than Dijkstra. This is because the path search process using the DFS algorithm is carried out by expanding to nodes in a graph.
After the node expansion stage, it is followed by a backtracking process to get the entire path from the source node to the destination. In Dijkstra's algorithm, the pathsearching process uses a greedy strategy where each traversed node is selected based on the smallest weight between the nodes selected in the path and other nodes not selected to produce the shortest path. From the results that have been analyzed, it can be concluded that multipath DFS is superior to Dijkstra's multipath algorithm when applied to SD-IoT with reference to the path generation time value. In addition, processing delay in calculating routing metrics does not impact the delay or packet loss value in the data transmission process. In future research, the authors plan to develop a routing model based on Artificial Intelligence (AI) that can automatically select a routing algorithm according to the realtime conditions on the network.