Proactive Load Balancing Mechanism for Fog Computing Supported by Parked Vehicles in IoV-SDN

Ahmed Jawad Kadhim,Jaber Ibrahim Naser

Ministry of Education,General Directorate for Education in Al-Qadisiyah,Al-Qadisiyah 58002,Iraq

Abstract:Internet of Vehicles(IoV)is a new style of vehicular ad hoc network that is used to connect the sensors of each vehicle with each other and with other vehicles’ sensors through the internet.These sensors generate different tasks that should be analyzed and processed in some given period of time.They send the tasks to the cloud servers but these sending operations increase bandwidth consumption and latency.Fog computing is a simple cloud at the network edge that is used to process the jobs in a short period of time instead of sending them to cloud computing facilities.In some situations,fog computing cannot execute some tasks due to lack of resources.Thus,in these situations it transfers them to cloud computing that leads to an increase in latency and bandwidth occupation again.Moreover,several fog servers may be fuelled while other servers are empty.This implies an unfair distribution of jobs.In this research study,we shall merge the software defined network (SDN)with IoV and fog computing and use the parked vehicle as assistant fog computing node.This can improve the capabilities of the fog computing layer and help in decreasing the number of migrated tasks to the cloud servers.This increases the ratio of time sensitive tasks that meet the deadline.In addition,a new load balancing strategy is proposed.It works proactively to balance the load locally and globally by the local fog managers and SDN controller,respectively.The simulation experiments show that the proposed system is more efficient than VANET-Fog-Cloud and IoV-Fog-Cloud frameworks in terms of average response time and percentage of bandwidth consumption,meeting the deadline,and resource utilization.

Keywords:IoV;parked vehicles;SDN;fog computing;load balancing

I.INTRODUCTION

Internet of Vehicle(IoV)represents a smart transportation area which consists of a vehicular mobile Internet,intra-vehicle network,and inter-vehicle network.There is a large number of sensors in this environment which connect with each other without human intervention[1].They generate large amounts of data that need storage,analysis,and execution.However,these devices suffer from lack of resources.Therefore,the traditional method of solving this challenge is sending the data and tasks to cloud computing servers which present different types of efficient applications and resources that can be used by different users via the internet[2][3][4].

IoV sensors send their tasks request processing from the cloud.However,there are multiple points to deliver the tasks to cloud servers since distances are long.Cloud computing suffers from high latency and this not only requires a high bandwidth but it may also cause a catastrophe in some cases[5].This means that cloud computing is not an optimal solution for hard real time tasks.This motivated the discovery of fog computing(FC)[6].The goal of fog computing is to use resources that are located at the network edge to execute launched jobs from the sensors locally.It provides storage,computation,and networking services between IoV sensors and cloud computing.Its benefits for IoV are awareness of vehicle location,supporting movement and applications that need quick responses[7].Fog computing focuses on time sensitive and data intensive IoV applications[8].Thus,it presents a flexible,efficient,and easy solution for IoV to enhance the quality of service because it helps decrease latency of cloud computing applications by using a single hop network.In addition,increases the throughput and avoids loss of bandwidth which appears due to forwarding job data to cloud computing[5,9].

SDN was designed to facilitate the old architecture of networks and increase programmability.It separates this architecture into data and control planes[10].In a SDN based network,the controller has a global view of network devices.Therefore,it has complete fresh information such as the remaining resources and jobs about them [9,11,12].SDN can enhance the Internet of Things (IoT) network,especially when it is merged with fog computing.It provides easy management of different fog nodes[6].The SDN concept abstracts available network resources and controls them using a centralized and intelligent authority that aims to optimize traffic flow in a flexible manner,which in turn increases availability [13].It was merged with IoV [5,9,14,15]and Vehicular Ad hoc Network(VANET)[16–21]to help solve several challenging issues in vehicular networks.Moreover,it increases flexibility,enhances management,applies VANET connections (V2V,I2I and V2I),and efficiently exploits network resources[22,23].

Generally speaking,IoT sensors forward jobs (i.e.,tasks)to nearby fog devices that respond in less time.In some cases,one or more fog devices are overloaded while other fog devices have few tasks or do not have any time sensitive jobs(i.e.,hard real time tasks).This is the inequitable distribution of duties and tasks which may lead to exceeding the deadline and decreasing resource utilization [2,24,25].There are many authors who have proposed solutions for these fog computing problems by transferring a number of jobs to cloud system for execution[24–32].However,latency and bandwidth cost problems will appear once again.Moreover,some other authors have used load balancing and allocation techniques to solve this problem[5,24,26,28,31].However,again the utilization of fog computing layer is low,and some hard real time tasks do not meet their deadline.This is because a few of the previous techniques are focused on load of the fog nodes in each cluster.When a fog node in a given cluster (say cluster 1),reaches the overloaded level,the next incoming tasks are sent to other fog nodes (under-loaded nodes) in cluster 1 if it is available.However,if there is no under-loaded node,the tasks will be transferred to the cloud whether there are any under-loaded fog nodes in other fog clusters or not.Moreover,the available resources of parked vehicles are not exploited in [5,9].It means that the fog computing resources are not efficiently utilized.Therefore,it is important to maximize the efficiency of the fog system layer and exploit the available resources in the best possible method.This can help in processing most tasks locally without sending them as much as possible to cloud computing.This will help reduce the bandwidth needed from the FC to cloud systems and meeting the deadline.This can be done by using parked vehicles as assistant fog nodes.In addition,saving some tasks and waiting for a fog node to be under-loaded if sending these tasks to the cloud servers does not meet their deadline.

The new aspects and contributions of this paper are as follows:

• Producing a new load balancing approach to improve utilization of resources of the FC layer and reduce the traveling of tasks to the cloud layer as much as possible to decrease bandwidth consumption.

• Exploiting the parked vehicles as an assistant for fog computing nodes to increase efficiency of the FC layer by processing more tasks.

• Comparing the waiting and execution time of a task in the fog layer with the sending and execution time of that task in cloud computing to decide which one will execute that task.This will decrease bandwidth consumption and increase the number of tasks that run within the deadline.

• The proposed system focuses on the very important challenge of IoV networks (i.e.,the vehicle mobility)to determine selection priority of an optimal fog server to improve the response time and meet the deadlines of hard real time tasks.

The other sections of the paper are:section II,III,IV,V,and VI illustrate respectively the previous research works,goals of this paper,problem definition and model of system,suggested load balance approach,simulation and evaluation while section 7 shows the conclusions and future aspects.

II.RELATED WORKS

During the last few years,several research works were carried out on vehicular ad hoc networks (VANET),IoV,SDN,parked vehicles and fog computing.This section focuses on a review of these works.Several researchers merged VANET with FC and SDN concepts to perform traffic analysis [16],resource management[17],surveillance services[18],data offloading [33],traffic signal optimization[34],data routing[18,19,21,35–39],information services[40]and security[41].

Moreover,there is a number of investigators who have focused on merging IoV with FC.X.He et al.[5]exploited the advantages of fog environment and SDN to improve performance of IoV networks by balancing the load between the fog and cloud computing systems.E.Lee et al.[42]explained the development of the IoV and converting steps of intelligent vehicle grid to autonomous systems.Moreover,they discussed the connection of fog platforms with IoV.S.Chun et al.[43]proposed an IoV-Fog architecture based on a publish/subscribe model.Furthermore,they proposed upper-level domain ontology in order to enhance the expressivity of knowledge and describe a variety of semantic properties that interlink spatial information in IoV.Finally,they present an active rule to support exchange of event-driven messages between publishing and subscribing fog nodes.T.S.J.Darwish et al.[44]introduced a new framework to analyze big data in IoV.They integrated fog,cloud,and big data analytics with IoV.Moreover,they described problems and issues that occur in applying FC and big data analytics in IoV.J.I.Naser et al.[45]suggested an authentication mechanism to secure connections in IoV supported by fog computing.A.J.Kadhim et al.[9]used SDN to optimize utilization of fog computing resources and response time.They applied a load balancing technique locally in each cluster manager and globally in the SDN controller.In addition to [5,9],the researchers in [24,26,28,31,46–51]focused on presenting load balance strategies.M.Verma et al.[24]presented a load balancing approach for fog computing.In their approach,each client forwards the requests to the nearest FC node which processes them based on data availability and workload.If a fog node fails in running a request,this request will be transferred directly to the cloud.R.Deng et al.[26]introduced techniques to allocate to edge and cloud systems depending on energy consumption and delay trade off.They applied generalized benders decomposition and interior-point approaches for this tradeoff and the Hungarian method to minimize dispatch delay.S.Verma et al.[28]proposed a load balancing algorithm based on replication.In their approach,if a fog device fails in running a specific request,it will forward it to all FC nodes.If there is a particular fog node that can execute this request,it sends the required data to the source fog device which replicates it to be exploited with other requests after running request.Otherwise,the request will be sent to the cloud.S.Ningning et al.[31]presented a load balancing method depending on dynamic graph partitioning for fog environment.They applied cloud atomization strategy on FC devices to produce virtual machines.E.Batista et al.[46]presented a load balancing strategy for distributed IoT-Fog networks to decrease response time based on SDN.They assumed that each set of IoT sensors are managed by an IoT gateway to reveal the failure cases and link overhead.N.T´ellez et al.[47]proposed an approach for fog-cloud environment load balancing depending on Tabu Search strategy.They used two cost functions to calculate execution cost of IoT tasks in cloud and fog computing.S.Sharma et al.[48]designed a four layer framework to schedule IoT tasks with minimum delay and balance load at the fog landscape to reduce delay and energy consumption.H.E.Refaat et al.[49]presented a model called decentralize load-balance scheduling for IoT-Fog system.It works based on the load of the fog devices and request’s type to distribute the load.F.Banaie et al.[50]presented a multi criteria decision making approach for load balancing of multiple gateways in IoT-Fog-SDN networks.Their main goal is to distribute load among network devices to reduce energy consumption and response time.F.M.Talaat et al.[51]proposed a load balancing approach for IoT-Fog networks based on genetic algorithm and reinforced learning.Depending on the collected information about the load of fog servers,it distributes the incoming tasks among the servers using a dynamic allocation technique.Table1 shows a comparison of the proposed approach and existing load balancing techniques based on several parameters.In this Table,the word“Comparison”refers to comparing waiting and execution times of a task in the fog layer with sending and execution times of the same task in cloud computing.

Moreover,several authors have proposed various resource management techniques to improve fog platform efficiency.O.Skarlat et al.[25]proposed a resource provision method for the IoT-Fog network.IoT devices send requested tasks to the FC node.If it cannot process a certain task,it will forward it to the fog orchestration device which will check other fog devices in the colony.If the required resources are not available,the request will be forwarded to the semi-central fog orchestration device which checks the resource availability in all colonies.If they are not found,then the request will be forwarded to the central cloudfog control middleware that will provide resources from cloud servers.X.Pham et al.[27]suggested a scheduling technique depending on the heuristic to make a tradeoff between cost and response time.IoT things forward the requests to the broker which schedules them before forwarding them to the appropriate fog or cloud servers.D.Zeng et al.[29]introduced architecture consisting of computation server,storage server,and embedded client tier.The needed data to process tasks is stored in storage servers.The more delay sensitive tasks are executed in embedded clients while computation servers run other tasks.The authors applied a scheduling algorithm and task placement approach to minimize computation delay and I/O interrupt delay.S.Agarwal et al.[32]proposed a resource allocation algorithm to enhance throughput and response time in fog-cloud systems.In this approach,the client propagates the request to the nearest fog node which checks for resource availability.If recourse is not available,then it will send the request to other fog nodes in its cluster.If the required resources are not found in other servers,then it will forward the request to the closest cloud device.M.Jia et al.[52]proposed a load balancing algorithm to minimize the average response time in each cloudlet.This algorithm determines the average response time threshold for each cloudlet.When the cloudlet receives a set of requests,it determines which requests it must execute itself and which requests must be transferred to other cloudlets based on given average response time taking network delay into account.M.A.Al Faruque et al.[53]implemented Energy Management-as-a-Service in the FC environment to manage energy consumption of IoT things.L.Huang et al.[54]merged fog computing and SDN with Wireless Sensor Networks(WSN).Each fog node collects information about the state of sensors such as energy and state of local WSN.Then,it processes this information and saves temporary sensing data and node states.Finally,it sends some information to the SDN controller which uses this information to improve latency and lifetime of WSN.K.Liang et al.[55]presented an architecture for SDN and virtualized radio access networks with FC.The authors presented a software as a service called Open Pipe which enables network-level virtualization with fog computing.H.A.G.Alsalman et al.[56]proposed a framework that continually considers the fog nodes’location and makes the right decisions in distributing the workload to get better performance of applications that run on top of fog computing resources and reduce latency.

According to previously reported studies in the literature,parked vehicles can be used in computation,connection,etc.as fog computing nodes.X.Wang et al.[57]focused on management of real-time traffic in fog computing based on IoV networks to minimize response time of tasks launched from vehicles.The authors used the closest moving or parked vehicles as fog computing nodes.Z.Ning et al.[58]integrated fog computing with IoV networks to manage traffic in a distributed manner with minimum response time for events in a smart city.Moreover,they used the parked and moving vehicles as fog nodes to address the offloading problems.R.Mahmud et al.[59]presented a survey about fog computing and demonstrated that parked vehicles with computation capabilities at the network edge could be employed to serve as fog nodes to increase the network scalability.X.Hou et al.[60]merged FC with VANET and produced a new idea named vehicular fog computing.The computation and communication are done based on parked and moving vehicles,respectively.

However,the previous research works suffered from several limitations and challenges that affect total performance of these proposed systems which can be explained as follows:

• Not Efficient Utilization of Resources

The architectures proposed in [24,32]for fog computing does not efficiently use available resources.In some cases,when a fog server suffers from the load while other FC servers are under-loaded,tasks will be transferred to cloud computing without using the resources of the nearest fog servers.This policy reduces resource utilization and incurring additional transmission delay and bandwidth consumption.

Table1.Comparison of load balancing approaches.

• Bandwidth Consumption

Sending all jobs to the cloud servers requires a tremendous amount of bandwidth and in some cases,may affect the overall performance of the system due to the bandwidth bottleneck phenomenon[61–63].

• The Cloud Latency

The main problem of using the resources of cloud computing to handle the tasks of IoT devices is the long distance between these devices and the cloud servers that leads to high latency.This latency reduces the overall performance of the IoT environment[61–63].

• Hard Real Time Tasks

The sensors generate a large number of tasks that need to be executed.These tasks are divided into time sensitive tasks that need to be responded to in a specified time called hard real time tasks and delay tolerant tasks that can execute within their deadline with adding a penalty called soft real time tasks [64].Previously reported techniques[24,28]do not take the priority and deadline of the tasks into consideration.

• Overhead and Delay of The Traditional Network

In[25],the traditional access point uses as orchestration.However,this orchestration caused high overhead and delay in the task execution time.Therefore,the SDN controller can be the optimal solution.

• Response Time

One of the most important factors is the response time to the tasks.In [24,26,27,29,30],when a FC server fails in executing a particular task,it will be forwarded to the cloud computing directly without checking the neighbor fog servers.This causes that cloud latency appears again and it will increase response time.

III.THE RESEARCH GOALS

• Maximizing the Resource Utilization

The main goal is to maximize utilization of resources of the FC layer and exploit them to execute tasks of IoV sensors.It will optimize the overall performance of the system and reduce the need for cloud computing resources.

• Meeting the Deadline

Exceeding the deadline of time sensitive tasks leads to a catastrophe in some cases.Priority of executing each hard real time task that is determined based on its deadline is determined in this study (giving high priority for the task with the minimum deadline and vice versa).

• Minimizing the Required Bandwidth

Minimizing transfer of tasks to the cloud and executing them locally in the fog computing layer will significantly reduce the required bandwidth between fog and cloud computing layers.Moreover,it reduces likelihood of occurrence of bandwidth bottleneck phenomena.

• Reducing the Latency

It is not good to send the tasks to cloud computing through the internet with multiple hops since it imposes high latency.Thus,it is necessary to send tasks to the nearest fog computing servers with single hop to reduce latency and help minimize response time which results in meeting task’s deadline.

IV.SYSTEM MODEL AND PROBLEM FORMULATION

The system shown in Figure1 is used to increase resource utilization of FC nodes and the number of tasks that meet their deadlines and decrease bandwidth consumption between the fog and cloud systems.This system consists of IoV vehicles,FC,SDN controller,and cloud system layer.It differs from the system reported in[9,18,36,55]since it considers parked vehicles as fog computing nodes.In addition to using parked vehicles,it differs from the architectures of the system reported in [5,19,48,50,51]in using a hybrid control layer(local and global control).The IoV vehicle layer contains a large number of moving vehicles.Each IoV vehicle has Wi-Fi,LTE,sensors and GPS.The vehicles located in a zone connect to the nearest fog node (i.e.,BS or RSU) via Wi-Fi or LTE in one hop.By sending beacon packets the closest fog node fc to the IoV vehicle v can be selected according to (1).The closest one is a node that responds with a minimum delay to the beacon packets.The IoV vehicles periodically forward the direction,speed and current location to the nearest fog node periodically.The second layer contains groups(i.e.,clusters).Each cluster has a number of fog nodes that are installed within a geographical area.These clusters are constructed based on the distance between the fog nodes.Moreover,in this paper parked vehicles are assumed to be part of the fog computing layer and work as assisting fog nodes.Like [57,60],we assume that the parked vehicles,BS and RSU have saving and processing capabilities.Moreover,the fog devices support the OpenFlow protocol.In each cluster,there is one local fog manager that connects to all fog computing nodes found in that cluster.It uses an SDN agent to locally generate the flow table for the fog node that is in its coverage area.The fog nodes periodically transfer their state information which includes average service time and the number of waiting tasks in the queue as well as the vehicle information to the local fog manager.The local fog manager stores this information in its data base and uses parked vehicles as assistant fog computing nodes.Moreover,each local fog manager connects to the SDN controller (i.e.,third layer) and sends its information for it.As a result,the SDN controller has complete information about the network.Thus,it can globally select an optimal fog computing for any task.In the fourth layer there are cloud computing servers.

wherec,δ,fcandFrefer to index,set of fog clusters,the fog node that exists in clusterc,and the set of fog nodes respectively.

In this paper,we assume that

• The servers in the fog computing layer are heterogeneous.

• All vehicles have Wi-Fi and LTE interfaces.

• The information about the deadline and the required resource of each task determined inside that task.

• The fog devices support the OpenFlow protocol.

• The parked vehicles,base stations(BS),and road side units(RSU)have storage and processing capabilities.

• The local fog managers and SDN controller have complete information about a load of fog devices,location of the parked vehicles,speed and location of the moving vehicles.

• The vehicles use the Google map to determine their locations.

Figure1.The proposed system.

To distinguish between parked and moving vehicles,each vehicle broadcasts information about its mobility and speed periodically.A few fog devices receive this information and save it in their data base and every time period they check the speed of each vehicle.We assumed that the time period is 4 minutes to distinguish between parked vehicles and those stopped in a traffic jam or behind a traffic light.If the speed of a vehicle is 0m/s and it does not change at all,then it is considered to be a parked vehicle.As a result,there are new created links between this parked vehicle and fog devices.These fog devices must send information about this parked vehicle to the closest fog managers.They update their information about the network links and inform the SDN controller about the updated information.At this point,this parked vehicle can work as an assistant fog node to execute the tasks.To model the system,T is the set of time sensitive tasks that are sent from vehicles using Poisson distribution.They vary in data volume and contain some parameters{∆i,∂i}.Table2 includes the other notations used.

Maximizing utilization of FC resources can help diminish latency and reduce needed bandwidth between the FC and cloud system.Therefore,it will help execute tasks within the deadline.The number of jobs processed by all FC nodes in a specific time period represents resource utilization of FC servers(U).The fog resource manager uses the value of U to get complete details about the resources and their utilization.Moreover,this value helps the manager use most of the available fog resources to run tasks without passing the deadlines and diminish the number of tasks sent to the cloud system.It is calculated as explained below[65]:

Maximizing the utilization of FC resources and considering the deadlines of time sensitive tasks is the objective of this paper.It is as shown below:

Table2.Notations.

In the first constraint(3),∆iis more than or equal to the response time of taski.The second constraint(4)explains that the required resources to perform all the arrived tasks (M) must be less than or equal to the capacity of all fog computing nodes(N)together.The third constraint(5)shows that any task cannot be run by more than one fog node.Therefore,the number of fog nodes that execute each taskiis always equal to 1.The fourth constraint (6) means that the needed resources to run a number of tasks should be smaller than or equal to the capacity of fog node allocated to it.In the fifth constraint (7),the value ofβijis 1 if taskiis processed in the fog nodej;else,it is 0.The response time of each job should be larger than 0 as shown in the last constraint(8).

The optimizer CPLEX that is a C++based system is used to solve the above proposed mathematical model.A set of examples is solved by using a computer with core i5,2.5GHz CPU and 6 GB RAM to study the strength of this model in optimizing the utilization of resources.For instance,supposeMis 30 with ∆iare 5 5 6 7 5 10 7 8 5 9 11 14 7 9 12 10 7 6 15 10 9 5 8 7 9 8 7 6 11 10 seconds.The number of fog nodes(N)is 9 with capacity(Cj)70,75,80,85,90,95,100,105 and 110.The needed resources to run the tasks(∂i) are 108911 10 8 9 11 5 6 8 5 5 9 10 4 5 7 5 9 12 6 6 8 5 4 3 5 9 5 7 7 5 4.The time period (τ) is 45 seconds.The time period (τ) is 45 seconds.The values ofpijare considered as random numbers are as the following(Ta=task):

The values ofβijare as follows:

The optimal solution (maximum U) of the above example is 0.5133.The goal is maximizing utilization.Thus,the value of U shows that fog resources are exploited optimally to run all jobs by taking into account the deadline constraint.The CPLEX system required around 14ms to find the optimal server to execute each task with maximum utilization.In addition to the above example,the spent time by CPLEX was investigated in several examples as briefly shown in Table3.

V.SYSTEM MODEL AND PROBLEM FORMULATION

A new load balance mechanism is proposed in this research paper.It balances load of fog computing nodes proactively by the local fog managers locally and the SDN controller globally.It is known that proactive methods increase overhead.However,they help in producing low response time and meeting deadline which are very important parameters in the proposed system.Moreover,the controller in SDN based networks has a global view of network devices and completely fresh information about them such as the remaining resources and jobs[9,11,12].Therefore,in this load balancing technique,the SDN controller and fog managers divide the capacity of each fog node logically into five levels:level 1,2,3,4,and 5 and assign a load threshold for these levels as illustrated in Figure2.(Note:the node that has load less than or equal to the threshold of level 3 is called an under-loaded node.Otherwise,it is called an overloaded node).The thresholds of level 1,2,3,and 4 are 30%,60%,80%and 90% of CPU utilization,respectively.The other resources (10%) represent the fifth level which is assigned for ideal use.An approximately similar division of operation of device capacity was achieved in[9,24].

Table3.Additional examples.

Figure2.The levels of fog node capacity and their thresholds.

• In the beginning,the SDN controller instructs all local fog managers to send a set of task requests to cloud servers.Each fog manager sends a set of tasks and saves time in a variable that represents the sending time.The cloud servers execute the tasks and send the responses to each fog manager,which saves the time of response in a variable named response time.After that,each local fog manager calculates the average required time to execute a task in a cloud server using(10).WhereiandXrepresent the taskiand number of sending tasks to the cloud servers respectively.• Each fog node in each cluster sends information about the IoV vehicles that are found in its domain,the average execution time of the task(avg_f),and the number of tasks in its queue to the local fog manager.Each manager monitors the vehicles and FC resources that are found in its domain and saves the information of fog servers and vehicles in tables namedfog_tableandveh_table,respectively.Moreover,it balances load of fog nodes.If a fog node (say node 1) reaches or exceeds the threshold of level 2,3 or 4 while load of another fog node(say node 2)in the same cluster is less than the threshold of level 1,2 or 3,respectively,then the fog manager will proactively notify node 1 using notifciation callednoti_1to send the next incoming tasks to node 2.Moreover,it sends the flow tables to node 1.After some time,if loads of all fog nodes in the cluster are approximately balanced,the fog manager will notify the fog nodes to work normally by sending the next incoming tasks to their queues.• Each local fog manager sends information about load of the fog nodes in its cluster to the SDN controller,which frequently monitors and checks clusters’ load.If it fnids a cluster (say cluster 1) near the overloaded level and another cluster(say cluster 2) at under-loaded level,then it will send information proactively by using a notifciation callednoti_2to the fog manager of cluster 1 about the under-loaded fog node(s)in cluster 2 and updated flow tables.The fog manager of cluster 1 will save this information in a cluster Table(C-T).If cluster 1 reaches its overload level,then the next incoming tasks in this cluster will be sent to cluster 2.After some time,if cluster 2 reaches its overload level,the SDN controller will send a new notifciation to the fog manager of cluster 1 to prevent it from sending other tasks.

• If there is more than one cluster that has underloaded fog nodes,then the local fog manager uses the Collaborative Vehicle Location Management Service (CVLMS) algorithm [66]and uses thevehtableto determine the new position of the car that launched the task and determines the new zone.ThenC_Tandvehtableare used to determine the optimal cluster depending on the priority.The local fog manager calculates the priority of the clusters depending on the vehicle’s mobility direction and its position.Thus,if the FC servers of the cluster domain of the new zone(i.e.,the expected area that the vehicle will go to it) are not overloaded,then it will send the task for the local fog manager of that cluster.Otherwise,it selects the nearest cluster to the new zone and sends the task to the fog manager of that cluster.

• Vehicle sensors forward the tasks to the closest fog node (see Figure3).The tasks,in addition to its data,contain the deadline.If this node is under-loaded and no notifciation(i.e.,noti1)has reached from the local fog manager,then it will let the task load into its queue.After that,it schedules the tasks based on the deadline.Else if the notifciationnoti_1has reached from the local fog manager,then the fog node will forward the task(s) to another fog node which is determined innoti1.Otherwise,it will forward the task(s)to the local fog manager.

• When the local fog manager receives a task,frist it schedules it based on deadline.Then,it checks whether there is an under-loaded parked vehicle in its cluster or not.If so,it will forward the incoming tasks to this vehicle.If there is more than one under-loaded parked vehicle,then the local fog manager selects an optimal one based on the load.The optimal one represents the parked vehicle with minimum load.The load can be computed using (12).The local fog manager uses parked vehicles as assistant fog nodes because these vehicles have limited capabilities and may move to another location at any moment which leads to new challenges.Thus,the local fog manager sends a limited number of tasks for each parked vehicle.

where

Wherewj,L(j)andavgjrefer to the number of waiting tasks,the load,and average service time of fog nodejrespectively.

Otherwise,if there are no under-loaded parked vehicles in its domain,it will check C-T.If there is another cluster that has fog node(s) in under-load level,then it will send the task to the local fog manager of that cluster using the flow tables which will then forward the incoming task(s) to the under-loaded fog node or parked vehicle.Otherwise,it must decide is sending this task to cloud servers is better than waiting for some time for them to be executed by the fog nodes in the same cluster.The decision is taken using Table4 based onavg_cand potential execution time (E-T) of the task in the fastest fog node (the fastest node is a node with minimum load or high capabilities).E-T is computed using(13):

Table4.Decision of local manager.

VI.SIMULATION AND EVALUATION

Figure3.Flow chart of executing a task.

To explain the efficiency of the proposed strategy,we need to simulate the environment and use some performance metrics to compare the proposed strategy with recently reported research works in various simulation scenarios.The next subsections explain the simulation framework,performance metrics,and simulation scenarios as follows:

• Simulation Framework:

In order to prove the efficiency of the proposed strategy,we will use the SUMO version 0.19.0 to model the network and the network simulator OMNeT++ version 4.6 to build the proposed system and program the proposed strategy.To add some realism to the simulation environment,the Baghdad map is imported from the www.openstreetmap.org and is added to the SUMO simulator.The general parameters of the simulation are in Table5[9].

Table5.The general simulation parameters.

• Evaluation

To study the behavior of the proposed approach,IoV-Fog-Cloud architecture[9]and VANET with Fog-Cloud architecture [26]are applied.Then we compared these architectures with the proposed system considering average response time and percentage meeting deadline,resource utilization,and bandwidth consumption.These performance metrics were used in[9]and can be explained as follows:

The Percentage of Resource Utilization:

It is the total number of jobs that are executed by the FC servers divided by the total number of launched tasks from the IoV sensors in a certain time period.It can be computed as follows:

whereQis the number of executed tasks by the fog computing nodes andMis the total number of launched tasks from the IoV sensors.

The Average Response Time(ART)

Response time represents the spent time between sending task requestifrom the IoV sensor and the response to that task and can be computed as follows:

Let M be the total number of tasks.The average response time(ART)represents the response time of all tasks divided by the number of those tasks as follows:

The Percentage of Meeting the Deadline

It is called the successfully transmitted rate and refers to the total number of handled jobs within their deadlines divided by the total number of jobs that are forward from the sensors of IoV vehicles.The percentage meeting deadline can be computed as follows:

whereKis the number of tasks that did not exceed the deadlines andMis the total number of launched tasks from the IoV sensors.

The Percentage of Bandwidth Usage:

The bandwidth usage represents the consumed bandwidth in a period of time.The percentage bandwidth usage is computed as follows:

• Simulation scenarios

In vehicular networks,there are many parameters that can affect total performance such as the number of vehicles and vehicle speed.However,in the proposed work,we have focused on showing the role of using the parked vehicles in enhancing the performance.Thus,in this section,we will study the performance of the proposed strategy and other previous strategies with increasing the number of parked vehicles and tasks.The simulation scenarios are as follows:

Scenario 1:The Number of Parked Vehicles

In this section,the effect of increasing the number of parked vehicles on the performance of the proposed technique and previous approaches in terms of percentage of resource utilization,ART,percentage meeting deadline,and percentage bandwidth consumption is investigated.Table6 shows the parameters of the scenarios.

Table6.Parameters of scenario 1.

Figure4,5,6,and 7 show the performance of the proposed system and other previous systems with various numbers of parked vehicles.

Figure4 illustrates the utilization ratio of fog computing resources when the proposed system is used plus IoV-Fog-Cloud,and VANET-Fog-Cloud in 90 seconds.Increasing the number of parked vehicles improves the ratio of resource utilization in the proposed system since increasing them leads to an increase in the number of tasks that are executed in the fog computing layer as seen in the Figure.Moreover,the proposed system is the best one since it exploits these parked vehicles in processing tasks while other systems do not do so.Also,it chooses an optimal FC server for each task while in the VANET-Fog-Cloud system and if a FC server cannot process a task,this task will be forwarded to the cloud directly without checking the rest of the fog servers.Thus,FC resource utilization will reduce according to(2).Figure5 explains that ART of the proposed system reduces when the number of parked vehicles increases since most tasks will execute in the fog layer and the same cluster.The suggested system minimizes the number of handled tasks by the cloud devices which reduces the transmission time.Thus,it is better than the other systems in this respect.The VANET-Fog-Cloud and VANET-Fog-Cloud systems send a number of jobs to the cloud layer.Therefore,spending time to send them is high which leads to high response time.

Figure4.Percentage of resource utilization.

Figure5.Average response time.

Figure6 illustrates that the ratio of tasks that do not pass the deadlines in VANET-Fog-Cloud architecture is acceptable because it runs several tasks in the fog layer.However,this percentage is high when using the proposed system as compared to other systems because it approximately handles all tasks in the FC nodes and uses parked vehicles as assistant fog nodes in executing these tasks.Moreover,it works proactively and checks the deadline of tasks before sending them to the cloud.In addition,this Figureshows that the ratio of tasks that do not pass deadlines increases in the proposed system by increasing the number of parked vehicles because it exploits these vehicles in executing tasks in the same cluster they are located.

The transfer of tasks to cloud system requires high bandwidth.Thus,it is better to execute tasks at the FC layer.The ratio of using the link bandwidth in VANET-Fog-Cloud and IoV-Fog-Cloud systems is higher than that of the proposed system as explained in Figure7.This is due to the fact that the number of tasks that are forwarded to the cloud devices is high.In the proposed system,increasing the number of parked vehicles reduces bandwidth consumption since it reduces the need for the cloud system and most tasks will be executed in the fog layer.Therefore,the bandwidth consumption ratio is lower than in other systems.As a result,the proposed system is suitable for networks with limited bandwidth.

Figure6.Ratio of executed tasks within the deadline.

It is shown in this scenario that the number of parked vehicles does not affect the results of VANET-Fog-Cloud and IoV-Fog-Cloud systems because they do not take these vehicles into account.

Figure7.Bandwidth consumption ratio.

Scenario 2:Number of Tasks

The behavior of the suggested system,VANETFog-Cloud,and IoV-Fog-Cloud systems with various numbers of tasks is studied in this scenario.Table7 explains the special parameters of this scenario.

Table7.Parameters of scenario 2.

Figure8,9,10,and 11 explain performance metrics of VANET-Fog-Cloud,IoV-Fog-Cloud and the proposed system with different number of tasks.Figure8 shows that the percentage utilization of FC resources reduces with increasing the number of tasks because some jobs are transferred to the cloud and are not processed in the fog layer.However,the proposed system presented amazing results in this metric since it exploits parked vehicles in executing the tasks when the fixed fog nodes become fully busy.Figure9 illustrates that the average response time reduces with increasing number of tasks.However,the proposed system uses parked vehicles when there is a lack of the fog resources.Therefore,it reduces the sending operations of tasks to the cloud system to the lowest level and executes all tasks approximately at the fog layer.Moreover,it balances the load proactively.As a result,it responds in a short period of time.

Figure8.Percentage of resource utilization.

Figure9.Average response time.

Figure10 shows that the percentage of tasks that meet the deadline reduces by increasing the number of tasks.However,the proposed system produces high percentage for this metric since it exploits the benefits of parked vehicles.Moreover,it focuses on the deadline of tasks before sending them to the cloud layer.Figure11 explains that the bandwidth consumption of the link from the fog to cloud layer increases due to increasing the number of transferred tasks to the cloud layer.However,this consumption in the proposed system is lower than in other systems.

Figure10.Ratio of executed tasks within the deadline.

Figure11.Bandwidth consumption ratio.

VII.CONCLUSIONS AND FUTURE DIRECTIONS

In this research fog computing and SDN with IoV networks were used.In addition,an approach was presented to use parked vehicles as assistants for fog computing servers to solve the problem of lack of resources.The parked vehicles are exploited as fog computing nodes to execute the launched tasks from the IoV vehicles.Moreover,a new load balancing mechanism is proposed which works locally in each cluster of the fog nodes and globally in the SDN controller to allocate tasks to suitable fog nodes.This mechanism differs from other previously proposed strategies in computing the waiting time of the task in the fog computing layer and compares it with the sending time of that task to the cloud computing layer to make a suitable decision to meet the deadline.This will reduce the number of jobs sent to the cloud system.Thus,it reduces the bandwidth consumption of the link between the fog and cloud systems and response time.It also meets the deadline.From the simulation scenarios with various effective parameters,we conclude that using parked vehicles as assistants fog nodes can help increase the effciiency of fog computing in executing the launched tasks from IoV vehicles.In addition,they can make the difference with other systems in producing amazing results in different performance metrics like percentage of bandwidth consumption,meeting the deadline,resource utilization and average response time.Unfortunately,bandwidth consumption and overhead at the fog computing layer are high due to transmission of information and tasks to fog nodes in the same cluster or other clusters in the cases of overloading.Reducing the bandwidth cost at the fog computing layer is the future direction of this research study Moreover,in the future,we plan to merge the concept of hierarchical SDN controlling with fog computing and IoV in an effciient architecture.Moreover,we plan to produce an assignment technique to assign each task to the appropriate fog server based on mobility direction and speed of IoV vehicle which sent the task.