Network perception task migration in cloud-edge fusion computing

With the development of cloud computing, edge computing has been proposed to provide real-time and low-delay services to users. Current research usually integrates cloud computing and edge computing as cloud-edge fusion computing for more personalized services. However, both cloud computing and edge computing suffer from high network consumption, which remains a key problem yet to be solved in cloud-edge fusion computing environments. The cost of network consumption can be divided into two parts: migration costs and communication costs. To solve the high network consumption problem, some virtual machines can be migrated from overloaded physical machines to others with the help of virtualization technology. Current network perception migration strategies focus more on the communication cost by optimizing the communication topology. Considering both communication and migration costs, this paper addresses the high network consumption problem in terms of the communication correlations of virtual machines and the network traffic of the migration process. It proposes three heuristic virtual machine migration algorithms, LM, mCaM and mCaM2, to balance communication costs and migration costs. The performance of these algorithms is compared with those of existing virtual machine migration algorithms through experiments. The experimental results show that our virtual machine migration algorithms clearly optimize the communication cost and migration cost. These three algorithms have a lower network cost than AppAware, an existing algorithm, by 20% on average. This means that these three algorithms improve the network performance and reduce the network consumption in cloud-edge fusion computing environments. They also outperform existing algorithms in terms of operation time by 70% on average.


Introduction
As a new type of pay-as-you-go computational model, cloud computing is widely studied and applied in academia and industry. Edge computing is an extension of cloud computing that provides lower delay and higher real-time services from the devices closer to users. Current research usually integrates cloud computing and edge computing as cloud-edge fusion computing to provide more personalized services than either of the two individual computational models. Cloud computing of the total cost [6]. Given that edge computing devices have much less computing and network resources than cloud computing, edge computing also suffers from a high network consumption problem. Therefore, in cloud-edge fusion computing environment, high network consumption is a key problem yet to be solved.
As the underlying technology of cloud computing, virtualization integrates users' applications as a virtual machine and allows users to manage and use this virtual machine conveniently [2]. It also makes the optimization of network consumption possible. Virtual machines can be migrated to a small number of physical machines and unused physical machines can be powered off to achieve a higher resource utilization ratio. This kind of migration strategy only focuses on the computing resources of the physical machines, such as the CPU, memory and disk storage. However, virtual machine migration is a resource-intensive procedure [7] and continuously demands not only computing, but also network resources. Most existing studies focus on the computing resources and limited studies have considered the influence of network resources on virtual machine migration [8]. Therefore, designing a virtual machine migration algorithm that can improve network performance and reduce network consumption would be significant in improving cloud-edge fusion computing performance.
V. Shrivastava et al. [3] proposed Application Aware (abbreviated as AppAware), as a network perception virtual machine migration strategy. It considers not only the computing resources, but also the topological structure of the underlying physical machines. AppAware migrates all virtual machines on a physical machine with a workload that exceeds its resource limits to other physical machines that have a better topological network communication structure. The migration of virtual machines faces several challenges in network performance. First, the migrated virtual machine needs a appropriate destination physical machine which has better communication correlations. Second, the virtual machine migration network traffic also reduces network performance, which need to be minimized. While AppAware optimizes the network performance, the migration also creates network traffic which in turn reduces network performance. Therefore, a virtual machine migration strategy should consider migration traffic and ensure the network performance of cloud-edge fusion computing in both virtual machine transmissions and normal working processes.
In cloud-edge fusion environment, edge computing devices have limited computing resources and network resources. Therefore, we consider that the virtual machine migrations create large volume of network traffic and limit the network performance of cloud-edge fusion computing. Considering that the less researches focus on the optimization of virtual machine migration network traffic, we propose a method which can balance the communication consumption and network traffic. Our contribution is summarized as follows: • We take both communication costs and migration costs into the virtual machine migration problem to have a better optimization on the cloud-edge fusion computing environment. Then we give the mathematical definition of these two costs and a linear programming definition of the virtual machine migration problem. • We propose three virtual machine migration algorithms, LM, mCaM and mCaM2, to solve the virtual machine migration problem. In the design of these algorithms, we use the weighted coefficients to balance the optimization of the communication cost and the migration cost. • By simulation experiments, we demonstrate that our algorithms have a better optimization on datacenters.
To compare the performance of these three algorithms, some experiments are conducted with various settings. In addition, we also compare the performance of the three algorithms with that of AppAware. Experimental results show that our dynamic algorithm reduces network and migration costs by 20% on average comparing with AppAware. And the proposed algorithms also outperform existing algorithms in terms of operation time by 70% on average.
For cloud-edge fusion computing, these algorithms significantly improve the network performance and reduce the power consumption of the edge network devices. It contributes to the development and further utilization of cloud-edge fusion computing. Moreover, the communication correlations between cloud center and edge devices is optimized, resulting in less network traffic and lower communication delay.
The paper is organized as follows: "Related work" describes related work on virtual machine migration cost and network perception migration methods. "Definition of the network-aware virtual machine migration problem" describes the model of the network perception virtual machine migration problem and improvements to this problem model. In "Optimization of the virtual machine dynamic migration algorithm", we introduce the design of the LM, mCaM and mCaM2. In "Experiment", we evaluate the performance of these three algorithms in terms of total cost, communication cost and migration cost. The performance results are compared with the existing algorithm AppAware. Finally, in "Conclusion" sections we summarize our algorithms and draw up some conclusions.

Related work
In this section, we summarize research on virtual machine migration problem. This is divided into two parts: the migration cost problem and the network perception migration problem. The research on the migration cost problem clarifies what can influence the performance degradation brought about by migration. The research on network perception migration focuses on power consumption, datacenter network congestion and communication correlations between application layer virtual machines.

Research on the cost of virtual machine migration
A. Verma et al. [9] studied the cost of virtual machine migration. Their experiment revealed that migration is irrelevant to workload and it is only decided by the virtual machine itself. Specifically, the migration cost can be measured by the throughput and performance reduction brought about by migration. L. Lefèvre [10] mentioned that online migration from the source physical machine to the destination physical machine involves a physical memory page exchange. If the virtual machine is running, it will be paused during the duplication from the first to the last memory page. H. Liu et al. [11] considered the online migration cost in their examination of datacenter performance and power consumption management. Through an experiment on XEN, they revealed that the power consumption of the migration of a virtual machine has a positive correlation with network communication traffic.

Research on network perception virtual machine migration
Network perception virtual machine migration is the key point of this study. Research on this topic has shown limited progress. Most current studies on virtual machine migration and physical machine integration have focused on the conservation of system resources, such as CPU, memory, and disk storage. However, the network costs of current datacenters accounts for an important proportion of the total cost and is reflected in the following aspects. First, Wen et al. [5] found that the network cost of a datacenter accounts for 10%-20% of the total cost. Second, a datacenter has a large network communication system, which requires high network bandwidth usage. Thus, network congestion and other issues occur at several switches and switch ports [3]. If no measures are in place to handle these problems, the performance of the datacenters will reduce sharply, and the maintenance costs will increase rapidly. However, research on network perception virtual machine migration is still in the initial stages and has not received sufficient attention. Here, we review the research on network perception virtual machines from different angles.

Research on the angle of network power consumption
F. Feleke et al. [12] proposed a virtual machine placement algorithm based on both bin-packing heuristics and physical machines' power efficiently Medium-Fit to reduce SLA violation. They found that the resource utilization level threshold of overloaded physical machines influences the power efficiency. The algorithm was proposed to meet different demands of power efficiency. B. Heller et al. [13] found that network power consumption accounts for 10%-20% of the total datacenter power consumption. They indicated that the ElasticTree model can serve as a topological graph of datacenter physical machines. To minimize network consumption, they introduced an algorithm for calculating the optimal network subset. The algorithm abstracts the optimization problem as a subset and flow assignment problem. They also compared the differences among a formal algorithm, a greedy algorithm, and a topological perception algorithm in terms of solution quality and extensibility. M. Zhani et al. [14] approached the allocation problem from virtual machines to the virtual machine datacenter and regarded several resources as the fitness value of optimization. This algorithm aims to maximize benefits of virtual machine migration and minimize the energy consumption of datacenters. Sukhpal Singh et al. [15] proposed a heterogeneous self-optimization system for cloud energy efficiency. To optimize the energy consumption of cloud computing, an algorithm was proposed to estimate the energy consumption of each task and assign each task to the virtual machine with the lowest energy consumption. Sambit Kumar Mishra et al. [16] proposed a virtual machine selection algorithm to minimize the energy consumption based on dynamic voltage frequency scaling technology. The proposed algorithm estimates the energy consumption and execution time of each virtual machine and assigns each task to the virtual machine with minimal energy consumption and execution time.
Next, for further optimization of the energy consumption, the algorithm finds the voltage frequency pairs that resulting in the minimal energy consumption for each virtual machine. Mohit Kumar et al. [17] proposed an architecture that acts as a cloud resource broker for dynamic task scheduling. To improve the optimization effect, an improved Particle Swarm Optimization (PSO) method was proposed to optimize not only the energy consumption, but also the QoS parameters. With the constraint of deadline, this method distinctly reduces the energy consumption based on the results of a simulation experiment.

Research on avoidance of datacenter network congestion
M. Bhatu et al. [18] proposed a task scheduling and resource allocation system to manage virtual machines and resources. The system uses an analytic hierarchy system to assign a rank to each task based on its length and runtime. Then they implemented a bandwidth-aware divisible scheduling method to allocate the computing resources and bandwidth, and a preemption methodology was proposed to check for overloaded virtual machines. Finally, the system uses divide-and-conquer methodology to break up tasks to solve the overloading problem. X. Wen et al. [5] investigated virtual machine migrations to alleviate the network congestion problem. They abstracted the problem into a secondary allocation problem and proposed Virtual Knotter, an online algorithm, to alleviate network congestion. The algorithm maintains low time complexity in the process of reducing the network communication cost. An experiment showed that the network cost of Virtual Knotter was only 5%-10% of that of the baseline algorithm. D. Kliazovich et al. [19] considered the reduction of network power consumption and the maintenance of network performance. They proposed the DENS algorithm to balance power consumption, the performance of each task, and the communication correlations to minimize the number of physical machines used. The algorithm maintains perfect communication correlations to avoid network congestion in the datacenter. In addition, Mohit Kumar et al. [20,21] proposed an architecture to serve as the cloud resource broker for dynamic task scheduling and load balancing with elasticity. This architecture manages task assignment and resource provision for each virtual machine. Based on this architecture, [20] proposed a dynamic load balancing method to minimize the makespan time, which increases the QoS of cloud computing. [21] proposed an improved PSO method to optimize multiple QoS parameters. To solve the trade-off problem for task scheduling, the improved PSO method generates several schedule plans based on Pareto efficiency theory.

Research on communication correlations between application layer virtual machines
X. Meng et al. [22] investigated the problem of virtual machine migration to a physical machine on the basis of network perception. Their experiments on datacenters indicated that the distribution of virtual machines' traffic in a datacenter is extremely uneven. Virtual machines that communicate with each other have high traffic, and virtual machines with minimal communication have low traffic. They abstracted the network perception allocation of virtual machines into an NP-hard optimization problem and proposed a two-layer approximate algorithm to solve this problem. Their experiment showed that their algorithm can improve network performance and reduce the network communication costs. D. Jayasinghe et al. [23] focused on the virtual machine migration problem with the server's resource constraints. They proposed a virtual machine migration algorithm to improve performance and availability of IaaS cloud services. This algorithm aims to minimize the network traffic of datacenters by satisfying the resource demands and availability constraints of physical machines. The authors also analysed the NP-completeness of this problem and divided the problem into three NP-completeness optimization problems. First,They abstracted the minization of the communication cost into a clustering problem, which is essentially a K-Min-Max-cut problem. Second, they abstracted the allocation of virtual machine clusters to physical machine sets into a cluster allocation problem. Third, they abstracted the allocation of each machine to each physical machine into an allocation problem. They also proposed four heuristic algorithms to solve these problems. However, they didn't consider the limitations in the physical machine's system resources (CPU, memory, disk storage, etc.). The resources that the server provides must be sufficient for the virtual machine.
To solve this problem, V. Shrivastava et al. [3] considered the limitations in network communication and server resources and analysed a situation in which virtual machines are overloaded. They abstracted a datacenter into a model that considers the dependencies of the virtual machines and the topological relationships of the underlying physical machine. They proposed a greedy algorithm AppAware for selecting a server with the minimum communication cost as the destination physical machine for each virtual machine migration.
With regard to the research on overloaded virtual machines, T. Wood et al. [24] proposed Sandpiper to solve the problem of overloaded virtual machine migration. Sandpiper includes two algorithms. The first is used to estimate the virtual machine migration time, and the second is used to determine the migration method and the destination physical machine. Sandpiper uses greedy and heuristic methods to migrate the virtual machine from the physical machine with the highest overload ratio to that with the lowest one. Sandpiper is a dynamic virtual machine migration algorithm that selects one or several virtual machines from the virtual machine set on the basis of load variety. However, Sandpiper ignores the cost of virtual machine migration. H. Liu et al. [11] mentioned that the data transferred by virtual machine migration are positively related to the energy consumption. Thus, the data transferred by virtual machine migration cannot be ignored.

Definition of the network-aware virtual machine migration problem
In 2011, V. Shrivastava et al. [3] proposed Application Aware, a virtual machine migration strategy. AppAware gives the basic definition of virtual machine migration problem as follows.
For the virtual machine migration problem, we should consider three factors:   Next is the definition of virtual machine resource demand and physical machine source. Load (V i ) is the resource demand of virtual machine V i , and Capacity(S i ) represents the CPU, memory, and disk storage capacity of physical machine S j . The migration cost of vir- represents the delay or hops between S k and S l .
To describe the relationship between virtual machines and physical machines, X ik indicates whether virtual machine V i is distributed to physical machine S k . If so, then X ik is equal to 1; otherwise, X ik is 0. Similarly, X jl ik = X ik * X jl indicates whether virtual machines V i and V j are distributed to physical machines S k and S l .
If the distribution exists, X jl ik = 1; otherwise, it is equal to 0.
On the basis of these definitions, the total communication cost of a datacenter is expressed as Meanwhile, the constraints of the problem are expressed as follows: Condition (2) indicates that every virtual machine must be allocated to a specific physical machine. Condition (3) indicates that the summation of the resource demands of the virtual machines in each physical machine must be lower than the amount of resources the physical machine provides.
In addition, AppAware presumes that the communication correlations between virtual machines is fixed. Thus, AppAware's optimization objective function is To simplify this problem, we presume that the allocation of underloaded virtual machines to physical machines is fixed. Thus, the allocation of underloaded virtual machine V i to physical machine S k is represented by x ik , which is defined as For convenience of description, we call the virtual machine migration problem VMIG. The virtual machine is called VM, and the physical machine is called PM.

Improvement of the problem model Communication cost model
In our work, the communication cost still follows the definition in (1). Here, we use Cost_Com to represent the total communication cost in the datacenter.
As the calculations of the communication cost and migration cost are totally different, the dimensions of these two costs are also different, which unbalances the influences of these two costs to the total cost. To eliminate this unbalance, we need to make sure the values of these two costs in objective function are in the same order of magnitude. Tiwari et al. [25], Zhao et al. [26], to increase the models' extensibility and adaptability and consider the migration cost in the objective function, we normal-ize the communication cost to a value between 0-1. The normalized communication cost is where e Topo_vm represents the number of VM pairs that are communicating with each other in the datacenter. It also represents the total number of edges in the network topological dependence graph. Given that each pair of VMs that communicate with each other corresponds to a Cost V i , S k , V j S l , we use the communication cost divided by the number of VM pairs that communicate with each other, the maximum VM communication demand max i,j∈V W ij and the maximum distance

Migration cost model
We define the migration cost as the product of the amount of VM migration data and the distance between the source and destination PMs. The VM is essentially a software container. It packages a complete set of virtual hardwares and softwares that includes the operating system and all applications. During VM migration, the migration operation is similar to the transfer and duplication of other software files. Therefore, the VM migration cost is related to the VM size, which is called Size. The function of the migration cost of VM V i from S l is where S l is the physical machine where the overloaded VM V i initially located. For an overloaded VM that needs to be reallocated, if X ik = 1, then VM V i will migrate to S k . The function of the total migration cost of a datacenter is As the communication cost need to be normalized, the migration cost also need to be normalized to the same order of magnitude with the communication cost. To increase the models' extensibility and adaptability, we also normalize the migration cost. The normalized value will be used in the improvement of the objective function. The normalized migration cost is

Improvement of the objective function
Next is the definition of the new objective function for this problem. The total cost, Cost_Total, is the weighted sum of the communication cost and the migration cost. This both enables our algorithm to estimate the network cost close to reality and ensures the extensibility and adaptability of the algorithm. We use the weight coefficients, α and β (which must sum to 1) to adjust the effect of these two factors in the optimization objective function.
These two coefficients can balance the optimization of the communication cost and the migration cost effectively. The administrators of datacenters can adjust these two coefficients based on the network condition to have different optimization of the communication cost and the migration cost. In the following experiment, we set α = 0.7 and β = 0.3. With these coefficient values, the performance of our proposed algorithms in simulated datacenters with different number of virtual machines is best in our experiments. We also finished the experiment in "Experiment" section to find the most appropriate coefficient values with the best optimization of the total cost.

Procedure of the dynamic migration algorithm
Initially, we provide the definition of an overloaded PM. For a PM S src , if the sum of the VM resource demands (Load) is larger than the sum of available resources (Capacity), S src will be referred to an overloaded PM. And the VMs on it are referred to overloaded VMs. Several of the VMs on overloaded PMs need to be migrated based on a migration algorithm, thus making the PMs underloaded. Following is the general procedure of the dynamic migration algorithm.

Direct optimization
Direct optimization refers to the optimization of the total cost, which is expressed as (11). We use a greedy strategy to migrate the overloaded VM with the minimum total if S dest = S src then 5: Add V mig , S dest to AllocationList 6: X mig src ← 0, X mig dest ← 1 7: end if 8: end while 9: return AllocationList cost Cost_Total, which guarantees that the total cost of the datacenter is also minimal.
Based on the direct optimization method, we propose the local search for min-cost algorithm (LM) to optimize the datacenter performance. For each overloaded VM V i , LM calculates the total cost to all the PMs and selects the destination PM with the lowest total cost, Cost (V i , S k ). By minimizing each VM migration total cost, the total cost of the datacenter can be made as low as possible.
Indicated below are the functions of communication cost and migration cost of VM V i to PM S k , which is where |TW (V i )| represents the number of VMs that communicate with V i , such as V j on the PM S l . max According to Functions (12) and (13), Cost_Total(V i , S k ) is the weighted summation of these two costs. The detailed function is expressed as follows On the basis of these functions, the LM algorithm is designed as shown below.  · n), where m is the number of VMs on PM S m and n is the number of PMs. Compared with AppAware, the advantages of the LM algorithm mainly lie in two aspects. The first aspect is the migration object. AppAware migrates all the overloaded VMs at once. By contrast, LM only migrates a small number of the VMs and makes the overloaded PM's capacity just larger than the sum of the load. The second aspect is the optimization effect of the network cost. AppAware only consider the VM communication cost, but the LM considers the optimization of both the communication cost and the migration cost.

Difference value optimization
LM optimizes the total network cost with the communication cost and migration cost, that is Before VM migration, the set of VMs has an unreasonable original allocation plan within the set of PMs. According to Function (4), the allocation plan corresponds to a communication cost C 0 . Based on the VM migration algorithm, overloaded VMs are migrated and a new VM allocation plan is generated with a communication cost called C. In theory, the purpose of VM migration is to minimize communication costs and migration costs. Specifically, the algorithm should maximize the difference in the communication cost, C = C 0 − C, and minimize the migration cost, Cost_Mig.
After importing C, the optimization object becomes min(α · C 0 − α · C + β · Cost_Mig). Here, we define the new optimization objective function as We define the new optimization objective function as the network benefit, Benefit.
In the new optimization objective function, the network benefit, Benefit (V i , S k ), is expected to be the maximum for each VM migration. This is essentially different from LM, whose object is the minimization of the communication cost and migration cost for each migration.

Algorithm mCaM
mCaM imports the prediction mechanism in LM. Initially, mCaM finds the overloaded PM S m based on the general procedure. Then, mCaM finds the overloaded VM migration with the maximum network benefit. In detail, mCaM calculates the minimum Cost_Total (V i , S k ) for migrating each VM V i to PM S k . Then, mCaM calculates the network benefit Benefit (V i , S k ) for all the overloaded VMs and migrates VM V i with the maximum network benefit to the destination PM S k .
Below is the detailed pseudocode of mCaM. mCaM can ensure that the migrated VM V i has the highest possibility of reducing the communication cost. The time complexity of this algorithm is o(m · n), where m is the number of VMs on PM S m and n is the number of PMs.

Algorithm mCaM2
Now we analyse the network benefit of each VM migration. mCaM migrates the VM with the maximum network benefit to the PM with the minimum total cost, until all PMs become underloaded. However, mCaM may never migrate those VMs with both high network benefit and high migration cost. For the datacenters with high communication costs, mCaM can only optimize the VMs that have low migration cost, which has a limited optimization on the communication performance. Here, we propose mCaM2 to solve this problem.
First, we describe the optimization objective of mCaM2. We define the reduction of communication cost as communication benefit, Benefit_Com = C. To describe the  16: if Benefit i > Benefit max then 17: Benefit max ← Benefit i 18:

19:
S dest ← Des i 20: end if 21: end for 22: return V mig , S dest benefit of the migration, we define the optimization object as Benefit_Ratio, which is Second, we describe the migration strategy of mCaM2. mCaM2 divides the optimization into two periods. In the first period, mCaM2 optimizes the communication cost. It calculates the communication benefit Benefit_Com for all overloaded VMs to all PMs. For each VM, mCaM2 selects the PM with maximum communication benefit as the destination PM. In the second period, mCaM2 optimizes the migration cost. mCaM2 calculates the benefit ratio Benefit_Ratio for each overloaded VM and migrates the VM with the maximum benefit ratio, until all PM become underloaded.
The time complexity of mCaM2 is o(m · n), where m is the number of VMs on PM S m and n is the number of PMs. The operation process of mCaM2 is shown below.

Physical topology structure
In the present datacenters, the network topology usually extends from triple-layer network structure [22]. In the bottom layer, every PM connects to one (or two) access switch. In the aggregation layer, the access switch connects to one or two switches, and every aggregation layer switch connects to multiple core layer switches. We call this structure a tree structure. The aggregation layer is usually a tree structure, and the root node is located in the core layer. If this triple-layer network structure is a multitree structure, the data package will be transferred based on the aggregation layer VLAN and the maximum weight spanning tree.

Algorithm experiment Algorithm parameter configuration
With different numbers of VMs and workloads, we finish the VM migration algorithm simulation. We compared the performance of AppAware, LM, mCaM and mCaM2 in the simulated environment.

Weight coefficient in the dynamic migration objective function
For these migration algorithms, the weight coefficients α and β are vital to the optimization effect. The weight coefficient can effectively balance the optimization of communication cost and migration cost. Therefore, we need to find the appropriate weight coefficient values. Given that the value of the objective function reflects the effect of migration, the difference between maximum and minimum needs to be as small as possible. Here, we use different values of α(from 0.0 to 1.0 in steps of 0.1) to calculate the total cost for the VMs (the number of VMs increased from 60 to 200 in steps of 20). To obtain a better weight coefficient value conveniently, we calculate the differences between the maximum and minimum total costs in the weight coefficient experiments, and the result is shown in Fig. 1, which describes the performance of each weight coefficient. As shown in Fig. 1, when α = 0.7, the total cost of the VIMG experiment has a minimum different value of 0.014733, in which the maximum value is 0.390296 from 100 VM application requests, and the minimum value is 0.379792 from 80 VM application requests. According to the results of the simulation experiment, the weight coefficients in the VMIG model are set to α = 0.7, and β = 0.3 in the following experiment.

Optimization algorithm performance comparison
Based on appropriate weight coefficients, we compared the total cost of each algorithm, including LM, mCaM and mCaM2. The detailed data are shown in Table 3 and Fig. 2. As shown in Table 3 and Fig. 2, mCaM has the minimum total cost among the three algorithms. This indicates that mCaM has better optimization for datacenter and has lowest migration cost in the VMIG. Given that LM directly optimizes the total cost for each VM migration, it only migrates VMs with high communication costs to PMs with low migration costs, which ignores the communication cost history information of VMs in the original network. Therefore, LM may never migrate VMs that have high communication cost, resulting in a limited optimization of the total cost. However, mCaM changes the form of the objective function. In the algorithm process, mCaM focuses on the network benefit to primarily optimize the communication performance. To have a better optimization, mCaM calculates the network benefits for each overloaded VM and migrates the VM with the maximum network benefit. Therefore, mCaM solves the limitations of LM. It maximizes the migration performance benefit and has a lower total cost after optimization. mCaM2 considers the reduction of communication cost after migration to maximize the network performance. mCaM2 separates the optimization into two periods: the selection of destination PM and the selection of migration VM. In the selection of the destination PM, only the reduction of communication cost is considered. Migration cost is considered in the selection of migration VM. It exerts a negative influence on the migration cost and increases the total cost. Therefore, the total cost of mCaM2 is higher than the other two algorithms.
To compare the communication cost and migration cost optimization effect, we also analysed the communication cost and the migration cost of these three algorithms. Figures 3 and 4 show the detailed communication cost and migration cost values with different numbers of VMs. Figure 3 shows that mCaM2 has the best communication cost optimization performance among the three algorithms, followed by mCaM. LM directly optimizes the total cost. In every VM migration, LM only focuses on the communication cost after VM migration and never considers the communication cost before migration.
In the migration VM selection, mCaM considers the VM cost information in the original network and migrates the VM with the greatest network benefit. Therefore, it has a better communication cost optimization than LM. mCaM2 considers the costs separately in different periods. In the destination PM selection period, mCaM2 only considers the optimization of the communication cost. It ensures that the communication cost can reduce as much as possible. Therefore, mCaM2 has the minimum communication cost.

Fig. 3 Comparison of communication costs
As shown in Fig. 4, LM has the smallest migration cost, followed by mCaM, but the difference is small. Given that the destination PM selection strategy ignores migration cost, mCaM2 has the largest migration cost. In conclusion, mCaM has the highest network optimization performance in terms of both communication cost and migration cost among the three algorithms. mCaM2 optimizes communication costs at most, but the migration cost of it is higher than that of mCaM. LM has the lowest migration cost, but the optimization in communication costs is the highest among the three algorithms.

Comparison with AppAware in terms of performance
We also compared the total cost of mCaM with that of AppAware. We list the statistical results in total cost in Table 4 and Fig. 5. Figure 5 shows that mCaM's total cost reduces sharply compared with that of AppAware, and Table 4 shows that mCaM has a 20% lower total cost than AppAware on average. Given that AppAware only optimizes the communication cost and ignores the migration cost, the result satisfies our expectations. Figure 6 shows the experimental results for AppAware and mCaM in terms of communication cost. As shown in this figure, given that AppAware only aims to optimize the communication cost, AppAware performs better than mCaM.
Next, we analysed performance in terms of the migration cost. Figure 7 presents the statistical results of these two algorithms in the simulation experiment.
The bar graph in Fig. 7 represents the number of migrations made by the two algorithms. The curves in Fig. 7 represent the corresponding migration costs. Evidently, mCaM has a much lower migration cost than AppAware, and its number of migrations is also smaller than that of AppAware by approximately 50%.   mCaM fully considers the influence of migration cost and uses the lowest migration cost to obtain the largest network benefit. AppAware only optimizes the communication traffic and directly migrates all VMs on the overloaded PM, which easily generates frequent VM migration. The curve of the number of migrations proves this point. Finally, we compare the operation times of these two algorithms. LM, mCaM and mCaM2 have the same time complexity. Therefore, LM, mCaM and mCaM2 has the similar operation times, and we only compared the operation times between mCaM and AppAware. The detailed data are shown in Fig. 8.
As shown in Fig. 8, mCaM exhibits a clear improvement in operation time compared with AppAware by 70% on average. AppAware migrates all the VMs on the overloaded PM. However, mCaM only migrates some of the overloaded VMs in order to just make the PM underloaded. Therefore, mCaM has a much smaller operation time. Furthermore, mCaM considers the balance between migration cost and network benefit. Therefore, mCaM migrates the VM with the highest benefit, which also decreases the operation time. In conclusion, the advantages of mCaM in operation time increase with the number of VMs and the scale of the problem.

Conclusion
In this paper, we focus on the network traffic created by the VM migrations, which profoundly limit the performance of the datacenters. To solve this problem, we propose three heuristics VM migration algorithms, LM, mCaM and mCaM2, to balance the optimization effect of communication cost and migration cost. The simulation experiments demonstrate that it efficiently decreases network traffic of the datacenters by 20% comparing with AppAware. And our algorithms also outperform AppAware in terms of operation time by 70% on average. Our simulation results show that the proposed algorithms can effectively decrease the energy consumption in the cloud-edge fusion computing environment. One promising future direction is to implement and evaluate our algorithms on a real cloud-edge fusion computing environment.