Collaborative task offloading and resource allocation with hybrid energy supply for UAV-assisted multi-clouds

Abstract

Cloud computing has emerged as a promising paradigm for meeting the growing resource demands of Internet of Things (IoT) devices. Meanwhile, with the popularity of mobile aerial base stations, Unmanned Aerial Vehicle (UAV) assisted cloud computing is essential for providing diversified service at areas without available infrastructure. However, it is difficult to meet the requirements of a number of IoT devices which distribute a large area through one single UAV cloud server, and thus multi-clouds have been applied in large-scale IoT environments. Due to the limited battery capacity of UAV, hybrid energy supply has been considered as an effective approach. How to allocate the computing resources and offload the tasks to the UAV-assisted clouds is a challenging task. In this paper, we study the trade-off between the energy consumption and system performance in a UAV-assisted multi-clouds system. Considering the transmission and execution cost, a dynamic optimization problem with the objective of minimizing the power consumption of UAVs with the constraint of queue stability is formulated, which is further decomposed into three sub-problems using stochastic optimization techniques. A collaborative task offloading and resources allocation algorithm (CTORAA) based on artificial intelligent (AI) technique is proposed to jointly determine task offloading and energy harvesting. We provide corresponding mathematical analysis showing that CTORAA can reach the arbitrary profit-stability trade-off. Finally, we conduct simulation experiments to validate the efficacy of our algorithm.

Introduction

Internet of Things (IoT) is an emerging technology that connects everything in the Internet. It has been widely applied in various domains such as healthcare, transportation, industry, etc. [1]. Many applications which require heavy computation and high energy consumption have been emerging, such as virtual reality, face recognition, and service recommendation [2]. However, due to the limited resources of IoT devices [3], such as limited battery and weak computing capability, computation-intensive and resource-consuming tasks cannot be fully performed on IoT devices [4, 5]. Combining the advantage of mobile computing and cloud computing, Mobile Cloud Computing (MCC) has been proposed to overcome the limited processing power and poor storage capacity of mobile devices [6]. In MCC, tasks generated by IoT devices can be transferred to multiple cloud servers, and thus the Quality of Service (QoS) can be guaranteed [7]. For edge-based IoT services, Zhang et al. [8] proposed a distributed edge QoS prediction model with privacy-preserving

In some extreme conditions (such as earthquake, flood, and typhoon), MCC deployed on the base station or centralized data centers may not work due to the failure of communication and computing infrastructures. To attack this challenge, Unmanned Aerial Vehicle (UAV) has been introduced to carry computing devices acting as a small cloud server as well as providing communication support [9]. Such promising technology, namely UAV-assisted cloud computing, has attracted extensive attention in both academia and industry.

Due to the limited battery capacity of the UAV, it is difficult to fly a UAV and let it provide services for IoT devices for a long time. Therefore, in order to prolong the service lifecycle, UAV with energy harvesting has been widely studied. However, energy is still the most limited resource, since both UAV itself and its payload (cloud server) consume energy while providing services. How to allocate the computing resources and schedule the tasks among the devices and multi-clouds with maximized energy utilization adapting to the dynamic environment and task arrivals is a challenging task.

In this paper, we study the problem of collaborative task offloading and resource allocation with hybrid energy supply for UAV-assisted multi-clouds. Our goal is to minimize the average energy consumption of the system meanwhile ensuring the performance. In order to deal with the dynamics of task arrival, we formulate a stochastic optimization problem, and further decompose the original problem into three independent sub-problems. Mathematical optimization methodology as well as AI-empowered approaches are presented to solve the sub-problems. We design a Collaborative Task Offloading and Resource Allocation Algorithm (CTORAA), and provide quantitative analysis to prove the effectiveness of our approach. Finally, we conduct simulation experiments to validate the performance of our CTORAA algorithm by comparing with other benchmark algorithms.

The rest of this paper is organized as follows. In Section 2, we survey the related works. In Section 3, we introduce the basic scenario, and present the system model and problem formulation. In Section 4, a collaborative task offloading and resource allocation algorithm is designed, and its mathematical analysis is provided. In Section 5, we conduct simulation experiments to validate our approach. Finally, in Section 6, we conclude this paper.

Related Work

It is practical to support IoT applications by transferring heavy computation tasks to powerful servers in the Edge-Cloud system. In [10], Almutairi and Aldossary introduced an Edge-Cloud system architecture that includes the required components to support scheduling offloading tasks of IoT applications. They provided a set of algorithms for offloading tasks scheduling in order to enhance service time and resource utilization in Edge-Cloud environments. Chen et al. [11] integrated the resources of Mobile Edge Computing (MEC) and cloud to study the collaborative and dynamic task offloading for IoT devices. They set one of the task offloading goals as maximizing the task finishing ratio with tolerable delay, and the other offloading goal is minimizing the power consumption. The two offloading goals are conflicting. The authors formulated the detailed task offloading problem to balance the two conflicting goals subject to resources constraints. In [12], Rong et al. presented the computing model and the architecture of Sophon Edge, an edge-cloud collaborative Artificial Intelligence of Things (AIoT) platform. While the cloud exploits its sufficient computing and storage resources to train sophisticated machine learning models, frequent but less complicated computation tasks are offloaded to the edge of the IoT network to provide low latency services, allowing the end users of the AIoT application to get near real-time responses, which really improves the user experience.

There is a growing interest in the study of UAV-assisted cloud computing system. Gao et al. [13] proposed a potential game combined multi-agent deep deterministic policy gradient (MADDPG) approach to optimize multiple UAVs’ trajectory with the consideration of ground users’ offloading delay, energy efficiency as well as obstacle avoidance system. In [14], with multi-UAV and ground mobile UEs in the system, heterogeneous performance requirement as well as fast-changing communication condition make the system more complicated. Wang et al. proposed a heuristic joint power and quality of experience (HJPQ) algorithm where the user equipments’ (GEs’) offloading delay, MIMO channel, transmission power, as well as UAVs’ placement are jointly optimized. Various aerial offloading data with different QoE requirements is stored and relayed in the multi-queueing architecture. Hence offloading rate differentiation is utilized to ensure the high-priority data a better QoE. In [15], Gao et al. proposed an energy-efficient resource allocation scheme with the ability of QoE enhancement. In [16], on the premise of ensuring that all user tasks are completed, the time-division multiple access protocols were adopted to minimize the energy consumption of all users. And the energy-performance tradeoff problem has always been a hot research topic in various aspects of wireless communication networks and IoT technology, as well as other network technologies.

The energy consumption of UAVs is also an important aspect to consider in the design of UAV systems. In order to extend the life of UAVs, energy must be used efficiently [17]. In recent years, there were many people have made efforts to improve the energy efficiency of UAVs. Du et al. [18] investigated a UAV-enabled mobile computing system in which the UAV used wireless power transfer (WPT) technology to power IoT devices. Then the data collected by each IoT device is transmitted to the UAV for processing. To improve the energy efficiency of the UAV, a new Time Division Multiple Access (TDMA) based workflow model was proposed that allows parallel transmission and execution in the UAV-enabled system. The total energy consumption of UAV is minimized by jointly optimizing the flight zone association, computation resource allocation, UAV hovering time, wireless power supply time, and flight zone order of service. In [19], they studied the energy-performance tradeoff for data transmission in UAV-enabled MEC, and they proposed a method that provides an effective scheme for realizing the energy-performance tradeoff in system design. In [20], the author investigated a multi-UAV collaborative data acquisition system, multiple UAVs perform data collection on the two-dimensional distributed devices in a flying or hovering mode to save energy.

Zhan et al. [21] concentrated on the completion time and energy optimization in UAV-enabled mobile computing system. To maximize the remaining energy, in [22], they combined the energy production model and energy consumption model to optimize the level flight trajectory. In [23], the authors investigated multi-UAV providing edge computing services by collaboration. They developed a multi-UAV cooperative edge computing framework. The tasks offloaded from IoT devices to UAVs are processed cooperatively by UAVs. To solve this problem, they jointly optimized the number of deployed UAVs and the offloading decision for each IoT device. Then, they determined the optimization objective of minimizing system energy consumption while ensuring that all offloading tasks were completed. And an efficient collaborative greedy task scheduling algorithm was designed to achieve the optimization objective.

However, little attention has been paid to the relationship between energy consumption and system performance in a UAV-assisted multi-clouds computing system. So we concentrate on the tradeoff between the energy consumption generated by transmission and computing and the system stability.

System Model and Problem Formulation

System Framework for UAV-assisted multi-clouds

Figure 1 demonstrates a basic scenario of a UAV-assisted multi-clouds system. We consider the system consisting of X energy-limited IoT devices denoted as $A=\{{1,2,\cdot \cdot \cdot ,X\}}$, and Y UAVs denoted as $B=\{1,2,\cdot \cdot \cdot ,Y\}$. We assume that all UAVs fly at a fixed height and all IoT devices randomly distributed in a flat ground. In this model, the ground equipment is divided into Y disjoint partitions, and each partition is associated with a UAV. The task requests from each IoT device can be transmitted to the UAV belonging to its region by the gateway.

Fig. 1

Basic scenario of UAV-assisted multi-clouds

In order to express the computing task offload strategy more clearly, we introduce $A_j$ to denote the set of IoT devices covered by UAV j. It yields the following constraints:

$$\begin{aligned}{}&A_{1} \cup A_{2} \cup \cdots \cup A_{j} \cup \cdots \cup A_{Y}=A \\ & A_{j} \cap A_{k}=\varnothing , \forall j, k \in B\ \& \ j \ne k\end{aligned}$$

(1)

In the above-mentioned UAV-assisted multi-clouds computing system, the computing task of each IoT device $i (i \in A_j, j \in B)$ can have two offloading strategies: local computing and cloud server computing. First, local computing means that the computing task is executed at IoT device i. Second, cloud server computing means that IoT device i offloading the computing tasks to UAV j. We consider a discrete time-slotted system $T=\{1,2,3,\cdot \cdot \cdot \}$. And the slot length is denoted by $\tau$. In Table 1, the main notations in the system model are provided.

Table 1 Notations and definitions

Task Model for IoT devices

In every time slot $t\in T$, the number of tasks that arrive at the ith IoT device can be expressed as $d_{i}(t), \forall i\in A$, $0\le d_{i}(t)\le d_{i}^{max}$. We only focus on computationally intensive tasks. The processing complexity of computationally intensive tasks can be considered to be proportional to the size of the input task. Then, the total number of tasks that arrive at the IoT devices set $A_j$ is calculated as

$$\begin{aligned} \mathrm {D}_{j}(t)=\sum \limits _{i \in A_{j}} d_{i}(t), j \in B. \end{aligned}$$

(2)

Within the tth time slot, IoT devices set $A_j$ will locally execute $D_{l,j}(t)$ tasks and will offload $D_{r,j}(t)$(the requests that the UAV allowed) tasks to the UAV j. At the beginning of the tth time slot, the local queue length of the jth IoT device set $A_j$ is $S_{j}(t)$. Initially, $S_{j}(0)=0$. At tth time slot, the amount of tasks which has arrived but not been executed locally or offloaded will be put in $S_{j}(t)$. Then, we can capture the following queue dynamics over time:

$$\begin{aligned} S_{j}(t+1)=max\{S_{j}(t)-D_{l,j}(t)-D_{r,j}(t),0\}+D_{j}(t),~j\in B. \end{aligned}$$

(3)

Then we set up a cache queue $M_{j}(t)$ to indicate the number of task requests of the UAV j that have not been processed by the cloud server. Initially, we assume that $M_j(0)=0$. The total number of tasks that have been executed by the cloud server can be expressed as $D_{u,j}(t)$. Thus, the queue length of each UAV evolves as the following equality

$$\begin{aligned} M_{j}(t+1)=max\{M_{j}(t)-D_{u,j}(t),0\}+D_{r,j}(t),~j\in B. \end{aligned}$$

(4)

Hybrid Energy Model for UAVs

For each UAV, it is equipped with a battery with limited capacity $E_H$. Denote the remaining battery capacity of UAV j at the beginning of the tth time slot as $H_j(t)$. And we assume that $H_j(0)=\frac{1}{2}E_h$. To ensure UAVs can continue to serve IoT devices, we provide UAVs with a hybrid energy supply [24]. We assume that energy source of UAVs includes two parts, which are the harvested solar energy and grid energy [25]. When the solar energy would be unable to afford the operations of the UAVs, they will obtain energy by wireless charging. At the beginning of the tth time slot, we assume that successive energy with size $W_j(t)$ arrives in UAV j. Thus, battery charging has the following constraint:

$$\begin{aligned} H_{j}(t)+W_j(t) \le E_H,~ j \in B. \end{aligned}$$

(5)

We consider that energy consumption of UAVs consists of computing energy consumption. Then, let $E_{u,j}(t)$ represents the energy consumption incurred by UAV j within the tth time slot. The battery life is affected by excessive discharging. Therefore, we introduce safe threshold as $E_{safe}$ , i.e. , the minimum battery energy level. Thus, we have

$$\begin{aligned} M_{j}(t)+W_{j}(t)-E_{u,j}(t) \ge E_{safe} ~,~j\in B. \end{aligned}$$

(6)

The evolution equation of energy for the jth UAV is

$$\begin{aligned} H_{j}(t+1)=max\{H_{j}(t)-E_{u,j}(t),0\}+W_j(t),~j\in B. \end{aligned}$$

(7)

Task Offloading and Resource Allocation Models for IoT devices

Within the tth time slot, the total number of tasks computed locally by the IoT devices set $A_j$ can be expressed as

$$\begin{aligned} D_{l,j}(t)=\sum \limits _{i\in A_j}\tau f_{l,i}(t)L_i^{-1},~j\in B, \end{aligned}$$

(8)

$$\begin{aligned} 0\le f_{l,i}(t) \le f_{max,i}~,i \in A, \end{aligned}$$

(9)

where $f_{l,i}(t)$ is the CPU-cycle frequency of the ith IoT device, and $f_{max,i}$ is the maximum value. $L_i$ (cycles/bit) denotes the CPU cycles of the ith IoT device when executing 1 bit task.

According to circuit theories, the CPU power consumption of the ith IoT device in the tth time slot is calculate as

$$\begin{aligned} p_{l,i}(t)= k_{mod,i} ~ f^{3}_{l,i}(t),~i\in A, \end{aligned}$$

(10)

where $k_{mod,i}$ is the effective switched capacitance of the CPU of the ith IoT device.

In the UAV-assisted multi-clouds computing system, the ith ($i\in A_j$) IoT device can send computing task to the jth UAV by a gateway. We denote the transmission power of the ith UAV in the tth time slot is $p_{r,i}(t)$. Thus, the transmission rate between the ith IoT device and the jth UAV is expressed as

$$\begin{aligned} &r_{i, j}(t)=B_{i} \log _{2}\left( 1+\frac{p_{r, i}(t) G_{i} Z_{0}}{B_{i} N_{0} d_{i, j}^{2}}\right) , j \in B, i \in A_{j}, \end{aligned}$$

(11)

$$\begin{aligned} 0 \le p_{r, i}(t) \le p_{\max , i}~ ,i \in A, \end{aligned}$$

(12)

where $B_i$ is the channel bandwidth of IoT device i, the $G_i$ is the channel power gain at a reference distance 1m of IoT device i. $d_{i,j}$ is denoted as the distance between the ith IoT device and the jth UAV. $N_0$ is the noise power spectrum density.$Z_0$ is a positive constant $(\approx 2.2846)$. $p_{max,i}$ denotes the maximum transmission power of the ith IoT device.

Within the tth time slot, the total number of tasks transmitted through IoT device set $A_j$ to UAV j can be expressed as

$$\begin{aligned} D_{r, j}(t)=\sum \limits _{i \in A_{j}} r_{i, j}(t) \tau , j \in B. \end{aligned}$$

(13)

The transmission energy consumption of the ith IoT device in time slot t is calculate as

$$\begin{aligned} E_{r, j}(t)=\sum \limits _{i \in A_{j}} p_{r, i}(t) \tau , j \in B \end{aligned}$$

(14)

In the tth time slot, the amount of tasks of the jth UAV is denoted as $D_{u,j}(t)$. Then, $D_{u,j}(t)$ is calculated as

$$\begin{aligned} D_{l,j}(t)=\tau f_{u,j}(t)L_j^{-1},j\in B, \end{aligned}$$

(15)

$$\begin{aligned} 0\le f_{u,j}(t) \le f_{max,j}~,j \in B, \end{aligned}$$

(16)

where $L_{j}$(cycles/bit) is the CPU cycles which the jth UAV uses to execute 1 bit task. $f_{u,j}(t)$ is denoted as the CPU-cycles frequency of the jth UAV, and $f_{max,j}$ is the maximum value.

The energy consumption of the computing of the jth UAV in time slot can be express as

$$\begin{aligned} E_{u,j}(t)=k_{mod,j} ~f_{u,j}^{3}\tau ,~j\in B, \end{aligned}$$

(17)

where $k_{mod,j}$ denotes the effective switched capacitance of the CPU of the jth UAV.

Task Offloading Problem Formulation

In UAV-assisted multi-clouds computing system, system stability is also an important factor to evaluate system performance. According to Little’s law [26], the average queue delay is proportional to the queue length. Furthermore, the queue length can be used as the metric to evaluate the performance of the system [27, 28]. We want all queues not to be congested. In other words, we require all the actual queues to be stable in the time average sense. Inspired by [29], we can obtain:

$$\begin{aligned} \overline{S} = \mathop {\limsup }_{T \rightarrow \infty } \frac{1}{T} \sum \limits _{t=0}^{T-1}\mathbb {E} \{\sum \limits _{j \in B}|S_j(t)| \} <\infty , \end{aligned}$$

(18)

$$\begin{aligned} \overline{M} = \mathop {\limsup }_{T \rightarrow \infty } \frac{1}{T} \sum \limits _{t=0}^{T-1} \mathbb {E} \{\sum \limits _{j \in B}|M_j(t)| \}<\infty , \end{aligned}$$

(19)

$$\begin{aligned} \overline{H} = \mathop {\limsup }_{T \rightarrow \infty } \frac{1}{T} \sum \limits _{t=0}^{T-1} \mathbb {E} \{\sum \limits _{j \in B}|H_j(t)| \}<\infty , \end{aligned}$$

(20)

where $\overline{S}$, $\overline{M}$ and $\overline{H}$ denote the time-averaged queue backlog. We hope that the value of the queue backlog is as small as possible.

For computationally intensive tasks, the energy computation of sending the results back is much smaller than the transmission and execution. Besides, we assume all UAVs consume the same energy for flight in each time slot. In the above-mentioned UAV-assisted multi-clouds computing system, we only consider the energy consumption for transmission and computing. First, we denote the system operation at the tth time slot with $X(t)\triangleq \{f_{l,i}(t), p_{r,i}(t), f_{u,j}(t),W_j(t)\}$. Then the objective function that minimize the average energy consumption of the system is defined as P1:

$$\begin{aligned} P1:&\min _{\{X(t)\}}\mathop {\lim }_{T \rightarrow \infty } \frac{1}{T} \sum \limits _{t=0}^{T-1}\mathbb {E} [\sum \limits _{j \in B}(E_{l,j}(t)+E_{r,j}(t)+E_{u,j}(t))], \\ {}&s.t. \quad (5), (6), (9), (12), (16), (18), (19), (20). \end{aligned}$$

(21)

Since the optimization goal and constraints depend on not only the current system states but also the future ones, the formulated problem P1 is a stochastic optimization problem. Based on Lyapunov optimization, the stochastic optimization problem is converted into a deterministic optimization problem. In next section, we will introduce an online algorithm for solving the above problem.

Collaborative Task Offloading and Resource Allocation Algorithm Design

In this section, we take advantage of the Lyapunov optimization techniques combining with an intelligent approach to design an online control algorithm. By leveraging the stochastic optimization techniques, the original problem (21) is decomposed into three independent sub-problems, corresponding to the three decisions that need to be made: 1) local computing control; 2) task offloading control; 3) cloud computing and charging control. Then mathematical analysis shows that the algorithm can reach the arbitrary optimal solution on the premise of maintaining the system stability.

Drift-plus-penalty Design

According to [30], first we define the vector of queue backlogs at time t by:

$$\begin{aligned} &S(t)=(S_{1}(t),S_{2}(t),\cdots ,S_{Y}(t)) ,\\&M(t)=(M_{1}(t),M_{2}(t),\cdots ,M_{Y}(t)),\\&H(t)=(H_{1}(t),H_{2}(t),\cdots ,H_{Y}(t)). \end{aligned}$$

(22)

Let $\Theta (t)=[S(t),M(t),H(t)]$ denotes the combined matrix of all the queues.

To model the problem as a Lyapunov optimization problem, we define the Lyapunov function $L(\Theta (t))$ as the sum of the squares of the current queue backlogs for each slot.

$$\begin{aligned} L(\Theta (t))&=\frac{1}{2}\sum \limits _{j\in B}[S^2_{j}(t)+M^2_{j}(t)+H^2_{j}(t)]. \end{aligned}$$

(23)

This function is a scalar measure of the total queue backlog in the network. It is called quadratic Lyapunov function on the queue state. Next we introduce the Lyapunov drift $\Delta (\Theta (t))$ as the change from one slot to the next:

$$\begin{aligned} \Delta (\Theta (t))=\mathbb {E} \{ L(\Theta (t+1))-L(\Theta (t))|\Theta (t)\}. \end{aligned}$$

(24)

The smaller $\Delta (\Theta (t))$, the more stable the queues. In order to make the network stable while minimizing the time average of the penalty $E_{j,t}+E_{j,v}$, the algorithm can be designed to minimize the upper bound of the drift-plus-penalty function in each time slot t. Then, we denote the Lyapunov drift-plus-penalty function as

$$\begin{aligned} \Delta (\Theta (t))+V\mathbb {E}[\sum \limits _{j \in B}(E_{l,j}(t)+E_{r,j}(t)+E_{u,j}(t))|\Theta (t) ] , \end{aligned}$$

(25)

where V is a non-negative parameter that is used to as adjusting the tradeoff between queue length and energy consumption [31, 32].

Deriving Upper Bound

Then a key derivation step is to obtain an upper of the drift-plus-penalty expression (25). The following lemma defines such an upper bound.

Lemma 1

In each time slot t, for any value of $\Theta (t)$ and any parameter value of V, we have:

$$\begin{aligned} \Delta (\Theta (t))+V\mathbb {E} [\sum \limits _{j\in B} (E_{l,j}(t)+E_{r,j}(t)+E_{u,j}(t))|\Theta (t) ]\le B\\ +\mathbb {E}[\sum \limits _{j\in B}S_{j}(t)(D_j(t)-D_{l,j}(t)-D_{r,j}(t)|\Theta (t)]\\ +\mathbb {E}[\sum \limits _{j\in B}M_{j}(t)(D_{r,j}(t)-D_{u,j}(t))|\Theta (t)]\\ +\mathbb {E}[\sum \limits _{j\in B}H_{j}(t)(W_j(t)-E_{u,j}(t))]\\ +V\mathbb {E}[ \sum \limits _{j\in B}(E_{l,j}(t)+E_{r,j}(t)+E_{u,j}(t))|\Theta (t)) ], \end{aligned}$$

(26)

where B is a constant.

Proof First, we define

$$\begin{aligned} \Delta Q_j(t)=&S_{j}^{2}(t+1)+M_{j}^{2}(t+1)+H_{j}^{2}(t+1) -S_{j}^{2}(t)-M_{j}^{2}(t)-H_{j}^{2}(t). \end{aligned}$$

(27)

From (3), (4) and (7), we have

$$\begin{aligned}\Delta Q_j(t)&=\left\{ \max \left\{ S_{j}(t)-D_{l, j}(t)-D_{r, j}(t), 0\right\} +D_{j}(t)\right\} ^{2} \\&+\left\{ \max \left\{ \mathrm {M}_{j}(t)-D_{u, j}(t), 0\right\} +\mathrm {D}_{r, j}(t)\right\} ^{2} \\&+\left\{ \max \{H_{j}(t)-E_{u, j}(t),0\}+W_{j}(t)\right\} ^{2} \\&-S_{j}^{2}(t)-M_{j}^{2}(t)-H_{j}^{2}(t). \end{aligned}$$

(28)

By applying the inequality that $(\max \{d,0\})^{2}\le d^{2})$, it can be obtained that

$$\begin{aligned} \Delta Q_j(t)&\le \left[ S_{j}(t)-D_{l, j}(t)-D_{r, j}(t)\right] ^{2}+D_{j}^{2}(t)+2 S_{j}(t) D_{j}(t) \\&+\left[ \mathrm {M}_{j}(t)-D_{u, j}(t)\right] ^{2}+D_{r, j}^{2}(t)+2 \mathrm {M}_{j}(t) \mathrm {D}_{r, j}(t) \\&+\left[ H_{j}(t)-E_{u, j}(t)\right] ^{2}+W_{j}^{2}(t)+2 H_{j}(t) W_{j}(t) \\&-S_{j}^{2}(t)-M_{j}^{2}(t)-H_{j}^{2}(t). \end{aligned}$$

(29)

After calculating, we can rewrite (29) as follow

$$\begin{aligned} &\Delta Q _j(t)\le D_{l, j}^{2}(t)+D_{r, j}^{2}(t)+D_{j}^{2}(t)+D_{r, j}^{2}(t)+D_{u, j}^{2}(t) \\&+E_{u, j}^{2}(t)+W_{j}^{2}(t)+2 S_{j}(t)\left( D_{j}(t)-D_{l, j}(t)-D_{r, j}(t)\right) \\&+2 \mathrm {M}_{j}(t)\left( \mathrm {D}_{r, j}(t)-\mathrm {D}_{u, j}(t)\right) +2 H_{j}(t)\left( W_{j}(t)-E_{u, j}(t)\right) . \end{aligned}$$

(30)

According to the previous definition, we can learn that $D_{l, j}(t)$, $D_{r, j}(t)$, $D_{j}(t)$, $D_{u, j}(t)$, $E_{u,j}(t)$ and $W_j(t)$ all have the upper bound, i.e., $D_{l, j}(t)\le D^{max}_{l,j}(t)$, $D_{r, j}(t)\le D^{max}_{r,j}(t)$, $D_{j}(t)\le D^{max}_{j}(t)$, $D_{u, j}(t)\le D^{max}_{u,j}(t)$, $E_{u,j}(t)\le E^{max}_{u,j}(t)$ and $W_j(t)\le W^{max}_{j}(t)$. Let B take the value of

$$\begin{aligned}B=&\frac{1}{2} \sum \limits _{j \in B}\left[ \left( D_{l, j}^{\max }(t)\right) ^{2}+2\left( D_{r, j}^{\max }(t)\right) ^{2}+\left( D_{j}^{\max }(t)\right) ^{2}\right. \\&\left. +\left( D_{u, j}^{\max }(t)\right) ^{2}+\left( W_{j}^{\max }(t)\right) ^{2}+\left( E_{u, j}^{\max }(t)\right) ^{2}\right] . \end{aligned}$$

(31)

Then, we can obtain that

$$\begin{aligned} \Delta (\Theta (t))&=\frac{1}{2}E(\sum \limits _{j \in B}\Delta Q_j(t)\mid \Theta (t))\\ \le B&+E\left[ \sum \limits _{j \in B} S_{j}(t)\left( D_{j}(t)-D_{l,j}(t)-D_{r,j}(t)\right) \mid \Theta (t)\right] \\&+E\left[ \sum \limits _{j \in B}\mathrm {M}_{j}(t)\left( \mathrm {D}_{r, j}(t)-\mathrm {D}_{u, j}(t)\right) \mid \Theta (t)\right] \\&+E\left[ \sum \limits _{j \in B}\mathrm {H}_{j}(t)(W_j(t)-E_{u,j}(t))\mid \Theta (t)\right] . \end{aligned}$$

(32)

The Lyapunov drift-plus-penalty function is defined as:

$$\begin{aligned}&\Delta _{v}(\Theta (t))=\Delta (\Theta (t))+V E\left[ E_{f, j}(t)+E_{t, j}(t) \mid \Theta (t)\right] \\&\le B+E\left[ \sum \limits _{j \in B} S_{j}(t)\left( D_{j}(t)-D_{l,j}(t)-D_{r,j}(t)\right) \mid \Theta (t)\right] \\&+E\left[ \sum \limits _{j \in B}\mathrm {M}_{j}(t)\left( \mathrm {D}_{r, j}(t)-\mathrm {D}_{u, j}(t)\right) \mid \Theta (t)\right] \\&+E\left[ \sum \limits _{j \in B}\mathrm {H}_{j}(t)(W_j(t)-E_{u,j}(t))\mid \Theta (t)\right] \\&+V E\left[ (E_{l, j}(t)+E_{r, j}(t)+E_{u, j}(t) )\mid \Theta (t)\right] . \end{aligned}$$

(33)

Notably, in time slot t, B and $S_{j}(t)D_{j}(t)$ are constants. To be more specific, parameter B is the constant defined in (31). Besides, $S_{j}(t)D_{j}(t)$ can be obtained and known at the beginning of each time slot t. Thus, the supremum bound minimization problem can be equivalently reformulated by eliminating the constant term.

$$\begin{aligned} P2:&\mathop {\min }_{\{X(t)\}} \sum \limits _{j\in B}[-S_j(t)D_{l,j}(t)+V E_{l,j}(t)]\\ +&\sum \limits _{j\in B}[D_{r,j}(t)(M_{j}(t)-S_{j}(t))+V E_{r,j}(t)]\\ +&\sum \limits _{j\in B}[H_j(t)(W_j(t)-E_{u,j}(t))-M_j(t)D_{u,j}(t)+V E_{u,j}(t)]\\&s.t.\quad (5),\quad (6), \quad (9), \quad (12), \quad (16). \end{aligned}$$

(34)

Collaborative Task Offloading and Resource Allocation Algorithm

Instead of solving the P2 optimally, we decompose it into three independent sub-problems, corresponding to the three decisions that need to be made. Next, we derive the optimal solutions to these three sub-problems separately.

Specifically, our algorithm is described in Algorithm 1.

Local Computing Control

We determine $f_{l,i}(t)$ by solving the problem SP1 as follows

$$\begin{aligned} SP1:&\min _{\left\{ f_{l, i}~(t)\right\} } \sum \limits _{j \in B}\left[ -S_{j}(t) \sum \limits _{i \in A_{j}} f_{l, i}(t) L_{i}^{-1} +\mathrm {V} \sum \limits _{i \in A_{j}} k_{mod, i}~ f_{l, i}^{3}(t)\right] .\\&s.t. \quad (9). \end{aligned}$$

(35)

It can been seen that SP1 is a simple convex optimization problem. Then, we can obtain the optimal solution:

$$\begin{aligned} f_{l, i}^{*}(t)=\min \left\{ \sqrt{\frac{S_{j}(t)}{3 V k_{\text{ mod } , i} L_{i}}}, f_{l, i}^{\max }(t)\right\} . \end{aligned}$$

(36)

Task Offloading Control

The optimal decision vector $p_{r,i}(t)$ can be obtained by solving the sub-problem SP2, as follow:

$$\begin{aligned} SP2:&\min _{\left\{ p_{r, i}~(t)\right\} } \sum \limits _{j \in B}\left[ \left( M_{j}(t)-S_{j}(t)\right) \sum \limits _{i \in A_{j}} B_{i} \log _{2}\left( 1+\frac{p_{r, i}(t) G_{i} Z_{0}}{B_{i}(t) N_{o} d_{i, j}^{2}}\right) +V \sum \limits _{i \in A_{j}} p_{r, i}(t)\right] .\\&s.t. \quad (12). \end{aligned}$$

(37)

SP2 is a convex optimization problem, which can be proved via the Second-order derivation. We can solve it by taking the derivative. Let us denote

$$\begin{aligned} \gamma _{i}(t)=\frac{\left( S_{j}(t)-M_{j}(t)\right) B_{i}}{V \ln 2}-\frac{N_{0} B_{i} d_{i, j}^{2}}{G \cdot Z_{0}}. \end{aligned}$$

(38)

Then, the $p_i^{*}(t)$ can be written as:

$$\begin{aligned} p_{r,i}^{*}(t)=\min \{\gamma _{i}(t),p_{max,j}~\}. \end{aligned}$$

(39)

Cloud Computing and Charging Control

$f_{u,j}^{*}(t)$ and $W_j^{*}(t)$ are defined as the optimal $f_{u,j}(t)$ and the optimal $W_j(t)$. We can obtain them by solving SP3.

$$\begin{aligned} SP 3: \min _{\{f_{u,j}~(t), W_j(t)\}}&\sum \limits _{j \in B}\left[ \left( V-H_{j}(t)\right) k_{mod , j} ~\tau f_{u, j}^{3}(t)-\frac{M_{j}(t) \tau }{L_{j}} f_{u, j}(t)+H_{j}(t) W_{j}(t)\right] .\\&s.t. \quad (5),\quad (6),\quad (16). \end{aligned}$$

(40)

SP3 is a combinatorial optimization problem. This objective function is determined by two variables, i.e., $f_{u,j}(t)$ and $W_j(t)$. To obtain the optimal solution, we propose an AI-empowered algorithm based on Simulated Annealing (SA), as described in algorithm 2. The algorithm produces near-optimal solution with less computation cost and avoids falling into local optimal solution.

Algorithm Analysis

In this subsection, we use mathematical analysis to prove that the algorithm we proposed can achieve the arbitrary optimal solution on the premise of maintaining the stability of the system.

Theorem 1

For all $t>0$ the time average profit and time average queue lengths satisfy:

$$\begin{aligned} \mathop {\lim }_{T \rightarrow \infty } \frac{1}{T} \sum \limits _{t=0}^{T-1}\sum \limits _{j \in B}\mathbb {E} \{E_{l,j}(t)+E_{r,j}(t)+E_{u,j}(t)\} \le e^*+\frac{B}{V}, \end{aligned}$$

(41)

$$\begin{aligned} \mathop {\lim }_{T \rightarrow \infty } \frac{1}{T} \sum \limits _{t=0}^{T-1}\sum \limits _{j \in B}\mathbb {E} \{S_{j}(t)+M_{j}(t)+H_{j}(t)\}\le \frac{B+V(\hat{e}-\breve{e})}{\varepsilon } , \end{aligned}$$

(42)

where $e^*$ is the optimal solution of the problem (21).

Proof Since we assume that arrival process is strictly within the network capacity region, there exists at least one stationary randomized control policy $\pi ^{*}$ that can stabilize the queue, which has following features:

$$\begin{aligned} \mathbb {E}\{ D_{l,j}(t)+D_{r,j}(t)|\Theta (t)\}\ge \mathbb {E}\{ D_{j}(t)|\Theta (t) \}=\lambda \\ \mathbb {E}\{ D_{u,j}(t)|\Theta (t)\}\ge \mathbb {E}\{ D_{r,j}(t)|\Theta (t) \}=\mu \\ \mathbb {E}\{ E_{u,j}(t)|\Theta (t)\}\ge \mathbb {E}\{ W_{j}(t)|\Theta (t) \}=\gamma \\ \end{aligned}$$

(43)

we can derive that there exists $\varepsilon \ge 0$ satisfying:

$$\begin{aligned} \mathbb {E}\{ D_{l,j}(t)+D_{r,j}(t)|\Theta (t)\}=\lambda +\varepsilon \\ \mathbb {E}\{ D_{u,j}(t)|\Theta (t)\}=\mu +\varepsilon \\ \mathbb {E}\{ E_{u,j}(t)|\Theta (t)\}=\gamma +\varepsilon \end{aligned}$$

(44)

Then, by applying (26), we obtain:

$$\begin{aligned}\Delta (\Theta (t))+V\mathbb {E} \{\sum \limits _{j \in B}(E_{l,j}(t)+E_{r,j}(t)+E_{u,j}(t))|\Theta (t)\}\\ \le B+\mathbb {E} [\sum \limits _{j \in B} (S_{j}(t)\mu )|\Theta (t)]-\mathbb {E} [\sum \limits _{j \in B} (S_{j}(t)(\mu +\varepsilon ))|\Theta (t) ]\\ \mathbb {E} [\sum \limits _{j \in B} (M_{j}(t)\lambda )|\Theta (t)]-\mathbb {E} [\sum \limits _{j \in B} (M_{j}(t)(\lambda +\varepsilon ))|\Theta (t) ]\\ \mathbb {E} [\sum \limits _{j \in B} (H_{j}(t)\gamma )|\Theta (t)]-\mathbb {E} [\sum \limits _{j \in B} (H_{j}(t)(\gamma +\varepsilon ))|\Theta (t) ]\\ +V\mathbb {E} \{\sum \limits _{j \in B}(E_{l,j}^{\pi ^{*}}(t)+E_{r,j}^{\pi ^{*}}(t)+E_{u,j}^{\pi ^{*}}(t))|\Theta (t)\}\\ =B-\varepsilon \mathbb {E}[\sum \limits _{j \in B}(S_{j}(t)+M_{j}(t)+H_{j}(t))|\Theta (t)]\\ +V\mathbb {E} \{\sum \limits _{j \in B}(E_{l,j}^{\pi ^{*}}(t)+E_{r,j}^{\pi ^{*}}(t)+E_{u,j}^{\pi ^{*}}(t))|\Theta (t)\}. \end{aligned}$$

(45)

Taking expectations on both sides and using iterated expectations, it can be obtained that:

$$\begin{aligned}&\mathbb {E}\{L(\Theta (t+1))-L(\Theta (t) )\} +V\mathbb {E} \{\sum\limits _{j \in B} (E_{l,j}(t)+E_{r,j}(t)+E_{u,j}(t))\}\\&\le B-\varepsilon \mathbb {E}[\sum\limits _{j \in B}(S_{j}(t)+M_{j}(t)+H_{j}(t))]+V e^*. \end{aligned}$$

(46)

Summing both sides of (46) over $t\in \{ 0,1,\dots ,T-1\}$, and then dividing both sides of T, we can yield:

$$\begin{aligned}&\frac{1}{T}\mathbb {E}\{L(\Theta (T)-L(\Theta (0)) \} +\frac{V}{T}\sum \limits _{t=0}^{T-1} \sum \limits _{j \in B} \mathbb {E}(E_{l,j}(t)+E_{r,j}(t)+E_{u,j}(t))\\&\le {B}-\frac{\varepsilon }{T}\sum \limits _{t=0}^{T-1}\sum \limits _{j \in B}\mathbb {E}(S_{j}(t)+M_{j}(t)+H_{j}(t))+V e^*. \end{aligned}$$

(47)

By taking the limit $T\rightarrow \infty$, we can obtain:

$$\begin{aligned}\mathop {\lim }_{T \rightarrow \infty }\frac{V}{T}\sum \limits _{t=0}^{T-1} \sum \limits _{j \in B} \mathbb {E}(E_{l,j}(t)+E_{r,j}(t)+E_{u,j}(t)) \le B\\+\mathop {\lim }_{T \rightarrow \infty }\frac{\varepsilon }{T}\sum \limits _{t=0}^{T-1}\sum \limits _{j \in B}\mathbb {E}(S_{j}(t)+M_{j}(t)+H_{j}(t))+V e^*. \end{aligned}$$

(48)

Dividing both sides of (48) by V and taking a limit as $\varepsilon \rightarrow \infty$, it can be obtain that (41). By dividing both sides of (48) by $\varepsilon$, we can yield (42).

The term (41) shows that as the value of V increases, the time averaged energy consumption decreases. However, The term (42) indicates that as the tradeoff parameter V increases, the time average queue congestion also increases. Therefore, we need to make a tradeoff between energy consumption and queue stability by adjusting the value of V. Besides, according to (41), there is an upper bound $\frac{B}{V}$ on the difference between CTORAA’s utility and the optimal utility. Furthermore, when V becomes large enough, the gap is small enough that the optimal solutions can be approximated by the final solutions.

Experimental Evaluation

Experimental Setting

In the experiments, we consider a system consisting of X IoT devices and Y UAVs. In general, there is $X=30$, $Y=5$. The $d_i(t)s$ are independent and identically distributed (i.i.d) in different time slots with the uniform distribution (i.e., $d_i(t)\sim U[0,10000]bit$), and $d_{i,j}\in [100,200]m$. We set the parameters of the experiment in Table 2.

Table 2 Experimental parameter setting

Simulation Results

Impacts of the Tradeoff Parameter V

Fig. 2

Time-average energy consumption with different V

Fig. 3

Time-average queue length with different V

We set $V=1\times 10^5, 1.5\times 10^5, 2\times 10^5, \cdot \cdot \cdot , 5.5\times 10^5$. Figure 2 show the time-average energy consumption and queue length of the UAV-assisted multi-clouds computing system with different V. In Fig. 3, it can be seen that the energy consumption decreases as the value of the tradeoff parameter V increases. And it converges to the minimum level for larger values of V. The agrees with the conclusion obtained by (41) in Theorem 1. In Fig. 3, it shows that the time-average queue congestion increases as the value of V increases, which is in accordance with (42) in Theorem 1. Together with Figs. 2 and 3, it can be seen that CTORAA can an achieve an arbitrary stability-profit tradeoff by changing the value of the tradeoff parameter V.

Impact of Task Arrival Rate

Figures 4 and 5 show task queue S and M with different task arrival rates respectively. In our experiments, the task arrival rate is denoted as $\alpha \cdot d_i(t)$. Specifically, we consider three different settings of $\alpha$, i.e. , $\alpha$ = 0.8, 1 and 1.2. From two figures, it can be seen that the length of task queues rises when the task arrival rate rises. We can see that the algorithm can adopt to the different task arrival rates and guarantee the queue stability.

Comparison Experiment

Fig. 4

Queue length of IoT devices with different task arrival rate

Fig. 5

Queue length of UAVs with different task arrival rate

In this subsection, in order to further present the performance of CTORAA, we compare our proposed algorithm with the two benchmark algorithms, namely random and fixed. Figure 6 shows the energy consumption with the three different algorithms.

Fig. 6

Energy consumption with different algorithms

It can be seen that the algorithm we proposed is closer to the optimal solution than the other two algorithms. In other words, our proposed algorithm is better than the other two algorithms.

Conclusions

In this article, we study the problem of cooperative task offloading and resource allocation with hybrid energy supply for UAV-assisted multi-clouds. We consider the tradeoff between the energy consumption of the system and system performance. To capture the high dynamics of the task and schedule each UAV more reasonably for conserving the energy, we take advantage of the Lyapunov optimization techniques to design an algorithm namely CTORAA. The goal is to minimize the average energy consumption of the system with the constraint of queue stability. Then mathematical analysis shows that CTORAA can reach the arbitrary profit-stability tradeoff. Finally, experiments are conducted to validate the performance of our approach.