A novel coordinated resource provisioning approach for cooperative cloud market

Cloud market

The idea that computers (and hence computing) can be provided as a utility, like electricity or telephony, dates back to the late 1970s [28]. Technological advancements over the next three decades or so in IC fabrication industry, multi-core processor architectures, and networked computing infrastructures gradually brought this grand vision closer to reality. These trends have enabled a vibrant IT industry that has shifted computing paradigms from traditional web-based computing to subsequent data-center based utility computing, grid and cloud computing, infusing numerous attributes and capabilities, like scalability, access ubiquity, autonomy of deployment and so on. The idea of having a market-oriented cloud dates back to almost a decade. Buyya et al. [29] were the first to propose the vision of a market oriented cloud and provided a high level architecture for supporting market-oriented resource management for trading services among cloud providers and consumers, bound by previously agreed upon agreements to ensure QOS based mechanisms [29]. This work invoked several research efforts to have apt scheduling policies for such market- oriented clouds [30, 31] and suitable toolkits for leveraging such market-oriented clouds [32].

The closest literature with respect to our work is presented by Haifei Li et al. [33], where they envision a marketplace model for cloud, namely, CCMarketplace. In CCMarketplace, computing resources of any kind, be it infrastructure (processors or memory), platform (IDEs on customized operating systems), or software (development tools like SDEs, databases, middle wares, etc.); become tradable commodities and the marketplace entities, like buyers and sellers of such commodities trade such resources to maximize their own profit [33].

However, in this paper a cooperative cloud market is dealt with for the purpose of reducing the cost of resource provisioning. The rationale behind this is the fact that the profitability and credibility of a cloud provider increases with the resource pool size it owns [2]. Since cloud users and their demands stochastically vary over time, thus to be able to tackle all such demand variations and still provision resources, it is pragmatic to have a cooperative cloud market, where the players share a common CMB entity for mediating provisioning requests. Such a concept also reduces the risk of failures to provide agreed services.

Resource provisioning

Virtualization has been the chief enabling technology that has led to successful realization of cloud computing [34]. Using virtualization, applications share underlying cloud resources by means of isolated Virtual Machines (VMs), such that each VM is endowed with cloud resources. Resource provisioning refers to the process of allocating succinct amount of resources to VMs [35] matched by their respective workloads. Usually cloud resources are provisioned dynamically in order to match workload fluctuations [36, 37] based on several criteria and are thereafter consolidated to a set of physical servers. Meng et al. [38] presents a joint criteria based provisioning, which is referred to as VM multiplexing. Zhan et al. [39] presents the idea of cooperative resource provisioning taking into account the heterogeneity of workloads. All the aforesaid articles deal with provisioning on a single cloud infrastructure. The research presented in this paper considers coordinated resource provisioning among a number of cloud providers via information sharing with a CMB entity by means of appropriate agents.

Pricing policies and cost optimization

H. Wang et al. [40] presented a very interesting interplay between distributed systems and economics related to pricing by decoupling the users from their specific cloud service providers. Providing multiple pricing options has enduring ramifications, both for the service providers as well as the users. Mazzucco et al. [41] have studied the effect of having flexible pricing options on revenue collection of cloud service providers. Menglan et al. [42] presents a case where the effect of reserved and on-demand instances are studied for minimizing user budget for deadline constrained jobs. They also present an equivalent formulation of execution time minimization for a budget constrained job. S. Khatua et al. [43] formulated the payable price for user of cloud services as an integer programming problem considering reserved and on-demand instances. They also presented a heuristic that finds the near optimal cost in polynomial time. Qian et al. [44] presents a stochastic integer programming model to optimize the cost of SLA-aware resource scheduling in the cloud. S. Chaisiri et al. [45] have considered uncertainty in terms of user demands over time and price uncertainties to arrive upon an optimum resource provisioning cost by means of stochastic linear programming. The scope of this paper is wider in the sense that the objective is to conduct optimal provisioning for a cloud market comprising of many cloud users, multiple cloud service providers as well as the two pricing policies, namely advanced reservation and on-demand provisioning. In order to do that, this paper employs a CMB entity that employs a multi-agent system among the cloud providers, such that the market players coordinate amongst themselves to form a cooperative cloud market.

Inter-cloud initiatives

In the past few years, there has been significant research attention targeted towards inter-clouds ranging from their interoperability issues to setting up of standard APIs and interfaces. Advantages of such approaches include lock-in situations over diverse geographical and legal demographics, improved reliability of applications and services, vendor lock-in avoidance [27]. Besides, there are other attractive incentives for cloud providers and users alike, such as option for on-demand workload expansion, better SLA offerings for customers and so on. There has been a number of projects receiving significant funding and attention, including Inter-Cloud [46], Contrail [47], Reservoir [48], Open Cirrus [49], OPTIMIS [50, 18], mOSAIC [51], Bonfire [52] etc. Grozev and Buyya [27] provide an extensive survey of these projects with a detailed taxonomical account from architectural and brokering perspectives. Another excellent review on this topic can be found in [26] wherein the authors delve on how standardization organizations view this domain and present the slightly varying nuances and goals of the aforementioned cloud projects.

In order that we can put our proposed cloud market in perspective, we observe that among the different projects the CMB proposed in this paper closely follows the third-party brokering model provided by OPTIMIS [18]. However, while [18] focuses on deployment issues of the third-party brokering model among other deployment model and architectures, our goal is to provide an infrastructure-level brokering model atop OPTIMIS provided services.

Background of the proposed cooperative cloud market (CCM)

Like in any marketplace, including the e-commerce paradigm, the CCM proposed herein also has four essential elements, namely buyers, sellers, intermediaries and interaction mechanisms [33]. In the context of the CCM, the distinction between buyers and sellers is blurred, in the sense that by accepting their respective terms and conditions, their individual roles may switch from time to time. An intermediary is referred to as a third party which offers intermediary services like information gathering and sharing, selection, negotiation, payment and so on [33]. A CMB has been proposed that fulfills the role of intermediary in the proposed CCM. A precise description for the choice of a multi-agent technology for deploying interaction mechanism in the CCM is presented in An illustrative example section. A simple schematic representation of the cooperative cloud market is depicted in Fig. 1.

The CMB entity acts as the intermediary. All cloud providers share required VM demand and QoS related information with the CMB via a multi-agent system (MAS). Details of this multi-agent system (MAS) and the interaction mechanisms in the proposed CCM have been presented in An illustrative example section with an demonstrative example.

Any cloud provider can join the cloud market by providing resources and service details to the cloud market broker. At any point of time, a cloud provider can leave the cloud market by fulfilling the agreements made to its consumers or by migrating their requested services along with SLA parameters to any other provider within the market without affecting the users’ service and by taking prior permission from the concerned users for any security concerns. The CMB is responsible for VM monitoring for the entire CCM. It is also responsible for resource provisioning decisions as well as consolidation decisions. The actors of the CCM have specific roles and the CMB is a broker, which coordinates among all cloud providers in the market for provisioning every user request or a group of requests. Any user can request for any kind of services by sending request-SLA (complete service request specification) to the cloud market broker. Each provider has a cloud broker for brokering within its resources and different cloud brokers may adopt different resource provisioning algorithms within their respective infrastructures. A high-level architecture of the proposed CCM has been depicted in Fig. 2. The cloud service consumers and the service providers, likened to buyers and sellers for any marketplace, constitute the first two layers in the architecture. The CMB entity along with these market actors form a multi agent system, the characteristics of which are presented in the following subsection. Based on information from its agents, the CMB takes decisions regarding which cloud service providers’ physical machines should be employed for the running VMs. It has to be noted that the bundled physical machines PM ₁ , PM ₂ ,…, PM _n etc., as depicted in Fig. 2 denote the computing resources within a single cloud provider.

The responsibility of resource provisioning for the CCM, as shown in Fig. 2, rests on the CMB, which executes appropriate algorithms to achieve its goal. Coordinated resource reservation and provisioning (CRRP) approach section presents a coordinated provisioning scheme for the proposed CCM that takes into account the variations in workload types as well as pricing schemes offered by different provider to achieve cost benefits for the entire market as a whole.

The users’ hour-wise demands for cloud resources are assumed to be known to the CMB. Also the prices of cloud resources pertaining to different cloud resource providers are assumed to be known ahead of time (i.e., provided by the providers before the provision period). In other words, uncertainties pertaining to users’ demands and prices offered by cloud providers have not been considered in the coordinated resource provisioning model for CCM.

Although resources can be reserved by a user under multiple contracts like, 1-year or 3-year contracts as provided by Amazon for their EC2 [30], in this paper we assume that all resources are reserved under a single contract for simplicity of the model.

An illustrative example

Additionally, CMB also collects the availability of VM instances of each VM type and their hourly on-demand prices to generate rpSLA. Based on these information, the CMB executes a heuristic for optimal on-demand provisioning. To do this, the CMB acts as a coordinator between different resource providers to decide upon how to allocate user demands from among the resource providers, considering their current availabilities and pricing information.

In case of user 2, all required 200 small VM instances are used from one resource provider rp3 for a fixed same price, i.e., 200 VMs from rp3 at the same price that has been quoted in the urSLA ($0.43/VM), whereas in case of 200 medium instances are used from two providers rp2 and rp3 at different prices that are slightly higher than what user quoted in SLA. Out of the 200 VMs, 150 VMs are used from rp2 at a price $0.46 per instance which is minimum among all providers available VMs within the CCM and rest 50 instances from rp3 at a bit higher rate than rp2 (i.e., $0.47/VM) as depicted in Table 3. In case of xLarge VM instances, the required 120 VMs are used from two providers at a lesser price that was quoted in the urSLA ($0.61/VM). i.e., out of 120 VMs, 100 VMs are used from rp1 at a price $0.60/ instance which is less than the user quoted price and the rest 20 instances have been used from rp2 with same price that the user quoted in urSLA. The CMB can allocate required 20 xLarge VMs to user2 from provider rp2 where as a minimum VM constraints specified in rpSLA is 50. The CMB can allocate required small amount of VMs to a user from a particular provider which is less than resource provider’s minimum constraint level using coordinated approach, So the rest of the required VMs of this type has to be allocated to other users within the CCM. In this, particular case, the next queued user (ur3) requires 200 VMs of same type which the CCM can provision but in case of non-coordinated approach the xLarge instances requested by ur2 cannot be provisioned during this time slot.

An optimal resource reservation approach

This section deals with an optimal reservation approach with the objective of finding the optimal number of reserved instances based on the present cloud provider resource, service status and available pricing models. It has been assumed that user service requests are processed batch-wise, i.e., the requests accepted within an interval of time are clubbed together to form a demand group and are considered for provisioning in cloud market. Suppose ‘m’ user requests are collected during a particular interval for provisioning resource reservation. These m = {ur₁, ur₂, ….ur_m} user requests have different SLAs. Moreover {ur₁ ⁿ¹, ur₂ ⁿ², ur₃ ⁿ³, ….ur_m ^nm} VM instances are required to serve the user requests of m = {ur₁, ur₂, ….ur_m} respectively.

In case the cloud users reserve some VM instances ahead of time, the cost model changes. Let, \( {C}_{\left({\mathrm{cp}}_i\right)}^{(f)} \) is the fixed reservation price offered by cloud provider CP_i for a single VM instance. Also let ur _i ⁿ be the i ^th user with n VM instance requests. Moreover, let fixed cost for allocating all urces of a user request from a single provider be \( u{r}_i^n*{C}_{\left({\mathrm{cp}}_i\right)}^{(f)} \) and additional hourly usage cost for the service be \( u{r}_i^n*{C}_{\left( t,{\mathrm{cp}}_i\right)}^{(r)} \). Thus, total cost of using resources from a single cloud provider is becomes \( u{r}_i^n*{C}_{\left({\mathrm{cp}}_i\right)}^{(f)}+{\sum}_{t=1}^T u{r}_i^n*{C}_{\left( t,{\mathrm{cp}}_i\right)}^{\left(\mathrm{r}\right)} \).

With these assumptions and using the aforementioned notations, the cost for using VM instances to be paid by an user can be expressed as follows:

In this section, we derive a heuristics for solving the optimal resource provisioning problem in polynomial time. It is possible to determine an optimal value for resource provisioning in a cooperative cloud market with S_rp number of resource providers and set of cloud users S_cu. If the demand vector of cloud users is available for stages or slots, t = 1, 2, 3,……t_p, it is possible to determine the optimal value for reservation. Although resources can be reserved by a user under multiple contracts like, 1 year or 3 year contracts provided by Amazon for their EC2 [54], in this paper we assume that all resources are reserved under a single contract for simplicity of the model. The discussion in this section is based on the assumption that duration of each stage or slot is 1 h, as is the norm with all prominent resource providers, including Amazon. The aggregate demand vector of each user for a predefined number of slots (typically monthly demand consisting of 30 × 24 = 720 slots) is thereafter sorted and using the heuristic presented in algorithm1 of [43] we can obtain an optimal number of VM resources to be reserved for every user. Details of this heuristic as in [43] is beyond the scope of this paper.

A coordinated resource provisioning approach for handling on-demand requests

Once the optimal reservation for every user demand vector is obtained in the first phase, a coordinated on-demand resource provisioning approach is employed. In this second phase, users’ VM instance wage costs as well as cloud providers’ service costs are optimized.

The following two algorithms present the resource reservation process under a single contract. First of all, every user requests are collected in a queue based on arrival time of requests and these are broadcasted to all the cloud providers in the CCM along with their specific SLAs agreed upon. Subsequently, a response is collected by the CMB. The scatter section of the pseudo code shown in algorithm 1 thus refers to the process of collecting all user demands by the CMB. Similarly the gather section in algorithm 1 refers to the VM instances that resource providers agree to provide based on CCM’s current demand.

Algorithm 2 presents a coordinated provisioning approach, where the CMB schedules the user requests among multiple cloud providers within the CCM. While provisioning a single user request among multiple providers, the market broker attempts to schedule the request with as less active datacenters as possible, a new datacenter is spawned only if remaining user requests exceeds a certain threshold value. This is motivated by individual resource providers’ cost reduction. In other words, this coordination mechanism takes care of the fact that all cloud providers maintain a minimal number of virtual machines (threshold) for starting a new datacenter and the threshold value may change depending upon the allocation of VM in datacenter. In our approach, a maxcount denotes the restriction that a single user request can be provisioned from a maximum of maxcount number of providers in order to reduce the intra-communication cost between the providers’ data centers. At the end of resource provisioning process, all the users’ un-provisioned VMs are inserted into a queue for re-provisioning.

Resource providers have data center with heterogeneous servers. The providers offering VMs at a lesser price per slot are considered as most cost effective providers. To enumerate the most effective providers, the CMB sorts the resource provider’s SLA based on their VM price, and assigns the VMs from most cost effective providers first and it then continues to allocate VM to the second most cost effective on the list, and so on, until all user requests are provisioned for or resource providers’ list is exhausted, in which case the CCM cannot accommodate all requests. For a cloud market with multiple VM type request, MCEPRF allocates a number VMs from more than one providers to one users and one user’s request can be provisioned with more than one providers.

Algorithm 2 presents the heuristic for arriving upon optimal hourly schedules and Table 7 shows a sample schedule only, where VMs are categorized into four groups based on the resource capacities required. The details of these algorithms are discussed hereafter.

The cloud market broker maintains a sorted list of resource providers with their SLA profiles. The resource provider vector is prepared based on their VM price offers. Where the most cost effective providers are placed on top of the resource providers list. Upon receiving VM request from cloud users, the cloud market broker assigns VMs to the users from the sorted rpSLA list from top to bottom. Once the required VMs allocated for provisioning then resource provider’s profile gets updated. As soon as the most cost effective resource provider’s VMs are saturated, they are moved from the list until they become non-saturated.

maxCount, an integral value, in algorithm 2 signifies that a users’ on-demand request for VMs can be fulfilled from different providers, but no more than maxCount providers. maxCount can vary, but for performance evaluation, we have used maxCount = 3. At stage ‘t’, if user resource demand is more than the reserved VM instances then rest of VM instances are provisioned under on-demand category. Algorithm 2 does necessary checks on the minimum number of VMs that a provider wish to furnish to a user. If a provider’s available VM resources is less than user requiredVms then the specific provider’s available VMs are marked for allocation with status some setting mechanism. rpCount is used to count the number of resource providers for satisfying the user requirements and this is cross checked with maxCount.

Illustrative example with CRRP approach

Experimental setup

In an effort to study the performance of CCM, the proposed provisioning algorithms have been simulated for performance evaluation. The algorithms for coordinated resource provisioning for CCM have been implemented using java SDK v6. The hardware platform on which these proposed algorithms have been executed include a sixteen-node cluster of Dell Power Edge R610 blades with Dual Intel Xeon Quad-Core E5620 (2.93 GHz Processor).

Setting up the simulation parameters

For conducting the experiments, certain simulation parameters have been assumed. Number of cloud users has been taken to be 100 and the number of resource providers in the CCM has been taken to be 10 (hereafter designated as rp1 through rp10). The resource providers’ varying prices have been taken from Amazon’s EC2 instances’ pricing, but for different locations [54]. The VM demands of users had to be constructed since cloud providers do not make them available publicly. Some researchers [43] have employed the huji parallel workload archive [55] data to represent cloud user demands. These data sets have been gathered from HPC systems that are normally operated as per a batch job model with node specific requests. We had to account for such information while preparing user demand data set. However, service providers do furnish distribution and relevant parameters pertaining to user demands from which workload data can be reconstructed, such as [45]. We also observed that considerable fraction of hourly demand request in [55] data set follow similar distribution parameters as that of [53]. Thus the synthetic demand data set for 1000 users was prepared using both, so that heterogeneity of demand range can be captured. We used a Monte Carlo Simulation for estimating the demand of these 1000 users at every slot thereafter. We repeat this for 25 times by varying the mean value of VM demands so that we test effect of aggregate demand variations on cost characteristics.

Simulation experiments conducted and results

With the slot-wise aggregate VM demands of users at hand, we move on to discuss experiments that can capture the efficacy of the proposed two-phased coordinated resource provisioning approach with its non-coordinated counterpart. The non-coordinated resource provisioning is the case where every cloud provider caters to the requests of its own users. It basically executes a portion of algorithm 1, the section between the two gather sections. This is intuitive, since in a non-coordinated scenario, user demands are kept within providers. Since the data pertaining to users demands are estimated using statistical techniques that are stochastic, hence, once the demands are generated, the demand vector for users are used for both the coordinated and non-coordinated counterparts. Besides, for the simulation purposes, the number of cloud providers have been limited to 10, so we do not use statistical methods to further complicate scenarios, rather consider a fixed set of combinations of users for each of the 10 cloud providers.

Accordingly simulations were carried out to study the effect of the proposed MCEPRF provisioning algorithms on user cost and resource provider costs with varying aggregate resource demands. The results of the proposed heuristics for coordinated resource provisioning in CCM have been compared with its non-coordinated counterpart. To this end, we have carried out a number of simulation experiments as detailed hereafter.

Experiment 1

A set of simulations were carried out to test the cost effectiveness of the proposed MCEPRF algorithm using VM demand data obtained in previous section and pricing data of Amazon [54]. It has to be noted that user costs have two components that are aggregated at the two distinct stages of the two-stage resource provisioning scheme presented in this paper, namely the reservation usage cost for the first stage and the on-demand usage cost for the second stage. These two has been depicted as Fig. 4a and b respectively.

Experiment 2

In order to test the efficacy of the coordinated provisioning in terms of uniformity of resource usage (i.e., within less number of providers in order to minimize the achieve the green computing), simulations were carried out to assess the percentage of computational resources usage for a variety of computational loads, taken in terms of VMs per slot. Figures 5a through c show that the variation of resource usage is more uniformly distributed in the coordinated provisioning approach when compared to its non-coordinated counterpart. Since the former approach reduces the number of active servers and attempts to involve a moderate percentage of servers to remain active throughout, this finding promises extended ramifications in terms of cost and energy savings.

Experiment 3

Another experiment has been carried out to test the effect of varying user demands on SLA violations. To do so, slot-wise varying demand data has been taken and user requests that could not be provisioned in terms of meeting SLA have been observed. Such cancelled requests are queued up for provisioning in the next slot. Figure 6 depicts a comparison of percentage of user requests cancelled and the proposed coordinated provisioning approach performs better from this perspective.

Experiment 4

Time taken to arrive at provisioning decision for every slot for the two provisioning approaches have been compared and the obtained results have been depicted in Fig. 7. It can be observed that the decision time for the coordinated approach is comparable to its non-coordinated counterpart. However, we have not studied the effect of increase in cloud provider in the market.

Experiment 5

An experiment has been carried out to test the effect of varying Max rpCount on SLA violations. To do so, slot-wise constant providers’ resource details data has been taken and user requests that could not be provisioned in terms of meeting SLA have been observed. Such cancelled requests are queued up for provisioning in the next slot. Figure 8 depicts a comparison of percentage of user requests cancelled gradually decreases in increase of rpCount, whereas execution time increases. In proposed coordinated provisioning approach, with maximum rpCount three and four gives an optimal performance in terms of percentage of users’ tasks cancellation and tasks execution time.

Performance analysis