A novel coordinated resource provisioning approach for cooperative cloud market
- K Hemant Kumar Reddy1,
- Geetika Mudali1 and
- Diptendu Sinha Roy2Email author
https://doi.org/10.1186/s13677-017-0078-z
© The Author(s). 2017
Received: 3 July 2016
Accepted: 6 March 2017
Published: 31 March 2017
Abstract
Cloud computing has been the enabling technology for shifting mass scale computation and storage requirements from individually owned clients towards an on-demand and utility styled alternative that provides many services. However, cost of maintaining datacenters, keeping the environmental ramifications of data centers at check, providing affordable computation alternative to users still needs to be addressed in a wholesome manner. One of the most exciting and recent research areas in cloud computing has been cloud federations that can mitigate the aforesaid problems. The past decade has seen immense efforts towards interoperability of clouds leading to realistic cloud federations. Motivated by these advancements and equipped with available technologies, this paper presents a detailed account of a cooperative cloud market. It delineates trading mechanisms of such cloud markets, extent of coordination among market players with illustrative examples. It also presents a novel two-phase coordinated resource reservation and provisioning (CRRP) approach that allocates cloud resources to users to meet the goal of minimizing users’ cost. To that end, this paper proposes a novel Most Cost Effective Providers’ Resources First (MCEPRF) algorithm. The efficiency of the proposed algorithm has been tested using synthetic data and the simulation results presented herein demonstrate the superiority of the proposed approach over its non-coordinated counterparts.
Keywords
Background
Cloud computing, with its underlying technologies, has been widely hyped as the enabling technology that could transform the vision of computing as an omnipresent utility into reality [1]. It has brought about a paradigm shift in the computing world by bringing about on-demand access to computing resources by leveraging enormous scale server-side computing in the form of massive datacenters. This trend however, has been driven by compelling reasons, like, ease of resources administration [2], access ubiquity [3], and economical scaling capabilities [3–5]; and not so much owing to user requirements. However, cost of maintaining datacenters, keeping the environmental ramifications of data centers at check, providing affordable computation alternative to users still needs to be addressed in a wholesome manner.
It has been an proven fact that cost of maintaining huge datacenters is remarkably high [6]. There have been a number of research articles, such as [3, 7, 8], that studied cost models of datacenters and concluded that server cost and datacenter’s power infrastructure together account for more than three fourths of datacenter’s total cost [9]. Moreover, both costs are decided by system capacity in terms of the number of servers [9].
Cloud providers ideally want to size data center capacity exactly to meet the demands, which is fairly impossible since user demands drastically fluctuate over time. Over sized data centers lead to increased cost of maintaining data centers, whereas undersized datacenters can hurt providers badly owing to possible unfulfilled service agreements with users.
As a natural extension, the concept of inter-cloud was conceived as a model that could support resource sharing and workload transfer between multiple clouds by means of coordination mechanism among providers to meet service-level-agreements (SLAs) of consumers [10]. This new approach, has been referred to as Inter Cloud by Buyya [11] and Bernstein [12]; Cloud Fusion [13]; sky computing [14, 15]; cloud-of-clouds [16]; and so forth. Based on which party initiates the collaboration, inter-clouds have been referred to as federated clouds [17–19] when collaborations are initiated voluntarily from cloud providers; or as multi-cloud when clients share the responsibility of resource management and scheduling [18]. Of course, these names do suffer from terminological ambiguity. As a natural consequence of the multifarious names for this collaborative cloud model, the term inter-cloud has been used in literature by authors to mean a broad variety of related things. The way inter-clouds are viewed and defined by standard bodies like the European Commission [20, 21], the European network for Information Security Agency (ENISA) [22], the NIST [23, 24] are related. The Cloud Computing Use Case Discussion Group [25], though adopts NIST models of brokering among multiple cloud providers, yet they do not specify guidelines regarding its implementation [26].
We argue that the past decade has seen remarkable research efforts for interoperability and standardization to make the cloud computing arena a truly global and market-oriented one. Since demand for inter-cloud is on the rise and technologies are available [27], research efforts need concentrate on effective mechanisms for resource brokering on such cloud markets at the pooled infrastructure level of multi clouds.
In this paper, we present a coordinated resource provisioning approach that assumes that mechanisms for coordination and trading by cooperation among cloud providers exist. We present the assumptions, trading mechanisms for such a market place and finally carry out experiments to demonstrate how such a cooperative market can be useful to attain higher benefits. The heart of this cooperative market is a Cloud Market Broker (CMB), a functional entity that coordinates the operations of various cloud providers, their workloads and users’ demand status by means of a multi-agent mechanism. The CMB is responsible for executing a coordinated resource provisioning algorithm that intelligently exploits workload heterogeneity and also uses some shared information among cloud service providers for cost-effective provisioning for the entire market.
The main contribution of this paper is a two phase coordinated resource reservation and provisioning (CRRP) strategy in a cloud market. In the first phase, the CMB reserves a near optimal number of resource instances for every cloud user by a heuristic that arrives at a solution at minimal user cost in polynomial time. In the second phase, a heuristic algorithm, namely Most Cost Effective Providers’ Resources First (MCEPRF), is presented that the CMB needs to execute to minimize users' usage cost and also minimizes the cost of provisioning of on-demand instances for cloud resource providers. The uniqueness of this work is that here we present a provisioning mechanism that can be deployed on top of existing inter-cloud deployment models and architectures already discussed previously.
The remainder of this paper is organized as follows. A brief account of the related research in this field has been presented in the Related work section. The section titled An overview of cooperative cloud market (CCM) explains the cooperative cloud market along with its functional components, its modality of functioning and contextual depiction. The functions of the CMB components have also been emphasized in that section. In Coordinated resource reservation and provisioning (CRRP) approach section, heuristics for the two-phase CRRP strategy has been presented and a detailed description of the MCEPRF has also been depicted. The Implementation and performance evaluation section presents the experiments carried out and presents the results. The conclusions are summarized in the Conclusion section.
Related work
The research presented in this paper builds on multifarious aspects of mathematical models and technologies related to cloud computing and its vision and thus in order to leverage the understanding of the related works, this section is presented in four sub-sections.
Cloud market
The idea that computers (and hence computing) can be provided as a utility, like electricity or telephony, dates back to the late 1970s [28]. Technological advancements over the next three decades or so in IC fabrication industry, multi-core processor architectures, and networked computing infrastructures gradually brought this grand vision closer to reality. These trends have enabled a vibrant IT industry that has shifted computing paradigms from traditional web-based computing to subsequent data-center based utility computing, grid and cloud computing, infusing numerous attributes and capabilities, like scalability, access ubiquity, autonomy of deployment and so on. The idea of having a market-oriented cloud dates back to almost a decade. Buyya et al. [29] were the first to propose the vision of a market oriented cloud and provided a high level architecture for supporting market-oriented resource management for trading services among cloud providers and consumers, bound by previously agreed upon agreements to ensure QOS based mechanisms [29]. This work invoked several research efforts to have apt scheduling policies for such market- oriented clouds [30, 31] and suitable toolkits for leveraging such market-oriented clouds [32].
The closest literature with respect to our work is presented by Haifei Li et al. [33], where they envision a marketplace model for cloud, namely, CCMarketplace. In CCMarketplace, computing resources of any kind, be it infrastructure (processors or memory), platform (IDEs on customized operating systems), or software (development tools like SDEs, databases, middle wares, etc.); become tradable commodities and the marketplace entities, like buyers and sellers of such commodities trade such resources to maximize their own profit [33].
However, in this paper a cooperative cloud market is dealt with for the purpose of reducing the cost of resource provisioning. The rationale behind this is the fact that the profitability and credibility of a cloud provider increases with the resource pool size it owns [2]. Since cloud users and their demands stochastically vary over time, thus to be able to tackle all such demand variations and still provision resources, it is pragmatic to have a cooperative cloud market, where the players share a common CMB entity for mediating provisioning requests. Such a concept also reduces the risk of failures to provide agreed services.
Resource provisioning
Virtualization has been the chief enabling technology that has led to successful realization of cloud computing [34]. Using virtualization, applications share underlying cloud resources by means of isolated Virtual Machines (VMs), such that each VM is endowed with cloud resources. Resource provisioning refers to the process of allocating succinct amount of resources to VMs [35] matched by their respective workloads. Usually cloud resources are provisioned dynamically in order to match workload fluctuations [36, 37] based on several criteria and are thereafter consolidated to a set of physical servers. Meng et al. [38] presents a joint criteria based provisioning, which is referred to as VM multiplexing. Zhan et al. [39] presents the idea of cooperative resource provisioning taking into account the heterogeneity of workloads. All the aforesaid articles deal with provisioning on a single cloud infrastructure. The research presented in this paper considers coordinated resource provisioning among a number of cloud providers via information sharing with a CMB entity by means of appropriate agents.
Pricing policies and cost optimization
H. Wang et al. [40] presented a very interesting interplay between distributed systems and economics related to pricing by decoupling the users from their specific cloud service providers. Providing multiple pricing options has enduring ramifications, both for the service providers as well as the users. Mazzucco et al. [41] have studied the effect of having flexible pricing options on revenue collection of cloud service providers. Menglan et al. [42] presents a case where the effect of reserved and on-demand instances are studied for minimizing user budget for deadline constrained jobs. They also present an equivalent formulation of execution time minimization for a budget constrained job. S. Khatua et al. [43] formulated the payable price for user of cloud services as an integer programming problem considering reserved and on-demand instances. They also presented a heuristic that finds the near optimal cost in polynomial time. Qian et al. [44] presents a stochastic integer programming model to optimize the cost of SLA-aware resource scheduling in the cloud. S. Chaisiri et al. [45] have considered uncertainty in terms of user demands over time and price uncertainties to arrive upon an optimum resource provisioning cost by means of stochastic linear programming. The scope of this paper is wider in the sense that the objective is to conduct optimal provisioning for a cloud market comprising of many cloud users, multiple cloud service providers as well as the two pricing policies, namely advanced reservation and on-demand provisioning. In order to do that, this paper employs a CMB entity that employs a multi-agent system among the cloud providers, such that the market players coordinate amongst themselves to form a cooperative cloud market.
Inter-cloud initiatives
In the past few years, there has been significant research attention targeted towards inter-clouds ranging from their interoperability issues to setting up of standard APIs and interfaces. Advantages of such approaches include lock-in situations over diverse geographical and legal demographics, improved reliability of applications and services, vendor lock-in avoidance [27]. Besides, there are other attractive incentives for cloud providers and users alike, such as option for on-demand workload expansion, better SLA offerings for customers and so on. There has been a number of projects receiving significant funding and attention, including Inter-Cloud [46], Contrail [47], Reservoir [48], Open Cirrus [49], OPTIMIS [50, 18], mOSAIC [51], Bonfire [52] etc. Grozev and Buyya [27] provide an extensive survey of these projects with a detailed taxonomical account from architectural and brokering perspectives. Another excellent review on this topic can be found in [26] wherein the authors delve on how standardization organizations view this domain and present the slightly varying nuances and goals of the aforementioned cloud projects.
In order that we can put our proposed cloud market in perspective, we observe that among the different projects the CMB proposed in this paper closely follows the third-party brokering model provided by OPTIMIS [18]. However, while [18] focuses on deployment issues of the third-party brokering model among other deployment model and architectures, our goal is to provide an infrastructure-level brokering model atop OPTIMIS provided services.
An overview of cooperative cloud market (CCM)
The idea of market-oriented cloud for trading computing infrastructures was envisioned by Buyya et al. [29]. High level market oriented architecture has been presented in [29, 53] whose major functional components include an admission control unit that interprets user requests, the QoS demands, tallies it with the available resources and finally accepts or rejects the user requests.
Many such cloud providers may enter into a marketplace under certain predefined agreements to allow trading of computing resources. This paper advocates the case of a cooperative cloud market, where the component cloud providers share information amongst themselves regarding their supply-demand details. To this end, a CMB entity has been proposed that is responsible for coordinating the information received from participating cloud service providers (CSPs) and service consumers (SCs). The service providers and consumers so mentioned constitute a multi-agent system along with the CMB. As mentioned earlier, CMB can be viewed as the third-party broker model, much like OPTIMIS as presented in Fig. 5 of [18]. The following subsections present a detailed account of motivations and characteristics of such a cooperative cloud market.
Background of the proposed cooperative cloud market (CCM)
Schematic Representation of the cooperative cloud market
The CMB entity acts as the intermediary. All cloud providers share required VM demand and QoS related information with the CMB via a multi-agent system (MAS). Details of this multi-agent system (MAS) and the interaction mechanisms in the proposed CCM have been presented in An illustrative example section with an demonstrative example.
High Level Architecture of the Cooperative Cloud Market
The responsibility of resource provisioning for the CCM, as shown in Fig. 2, rests on the CMB, which executes appropriate algorithms to achieve its goal. Coordinated resource reservation and provisioning (CRRP) approach section presents a coordinated provisioning scheme for the proposed CCM that takes into account the variations in workload types as well as pricing schemes offered by different provider to achieve cost benefits for the entire market as a whole.
The users’ hour-wise demands for cloud resources are assumed to be known to the CMB. Also the prices of cloud resources pertaining to different cloud resource providers are assumed to be known ahead of time (i.e., provided by the providers before the provision period). In other words, uncertainties pertaining to users’ demands and prices offered by cloud providers have not been considered in the coordinated resource provisioning model for CCM.
Although resources can be reserved by a user under multiple contracts like, 1-year or 3-year contracts as provided by Amazon for their EC2 [30], in this paper we assume that all resources are reserved under a single contract for simplicity of the model.
An illustrative example
A sample of User demands and VM SLA reserved in advance
Urid | S: Small VM | M: Medium VM | L: Large VM | xL:xLargeVM | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
#VMReq | #Reserved | rp | @Price | #VMReq | #Reserved | rp | @Price | #VMReq | #Reserved | rp | @Price | #VMReq | #Reserved | rp | @Price | |
ur1 | 700 | 300 | rp1 | $20.24 | 700 | 350 | rp2 | $28.4 | 600 | 400 | rp2 | $34.5 | 380 | 280 | rp1 | $40.2 |
ur2 | 500 | 300 | rp2 | $20.24 | 460 | 260 | rp3 | $28.2 | 560 | 340 | rp1 | $34.2 | 400 | 280 | rp3 | $40.2 |
ur3 | 530 | 150 | rp2 | $20.24 | 550 | 240 | rp1 | $28.5 | 450 | 270 | rp1 | $34.8 | 400 | 250 | rp2 | $40.4 |
Sample of on-demand user requests collected by the CMB (urSLA)
Urid | S/Price | M/Price | L/Price | xL/Price |
---|---|---|---|---|
ur1 | 400/$0.42 | 350/$0.44 | 200/$0.48 | 100/$0.59 |
ur2 | 200/$0.43 | 200/$0.45 | 220/$0.47 | 120/$0.61 |
ur3 | 380/$0.42 | 310/$0.45 | 380/$0.48 | 250/$0.62 |
Additionally, CMB also collects the availability of VM instances of each VM type and their hourly on-demand prices to generate rpSLA. Based on these information, the CMB executes a heuristic for optimal on-demand provisioning. To do this, the CMB acts as a coordinator between different resource providers to decide upon how to allocate user demands from among the resource providers, considering their current availabilities and pricing information.
A sample output of resource allocation done by the CMB
urId | S : Small VM | M: Medium VM | L: Large VM | xL: xLarge VM | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
#vms | RP | urprice | rpPrice | #vms | RP | urprice | rpPrice | #vms | RP | urprice | rpPrice | #vms | RP | urprice | rpPrice | |
ur1 | 200 | rp1 | $0.42 | $0.42 | 300 | rp1 | $0.44 | $0.45 | 200 | rp2 | $0.48 | $0.49 | 100 | rp1 | $0.59 | $0.60 |
ur1 | 200 | rp2 | $0.42 | $0.42 | 50 | rp2 | $0.44 | $0.46 | ||||||||
ur2 | 200 | rp3 | $0.43 | $0.43 | 150 | rp2 | $0.45 | $0.46 | 200 | rp2 | $0.47 | $0.49 | 100 | rp1 | $0.61 | $0.60 |
ur2 | 50 | rp3 | $0.45 | $0.47 | 20 | rp1 | $0.47 | $0.50 | 20 | rp2 | $0.61 | $0.61 | ||||
ur3 | 100 | Rp3 | $0.42 | $0.43 | 310 | Rp3 | $0.45 | $0.47 | 280 | rp1 | $0.48 | $0.50 | 80 | rp2 | $0.62 | $0.61 |
ur3 | 180 | Rp4 | $0.42 | $0.44 | 100 | rp3 | $0.48 | $0.51 | 170 | rp3 | $0.62 | $0.61 |
In case of user 2, all required 200 small VM instances are used from one resource provider rp3 for a fixed same price, i.e., 200 VMs from rp3 at the same price that has been quoted in the urSLA ($0.43/VM), whereas in case of 200 medium instances are used from two providers rp2 and rp3 at different prices that are slightly higher than what user quoted in SLA. Out of the 200 VMs, 150 VMs are used from rp2 at a price $0.46 per instance which is minimum among all providers available VMs within the CCM and rest 50 instances from rp3 at a bit higher rate than rp2 (i.e., $0.47/VM) as depicted in Table 3. In case of xLarge VM instances, the required 120 VMs are used from two providers at a lesser price that was quoted in the urSLA ($0.61/VM). i.e., out of 120 VMs, 100 VMs are used from rp1 at a price $0.60/ instance which is less than the user quoted price and the rest 20 instances have been used from rp2 with same price that the user quoted in urSLA. The CMB can allocate required 20 xLarge VMs to user2 from provider rp2 where as a minimum VM constraints specified in rpSLA is 50. The CMB can allocate required small amount of VMs to a user from a particular provider which is less than resource provider’s minimum constraint level using coordinated approach, So the rest of the required VMs of this type has to be allocated to other users within the CCM. In this, particular case, the next queued user (ur3) requires 200 VMs of same type which the CCM can provision but in case of non-coordinated approach the xLarge instances requested by ur2 cannot be provisioned during this time slot.
A sample output of resource allocation done by individual cloud users
urId | S : Small VM | M: Medium VM | L: Large VM | xL: xLarge VM | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
#vms | RP | urprice | rpPrice | #vms | RP | urprice | rpPrice | #vms | RP | urprice | rpPrice | #vms | RP | urprice | rpPrice | |
Ur1 | 400 | rp4 | $0.42 | $0.45 | 350 | rp3 | $0.44 | $0.47 | 200 | rp2 | $0.48 | $0.49 | 100 | rp1 | $0.59 | $0.60 |
Ur2 | 200 | rp1 | $0.40 | $0.45 | 200 | rp2 | $0.45 | $0.47 | 220 | rp4 | $0.47 | $0.50 | 120 | rp3 | $0.61 | $0.61 |
Ur3 | 380 | not provisioned | $0.42 | not provisioned | 310 | rp3 | $0.45 | $0.47 | 380 | rp2 | $0.48 | $0.51 | 250 | $0.62 | not provisioned |
A sample of provisioning cost of CCM vis-à-vis its non-coordinated counterpart
urId | S : Small VM total provisioning cost | M: Medium VM total provisioning cost | L: Large VM total provisioning cost | xL: xLarge VM total provisioning cost | ||||
---|---|---|---|---|---|---|---|---|
CCM | Non-CCM | CCM | Non-CCM | CCM | Non-CCM | CCM | Non-CCM | |
Ur1 | 4407.2 | 4695.2 | 4375.3 | 4531.3 | 2752.8 | 2752.8 | 2097.44 | 2097.44 |
Ur2 | 2439.2 | 2535.2 | 2651.6 | 2687.6 | 2992.8 | 3040.8 | 2390.24 | 2414.24 |
Ur3 | 3120.4 | Not provisioned | 3897.6 | 3897.6 | 3956.4 | 4943.6 | 4236.6 | Not provisioned |
Coordinated resource reservation and provisioning (CRRP) approach
Two-Phase Coordinated Resource Reservation and Provisioning (CRRP) Model
In the second phase the Cloud Market Broker uses the individual cloud users’ demand data along with SLAs with their respective resource providers to arrive at optimal on-demand resource allocation. Detailed procedure of this phase has been presented in A coordinated resource provisioning approach for handling on-demand requests section. In the following section we formulate the coordinated resource provisioning in CCM using a heuristic coordinated approach.
An optimal resource reservation approach
This section deals with an optimal reservation approach with the objective of finding the optimal number of reserved instances based on the present cloud provider resource, service status and available pricing models. It has been assumed that user service requests are processed batch-wise, i.e., the requests accepted within an interval of time are clubbed together to form a demand group and are considered for provisioning in cloud market. Suppose ‘m’ user requests are collected during a particular interval for provisioning resource reservation. These m = {ur1, ur2, ….urm} user requests have different SLAs. Moreover {ur1 n1, ur2 n2, ur3 n3, ….urm nm} VM instances are required to serve the user requests of m = {ur1, ur2, ….urm} respectively.
Notation summary
P | Number of reservation contracts offered by cloud market |
T/t | Total number of stages of collected demand vector |
h | Duration of each stages, ideally in hour basis |
VmDt | VM instance Demand vector for a stage ‘t’ |
C(f) | Upfront or one time reservation cost of reserved instance |
C(r) | Resource usage cost for reserved VM instances per hour |
C(o) | Resource usage cost for on-demand VM instances per hour |
vm(r) | Number of reserved VM instances |
vmt(r) | Number of reserved VM instances for a stage ‘t’ |
vm(u) | Total number of usage VM instances |
vm(o) | Number of on-demand VM instances |
vm(D) | Cloud user request demand vector for a duration ‘T’ |
vm(d) | Cloud user request demand vector for a duration ‘t’ |
Totc | Total cost corresponding to demand vector ‘Vm(D)’ |
vmDsort | Sorted cloud user demand vector for a duration ‘T’ |
TotDc | Total cost of demand vector |
TotDjsort | Total cost of sorted demand vector |
Tc | Total cost of usage resources of all cloud users |
R-cost | Reserve VM instance cost for entire contract period |
U-cost | Usage VM instance cost per slot |
O-cost | Ondemand VM instance cost per slot |
In case the cloud users reserve some VM instances ahead of time, the cost model changes. Let, \( {C}_{\left({\mathrm{cp}}_i\right)}^{(f)} \) is the fixed reservation price offered by cloud provider CPi for a single VM instance. Also let ur i n be the i th user with n VM instance requests. Moreover, let fixed cost for allocating all urces of a user request from a single provider be \( u{r}_i^n*{C}_{\left({\mathrm{cp}}_i\right)}^{(f)} \) and additional hourly usage cost for the service be \( u{r}_i^n*{C}_{\left( t,{\mathrm{cp}}_i\right)}^{(r)} \). Thus, total cost of using resources from a single cloud provider is becomes \( u{r}_i^n*{C}_{\left({\mathrm{cp}}_i\right)}^{(f)}+{\sum}_{t=1}^T u{r}_i^n*{C}_{\left( t,{\mathrm{cp}}_i\right)}^{\left(\mathrm{r}\right)} \).
- i.
Number of reserved VM instances and number of VM instances launched from reserved instances as well as on-demand instances at any stage are non-negative integers.
- ii.
Cloud user can place one pricing policy request as one request, if a user is interested to use different pricing model, then the user need to put multiple requests for multiple pricing schemes.
With these assumptions and using the aforementioned notations, the cost for using VM instances to be paid by an user can be expressed as follows:
- 1.
\( v{m}_{\left(\mathrm{cp}\right)}^{\left(\mathrm{r}\right)}, v{m}_{\left({\mathrm{cp}}_i\right)}^{\left(\mathrm{o}\right)}, v{m}_{\left({\mathrm{cp}}_i\right)}^{\left(\mathrm{D}\right)} \) > = 0 and is an integer, signifying constraint i mentioned earlier.
- 2.
\( {C}_{\left({\mathrm{cp}}_i\right)}^{\left(\mathrm{o}\right)}+{\displaystyle {\sum}_{t=1}^T{C}^h>= v{m}^{(D)}\kern0.5em \forall t=1,2,\dots \dots T} \), signifying constraint ii mentioned earlier.
In this section, we derive a heuristics for solving the optimal resource provisioning problem in polynomial time. It is possible to determine an optimal value for resource provisioning in a cooperative cloud market with Srp number of resource providers and set of cloud users Scu. If the demand vector of cloud users is available for stages or slots, t = 1, 2, 3,……tp, it is possible to determine the optimal value for reservation. Although resources can be reserved by a user under multiple contracts like, 1 year or 3 year contracts provided by Amazon for their EC2 [54], in this paper we assume that all resources are reserved under a single contract for simplicity of the model. The discussion in this section is based on the assumption that duration of each stage or slot is 1 h, as is the norm with all prominent resource providers, including Amazon. The aggregate demand vector of each user for a predefined number of slots (typically monthly demand consisting of 30 × 24 = 720 slots) is thereafter sorted and using the heuristic presented in algorithm1 of [43] we can obtain an optimal number of VM resources to be reserved for every user. Details of this heuristic as in [43] is beyond the scope of this paper.
A coordinated resource provisioning approach for handling on-demand requests
Once the optimal reservation for every user demand vector is obtained in the first phase, a coordinated on-demand resource provisioning approach is employed. In this second phase, users’ VM instance wage costs as well as cloud providers’ service costs are optimized.
The following two algorithms present the resource reservation process under a single contract. First of all, every user requests are collected in a queue based on arrival time of requests and these are broadcasted to all the cloud providers in the CCM along with their specific SLAs agreed upon. Subsequently, a response is collected by the CMB. The scatter section of the pseudo code shown in algorithm 1 thus refers to the process of collecting all user demands by the CMB. Similarly the gather section in algorithm 1 refers to the VM instances that resource providers agree to provide based on CCM’s current demand.
Availability and price information of resource providers collected by the CMB (rpSLA)
S : Small VM | M: Medium VM | L: Large VM | xL: xLarge VM | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
rp | #VM | Available | min | VMcost | rp | #VM | Available | min | VMcost | rp | #VM | Available | min | VMcost | rp | #VM | Available | min | VMcost |
rp1 | 1000 | 200 | 50 | $0.42 | rp1 | 800 | 300 | 50 | $0.45 | rp2 | 800 | 400 | 100 | $0.49 | rp1 | 600 | 200 | 50 | $0.60 |
rp2 | 1000 | 200 | 50 | $0.43 | rp2 | 900 | 200 | 50 | $0.46 | rp1 | 1200 | 300 | 50 | $0.50 | rp2 | 800 | 100 | 50 | $0.61 |
rp3 | 1200 | 300 | 50 | $0.43 | rp3 | 800 | 400 | 100 | $0.47 | rp3 | 1000 | 200 | 50 | $0.51 | rp3 | 900 | 200 | 50 | $0.61 |
rp4 | 900 | 410 | 50 | $0.45 | rp4 | 1000 | 200 | 50 | $0.47 | rp4 | 900 | 400 | 100 | $0.51 | rp4 | 800 | 200 | 50 | $0.62 |
Algorithm 2 presents a coordinated provisioning approach, where the CMB schedules the user requests among multiple cloud providers within the CCM. While provisioning a single user request among multiple providers, the market broker attempts to schedule the request with as less active datacenters as possible, a new datacenter is spawned only if remaining user requests exceeds a certain threshold value. This is motivated by individual resource providers’ cost reduction. In other words, this coordination mechanism takes care of the fact that all cloud providers maintain a minimal number of virtual machines (threshold) for starting a new datacenter and the threshold value may change depending upon the allocation of VM in datacenter. In our approach, a maxcount denotes the restriction that a single user request can be provisioned from a maximum of maxcount number of providers in order to reduce the intra-communication cost between the providers’ data centers. At the end of resource provisioning process, all the users’ un-provisioned VMs are inserted into a queue for re-provisioning.
Algorithm 2 presents the heuristic for arriving upon optimal hourly schedules and Table 7 shows a sample schedule only, where VMs are categorized into four groups based on the resource capacities required. The details of these algorithms are discussed hereafter.
The cloud market broker maintains a sorted list of resource providers with their SLA profiles. The resource provider vector is prepared based on their VM price offers. Where the most cost effective providers are placed on top of the resource providers list. Upon receiving VM request from cloud users, the cloud market broker assigns VMs to the users from the sorted rpSLA list from top to bottom. Once the required VMs allocated for provisioning then resource provider’s profile gets updated. As soon as the most cost effective resource provider’s VMs are saturated, they are moved from the list until they become non-saturated.
Illustrative example with CRRP approach
A sample of users’ total resource utilization cost with the coordinated provisioning approach
urId | S : Small VM | M: Medium VM | L: Large VM | xL: xLarge VM | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
R-cost | u-cost | o-cost | Total | R-cost | u-cost | o-cost | Total | R-cost | u-cost | o-cost | Total | R-cost | u-cost | o-cost | Total | |
Ur1 | 202.4 | 172.8 | 4032 | 4407.2 | 331.3 | 252 | 3792 | 4375.3 | 228 | 172.8 | 2352 | 2752.8 | 375.2 | 282.24 | 1440 | 2097.44 |
Ur2 | 202.4 | 172.8 | 2064 | 2439.2 | 244.4 | 187.2 | 2220 | 2651.6 | 228 | 172.8 | 2592 | 2992.8 | 375.2 | 282.24 | 1732.8 | 2390.24 |
Ur3 | 101.2 | 86.4 | 2932.8 | 3120.4 | 228 | 172.8 | 3496.8 | 3897.6 | 313.2 | 259.2 | 3384 | 3956.4 | 336.6 | 240 | 3660 | 4236.6 |
A sample of users’ total resource utilization cost with the non-coordinated provisioning approach
urId | S : Small VM | M: Medium VM | L: Large VM | xL: xLarge VM | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
R-cost | u-cost | o-cost | Total | R-cost | u-cost | o-cost | Total | R-cost | u-cost | o-cost | Total | R-cost | u-cost | o-cost | Total | |
Ur1 | 202.4 | 172.8 | 4320 | 4695.2 | 331.3 | 252 | 3948 | 4531.3 | 228 | 172.8 | 2352 | 2752.8 | 375.2 | 282.24 | 1440 | 2097.44 |
Ur2 | 202.4 | 172.8 | 2160 | 2535.2 | 244.4 | 187.2 | 2256 | 2687.6 | 228 | 172.8 | 2640 | 3040.8 | 375.2 | 282.24 | 1756.8 | 2414.24 |
Ur3 | 101.2 | 86.4 | Not provisioned | 228 | 172.8 | 3496.8 | 3897.6 | 313.2 | 259.2 | 4651.2 | 4943.6 | 336.6 | 240 | Not provisioned |
Implementation and performance evaluation
Experimental setup
In an effort to study the performance of CCM, the proposed provisioning algorithms have been simulated for performance evaluation. The algorithms for coordinated resource provisioning for CCM have been implemented using java SDK v6. The hardware platform on which these proposed algorithms have been executed include a sixteen-node cluster of Dell Power Edge R610 blades with Dual Intel Xeon Quad-Core E5620 (2.93 GHz Processor).
Setting up the simulation parameters
For conducting the experiments, certain simulation parameters have been assumed. Number of cloud users has been taken to be 100 and the number of resource providers in the CCM has been taken to be 10 (hereafter designated as rp1 through rp10). The resource providers’ varying prices have been taken from Amazon’s EC2 instances’ pricing, but for different locations [54]. The VM demands of users had to be constructed since cloud providers do not make them available publicly. Some researchers [43] have employed the huji parallel workload archive [55] data to represent cloud user demands. These data sets have been gathered from HPC systems that are normally operated as per a batch job model with node specific requests. We had to account for such information while preparing user demand data set. However, service providers do furnish distribution and relevant parameters pertaining to user demands from which workload data can be reconstructed, such as [45]. We also observed that considerable fraction of hourly demand request in [55] data set follow similar distribution parameters as that of [53]. Thus the synthetic demand data set for 1000 users was prepared using both, so that heterogeneity of demand range can be captured. We used a Monte Carlo Simulation for estimating the demand of these 1000 users at every slot thereafter. We repeat this for 25 times by varying the mean value of VM demands so that we test effect of aggregate demand variations on cost characteristics.
Simulation experiments conducted and results
With the slot-wise aggregate VM demands of users at hand, we move on to discuss experiments that can capture the efficacy of the proposed two-phased coordinated resource provisioning approach with its non-coordinated counterpart. The non-coordinated resource provisioning is the case where every cloud provider caters to the requests of its own users. It basically executes a portion of algorithm 1, the section between the two gather sections. This is intuitive, since in a non-coordinated scenario, user demands are kept within providers. Since the data pertaining to users demands are estimated using statistical techniques that are stochastic, hence, once the demands are generated, the demand vector for users are used for both the coordinated and non-coordinated counterparts. Besides, for the simulation purposes, the number of cloud providers have been limited to 10, so we do not use statistical methods to further complicate scenarios, rather consider a fixed set of combinations of users for each of the 10 cloud providers.
Accordingly simulations were carried out to study the effect of the proposed MCEPRF provisioning algorithms on user cost and resource provider costs with varying aggregate resource demands. The results of the proposed heuristics for coordinated resource provisioning in CCM have been compared with its non-coordinated counterpart. To this end, we have carried out a number of simulation experiments as detailed hereafter.
Experiment 1
a Reserved Usage Cost for Cloud Users in CCM. b On-demand Usage Cost for Cloud Users in CCM
Experiment 2
a Resource usage (in %) with a Market Demand of 1000 VMs/Slot. b Resource usage (in %) for a Market Demand of 5000VMs/Slot. c Resource usage (in %) for a market demand of 10000 VMs/Slot
Experiment 3
Percentage of cloud users’ requests cancelled
Experiment 4
Provisioning decision time in mili-sec /slot
Experiment 5
Un-provisioned Users’ VM Request (in %) w.r.t different rpMaxCount
Performance analysis
- A.
Cloud User Cost: Fig. 4a depicts the cloud users’ reserved usage cost difference between coordinated and non-coordinated approach; whereas Fig. 4b depicts the on-demand cloud users’ usage cost of the same. These two cost components have been explained in the last section under experiment 1. In both cases, it can be observed that as the number of users’ requests increases the difference between coordinated and non-coordinated approach cost increases. The pattern of plots varies widely depending upon the data, specifically the extent of difference between reserved VMs by users and total VMs requested. We consider this variation between the VMs reserved in advance and total VMs requested to be restricted within ±10% in a normal distribution. Additionally we observed that at higher variations, the coordinated approach performs better. Intuitively, in terms of spawning fewer servers for green computing purposes, the coordinated approach outperforms its on-coordinated counterpart, though we have not made a quantitative evaluation in this regard.
- B.
Resource Usage: Fig. 5 depicts the percentage of resource usage during the users’ requests provisioning. Figure 5a, b and c presents the percentage of resource usage with varying market demands. It can be observed that coordinated approach requires lesser number of resource providers to cater to the users’ requests than the non-coordinated approach. This is due to the fact that in coordinated approach, a new server is spawned only if it can be loaded to a minimum threshold, or else, the user requests are dealt with by some previously allocated server, either from the same resource provider or otherwise. Threshold values pertaining to limits for spawning new servers for energy saving reasons are between 30 and 80%, as has been found in literature [3, 6–9]
- C.
Percentage of Cancelled Requests: Fig. 6 depicts the percentage of cloud users’ requests cancelled in coordinated approach is much lesser then the non-coordinated one, because, most of un-provisioned users’ requests with single resource provider are provisioned with the help of multiple providers in coordinated approach.
- D.
Provisioning Decision Time: Fig. 7 depicts the performance of provisioning decision time. It can be observed that with lesser number of users’ requests non-coordinated approach decision time is almost same the coordinated one, but as number of users’ requests increases the coordinated approach takes less time as compared to non-coordinated one.
- E.Execution Performance: Figs. 8 and 9 depicts the performance of coordinated approach in CCM in terms of un-provisioned users’ VMs request and execution time with respect to different rpMaxCount respectively, it can be observed that provisioning of users’ requests within three to four providers gives the best result in terms of both execution time and un-provisioned users’ requests.Fig. 9
Execution time of provisioned users’ VMs request with respect to different rpMaxCount
Conclusion
Proliferation of open standards and research initiatives towards federated clouds have opened avenues for conceiving truly global cloud markets that can have great impact in delivering economic services, customer satisfaction and environmental implications. To this end, this paper proposes a cooperative cloud market, defining the roles and modalities of their operations. This paper presents a novel, two-stage, coordinated resource provisioning approach that is driven by cost-effective service provisioning for consumers. Simulated results presented in this paper using existing pricing mechanisms demonstrate the efficacy of our proposed work over its non-coordinated provisioning counterpart. However, the model presented in this paper ignores issues such as customer subscriptions for multiple contracts, uncertainties pertaining to demand and prices offered by resource providers which can be considered in subsequent research.
Declarations
Authors’ contributions
KHKR carried out the literature review. GM developed the mathematical models. KHKR and GM jointly developed the simulations and drafted the manuscript. DSR provided useful insights and guidance and critically reviewed the manuscript. All authors read and approved the final manuscript.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
- Buyya R, Yeo CS, Venugopal S, Broberg J, Brandic I (2009) Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Futur Gener Comput Syst 25(6):599–616View ArticleGoogle Scholar
- Marston S, Li Z, Bandyopadhyay S, Zhang J, Ghalsasi A (2011) Cloud computing-The business perspective. Decis Support Syst 51(1):176–189View ArticleGoogle Scholar
- Kondo D, Javadi B, Malecot P, Cappello F, Anderson DP (2009) Cost-benefit analysis of cloud computing versus desktop grids. InParallel & Distributed Processing. IPDPS 2009. IEEE International Symposium on 2009 May 23. IEEE, Rome, pp 1–12Google Scholar
- Barroso LA, Clidaras J, Hölzle U (2013) The datacenter as a computer: An introduction to the design of warehouse-scale machines. Synthesis Lectures Computer Architecture 8(3):1–54View ArticleGoogle Scholar
- Wang L, Zhan J, Shi W, Liang Y, Yuan L (2009) In cloud, do mtc or htc service providers benefit from the economies of scale?. In Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers. ACM, Portland, p 7Google Scholar
- Hamilton J (2009) Internet-scale service infrastructure efficiency. InACM SIGARCH Computer Architecture News. ACM, Vol. 37, No. 3, pp 232–232Google Scholar
- Karidis J, Moreira JE, Moreno J (2009) True value: assessing and optimizing the cost of computing at the data center level. In Proceedings of the 6th ACM conference on Computing frontiers. ACM, Ischia, pp 185–192Google Scholar
- Fan X, Weber WD, Barroso LA (2007) Power provisioning for a warehouse-sized computer. In ACM SIGARCH Computer Architecture News. Vol. 35, ACM, No. 2, pp 13–23Google Scholar
- Campegiani P, Presti FL (2009) A general model for virtual machines resources allocation in multi-tier distributed systems. In 2009 Fifth International Conference on Autonomic and Autonomous Systems. IEEE, Valencia, pp 162–167Google Scholar
- Cases U (2010) Functional Requirements for Inter-Cloud Computing. InGlobal Inter-Cloud Technology Forum, GICTF White PaperGoogle Scholar
- Buyya R, Ranjan R, Calheiros RN (2010) Intercloud: Utility-oriented federation of cloud computing environments for scaling of application services. In: International Conference on Algorithms and Architectures for Parallel Processing. Springer Berlin Heidelberg, Heidelberg, pp 13–31View ArticleGoogle Scholar
- Bernstein D, Ludvigson E, Sankar K, Diamond S, Morrow M (2009) Blueprint for the intercloud-protocols and formats for cloud computing interoperability. InInternet and Web Applications and Services. ICIW’09. Fourth International Conference on 2009 May 24. IEEE, pp 328–336Google Scholar
- Sakashita Y, Takayama K, Matsuo A, Kurihara H (2012) Cloud fusion concept. Fujitsu Sci Tech J 48(2):143–150Google Scholar
- Keahey K, Tsugawa M, Matsunaga A, Fortes J (2009) Sky computing. IEEE Internet Comput 13(5):43–51View ArticleGoogle Scholar
- Petcu D, Crăciun C, Neagul M, Panica S, Di Martino B, Venticinque S, Rak M, Aversa R (2010) Architecturing a sky computing platform. In: European Conference on a Service-Based Internet. Springer Berlin Heidelberg, Heidelberg, pp 1–13Google Scholar
- Kelly K. The Technium: A Cloudbook for the Cloud. Available from: http://kk.org/thetechnium/a-cloudbook-for/. Accessed 17 Feb 2017
- Rochwerger B, Breitgand D, Levy E, Galis A, Nagin K, Llorente IM, Montero R, Wolfsthal Y, Elmroth E, Caceres J, Ben-Yehuda M (2009) The reservoir model and architecture for open federated cloud computing. IBM J Res Dev 53(4):4–1View ArticleGoogle Scholar
- Ferrer AJ, HernáNdez F, Tordsson J, Elmroth E, Ali-Eldin A, Zsigri C, Sirvent R, Guitart J, Badia RM, Djemame K, Ziegler W (2012) OPTIMIS: A holistic approach to cloud service provisioning. Futur Gener Comput Syst 28(1):66–77View ArticleGoogle Scholar
- Arjuna Agility. What Is Federation? Available from: http://www.arjuna.com/cooperation-with-federation. Accessed 17 Feb 2017
- Schubert L, Jeffery K, Neidecker-Lutz B (2010) The Future of Cloud Computing-Report from the first Cloud Computing Expert Working Group Meeting. Cordis (Online), BE: European Commission. Available online: http://cordis.europa.eu/fp7/ict/ssai/docs/Cloud-report-final.pdf
- CaaSt EU FP7 project, PaaS Cloud Platform (2013) http://www.4caast.eu/. Accessed 17 Feb 2017
- Catteddu D, Hogben G (2009) Cloud computing Risk Assessment: Benefits, risks and recommendations for information security, ENISA report. https://www.enisa.europa.eu/activities/risk-management/files/deliverables/cloud-computing-riskassessment/at_download/fullReport
- Mell P, Grance T (2016) The NIST definition of cloud computing. NIST Special Publication 800–145. Online: http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-145.pdf
- Liu F, Tong J, Mao J, Bohn R B, Messina J V, Badger M L, Leaf D M, (2011), NIST Cloud Computing Reference Architecture, NIST Special Publication 500–292. Online: http://www.nist.gov/customcf/get_pdf.cfm?pub_id=909505. Accessed Sept 2016
- Ahronovitz M, Amrhein D, Anderson P, Andrade A D, Armstrong J, (2010). A white paper produced by the cloud computing use case discussion group. Cloud Computing Use Cases, 10, 66. Available at: http://www.cloud-council.org/Cloud_Computing_Use_Cases_Whitepaper-4_0.pdf. Accessed 17 Feb 2017
- Kertesz A (2014) Characterizing Cloud Federation Approaches. In Cloud Computing. Springer International Publishing, pp 277–296Google Scholar
- Grozev N, Buyya R (2014) Inter‐Cloud architectures and application brokering: taxonomy and survey. Software: Pract Experience 44(3):369–390Google Scholar
- Kleinrock L (2005) A vision for the Internet. ST J Res 2(1):4–5Google Scholar
- Buyya R, Yeo C S, Venugopal S (2008) Market-oriented cloud computing: Vision, hype, and reality for delivering IT services as computing utilities. In High Performance Computing and Communications, 2008. HPCC’08. 10th IEEE International Conference on. IEEE, Dalian, pp 5–13Google Scholar
- Wu Z, Liu X, Ni Z, Yuan D, Yang Y (2013) A market-oriented hierarchical scheduling strategy in cloud workflow systems. J Supercomputing 63(1):256–293View ArticleGoogle Scholar
- Salehi MA, Buyya R (2010) Adapting market-oriented scheduling policies for cloud computing, Algorithms and Architectures for Parallel Processing. Springer Berlin Heidelberg, Heidelberg, pp 351–362Google Scholar
- Buyya R, Pandey S, Vecchiola C (2009) Cloudbus toolkit for market-oriented cloud computing, Cloud Computing. Springer Berlin Heidelberg, Heidelberg, pp 24–44Google Scholar
- Li H, Jeng JJ (2010) CCMarketplace: a marketplace model for a hybrid cloud. In Proceedings of the 2010 Conference of the Center for Advanced Studies on Collaborative Research. IBM Corp, Toronto, pp 174–183Google Scholar
- Sotomayor B, Montero RS, Llorente IM, Foster I (2009) Virtual infrastructure management in private and hybrid clouds. IEEE Internet Comput 13(5):14–22View ArticleGoogle Scholar
- Zhang Q, Cheng L, Boutaba R (2010) Cloud computing: state-of-the-art and research challenges. J Internet Serv Applications 1(1):7–18View ArticleGoogle Scholar
- Padala P, Shin KG, Zhu X, Uysal M, Wang, Z, Singhal S, Salem K (2007) Adaptive control of virtualized resources in utility computing environments. In ACM SIGOPS Operating Systems Review. Vol. 41, ACM, Stevenson, No. 3, pp 289–302Google Scholar
- Bennani MN,Menasce D (2005) Resource allocation for autonomic data centers using analytic performance models. In Autonomic Computing, 2005. ICAC 2005. Proceedings. Second International Conference on. IEEE, Seattle, pp 229–240Google Scholar
- Meng X, Isci C, Kephart J, Zhang L, Bouillet E, Pendarakis D (2010) Efficient resource provisioning in compute clouds via VM multiplexing. In Proceedings of the 7th international conference on Autonomic computing. ACM, Washington DC, pp 11–20Google Scholar
- Zhan J, Wang L, Li X, Shi W, Weng C, Zhang W, Zang X (2013) Cost-aware cooperative resource provisioning for heterogeneous workloads in data centers. IEEE Trans Comput 62(11):2155–2168MathSciNetView ArticleGoogle Scholar
- Wang H, Jing Q, Chen R, He B, Qian Z, Zhou L (2010) Distributed systems meet economics: pricing in the cloud. In Proceedings of the 2nd USENIX conference on Hot topics in cloud computing. USENIX Association, pp 6–6Google Scholar
- Mazzucco M, Dumas M (2011) Reserved or On-demand instances? a revenue maximization model for Cloud providers. In Cloud Computing (CLOUD), 2011 IEEE International Conference on. IEEE, pp 428–435Google Scholar
- Menglan H, Jun L, Veeravalli B (2012) Optimal provisioning for scheduling divisible loads with reserved cloud resources, In 18th IEEE International Conference on Networks (ICON). IEEE, Singapore, pp 204–209Google Scholar
- Khatua S, Sur PK, Das RK, Mukherjee N (2014) Heuristic-based Optimal Resource Provisioning in Application-centric Cloud. arXiv preprint arXiv:1403.2508Google Scholar
- Li Q, Guo Y (2010) Optimization of Resource Scheduling in Cloud Computing, SYNASC., pp 315–320Google Scholar
- Chaisiri S, Lee BS, Niyato D (2012) Optimization of resource provisioning cost in cloud computing. IEEE Trans Serv Comput 5(2):164–177View ArticleGoogle Scholar
- http://www.intercloudsys.com/. Accessed 17 Feb 2017
- http://www.opencontrail.org/. Accessed 17 Feb 2017
- https://ercim-news.ercim.eu/en83/special/reservoir-a-european-cloud-computing-project. Accessed Sept 2016
- http://opencirrus.org/. Accessed 17 Feb 2017
- http://www.optimis-project.eu/. Accessed Sept 2016
- http://www.mosaic-cloud.eu/. Accessed 17 Feb 2017
- http://www.bonfire-project.eu/. Accessed 17 Feb 2017
- Mudali G, Patra MR, Reddy KHK, Roy DS (2015) Cooperative Resource Provisioning for Futuristic Cloud Markets, Computational Intelligence in Data Mining-Volume 3. Springer India, India, pp 607–616Google Scholar
- (2014) http://aws.amazon.com/ec2/pricing/
- http://www.cs.huji.ac.il/labs/parallel/workload/. Accessed Sept 2016