AWESOME: an auction and witness enhanced SLA model for decentralized cloud marketplaces

In recent decades, the world has witnessed cloud computing as an essential technology that changes the traditional application Development and Operation (DevOps) lifecycle. However, current cloud software DevOps and Service Level Agreement (SLA) management often face challenges of 1) selecting the best fitting service providers, customizing services and planning capacities for large-scale distributed applications; 2) guaranteeing high-quality and trustworthy SLAs among multiple service providers; 3) enhancing the interoperability of cloud services across different providers; and 4) designing effective incentive models among stakeholders. This paper proposes a novel framework called Auction and Witness Enhanced trustworthy SLA for Open, decentralized service MarkEtplaces (AWESOME) to build a trustworthy cloud marketplace and address the above challenges. The proposed framework contains four subsystems: a customizable graphical user interface, an auction-based service selection model, a witness committee management mechanism, and a smart contract factory orchestration. We developed a prototype AWESOME decentralized application (DApp) based on the Ethereum blockchain. Extensive experiments are designed to evaluate the latency and cost of our model. The experimental results demonstrate that our model is economical and feasible.


Introduction
The cloud computing paradigm provides flexible services based on pay-as-you-go business models [1]. In the current cloud marketplace, several well-known service providers maintain the traditional cloud marketplace, and the share of these top providers is continuously growing. According to a report, as of October 2020, AWS, Azure, Google, and Alibaba control 63% of the entire cloud marketplace, whereas all other providers only share 37% 1 . Since product migration is complex, consumers become locked in a particular provider's ecosystem. In the future, *Correspondence: huanzhou@nudt.edu.cn; z.shi2@uva.nl z.zhao@uva.nl 3 School of Computer Science, National University of Defense Technology, 410073 Changsha, China 1 Informatics Institute, University of Amsterdam, Science Park 904, 1098 XH, Amsterdam, the Netherlands Full list of author information is available at the end of the article 1 https://www.canalys.com/newsroom/worldwide-cloud-market-q320 however, we can expect a more open, fair, and trustworthy cloud resource trading marketplace for all service providers and customers.
There are typically two cloud transaction models: centralized and decentralized [2]. In a centralized cloud environment, all service trading and trust-related issues rely on trusted third parties (TTPs), e.g., some well-known cloud service providers with good reputations and track records. However, those providers are not always trustworthy in practice and can be biased or conspire with any party. On the other hand, in a decentralized trading environment, all sellers or buyers perform transaction management and operations, avoiding the concentration of power and making the transactions more trustworthy. In this case, all trust assurance comes from a decentralized platform (e.g., blockchain), which needs to be appropriately designed, implemented, deployed, and monitored. Traditionally, a Service Level Agreement (SLA) is a business concept that defines the contractual financial agreements between the roles engaging in the business activity. In the context of a cloud marketplace, it is an agreement between the cloud customer and provider regarding the cloud service quality [3]. For instance, the IaaS (Infrastructure-as-a-Service) provider, Amazon Elastic Compute Cloud (Amazon EC2), claims that the availability of its data center is no less than 99%. If this is not achieved, it will pay back 30% credits to its customers as compensation. In practice, however, this agreement is hard to enforce fairly and transparently; it is usually performed manually and dominated by giant providers in the traditional SLA management process.
Blockchain technology can be used to support decentralized applications (DApps), bringing new hints of possible solutions to address these challenges [4,5]. It inspires the emergence of a new decentralized cloud marketplace that encourages greater inclusivity and participation from different service parties. We can foresee that such a decentralized marketplace will provide more choices and opportunities for both providers and consumers. Besides, the smart contract makes it possible to manage and automate the SLA process on the blockchain in a fair and tamperproof way [6]. However, reaching a consensus on events that occur outside the blockchain is another possible challenge. Cloud customers or providers can still violate the agreed SLA despite using the blockchain to complete cloud transactions. For example, the provider may not provide the QoS (Quality of Service) they promised, and the customer may refuse to pay for the claimed cloud resources. In the blockchain community, the bridge between on-chain and off-chain events is called "oracle" [7]. One of the solutions to build this bridge is to retrieve data from Oraclize 2 , a third-party company that performs as a trusted data source for the blockchain. However, this solution suffers from a single point of failure and needs extra commission fees. In this case, a decentralized witness mechanism is promising to judge SLA violations that occur off-chain.
This paper expects to use blockchain to enhance the cloud marketplace and SLA management lifecycle by introducing a novel Auction and Witness Enhanced trustworthy SLA for Open, decentralized service MarkEtplaces (AWESOME) framework. Specifically, a new role called auction witness is involved in the entire cloud service trading process, as shown in Fig. 1. In our model, the decentralized blockchain users can join the SLA judgment and work as witnesses through an incentive mechanism that motivates them to make truthful judgments to win profits. This paper is based on our previous conference paper [8], and we have extended the framework, protocols, and 2 http://www.oraclize.it/ experiments extensively. In brief, the main contributions of this paper can be summarized as follows: • A novel auction and witness enhanced SLA model called AWESOME for decentralized cloud marketplaces. The model can support interactions between service providers, customers, and witnesses to complete trustworthy transactions and SLA enforcement. • A prototype DApp based on the AWESOME model is fully developed on the Ethereum blockchain 3 . It contains customizable graphical user interfaces (GUIs) and advanced smart contract protocols to support the SLA business process. • Extensive experiments are designed to evaluate the execution latency and cost of the proposed model and DApp. The experimental results demonstrate that our model is economical and feasible to implement.
In the rest of this paper, we first demonstrate the related works in "Related work" section. Then, in "The AWESOME framework" section, we analyze the system requirements, objectives, actors, on-chain and offchain interactions, and then present the overall AWE-SOME system architecture. Next, we introduce the design choices of our AWESOME DApp and show the details of how the DApp works with a business process model in "The AWESOME DApp demonstration" section. "Experiments and validations" section shows the experimental results to demonstrate the feasibility of the AWESOME framework and DApp. Finally, we discuss key considerations and future development plans in "Discussion and future work" section, and conclude the whole paper in "Conclusion" section.

Related work
This section provides an overview of the state-of-the-art technologies that are related to the AWESOME framework.

Smart contract-based SLA management
Smart contracts are computer programs designed to execute contracts automatically on a blockchain. In this regard, many studies have discussed the use of smart contracts and blockchain for SLA management. Uriate et al. [9] proposed a domain-specific language known as SLAC, which is designed for SLA specification in the cloud computing domain to avoid ambiguities that come with SLAs. The core purpose of SLAC is to describe the contract, specify the contract terms, and define the guarantees of those terms. Gomes et al. [10] developed a smart contract that makes use of web3.js to connect web applications to the blockchain. They use this smart contract to determine if an SLA violation has occurred and to enforce fines and compensation. Their proof of concept demonstrated that service operators could use DApps in cloud environments to simplify the process of SLA validation. The same research group also highlighted transparency and trust between different parties as a major issue in SLA validation [11]. They proposed a solution that uses a decentralized file system to store the generated agreements. An oracle, in this case, is used as a real-time data channel that inserts real-world data into the blockchain to determine if there is any loss of connectivity.

SLA monitoring solutions
There are various monitoring solutions in the SLA domain. For example, the Web Service Management Agent [12] implemented SLA monitoring of web services using agents from both server-side and customer-side. In addition, monitoring is done in practice by SLA decom-position, where low-level system performance metrics are mapped to high-level SLA parameters [13]. From the consumer's perspective, SLAs should be understandable and avoid ambiguity to ensure QoS, so providers often write SLA documents in legal language. Besides, formal metrics need to be defined in order to facilitate successful monitoring [14,15]. To ensure proper SLA modeling and monitoring, the authors in [15] suggested an SLA modeling process that defines the ontology to represent the SLA concepts. After that, monitoring can occur to measure different SLA parameters and detect violations. In [16], the authors stated that a monitoring engine should mainly consist of two different phases. The first phase is the runtime monitoring of SLA parameters, where the monitoring agents continuously record the application performance. In the second phase, an analysis engine verifies the values and compares them to predefined thresholds from the agreement's service-level objectives. Then, the engine identifies violations, and the provider is penalized in some cases.

Blockchain-based auction models
Auction is a well-studied research topic due to its effective and fair transaction properties. In cloud computing, classic auction models, e.g., English auction, Dutch auction, and seal-bid auction, have been widely leveraged to optimize cloud resource allocation. For example, the authors in [17,18] used different auction models to achieve optimal cloud customer/provider selection. While blockchain-based auction models have great potential in optimizing cloud transactions and marketplaces, most existing solutions focus only on the design of the auction models without considering the trustworthy enforcement of cloud SLAs [19].

DApp development challenges
The contribution of this paper is to provide DApp developers with a framework for a decentralized service marketplace in which users can customize to their own business needs and technologies. Therefore, the challenges in DApp development that currently exists should be identified. An initial challenge with AWESOME DApp development stems from the fact that it is new and unexplored. As a result, DApp requirements are not fully understood or require continuous change during production. This is why the use of Agile methods in DApp Development and Operation (DevOps) is often encouraged [20]. Unfortunately, few development frameworks can aid DApp developers in making service marketplaces.
To the best of our knowledge, this paper is the first integrated model and DApp attempt to study generic auctions and witness mechanisms for decentralized cloud marketplaces.

The AWESOME framework
In this section, we first perform the requirement analysis using two industrial use cases and show the design objectives of the system. Then, we describe our AWE-SOME model in detail, including actor identification, onchain vs. off-chain interactions, and system components & workflows.

Requirements analysis
In industrial innovations (e.g., crowd journalism and disaster early warning) and scientific applications (e.g., research data management), cloud services are playing an increasingly important role in real-time data processing (e.g., multimedia acquired by mobile devices), running simulations (e.g., for predicting possible disasters), and for enabling extensive scale collaborations (e.g., for running distributed scientific workflows). Therefore, it is necessary to employ multiple data centers or providers to handle decentralized collaboration between resource providers and customers in several industrial use cases.
1. Use case 1. Decentralized cloud marketplace for social media (taken from EU ARTICONF project 4 ): crowd journalism for real-time news reporting during live sports, music events, or natural disasters. Individual citizen journalists make photos or videos of the news and trade them via the news platform.
The system has to detect fake news from those crowdsourced contents by running real-time processing from multiple cloud providers or engaging human experts to review them. 2. Use case 2. Decentralized service marketplace for medical data management (taken from EU CLARIFY project 5 ): sharing and utilizing pathology data provided by hospitals or individuals from different countries, where various medical data access constraints are often applied. When a machine learning application for studying breast cancer must use data from multiple hospitals, the application developer has to select cloud providers from a decentralized marketplace that meet the application needs (e.g., geolocation, capacity, and price).
We can therefore highlight the following requirements and challenges from those use cases: • Provider selection, service customization, and capacity planning challenges. The developer has to select cloud services from different providers (very often multiple ones) due to distributed data locations (e.g., sensors or repositories), various data access constraints (e.g., for medical data), and performance constraints (e.g., for real-time decisions in early warning). The various price and reputation models make the selection time-consuming and challenging to be optimal. • SLA interoperability and guarantee challenge. The time-critical application constraints, e.g., processing media contents during crowd news reporting and real-time decision-making, require the profound optimization of the application logic and the quality guarantee of the cloud services. However, the diverse SLA terms among providers and the uncertainties in the SLA guarantee make performance optimization difficult. • Difficulties in setting up incentive models and customizing witness games in a decentralized marketplace. The business logic in a decentralized marketplace is often realized by smart contracts, which are supposed to be immutable after being deployed on blockchains. However, any careless design or mistake may cause unexpected loss. • Virtual infrastructure automation challenge. When an application involves multiple providers or data centers, the provisioning of the virtual infrastructure, deployment of the software platform and application components, monitoring, and adaptation of the application need to be ideally automated. However, the diverse Application Programming Interfaces (APIs) from different providers and the interoperability issues across those providers make automated provisioning and deployment a challenge. This leads to a high level of complexity in monitoring runtime infrastructure quality, detecting SLA violations, and adapting the infrastructure when violations happen.
To tackle these challenges in a decentralized cloud marketplace, we propose the AWESOME framework. The AWESOME software architecture consists of novel technologies in DApp DevOps, game theory, blockchain, and smart contracts.

System objectives
We aim to provide guidelines, tools, and templates that can aid developers in developing a DApp for a specific business that can benefit from a decentralized architecture. The system needs to provide service customers and providers a platform that can facilitate easy SLA generation and operation, and allow decentralized witnesses to monitor such SLAs. Furthermore, the system needs to be generic and modular. DApp developers who use the system could customize the model for their own business needs and adapt it to other blockchain use cases. Specifically, the objectives of the AWESOME model can be summarized as follows: • Objective 1: Improve the customer/provider selection in a decentralized ecosystem by developing an automated service auction framework to enable dynamic business relations between a consumer(s) and providers and establish SLAs. • Objective 2: Improve the service quality and SLA's trustworthiness between consumer(s) and providers by establishing a decentralized witness mechanism to monitor the SLA violations and automate the procedure for SLA compensation and payment. • Objective 3: Improve the model usability by developing easy-to-use customizable DApp GUIs for general cloud users to interact with different smart contracts. • Objective 4: Improve the continuous DevOps of DApps by providing an integrated contract factory to improve smart contracts' security and efficiency.

Actor identification
Actors which interact with the AWESOME model are human roles, external systems, or devices that exchange information with the DApp. With this in mind, we identify the following actors: • Service Customer: Service customers use the DApp to find providers that can offer them services. They should be able to make listings on the platform and sign an SLA with a service provider. They pay for these services but can receive compensation in case of SLA violations. • Service Provider: Service providers use the DApp to list their available services on the platform for auction. They earn monetary rewards for these services but may be penalized in case of SLA violations. • Auction Witness: Witnesses can use the DApp to monitor SLAs and receive monetary rewards for their efforts. The judgment from the witness committee can ensure that auctioned cloud services are delivered as agreed in SLAs. • AWESOME Operator: An AWESOME operator could modify the developed framework for a specific business use case. They need to read the provided documentation, edit the user interfaces, blockchain APIs, and smart contract templates, and then deploy a custom decentralized service marketplace 6 .
In our model, customers are motivated to receive services at a low cost, and providers are incentivized to provide high-quality services in return for service fees. Two parties complete transactions and make profits by means of customized on-chain auctions. The payoffs of both parties are always positive; otherwise, an auction will never be settled. The incentive for witnesses, on the other hand, can include the following three parts: 1) a reward for their monitoring efforts; 2) a penalty if their reports about SLA violations fail to match the results of others (based on the majority rule); and 3) a blockchain transaction fee. We specify in our model that the monitoring reward is large enough so that the witness's payoff is always positive. In this way, witnesses are always motivated to participate in our model and earn their rewards.

On-chain vs. off-chain interactions
After identifying system objectives and actors, we can now design the system architecture as a whole. The ABCDE DApp development method [20] suggests first dividing the system into two subsystems. Namely, smart contracts running on top of the blockchain and external systems that interact with the blockchain. The authors also recommended formalizing what transactions and data should be placed on-chain and off-chain. In practice, the information that should be included in the blockchain is that with critical trust requirements. This is because on-chain information is immutable and enforces non-repudiation. In addition, not only "big data" is not suitable for the blockchain, but even "not-tiny" data should not be stored on the blockchain. This is mainly due to cost and scalability considerations; the computing power and data storage space available on the blockchain is limited. One of the typical solutions is to store raw data off-chain due to its size while storing small critical data and hashes on-chain. In terms of computation, blockchains are not the best for complex, intensive computations; however, they provide a benefit in their interoperability properties as many systems can access it [21]. The authors of the ABCDE development methodology suggest as a general guideline that data with transparent and immutable requirements for DApps should be managed on the blockchain system [20]. In this section, we follow this idea to demonstrate our design choices.

On-chain activities
The data and activities the system should keep on-chain include: • Auction: To support transparent trade, the auction of service tasks should be conducted on the blockchain to maintain fairness and prevent fraud. Implementing decentralized auctions on the blockchain can also avoid the cheating auctioneers of traditional centralized auctions and save commission fees. • Witness reports: In order to avoid tampering with the SLA reports, they should be placed on-chain. While this may lead to witnesses viewing each other's reports and reporting accordingly, the reports should be transparent. One effective way to address this issue and protect privacy is to submit the hashes of the reports in a specified time window. • SLAs: It is essential to include SLA details between the service provider, customer, and witnesses in the blockchain, as all these data have trust requirements. However, since SLAs may be larger textual files that could give a large load to the blockchain, a possible solution is to place SLA metadata that can unlock the SLA off-chain while keeping the cryptographic hash on-chain. • Payment enforcement: In case of an SLA violation, a smart contract should be used to facilitate payments to providers, customers, and witnesses automatically. Blockchain cryptocurrencies can support the secure and fair enforcement of money payments.

Off-chain activities
The data and activities the system should keep off-chain include: • User interface: Due to data loads, the way in which providers, customers, and witnesses interact with the platform should mostly be off-chain. • Cloud Services. Cloud services offered and used by providers and customers should be off-chain. • Pre-monitoring communication: The platform should facilitate communications between providers and customers before entering an SLA agreement so that they can privately agree upon service and monitoring terms.

Overall system components
Based on the above analysis, we designed the system architecture of AWESOME. The AWESOME framework consists of four subsystems in response to the proposed objectives, as shown in Fig. 2: 1. DApp Graphical User Interface (DGUI) provides a flexible and customizable DApp interactive environment for different AWESOME users to connect on-chain and off-chain activities. It is designed to provide a bridge between customers, providers, and witnesses who do not have IT development knowledge and assist them in calling function interfaces between different smart contract agents. Furthermore, the usability of the AWESOME DApp is ensured through a customizable interface design for business needs. More specifically, DGUI is designed with interfaces regarding 1) auctions by customers and providers; 2) SLA monitoring activities by witnesses; and 3) DApp management and maintenance by operators.

Auction-Based Service Selection (ABSS) provides
an auction-based service customer/provider selection solution. This subsystem will first diagnose the use case requirements and help the user select the most suitable auction model and algorithm to achieve desirable results. Then, the management of the auction process and the enforcement of the service fee payment (in the form of cryptocurrency) are executed on the blockchain, ensuring that the whole auction is open and trustworthy. Finally, ABSS also audits bidder candidates to prevent malicious actors from joining the auction.

Decentralized Witness Committee Management
(DWCM) provides a trustworthy incentive framework to manage decentralized auction witnesses. First of all, an appropriate number of witness candidates will be selected in an unbiased way to perform off-chain monitoring of cloud SLAs. DWCM will then invoke a game theory incentive mechanism based on customized payoff functions to enable selected witnesses to make correct judgments to win more profits. The subsystem will also audit the witnesses' reputations to reward/restrict their participation in future monitoring activities.

Smart Contract Factory Orchestration (SCFO)
provides tools and APIs for AWESOME operators to set the necessary smart contracts on the blockchain. More specifically, it is responsible for automating the process of planning, provisioning blockchain infrastructure, and deploying AWESOME business smart contracts. In addition, the SCFO subsystem also monitors and diagnoses smart contracts and the underlying blockchain infrastructure at runtime to provide effective adaptation solutions.
As shown in Fig. 3, the overall workflow of the AWE-SOME framework can be described as the following steps (using a reverse auction example). First of all, an AWE-SOME operator calls the DGUI subsystem to customize and generate customer, provider, and witness user interfaces (UIs) for the current use case. The AWESOME operator then calls the SCFO subsystem to initiate a contract factory. This contract factory automatically generates the required auction, witness, and SLA smart contracts to ensure trustworthy interaction between different participants. Meanwhile, it also invokes a runtime monitor for these contracts and returns the monitoring result to the AWESOME operator. Next, an AWE-SOME customer invokes the UI to transfer the specific business requirement to the ABSS subsystem. Based on this requirement, an auction model is selected and configured to wait for providers to submit their bids. The decentralized service providers then start to registerer in the ABSS subsystems through their UIs. When there are enough bidder candidates, smart contracts automatically start the auction process to find qualified providers. Then, the witnesses invoke their UIs to register in the DWCM subsystem. The DWCM subsystem also performs game design and an unbiased witness screener to choose the appropriate witness to form the witness committee. Finally, the selected providers collaborate to provide cloud services, and selected witnesses monitor the SLAs to win profits. The service fee and witness fee will be paid and enforced with cryptocurrency using smart contracts when the cloud service ends.
In the entire AWESOME workflow, DGUI provides a customizable graphical interaction environment to support user-to-user interactions in business processes. ABSS selects candidate providers through an effective auction mechanism. DWCM ensures trustworthy SLA enforcement through truth-telling witness monitoring. Finally, SCFO provides automated smart contract support for the entire process. The four subsystems form a dynamic ecosystem that provides services to AWESOME users collaboratively.

The AWESOME DApp demonstration
This section presents a prototype system based on our AWESOME framework. The prototype output is a DApp system that helps AWESOME operators and users trade cloud services in a decentralized service marketplace. This DApp will contain customizable auction and witness models for service provisioning and SLA violation monitoring. Specifically, we first examine different design choices and implementation options. Then we leverage a use case to demonstrate how the different roles would utilize the DApp.

Design choices
We discuss the design choices regarding the auction models, the blockchain infrastructure, and the smart contract protocols.

Auction models
In order to support dynamic business requirements, we should allow both customers and providers can be initiators and bidders. This produces a more customizable system that aids in defining a broader range of auction possibilities. Therefore, at the highest level, the two types of auctioning that the system supports are: 1) forward auctions, where the provider is the initiator and the customers submit competing bids; 2) reverse auctions, where the customer is the initiator, and the providers submit competing bids. Specifically, AWESOME is designed to support the following eight auction models: • English Auction: This is an ascending bid auction, where the price is gradually raised until only one final bidder remains, and that bidder wins the service at the finalized price.  • Second Price Sealed Bid Auction: In this auction, bidders submit sealed bids and the highest bidder wins again, but the price they pay is the value of the second-highest bid. It is also known as the Vickrey auction, which encourages truthful bidding in terms of mechanism design [22].
We can also implement the symmetrical version of these four types for reverse auctions.
• Reverse English Auction: In this auction, the price is decremented by competing providers until no one bids at a lower price. The provider offering the lowest price wins the auction and provides the service at that price. • Reverse Dutch Auction: This auction begins at a very low price for the service and gradually increases until a service provider accepts to provide the service at that specific price. • Reverse First Price Sealed-Bid Auction: In this auction, providers submit sealed bids simultaneously to the customer. The one with the lowest bid wins and pays the value of their bid. • Reverse Second Price Sealed Bid Auction: In this auction, providers again submit sealed bids, the lowest bid again wins, but the price he should pay is the value of the second-lowest bid.

Blockchain infrastructure
In general, blockchains can be permissionless or permissioned. Our modular AWESOME framework is not limited to the underlying blockchain infrastructure. AWE-SOME aims to build a decentralized cloud marketplace using both permissionless and permissioned blockchains. Some existing popular platforms may include: • Ethereum: It is an open-source permissionless blockchain platform with smart contract and cryptocurrency functions. It provides a decentralized mechanism to handle peer-to-peer smart contract transactions through its proprietary Ethereum Virtual Machine. We are currently using the Ethereum blockchain to develop AWESOME smart contracts and DApps. • Hyperledger Fabric: It is an open-source permissioned blockchain platform. The main reason for choosing a permissioned blockchain is the availability of a more efficient consensus mechanism, which increases scalability and reduces wasted resources. • Other promising blockchain platforms, e.g., Hyperledger Sawtooth, Hyperledger Iroha, Hyperledger Besu, and IOTA, can also be implemented to support our decentralized cloud marketplace. These platforms have been investigated in many research studies and commercial projects [23].

Smart contract protocols
To meet the requirements of the AWESOME model to build a decentralized cloud marketplace, we designed three smart contracts (i.e., auction contract, witness contract, and SLA contract) to support trustworthy and fair interactions between different stakeholders, as shown in Figs. 4, 5 and 6. Specifically, the auction contract is responsible for managing the auction process. The witness contract is responsible for the registration and management of auction witnesses, as well as the calculation of witness reporting results and corresponding witness fees. Furthermore, the SLA contract is used to build and manage a trustworthy SLA lifecycle. It should be noted that we leverage a contract factory to manage and generate subcontracts instead of developing different contracts separately in the AWESOME framework, as this is a more secure and efficient way [24]. The sequence diagram in Fig. 7 shows the interaction between the contract factory and different subcontracts. First, an AWESOME operator calls the contract factory to create a new auction contract. Next, an auction contract with a customized auction rule for business requirements is built to support a transparent and automated auction process. In this case, service providers can register and submit their bids for services on the blockchain. The auction contract then selects the winning providers based on the highest k bids and generates k SLA contracts for each provider. When the auction is settled (note that the services have not been delivered yet), the AWESOME operator calls the contract factory again to generate a witness contract that contains customized incentive mechanisms to encourage truth-telling witnesses. More details about a game theory-based witness payoff design are discussed in our previous research [25]. Then, different winner providers can start to deliver cloud services offchain while the witnesses start to monitor all the services; if the QoS satisfies the requirements in the SLA contract, there is no violation; otherwise, there is a violation. The result of service monitoring is also returned to the auction contract to determine the status of the auction.

Use case demonstration
We use the business process model in Fig. 8 to demonstrate how our AWESOME DApp works. There are three parties of stakeholders who interact with the AWESOME DApp to complete a reverse auction. In this auction, the customer acts as the purchaser of the cloud service and the initiator of the auction. The providers need to compete for bids to get the right to sell their services. The entire AWESOME business process is described as follows. When the AWESOME DApp is launched, it first invokes the AWESOME contract factory to generate the auction contract to support the auction process management. At this time, a customer can register, set up, and post an auction through the customer UI, as illustrated in Fig. 9a. An example output of the JSON data structure when setting an auction object is shown in Listing 1. This auction invitation is displayed on the AWESOME DApp to make it visible to providers. When noticing the auction invitations, different providers can register as bidders and submit their bids through the provider UI, as shown in Fig. 9b. When enough bids are received to meet the customer's requirement, the auction is settled, and the winning providers are selected. After that, the AWESOME contract factory will generate SLA contracts to prepare for service delivery and monitoring. The customer and providers need to sign and confirm the SLA contracts in AWESOME DApp, respectively. At the same time, a witness contract is also generated. At this point, the auction initiator (i.e., the customer) needs to define the rules for witnesses, including the number of witnesses, the minimum consensus percentage required to confirm a violation, the time window for submitting reports, and the rewards and penalties for each witnesses' report. This is illustrated in Fig. 10a and b. Then, witnesses can register and interact with the contract through the Witness UI. The cloud service is officially launched only when all sub-SLAs are confirmed, and there are enough witnesses for SLA monitoring. Next, the customer and providers perform off-chain cloud service provisioning and consumption. Witnesses perform continuous SLAs monitoring and report the monitoring results to the AWESOME DApp, as shown in Fig. 11. Based on the results reported by the witness committee, SLA violation is confirmed: when there is a violation, the service fee prepaid by the customer is refunded; while when the service is completed as agreed in the SLA, the providers can withdraw their corresponding service fees. Finally, witness fees are calculated and allocated based on the monitoring results of the witness game.

Experiments and validations
In this section, we present the evaluation and validation of the AWESOME framework. We identified two key metrics that affect the performance of our AWESOME model: latency and cost.

Latency analysis
To evaluate the performance of the AWESOME DApp and the feasibility of the model, we first simulated cloud Then, the latency is tested in the local Ethereum blockchain and calculated in two ways: 1) the response time of the AWESOME DApp; 2) the difference between the block timestamps [26]. The former reflects the latency of executing transactions in our DApp, and the latter demonstrates the transaction processing time of the underlying blockchain. In addition, we simulated two cases of blockchain mining network congestion. The "best" case implies that there are enough miners to process transactions promptly. In contrast, the "average" situation indicates seconds of delays for transaction processing due to network congestion. Figure 12 consists of 20 plots representing the performance of 20 function interfaces in a "best" blockchain mining network. Overall, the response time of the AWE-SOME DApp API is a few seconds longer than the blockchain block time, which is in line with our expectations. It can be seen that the execution time of most of the function interfaces increases linearly with the number of players. One exception is Place Bids, which has an exponentially increasing trend. This is because this function requires on-chain calculations to place bids and therefore takes significantly more time when the number of players increases or the auction data becomes complex. Some other functions that take longer time include Setup Auction, Calculate Witness Fee, Publish Service, and Setup SLA. Except for these functions, the latency of all others holds at a low level (10 to 15 seconds with 100 players).
Similarly, Fig. 13 shows the performance of 20 functional interfaces on an "average" mining network. It can be found that the DApp API response time and the blockchain block time have both increased significantly; the two latency values also tend to be close to each other because of the huge increase in the total time needed to process blockchain transactions. Nevertheless, all latency is maintained within a few minutes when the number of players increases. A strange observation is that the latency is high in the Cancel SLA function when the player number is 1. By analyzing our experimental workflow, we found that this is because he needs to process a large number of SLAs generated from the previous step. This also proves that the latency is influenced by the player numbers and the complexity of the data to be processed. In summary, we conclude that the AWESOME framework has a good performance in terms of execution latency. Some individual functions that require complex calculations have longer time delays, but they may not be time-critical in the auction process. We also conclude that the main factor affecting DApp latency is the performance of the underlying blockchain; namely, the latency is heavily dependent on the congestion of the blockchain mining network. A more detailed comparison is also presented in Table 1. The sequence diagram of the AWESOME smart contracts interactions

Cost analysis
Next, we measured the gas consumption and transaction fee (Ether) of each function interface in AWESOME smart contracts, as shown in Fig. 14 and Table 2. These functions have built-in access control mechanisms; only specific stakeholder groups can access and call them. Specifically, three transaction submission speed modes (i.e., low, average, and high) were tested 7 . By analyzing the testing results, we can find that the transaction fees of most function interfaces are maintained at a relatively low level (less than 0.01 ether), except for only three special cases, namely Place Bids (auction contract), Calculate Witness Fee (witness contract), and Check SLA Violation (SLA contract). These function interfaces require specific computational tasks onchain and therefore need more transaction fees to pay for miners. 7 The estimated transaction confirmation duration for three modes are 16 minutes, 2 minutes and 19 seconds, and 30 seconds, respectively. Data was collected on April 30, 2021, at https://etherscan.io/gastracker Table 3 further shows the transaction fee of each AWE-SOME user (converted into USD). Overall, the customer is the beneficiary and initiator of the service auction and should therefore bear more commission fees. Providers have lower transaction costs since they only join the model as bidders. Besides, the transaction fee per witness is economical (about $ [15][16][17][18][19][20], which ensures that they have sufficient incentive to join the SLA monitoring activities. It is worth noting that although the customer pays over $200 to initiate the auction, this fee is fixed and independent of the final service price. In contrast, some popular online auction platforms charge a percentage of the sale price as a commission (e.g., eBay charges 12.9% of the sale price). Therefore, our model has a price advantage, especially when the price of the auctioned cloud services becomes very expensive. On the other hand, if the commission is higher than the cost of the cloud service itself, our model will not be suitable for a public blockchain like Ethereum. A fee-free permissioned blockchain (e.g., Hyperledger Fabric), in this case, can be used as an alternative and the whole model is still valid. Discussion and future work AWESOME aims to build a trustworthy cloud marketplace using blockchain technologies. In this section, we discuss the design choices and future development plans.

Blockchain technologies
The first consideration is the tradeoff between choosing permissionless and permissioned blockchain technologies. Permissioned blockchains can address the huge operational cost and low-performance issues of permissionless blockchains, but this is often considered at the cost of transaction security, especially in a low trust environment [27]. We used a static sealed-bid auction model to demonstrate and evaluate our AWESOME DApp in "The AWESOME DApp demonstration" and "Experiments and validations" sections. Such an auction does not require high performance and scalability of the blockchain since each bidder only needs to submit a bid once. However, in a high-frequency and large-scale dynamic auction, the blockchain's performance is a key factor; therefore, we believe that a permissioned blockchain is a better choice. Another consideration is to support auction payments using cryptocurrencies. Ethereum's widely recognized token Ether can be commonly regarded as fiat money, which primarily motivated us to use the Ethereum blockchain to develop the current DApp. We leave the implementation of permissioned blockchain platforms (e.g., Hyperledger Fabric) in the AWESOME framework as our future work.

Auction models
In order to support dynamic business requirements, we design our AWESOME framework to allow both forward and reverse auctions. This gives users a more customizable system that aids in defining a broader range of auction possibilities. Introducing both forward and reverse auctions also increases the possibilities for users on the DApp and promotes fairness between parties as both can use the DApp to serve their business needs in their own way. Besides, many other popular auction models, e.g., double auction, combinatorial auction, and VCG auction, have demonstrated their great potential for integration with blockchain and cloud computing [19]. We leave the implementation of these auctions as our future work. In addition to the eight auction models mentioned in "Auction models" section, AWESOME users can also customize various auction models to support different business needs. We expect a richer AWESOME ecosystem to be built with the contribution of more auction models from users.

Smart contract optimization
We do not let users deploy AWESOME smart contracts directly in the current model. Instead, users need to invoke contract templates through the DApp UI to gen-erate auction, witness, and SLA contracts automatically. This design provides users an easy way to customize and deploy contracts while helping the DApp operator keep track of all deployed contracts. We believe such a design is reasonable and effective. Besides, the gas consumption of contracts is a huge challenge. In "Cost analysis" section we show that although the contract code has been optimized and the overall cost of the AWESOME DApp is economical, there are some functional interfaces such as Place Bids and Calculate Witness Fee may invoke a large amount of gas consumption. This will, to some extent, hinder the widespread utilization of the AWESOME DApp. To handle this issue, some off-chain solutions (e.g., state channels and trusted execution environments) can be leveraged to offload the on-chain computation tasks to off-chain networks. In addition, contract vulnerabilities should be carefully investigated in the future to prevent diverse security attacks from causing huge losses to users.

Witness mechanism
Regarding the witness mechanism of the SLA, a possible optimization solution is to perform a random and fair selection of the registered witnesses, which can effectively avoid Sybil attacks and malicious witnesses. In this regard, an unbiased screening algorithm proposed in our previous work [25] could be integrated into the current model to improve trustworthiness. Besides, a reputation system could also be designed to prevent the inclusion of disreputable witnesses. Furthermore, the current judgment of the SLA violation is implemented based on a static game model between different witnesses. In the future, we plan to design more complex and effective game models and incentive mechanisms to motivate witnesses to report trustworthy monitoring results.

Privacy concerns
All data stored on the blockchain must be public to all peer nodes to ensure traceability, verifiability, and immutabi-lity. This conflicts with the privacy requirements of auction users, especially for those applications with critical business secrets [28]. Our AWESOME model will not be widely used if privacy and security are not adequately safeguarded. In general, blockchain-based auction models have two privacy concerns: identity privacy and transaction privacy. The former considers preventing transactions from being associated with specific auction users and their blockchain addresses. The latter covers the privacy of auction information regarding bids, contracts, payments, and other transaction details. Several privacy protection solutions are already available in this context, e.g., cryptographic techniques, mixing/tumbler service, differential privacy, trusted execution environment, and permissioned blockchain [19]. AWESOME does not restrict users from choosing auction models and application scenarios. Therefore, users should choose and implement appropriate privacy protection solutions for their specific needs in practice.

Conclusion
This work proposes a novel AWESOME model for building a decentralized cloud marketplace, which is enhanced by auction models and witness mechanisms. The model can support interactions between service providers, customers, and witnesses to complete trustworthy auctions and SLA enforcement. Compared with classic cloud service models, our model uses blockchain and smart contracts for decentralized service auctions; transaction efficiency is ensured through customizable auction models, and auction trustworthiness is guaranteed by the monitoring of decentralized witnesses. We also developed a prototype AWESOME DApp based on the Ethereum blockchain. It contains customizable GUIs and several advanced smart contract protocols to support the SLA business process. Extensive experiments are designed to evaluate the latency and cost of our model and DApp.
The experimental results demonstrate that our model is economical and feasible. It is worth noticing that the AWESOME framework aims to develop generic software architecture for a decentralized cloud ecosystem. Some features, such as the integration with other blockchain platforms (e.g., Hyperledger Fabric) and auction models (e.g., VCG auction and double auction), are still under development. In the future, we will continue to test our framework and demonstrate its feasibility in two ongoing industrial projects (i.e., EU ARTICONF and CLARIFY).