Dynamic resource provisioning and secured file sharing using virtualization in cloud azure

Abstract

Virtual machines (VMs) are preferred by the majority of organizations due to their high performance. VMs allow for reduced overhead with multiple systems running from the same console at the same time. A physical server is a bare-metal system whose hardware is controlled by the host operating system. A physical server runs on a single instance of OS and application. A virtual server or virtual machine encapsulates the underlying hardware and networking resources. With the existing physical server, it is difficult to migrate the tasks from one platform to another platform or to a datacentre. Centralized security is difficult to setup. But with Hypervisor the virtual machine can be deployed, for instance, with automation. Virtualization cost increases as well as a decrease in hardware and infrastructure space costs. We propose an efficient Azure cloud framework for the utilization of physical server resources at remote VM servers. The proposed framework is implemented in two phases first by integrating physical servers into virtual ones by creating virtual machines, and then by integrating virtual servers into cloud service providers in a cost-effective manner. We create a virtual network in the Azure datacenter using the local host physical server to set up the various virtual machines. Two virtual machine instances, VM1 and VM2, are created using Microsoft Hyper-V with the server Windows 2016 R. The desktop application is deployed and VM performance is monitored using the PowerShell script. Tableau is used to evaluate the physical server functionality of the worksheet for the deployed application.

The proposed Physical to Virtual to Cloud model (P2V2C) model is being tested, and the performance result shows that P2V2C migration is more successful in dynamic provisioning than direct migration to cloud platform infrastructure. The research work was carried out in a secure way through the migration process from P2V2C.

Introduction

A virtual machine is a device that runs on a local host system. The Virtual machine on the host allows user to configure the settings, customize user interface access and to access the services provided by the cloud service provider. Virtual machines allow for a great deal of flexibility and creativity. The Guest VM resides on the host machine and all the information can be accessed from the cloud. This VM architecture enables businesses to backup entire VMs at once. Backing up the entire virtual machine is possible. Virtual machines (VMs) allow users to work on multiple platforms [1]. Users can connect to various operating systems without installation on their local host machines. Automatic update patches will be invoked in the applications and projects.

Virtualizing involves creating virtual services such as servers, desktops, operating systems (OSs), directories, storage, or networks. Every physical server is partitioned into many virtual servers and different OS platforms for every server that can be ultimately provided to the user. The functionality of virtual servers is like that of local servers with high resource availability and scalability. The cloud server resources are shared for end users as if they had their own physical server with hardware, software and running multiple applications [2].

Most servers are idle because the workload is spread across only a few network machines. A lot of power is affecting the physical servers, and separate maintenance is required. This free migration can be deployed by VMware, which supports Linux and Windows client server configurations. Hyper-V transforms remotely configured physical computers to virtual configured servers and the performance of downtime is minimized.

The introduction to this research topic is about the process of converting and turning a physical server into a cloud. This can be achieved by transforming the virtual disk that is installed on the VMware workstation into a virtual hard disk. There is a data transfer to the cloud to reach desktop workstations at anytime and anywhere. Additionally, this gives the user the benefit of accessing the appropriate data. The effort is made by creating a virtual machine in VMware and then converting the server to a virtual hard disk and running the Visual Studio Desktop Application program. The application is developed and deployed on this newly created platform using C # code. The application workflow is to upload and download the file as securely as possible. The process is done using the Key Generation Algorithm to encrypt the file as a security layer. The source file can be removed to prevent any hacker attack on our local system, and then the file can be recovered by entering the key that refers to that file. Syncing the local application server with the cloud is one of the most important parts of this work [3].

In the Cloud Multi-VM model, one VM is for managing your desktop using Hyper-V, and the other VM is for big data. Big data is rendered by Horton Works Sandbox to execute Hive queries. Operations are handled by the database and file processes. Tableau is used to obtain an analytical view of the performance. Overall, monitoring is performed for the generated virtual server. The synchronization process is also achieved here by using the PowerShell script to connect to the local machine. Physical machines running Linux or Windows can be converted as virtual machines that function like real machines and data migration is performed. This method of transformation is known as a “virtual migration.”

The physical server operating system will migrate to a virtual machine on a physical server. In the virtual server to virtual server migration process, the virtual machine operating system can be migrated from one platform to various virtualization platforms. However, virtual machines made on one hypervisor can migrate to other rival virtual server applications.

Hyper-V is a Windows hypervisor that allows running multiple virtual machines on a single physical host. Microsoft added Hyper-V to Windows Server 2008. There are several ways to manage Hyper-V hosts. The Hyper-V Manager is the primary tool used to handle Hyper-V. This tool is also used to control remote hosts. This allows control of virtual machines as well as network setup and all Hyper-V features. The Hyper-V PowerShell application makes it simple to install and use Windows PowerShell within virtual machines. PowerShell can perform the majority of the functions that the GUI would perform. PowerShell is used to monitor the Hyper-V framework. However, PowerShell has several advantages. It allows jobs to be streamlined and provides more flexibility. In some cases, PowerShell features cannot be used because Microsoft does not always need all Hyper-V features in the Virtual Server.

The data warehousing system Apache Hive was built on top of Hadoop. Large datasets can be queried using Hive, which defines data for searching and analyzing data stored in Hadoop clusters, Hive offers the Query Language as similar to the SQL language. Writing and executing Hive queries more quickly is made possible by Hive view. Numerous calculations are required for data analysis. Tableau can be used to do calculations on the fields. There are numerous built-in functions in Tableau that facilitate generating calculations. Tableau is used to perform the worksheet functions analytically.

The remaining article is structured as follows. The detailed discussion of the virtual machine application challenges for deploying on various cloud platforms is discussed in “Challenges for virtual machine server in various cloud platforms” section. The related works are briefly discussed in “Related works” section. “The proposed framework” section defines the system for the proposed framework architecture and the workflow to build Azure virtual machines. In “Experimental testing in azure environment” section, experimental tests are carried out in the Azure environment, and the findings are illustrated. The Tableau analytical view is represented in “Tableau on azure” section. “VM performance tracking using PowerShell script” section visualises the performance metrics. The results and discussion are discussed in “Results and discussions” section. The conclusion and future work are outlined in “Conclusion and future work” section.

Challenges for virtual machine server in various cloud platforms

The findings of the research from the existing VM migration models are network link, seamless node connectivity, application deployment performance, resource heterogeneity, unidentified workload, and stable network connections. As a result, we propose physical to virtual and cloud migration to ensure minimal network disruption.

First, we virtualize the server and storage resources with VM, then host the application and test its performance. The Azure cloud framework is proposed to manage the virtual cloud infrastructure.

The main challenge in the existing framework is finding a third-party collaboration tool for efficient remote access to physical server resources. The physical server hardware and software resources should be accessible remotely based on availability. They should be turned on or off from the source to the destination using a remote access link mechanism. Resources can be accessed based on the performance of the VM server. Thus, to manage resource usage, the VM server performance report should be monitored.

The cloud migration process may be broken down into four steps: server migration, data and database migration, and migration planning. For the VM migration process, services are offered that may be divided into four stages: preparation for migration, record migration, server migration, and database migration [4].

The major IaaS providers, namely Amazon Web Services (AWS), Microsoft, and Google, offer free trial rates of cloud migration or inexpensive cloud migration services. Experimental testing is done with Azure cloud VM migration. The benefit of Azure migration is that it allows customers to evaluate the performance of VMware load on the Azure platform before committing to a full migration. Customers can also transfer VMs to Azure. This lets user to see how setup, application use, requirements, capacity, and resource use are being handled [5].

Google also offers some tools for cloud migration. Google Cloud Storage switches programs and applications to move the data into the cloud. Google Cloud provides offline transfer services. The AWS Cloud Migration Center provides users with tracking of the progress of their application migration. The AWS cloud migration service provides information about configuration, data usage, dependencies, storage, and use of resources.

Related works

Monge et al. [6] suggested a multi-objective workflow called Cloud Multi-objective Intelligence (CMI) to minimize the makespan, operational costs, and errors arising from unreliable VMs. The authors conduct simulation experiments in which CMI dramatically outperforms the state-of-the-art auto scaler with special features called Spot Instances Aware Autoscaling (SIAA).

Mao et al. [7] presented VM live migration as essential to cloud delivery to allow fast and consistent rescheduling of workloads with unevenly used nodes targeted at low energy consumption in utilizing server resources. The authors suggest an IO Follow scheme to increase VM efficiency and migration performance at the same time. Agarwal et al. [8] focused on secure VM positioning algorithms that the cloud provider automatically applies against such co-location-based attacks.

A system that identifies the data loss and prevents it during the data manipulation from a physical server to a cloud server during the migration process was proposed by Alkhonaini and El-Sayed [9]. This strategy will guarantee that the data sent to the cloud server is an exact replica of the physical server data. This will be achieved by algorithms that encrypt, compare, and segment the original data with transferred data.

Mavridis et al. [10] examined log analysis for the cloud computing systems Spark and Hadoop. The authors built log analysis programs in Hadoop platforms and ran queries against the Apache Server. Numerous experiments with distinct parameters have been evaluated to evaluate the efficacy of the schemes.

Ahmad et al. [11] examined cutting-edge optimization for bandwidth server optimization and power optimization by Dynamic Voltage and Frequency Scaling (DVFS) and techniques for maximizing storage across WAN lines. Using a thorough study, the essential components of the migration of virtual machine strategies are investigated.

Varasteh et al. [12] focused on the virtualization of virtual machines (VM) in cloud service provider data centers. Virtualization approaches enable the live migration of virtual machines between physical servers with minimal failure. Thus, many virtual machines are loaded onto a single physical server, allowing the physical server to operate in a highly energy-efficient mode. Server optimization can be accomplished in a variety of ways, with varying consequences taken into account. The authors discuss the implementation methods and algorithms for virtual machines on physical servers.

Amann et al. [13] created a method for practicing visual studio practice. The authors have taken the formerly unexplored Visual Studio Pro and traced the connection of the coders to the manufacturer’s IDE. The authors got interactions of more than 6300 hours of work time, between twenty-seven and eighty-four professional level C# programming summaries in Code. This work shows how long coding uses the repeated tools provided for the Interactive Development Environment (IDE). Wen et al. [14] have developed a method for secure and open cloud modeling of data replication. Cloud storage varies from objects stored in containers to blobs, files, etc. Cloud apps must store secure and cached content in the repository. The authors suggest that cloud storage could be made better by migrating containers and copying backup data.

Jordan et al. [15] developed a model that facilitates training of prediction algorithms on another framework. Microsoft Windows does not have a load generator optimized for an active directory context. The authors present a tool to apply this new framework in a Microsoft environment and show the data generated by the machine to illustrate its effectiveness. Liu et al. [16] claimed that the Apache Hive has been commonly used by many companies to process Big data over massive clusters. It offers a declarative query language called HiveQL. The efficiency of filtering out non-relevant data from the Hadoop Distributed File System (HDFS) has a similar impact on the performance of database processing. It is especially valid for multi-dimensional, high-selective, and few query-involved columns that have ample information to minimize the number of reading bytes. To achieve this, indexing and columnar storage are appropriate strategies. However, it is not easy to select an optimal index and columnar storage based on data and database features. The authors compare data filtering by performing comprehensive tests using uniform and skewed TPC-H datasets and different multi-dimensional queries.

Choudhary et al. [17] surveyed the VM migration on single or multiple virtual machines for duplication, replication, redundancy, dependency, compression among VM’s dirty pages and page faults caused by the cluster of VM pages.

The proposed framework

Our study contributes to the fact that various business systems and apps can be run on a single server. Therefore, the company’s effectiveness and economy can be observed. Virtualization enables businesses to operate applications over a virtual network rather than a physical server network. Virtualization is a simple and cost-effective process. The data from several physical servers can be transferred to a single virtual storage. Administrators do not need to take time to find data by digging out hard disks. Using a specific server and storage, it is easy for the cloud to find the files it needs and send them.

Physical to virtual to cloud (P2V2C) framework

VMware Workstation is installed and mounted on a physical server. The VM on Win Server 12 R2 Data Center must be built on this, and the application must be evaluated. Then migrate the physical server disk to Virtual Hard Disk (VHD) through the commands. The virtualization is uploaded on the cloud, and the virtual infrastructure is converted to cloud infrastructure [18]. The synchronization process is followed by tracking the multiple VM servers’ performance reports. In this circumstance, a physical to virtual to cloud migration is better resource management for utilizing the resources. The virtualization of data center from physical to cloud provides the full-time accessibility, resulting in specific cost-management features [19]. A physical server application is deployed on instance VM1 in a cloud infrastructure, and analytics are performed on VM2 using Hortonworks Sandbox. Virtual machine performance is monitored in the cloud for the admin to manipulate or maintain the background services which are running on the state.

Physical to virtual to cloud (P2V2C) architecture diagram

The proposed physical to virtual to cloud architecture is shown in Fig. 1.

Fig. 1

Physical to virtual to the cloud architecture diagram

Figure 1 shows the proposed physical server to virtual server to cloud architecture. The desktop application is using Visual Studio as front-end and SQL server as back-end is migrated to the Azure cloud through the Hyper-V hypervisor connecting to windows server. The other services are availed from cloud server. The local server acts as the host system. VM Windows or Linux server will act as a guest machine to migrate or host, or run the applications. The performance of VM1 and VM2 can be measured using powerShell script.

The transformation is not an easy process to follow, commands should be checked, and instead, the Azure subscription is required to link virtualized cloud networking to the local application server for full-time access to the physical server. The system can be used as a login or for web browsing. The hardware virtualization and synchronization processes are some of the components that make it more useful to use a customizable system.

Workflow to create azure virtual machine

Install VMWare Workstation and build the Windows server edition 2012 R2. When Server 2012 R2 hits the data center, using Microsoft Hyper-V creates a new kind of VM. Design the application using Visual Studio with C# code as front end, SQL Server Management Studio as the back end (Database). Sample application code is written for secure file management; upload the file and a key is generated in the mail to download. Then convert that VM disk to a virtual hard disk for physical to the virtual conversion process. Get an Azure subscription for uploading the VHD in storage account files. Build a Windows Server 2016 Datacenter VM in Azure to set up the base machine for the physical server. Map the VHD using network file share commands and build Hyper-V using PowerShell script for VHD mounting. Synchronize from the cloud to local using a PowerShell script. Build another VM (Winserver16) for analytics. The processing of data is performed on Hortonworks Sandbox Hive. Queries are performed using HQL. Tableau can be used to get an analytical view of desktop worksheet functions. A virtual server’s performance can be monitored and tracked using PowerShell script. The workflow to create an Azure virtual machine is mentioned below.

Step 1: Login module for Microsoft Azure account.

Step 2: New VM with instance created.

Step 3: Tune the parameters of the virtual machine.

Step 4: Review the parameters and create the virtual machine.

Step 5: Connect with login credentials to the virtual machine.

Experimental testing in azure environment

Virtual Machine1 (VM1) and Virtual Machine 2 (VM2) instances are created using Hyper-V. The application is deployed, and the performance of the VM is monitored using the PowerShell script. The process flow is illustrated in the following subsections.

VM1 instance creation under myResourceGroup in myVirtualNetwork1/Subnet1

Virtual Machine creation using PowerShell commands are

The Virtual Machine VM1 is created through myVirtualNetwork1/Subnet1 where VM1 Name = “TEST1” and the User = “ABC”. TEST1 VM1 instance and its properties is shown in Fig. 2.

Fig. 2

TEST1 VM1 and properties

VM2 instance creation under myResourceGroup in myVirtualNetwork2/Subnet2

Virtual Machine creation using PowerShell commands are

The Virtual Machine VM2 is created through myVirtualNetwork2/Subnet2 where VM2 Name = “TEST2” and the User = “XYZ”. TEST2 VM2 instance and its properties is shown in Fig. 3.

Fig. 3

TEST2 VM2 and properties

File share through network mapping in VM1

The VHD is uploaded and connected with the PowerShell command on the Azure VM to map the network drive to access the file [20]. The PowerShell script is used to map the storage account container file share to VM connect. Then network drive ‘Z‘is mapped to access it as any other normal drives. VHD is deployed in Azure and mounted using a PowerShell script as the Hyper-V role is set up. The Hyper-V VM 1 represents that the physical machine is migrated to the cloud. The application is deployed for secure file uploading and downloading by key generation. The drive mapping and key generation PowerShell Script for VM1 are illustrated below.

Figure 4 shows output of PowerShell script to map a local host physical machine drive ‘Z’ in VM 1 and to access file from Z drive to VM1 with the key generation.

Fig. 4

PowerShell Script to Map Network Drive ‘Z‘

File share through network mapping in VM2

Big data and Hortonworks framework setup are made by installing and configuring the Sandbox. The platform known as the Ambari view login can be used to process Hive data. Files are submitted to Hive view to process data as a CSV file [21].

In the Azure portal, log in to VM1 configure storage services and create a document named “Cloud Computing“. Then navigate to the ‘qsfileshare‘ and connect to map the Azure file to a windows drive ‘Z‘. The created file “cloud computing” view in Z drive mapped to path \\kumaresan.file.core.windows., in VM1 is shown in Fig. 5.

Fig. 5

File view in VM1

The ‘qsfileshare‘ is mounted on drive ‘Z in VM1. The shared drive shown in Fig. 6 is mounted and shared to VM2. where cloud computing document stored in qsfileshare can be accessed in both VM1 and VM2.

Fig. 6

Mounting Z drive in VM1 and shared to VM2

Tableau on azure

The analytical view is performed using Tableau to show the physical server worksheet functions. The various options of the output view are to extract the result in different visualization. Figure 7 shows the analytical view of the worksheet using Tableau running on a physical server hosted in VM1 and VM2.

Fig. 7

Analytical view of physical server worksheet functions using Tableau

VM performance tracking using PowerShell script

At the enterprise level, once the user gets to the Azure cloud as a network means, there are several virtual servers running in the background on that database platform [22]. The performance of a VM utilizing resources can be easily monitored and controlled as high or low and so on. With the support of this PowerShell script and its subsequent output, multiple performance reports could be generated for VMs in a single file. Figure 8 shows the diagnostics report of performance tracking and monitoring of VM1 TEST1 using PowerShell commands. The resource utilisation is also shown in Fig. 8. Validation in Azure in PowerShell is performed by monitoring the virtual machines with the PowerShell “cmdlet“.

Fig. 8

Performance diagnostic report of VM1 TEST1

To enable the virtual machine the cmdlet is

The end-to-end synchronization and performance tracking of VM1 were successfully performed using the PowerShell script. Figure 9 shows the monitoring metrics for Z drive on VM1 based on the total number of requests received to access the drive.

Fig. 9

Monitoring metrics for VM1 TEST1 Z Drive

The Blobs (Sum) is the total of Blob storage used in the storage account. The Queues (Sum) is calculated with the storage account used by the Queue. The Tables (Sum) is calculated with the storage account use by the Table storage. The files (Sum) in the storage account are calculated with the number of files. FileCapacity is the amount if FileStorage is utilized by the storage account. The number of files in the storage account is file count. The number of queries made to a storage service or the specified application program interface constitutes a transaction. This statistic includes both successful and unsuccessful queries, as well as those that resulted in errors.

Results and discussions

Test results are obtained from the initial setup to the final phase. The commands are efficiently performed for the migration process from a local physical server to a virtual hard disk format and then to Hyper-V VM. The secure file access application is deployed effectively using a key generation algorithm. The code is written in C#, and the SQL server is used as the backend database. The mail service is configured to get the key for viewing and downloading the file. The application runs synchronously from the physical server to the cloud framework. Because of this users can easily connect to remote physical servers with the utilization of all the resources. Hive facilitates data processing efficiency to be achieved. Tableau represents the analytical view of physical server worksheet functions. The end-to-end synchronization and performance tracking of VMs are successfully performed using the PowerShell script. The performance results show that the proposed P2V2C migration is more efficient than direct migration to the cloud platform infrastructure. The comparative study of our proposed work is given in Table 1. The proposed method P2V2C using Hyper-V hypervisor outperforms existing methods in dynamic provisioning of resources and is cost-effective.

Table 1 Comparison of VM migration methods with the proposed work

Conclusion and future work

The proposed work of converting physical server to virtual server and to the cloud is successfully implemented using Azure cloud services. The Azure cloud framework is developed, and virtual machines VM1 and VM2 are created using Hyper-V. Secured Dynamic Provisioning of resources is implemented during the migration process from physical to virtual to a cloud model. At each point, security measures are taken to ensure that the data is transferred without interruption. Then the application is deployed successfully with synchronization and minimal disruption. The evaluation and performance results show that P2V2C migration is more successful in dynamic provisioning of resources than direct migration and is also cost-effective. The drawback of this system is migrating the complete physical server to a cloud service provider.

The future work is to implement full virtualization to enhance the complete accessibility of physical servers anywhere and at any time. The performance analysis should be conducted on multiple numbers of virtual machines with multiple applications installed.