To obtain empirical evidence to verify and demonstrate the theoretical discussion in “Theoretical discussion” section, we conducted an experimental study on different efficiencies of data communication of an edge cloud application. The application scenario is assumed to be remote health monitoring that was also used in our earlier study [13], as shown in Fig. 4. In this scenario, we consider the application’s business logic to be: (1) wearable sensors continuously detect the host patients’ health parameters and send their values to a closeby broker; (2) the broker periodically processes the collected raw data into prediagnosis reports that are atomic data entities in this case; and (3) once a prediagnosis report is ready, the broker immediately sends it to a hospital who remotely monitors the health status of those patients. Moreover, we constrain that each prediagnosis report should be small enough to satisfy Theorem 1. Then, we focus on the one-way communication from the broker to the hospital for our investigation, which essentially corresponds to the relay-destination transmission described in “Related work” section.
Testbed setup
We distinguish between the application and its runtime environment when building up a testbed for this experimental investigation. In particular, the runtime environment is further decomposed into two parts for the sender (i.e. relay instead of original data source, as represented by Prediagnosis Broker in Fig. 4) and receiver (i.e. destination, as represented by Cloud (Hospital) in Fig. 4) of the prediagnosis reports respectively, as specified below.
-
Application. We developed a mobile app with Android Studio to simulate the health monitoring application.Footnote 1 To simplify the sensor data collection (from Wearable Sensors in Fig. 4) that is out of this study’s focus, we pre-stored a group of “people’s” health parameters in a JSON-format file to mimic the moment when the broker is ready to do prediagnosis. The prediagnosis is composed of simple comparison measures against standard values, which can be quickly done including generating PDF-format reports.Footnote 2
-
Report Sender (i.e. Prediagnosis Broker that plays the relay role in the application). The developed mobile app is deployed on a Redmi Note 8 Pro handset with the CPU Octa-core (2x2.05 GHz Cortex-A76 & 6x2.0 GHz Cortex-A55) and the operating system Android 9.0 (Pie). Here we let the handset play the broker role who tries to send out prediagnosis reports to the remote cloud (hospital) at soon as possible. In particular, the handset works in a stable WiFi environment with about 101 Mbps download speed and 5.6 Mbps upload speed (measured by using the Speedtest appFootnote 3).
-
Report Receiver (i.e. Cloud (Hospital) that plays the destination role in the application). Since this study is focused on the data communication only, the post-transmission activities of health monitoring (e.g., cumulative data analytics, setting off alarms, etc.) are out of the scope of this paper. Therefore, we assume that the role of remote hospital merely keeps the prediagnosis reports after receiving them. In this testbed, we employed Firebase StorageFootnote 4 to act as the report receiver. Considering that Firebase Storage is backed by Google Cloud StorageFootnote 5, the transmitted data are eventually stored in the Google cloud, which essentially enables our testbed to reflect real edge-cloud application features.
Note that, following the definition of atomic data entity, here we treat each prediagnosis report as a coherent and irreducible message from the sender to the receiver. Since each report has about 1.4 KB of data, the atomic data entity in this testbed clearly satisfies the aforementioned size constraint.
Two transmission mechanisms
As demonstrated in Fig. 3 via different communication options, we take into account two mechanisms for transmitting prediagnosis reports in the scenario of remote health monitoring, namely FIFS transmission and buffer-aided transmission.
-
FIFS transmission is part of the traditional and straightforward implementation of the application’s business logic. In this implementation, an instance of data transmission (i.e. an uploading request) starts immediately after the prediagnosis processing delivers a report. Given a set of jobs of pre-diagnosing a group of people’s health status, the delivered reports are transmitted consecutively in the FIFS manner, as shown in Fig. 5(a). It is noteworthy that the FIFS transmission mechanism corresponds to the illustrated Option 1 of data communication in Fig. 3, and it also acts as the baseline for us to study the buffer-aided transmission mechanism.
-
Buffer-aided transmission temporarily keeps the generated prediagnosis reports and delays their transmission until reaching a predefined buffer threshold. Then, the buffered reports will be transmitted all at once rather than individually, as shown in Fig. 5(b). It is noteworthy that the buffering mechanism here makes common sense, without requiring unique techniques or sophisticated manipulations. As such, we expect this research to draw relatively generic conclusions with respect to the time efficiency of buffer-aided relaying in edge-cloud data transmissions. Particularly, in our developed app, since Firebase Storage allows an upload task to include a single file only, we merge the generated individual reports into one PDF file when buffering them. Correspondingly, we predefine and adjust the buffer threshold by varying the number of reports to be merged. In other words, we manually control the eventually programmatically adjustable buffer size in this research. As such, we are able to gradually investigate the effective buffering range, as demonstrated in Fig. 3 with regard to Option 2, Option 3, and Option 4 of data communication.
Measurement metrics
This study on data transmission’s time efficiency is essentially to engineer the communication performance of edge cloud applications. Since the selection of measurement criteria or metrics plays a prerequisite role in performance engineering for communication and computing systems [37], practitioners need to decide what to measure before conducting the experiments.
Following the theoretical timing discussion about data communication (cf. “Theoretical discussion” section), we choose the primary metric to be Latency that is widely used to measure the elapsed time of an activity, a process, or a task/job. Thus, ideally, we can use different timers in the testbed to obtain the latencies of data processing, waiting/buffering, and transmitting report for every single prediagnosis task.
However, since the public cloud environments are uncontrollable (often invisible) from the perspective of cloud users [36], it is impossible for us to set any timer inside Firebase Storage from the cloud. Instead, Firebase Storage allows app development to use an OnSuccessListener to know if a report is uploaded successfully or not. As such, although we cannot obtain the exact timestamp when a report arrives to its receiver (in the cloud), it is still convenient to understand each report’s successful uploading latency, at least from the sender’s perspective. In detail,
-
under the FIFS mechanism, we employ four timers in every prediagnosis task: (i) when the prediagnosis processing starts; (ii) when the prediagnosis report is generated; (iii) when the report uploading starts; and (iv) when the report is uploaded successfully. The expected measurement results are then named as Processing Latency from Timer (i) to (ii), Waiting Latency from Timer (ii) to (iii), and Uploading Latency from Timer (iii) to (iv). In particular, here we define Transmission Latency to be equivalent to the uploading latency in this case.
-
under the buffering mechanism, we employ four timers with respect to each uploading event: (i) when the prediagnosis processing starts; (ii) when the buffering starts (as soon as the buffer size is saturated); (iii) when the uploading event starts (as soon as the buffering is done); and (iv) when the buffered reports are together uploaded successfully. The expected measurement results are then named as Processing Latency from Timer (i) to (ii), Buffering Latency from Timer (ii) to (iii), and Uploading Latency from Timer (iii) to (iv). Note that, unlike under the FIFS mechanism, the processing latency here covers the generation of multiple prediagnosis reports (depending on the buffer size). Moreover, to be consistent with the discussion in “Related work” section, we define Transmission Latency in this case to be the combination of buffering latency and uploading latency.
Experimental analysis of FIFS transmission
Following the principles of Design of Experiments (DOE) [38], we repeated the FIFS health monitoring experiment multiple times (five times in specific) for 50 made-up individuals, and then calculated average values of latency indicators. According to the experimental results, there is no waiting latency between the data processing and report uploading in the individual prediagnosis tasksFootnote 6, as illustrated in Fig. 6. In terms of “waiting”, there are merely queuing delays for the later tasks due to the FIFS nature. Such a phenomenon is the result from a combination effect of the asynchronous mechanism of Firebase Storage and the single-thread generation of prediagnosis reports. Since the performance bottleneck is data processing rather than data transmission in this case, none of the generated prediagnosis reports needs to wait to be uploaded.
We believe that the waiting time in the prediagnosis tasks will appear when we speed up the report generation, e.g., through multi-thread processing. However, the investigation into multi-thread report generation has been intentionally excluded from this study. Recall that the broker role is played by an edge device in this research. Given the huge heterogeneity in edge devices and technologies [39, 40], we have to conservatively relax the assumption of brokers’ multi-thread processing capacity. For example, the broker in this research can also be implemented on a WiFi-supported Arduino YUNFootnote 7 that has limited compute resources.
To facilitate discussing our observations, we demonstrate the average measurement results for the first 10 reports in Fig. 6. In specific,
-
1)
The dash green part of each line indicates the queuing delay before processing the corresponding patient’s data. It is clear that data processing for the first patient does not have any delay.
-
2)
The tiny red part of each line represents the data processing latency until delivering the prediagnosis report. Given the simple prediagnosis logic in our testbed, generating one report takes from about six ms to 22 ms.
-
3)
The long blue part of each line shows the report uploading latency between the moment when the prediagnosis report starts transmission and the moment when the successful uploading is heard (by the OnSuccessListener).
Such measurement results have denied the previous hypothesis of serial data transmission illustrated in Fig. 3, i.e. there are only task queuing delays instead of any waiting latency before transmitting individual reports. This also justifies the needs of empirical investigation in addition to theoretical discussion. Benefiting from the asynchronous strategy of Firebase Storage (i.e. putFile() in our testbed), the successor prediagnosis task can be launched right after starting the precursor report transmission. Thus, the individual reports are generated regularly about every 10 ms on average, indicating roughly the same overhead of single-thread data processing. Meanwhile, the transactions of report uploading are also issued regularly, with overlapping time windows, in parallel.
However, there exhibits a trend that although the data transmission transactions start regularly, it would take longer and longer time (wall-clock latency) to finish the continuous prediagnosis tasks, especially after the first several ones. This phenomenon can be narrowed down to the uploading latency, as portrayed in Fig. 7.
The shape composed of plotted uploading latencies is similar to a hockey stick. The measurement result keeps increasing regularly from uploading the fourth report on. Considering the low network utilization in this case, we believe this phenomenon is because of the incremental resource competition when saving reports and/or the queuing delay of issuing response signals from the receiver side (in the cloud).
To verify such an explanation, we further tested serial data transmission by forcing the later uploading events to wait for the former success signals. Ideally, there should not be resource competition anymore in the new test. Thus, according to the illustrated results in Fig. 8, we claim that the plain uploading latency of a single report can be as short as less than 300 ms on our testbed. This confirms that the longer latency of uploading individual reports via Firebase Storage’s asynchronous approach must have involved extra noises, and one noise source must be the resource competition when uploading data asynchronously. However, by focusing on the wall-clock latency of multiple tasks from the holistic perspective, the asynchronous FIFS transmission is still more efficient than the serial data transmission. Therefore, we stick to the asynchronous uploading for investigating the FIFS transmission mechanism in the rest of our study.
Experimental analysis of buffer-aided transmission
Given the above lessons from the experimental analysis of FIFS transmission, it is foreseeable that the asynchronous buffer-aided transmission will also come with extra measurement noises. To better isolate and compare different buffering effects, we decided to emphasize the standalone communication transactions’ latencies, by experimenting with serial transmission of buffered data.
When it comes to observing different buffering effects, since controlling the buffering time will introduce higher overhead (e.g., frequent timestamp comparisons) in the testbed, we vary the buffering threshold by modifying the amount of prediagnosis reports to be buffered, i.e. adjusting the buffer size instead of controlling the buffering time. In fact, when successively including reports in one communication transaction, the buffering time window will correspondingly be expanded in a fine-grained manner, as demonstrated from Option 2 to Option 3 in Fig. 3. Thus, focusing on the buffer size still allows us to match our empirical investigation with the previous theoretical discussion about the buffering time span (e.g., Lemma 1).
In detail, we performed 49 experimental trials of buffering two, three,..., and 50 prediagnosis reports respectively. Note that “buffering” one report is meaningless in practice, as justified in Theorem 2. For each buffer size, the corresponding trial continuously buffered and transmitted data entities 50 times in the serial manner, and also recorded relevant timestamps of the 50 transactions. After using the function TRIMMEAN (array, 20%) to exclude potential outliers, we calculated the average latencies of a single transaction with respect to different buffer sizes, as plotted in Fig. 9.
As can be seen, along with the increasing buffer size, the latency trajectories generally show an upward trend. In particular, the lower trajectory represents the uploading latency only. Its upward trend indicates that the data size still proportionally influences the uploading latency, even at the kilobyte level. The upper trajectory represents the combination of both the buffering latency and the uploading latency. Thus, the distance between those two trajectories reflects the influences of buffering on the whole transmission latency. Despite the common sense that buffering more data would take longer time, the buffering overhead (against the data size) will gradually dominate the transmission latency when the buffer size keeps increasing, as revealed by the slope difference between the lower and the upper trajectories.