Skip to main content

Advances, Systems and Applications

Journal of Cloud Computing Cover Image

Table 1 Existing benchmarks studies for performance evaluation

From: Towards a unified customer aware figure of merit for CSP selection

Study Benchmarks Metric/unit
[26] WSTest Overall performance/transactions per second (jobs/s)
[6] GridNPB/ED + NAS/NGB Overall performance/transactions per second (jobs/s)
[27] Scientific workflow Overall performance/transactions per
  application second (jobs/s)
[28] Modified Grenchmark Overall performance/queue waiting and response time per second
  Developed by the authors Elasticity/VM acquisition and release time (s)
[10] TPC-E Overall performance/average transaction time (s)
  Unknown Elasticity/VM acquisition and release time and time per web role action (s)
[11] Developed by the authors Elasticity/VM acquisition and release time (s)
[29] CPU-intensive web CPU/duration of operation (s)
  Database read/write intensive Disk storage/duration of operation (s)
[30] Phoronix/crafty, dcraw CPU/test duration, MFLOPS
[31] TORCH/Dhrystone, spectral, particle CPU/total execution time (s)
[32] EnKF-based matching CPU/total execution time (s)
[8] Bonnie++ Disk storage/speed (KB/s)
[8] Compilation Linux Kernel CPU/duration of operation (s)
  iperf Network/transfer rate (KB/s)
  ping Network/RTT (ms)
[5] CSFV, NPB, HPCC/HPL CPU/total execution time (s), GFLOPS
  iperf Network/message latency (s), TCP throughput (bps)
[4, 13] HPCC/HPL, DGEMM CPU/GFLOPS
  HPCC/Random Access Network/speed (MB/s)
  HPCC/Stream, Cachebench, HPCC/ b eff Memory/GB/s