Solving EBS Performance Bottlenecks: IOPS and Bandwidth Challenges on gp3 Volumes

To understand how IOPS and Bandwidth impact each other, we’ll need to first understand the “B” from EBS, and that is – the block. Block storage is a generic name for storage systems that take any data and divide it into equally sized blocks (or at least tries to be equal). 

The way that data is translated into blocks and handled is done by the file-system itself in most cases (some applications like DB’s write directly to the block device and control it themselves but that’s uncommon). It is something we can’t really control, however the block size is what connects between IOPS and BW.

let’s look at an example using an EBS volume with a baseline of 3000 IOPS (gp3 volume baseline):

• Block Size of 16KiB:
  • IOPS = 3000
  • Block Size = 16KiB
  • Bandwidth Calculation: 3000 IOPS x 16KiB = 48MiB/s

Here, the resulting bandwidth of 48MiB/s is within the gp3 baseline bandwidth of 125MiB/s, meaning the EBS volume can handle this workload efficiently without hitting any limits. But what happens when we increase the Block Size to 64KiB? 

• Block Size of 64KiB:
  • IOPS = 3000 (same as before)
  • Block Size = 64KiB
  • Bandwidth Calculation: 3000 IOPS x 64KiB = 192MiB/s

In this scenario, the bandwidth requirement jumps to 192MiB/s, which exceeds the gp3 baseline bandwidth of 125MiB/s. So unless you purchase additional gp3 bandwidth, the EBS volume will be limited by its bandwidth. This means that despite having IOPS within the range of the gp3 baseline, the larger block size causes the bandwidth limit to become a bottleneck. As a result, to avoid exceeding the bandwidth limit, the effectiveness of the IOPS will be capped at 2000.