As IT staff deploy virtual machines at an exponential rate, they inevitably reach the performance limits of the spinning disks, exceeding maximum storage IOPS before reaching maximum capacity. As a result, a significant amount of capacity is left unutilized.

Consider the four main components of virtualization: CPU, memory, network and disk storage. CPU power continues to increase in accordance with Moore’s Law, doubling approximately every two years by providing an ever increasing number of processing cores and clock speeds. We also see memory sizes increasing rapidly owing to low-cost RAM modules and faster connectivity such as 10 Gigabit Ethernet, 8Gb Fibre Channel and 40Gb InfiniBand. All of this allows more VMs to be deployed at increased levels of performance.

When we look at disk technology, however, classic spinning disks remain limited by their mechanical components, constraining the number of IOPS and thus the number of VMs they can support. Although SSD technology will alter this relationship, it is unlikely to become mainstream as primary storage for several years because of cost/capacity constraints and reliability concerns. So, as the IT industry’s thirst for greater numbers of VMs grows, it puts significant pressure on storage infrastructure—an issue that can only be addressed by more-efficient design.

When did storage become so critical?
By making servers ‘virtual’ users can move them between hardware swiftly and without downtime, leaving the physical machine to be viewed as little more than a CPU with a bit of memory, and I/O. Conversely, in a VM environment, the storage system grows in importance, as it becomes the underpinning of the entire infrastructure. In a virtualized environment the traditional system disks are provisioned from the central storage, not only adding load but also adding to the randomized data access pattern as many virtual servers concurrently contend for disk resources.

Consider this example: Administrator ‘A’ wants to consolidate and virtualize his infrastructure. He has 25 Windows servers, 5 Linux servers, MS Exchange, an ERP system, two small SQL databases and user home directories. The administrator in this scenario will often invest in several new servers with a drastically increased number of CPU cores and provide needed memory but may neglect to size the storage system properly. The problems start here because many factors, including capacity, types of drives, RAID levels, throughput and random I/O performance, need to be considered.

Performance versus capacity
In the classic disk-drive market, significant capacity increases occur every 18 months. But we don’t see a significant increase in spinning-disk performance when benchmarking SATA 1 versus SATA 2 drives or a 3Gb SAS versus 6Gb SAS drives. Performance results remain the same. In the past this may not have presented a challenge, as disk capacity remained so low that most SAN solutions included upwards of 50+ disks to provide any useful capacity. This many disks provided plenty of IOPS per GB of capacity.

In the current technology climate, a prevalence of cost-effective SATA drives could provide the same capacity with one-fifth or one-sixth the number of disks required, compared with SAS drives. This use of SATA drives significantly reduces the number of IOPS per GB and, if used in a demanding random-I/O environment like a highly transactional database or large virtual server infrastructure, SATA disks and their IOPS capability will bottleneck long before the capacity limit is reached, unless they are fronted with solid-state cache, which can increase the system’s random-I/O performance by three to ten times.

It’s also worth looking at the approximate breakeven cost points of different disk technologies:

• In the 100–3,000 IOPS range, SATA drives provide a very cost-effective platform, with pricing usually provided in dollars per GB.
  
  In the 3,000–10,000 IOPS range, SAS drives are usually the default technology, as reaching this performance level with SATA requires a vast number of spindles or amount of SSD caching. High-performance disks are typically priced higher per GB.
  
  In the 10,000+ IOPS range, SSD begins to make financial sense, as only a fraction of a customer’s overall storage requires such levels of performance. But the best use of flash is as cache.
  
  I/O Data Patterns

The pattern in which an application or ‘host’ server reads or writes its data can significantly affect the performance of the storage system.

Data patterns are usually referred to as either random or sequential. A random data pattern implies that the data is written or read from random areas of the disk platter. This has two main effects on the performance of a RAID system. First, it drastically reduces the effectiveness of the controller cache, which relies on patterns to ‘guess’ which data blocks will be read or written next. In a random data pattern this is not possible, since a random sequence of events can never be ‘guessed’ and, as such, cached, but ‘hot data’ does tend to gravitate into the cache.

The second crucial effect of random patterns is an increased number of ‘seeks’: the point at which a disk head must move to the next requested data block. If this block is randomly placed, the disk’s actuator arm and head must move a significant distance to seek the block for each read or write. This situation adds significant overhead and reduces performance.

For example, SATA drives, which use larger disk platters, suffer under very random workloads, as they only spin at 7,200rpm and have longer seek and access times (8.1ms on average). SAS drives are better suited because they have smaller platters and spin at 15,000rpm, and they seek in about half the time (3.3ms on average). In extreme high-performance applications, SSD are an alternative because they have no moving parts, making seek times near nonexistent.

With a random workload, spin speed and access time are crucial to spinning-disk performance. The faster a disk spins, the more IOPS it will provide. By contrast, a sequential data pattern is one of structure and predictability: for example, data backup and video streaming. In these applications, the files are typically large and are written to the disk in continuous blocks and sectors.

With this in mind, the RAID controller and disks can more easily ‘guess’ and/or cache the impending data blocks to increase performance. In addition, the disk actuator arm and head need not move a great distance to seek the next requested block. Such sequential applications are usually designed around MB/s (throughput). This design is rarely limited by disk speed and more commonly limited by the controller and interconnect. So, in a storage design for sequential applications, SATA, SAS and SSD disks provide very similar performance levels. The quick rule of thumb is that sequential patterns are those with large or streaming files and are best suited to SATA drives. Random workloads are typically those with very small files or storage requests that have no consistent structure (virtual servers, virtual desktops, transactional databases and so on) and are best suited to SAS or possibly SSD.

The impact of RAID
Understanding data patterns and disk types is crucial when designing storage for specific applications, but RAID level/type must also be considered. The storage concept of ‘parity penalty’ refers to the performance cost or impact of protecting data via RAID. This penalty only exists on writes, so it is important to know if the environment is write- or read-intensive. These are the RAID protection parity penalties:

• RAID0: ~0% overhead vs. reads
• RAID1+0: ~50% overhead vs. reads
• RAID5: ~75% overhead vs. reads
• RAID6: ~85% overhead vs. reads
  
  With these write overhead costs in mind, consider the following:

• SSD drives are designed for random workloads, so they should typically be configured as RAID1+0 to maximize performance (unless an environment is 100% read).
• SAS drives are also aimed at performance. Therefore, RAID1+0 or RAID5 should be used.
• SATA drives are aimed at capacity with throughput, and because of their huge capacities should be configured as RAID6.
• RAID6 also provides additional security and peace of mind during rebuilds for backup applications where SATA drives are preferred.
  
  Note: RAID5 can be considered when using 2TB drives or smaller. RAID1+0 can also be considered in very high-scale virtual infrastructures of 2,000+ virtual machines.
  
  Designing the storage system
  The key to designing an efficient storage solution is understanding application and environment requirements. Developing the knowledge to design the right architecture can come from technical meetings and discussions, remote analysis, on-site professional services and studying application best-practice guides on IOPS requirements for Exchange, SQL, VMware View or other applications specific to the environment. In every case, the basic goal is to determine if the environment/application is sequential or random in nature. Next, discover the requirements for capacity, throughput (MB/s) and/or IOPS. There may also be requirements for storage functions such as snapshots and replication.
  
  If the full range of data is unavailable, simply knowing the operating system and applications will give you a direction in the design. The most demanding servers in the customer environment can be monitored using I/OStat (Unix) or Perfmon (Windows). When used correctly, these built-in tools can provide all the data needed.
  
  Another option is to use third-party monitoring applications such as VMware Capacity Planner. Such applications will gather detailed performance information and produce storage reports. Finally, you may gather performance statistics from your existing storage system.
  
  All this data offers a starting point for designing the storage solution. In a random-I/0 environment, you will want to balance IOPS and capacity. In a sequential environment, the design will focus on capacity and throughput or MB/s. It is worth noting that sequential storage systems are much easier to configure, as the MB/s ratings almost always exceed the requirements.
  
  Now that disk type, RAID level and capacity have been determined, it is time to choose the interconnect type. Although this choice is typically dictated by the existing infrastructure or preference, there are performance implications to consider. Redundancy is also an important factor when it comes to connectivity. Some systems are shipped with dual RAID controllers, and each connected server must detect and address at least one data port from each controller. This arrangement enables continuous access to data even in the event of a controller failure. In most cases it requires multipathing to be configured at the host.