Performance figures are sometimes quoted including clock speeds in Megahertz along with other measures. Some years ago, the other measure was often millions of instructions per second or MIPs and, for scientific work, thousands or millions of floating point operations per second or KFLOPs/MFLOPs. The MIPs measure fell into disrepute over time as systems with an apparently superior speed in MIPs were found to be inferior when measured by a different, more realistic, benchmark.

The original system which was quoted in MIPs was the DEC PDP 11/70 (1 MIP processor). As the MIP began to lose credibility, DEC VAX systems were sometimes quoted in VUPs or VAX Units of Power. The same PDP 11/780 was used as the base of 1 VAX Unit of Power (VUP).

It can be seen from various sources that MIPs per MHz increases as the years go on and processors evolve and the reason is enhancements in architecture and chip design (Moore’s Law here).

Misleading MIPs

What about millions of instructions executed per second (MIPs) as a performance metric? Firstly, it is often difficult to know how many instructions are being executed by the operating system and other subsystems and how many go on useful work. Assuming that MIPs was a valid measure of performance, a 2 MIP machine is not necessarily capable of twice as much work as a 1 MIP machine of a different architecture since the efficiency of the instruction use may be different on the two machines.

For example, the following table shows results for ‘millions of instructions per second’ or MIPs for some IBM processors. 

Table  Which vintage IBM CPUs are represented here by their MIPs ratings?

The answer in all seven cases is the IBM 3081D, announced in 1981. The reason for the variation in answers for ‘MIP’ ratings is the fact that the instruction mixes are different in each case.  The ‘big MIP’ results could be accounted for by a mix of fast register-to-register instructions, whereas the ‘low MIP’ results can be accounted for as mixes of complex or heavy duty  diagnostic instructions. The metrics of this simple benchmark are so simple that it is easy to deceive people by quoting the performance numbers in isolation.

MIPs then are only useful in perhaps comparing the performance of different models of the same range of processors but outside that the measure open to much ridicule, except by people who rely on them to sell hardware.

I/Os per second legends

The performance of an application depends on the availability of adequate IT resources, such as CPU, memory, storage and so on. Storage metrics of interest are:

- Data capacity

- Input/output capacity (I/O performance)

- Durability, space, cooling, cost, ROI and other mainly commercial factors.

We are concerned in this section with the second item, I/O capability, which is not as simple as “My system does X input/output operations per second (IOPs)”. First, let us look at some background to input/output.

Records:  A record to an application usually means a logical record, for example, the name and address of a client. This can be made up of more than one physical record, which is normally retrieved as a block of a certain size, for example, 2048 bytes. Some though, a physical record may contain more than one logical record.

Disk Access:  An I/O operation consists of several activities and the list of these depends how far you go back in the chain from data need   to fulfilment. This is shown in the I/O time equation above.

Myth 1

This myth is propagated widely in internet articles and is totally erroneous, so beware. The misconception is a follows: if an I/O operation (seek, search, read) takes x milliseconds, then that disk arm is capable of supporting  1000/x  I/Os per second (IOPs). Yes it is if you don’t mind a response time of approximately infinity, give or take a few ms as the disk actuator would be running at 100 percent utilization.

A sensible approach would be to do this calculation and settle for, say, 40% of this IOPs rate as an average which might be sustained.

Myth 2

If we make the allowance above, then a storage subsystem supporting x IOPs will perform better than one supporting 0.8x  IOPs. In its raw form, this statement is not true I’m afraid, since the I/Os needed to satisfy an application’s request for data depends on other factors, many within the designer’s control.

1. the positioning of the physical data and its fragmentation, the former no longer in the control of the programmer, the latter a fact of life, except for the ability to defragment when necessary

2. the type of application (email, query, OLTP etc.) and access mode (random, sequential, read or write intensive)

3. block sizes and other physical characteristics, such as rotational speed (up to 15,0000 rpm)

4. the use of memory caching or disk caching, which can eliminate some I/Os

5. the design of the database layout, which is crucial and trees have been sacrificed writing about this topic

6. what RAID level, or other access method, is employed

7. the program’s mode of accessing logical records (see below) might be sub-optimal (to be mild about it); does he chain reads, save records or retrieve them again and so on

8. the key and indexing should be optimized to avoid long synonym chains to compose a single record. I saw (and resolved) a customer issue where his response times were diabolical until we realized that he was using 3-byte keys which generated dozens of synonyms (identical record locations for different records, causing long searches though chains to access the correct data.

9. Other factors and storage subsystem parameters

The upshot of this is that very fast I/O performance can be negated by poor design and often is. If the items above are properly thought through then, and only then, will the system supporting x IOPs outperform the system supporting 0.8x IOPs.

These areas are covered in Dr Critchley’s book, ‘High Availability IT Services’ .