The Issues that Won't Die

Through the history of computer memory, some key issues have not changed.

speed vs. cost --the fastest memory you need is that which is as fast as the processing ability of the computer. Such memory is usually too costly to use in the quantity you would like
latency vs. bandwidth -- latency is the time to complete one operation; bandwidth is the number of operations that can be completed over time. Latency is hard to make fast; bandwidth can often be bought (e.g., by doing more at once)
sequential vs. random access -- it's always easier to access a large amount of information in a steady stream, than to access information scattered over different parts of memory

It's interesting that these issues have existed from the very early days of computing -- at least from the generation after ENIAC, when economical, commercially viable technologies began to be explored.

The concept of virtual memory in which part of the memory consists of a slower device (in modern systems, a disk), but the programmer can pretend that all memory used is the fastest memory on the system, dates back to 1946. Virtual memory exploits the principle of locality, by which in essence a small part of the total memory used is accessed at a time.

What's Changed?

For one thing, the relative speeds of various parts of the memory system have grown more widely divergent. In the 1950s, a fast drum system may have had an access time of around 1ms, with the fastest memory available maybe 10 to 100 times faster. In today's world, the very fastest memory keeps up with the processor clock speed, only a few nanoseconds. A disk on the other hand has an access time of about 10ms (actually slower than a fast drum of the 1950s -- though able to store thousands of times more information than a drum). The speed difference today between the fastest and slowest memories on a given system is a factor of a million or more.

The practical effect of this change is that modern systems tend to have many more levels of memory to bridge the speed gap between fastest and most affordable. A typical modern computer's memory consists of

first level cache -- made of fast static RAM (SRAM), and usually on the same chip as the processor; access time maybe under 5ns, depending on processor speed
second level cache -- made of a larger amount of slightly slower SRAM than the first level cache; usually not on the same chip as the processor; access time 5 to 15ns, depending on how fast the overall system is
main memory -- made of dynamic RAM (DRAM) which is slower than SRAM, but also much cheaper; access time typically 60ns
disk -- spinning platters with a magnetically recorded surface; much cheaper than DRAM but also much slower; access time typically 10ms

The way the memory system works is by exploiting the principle of locality. As far as possible, memory accesses should happen in the smallest, fastest level of the memory system. The slowest but largest part should be accessed as little as possible. Any software which doesn't have good locality will run very slowly, especially if the slowest part of the memory hierarchy (the disk) is used often.

Memories are Made of This 17:30 Thurs 12 September 1996
SHB5, Senate House, University of the Witwatersrand, Johannesburg, South Africa