Skip to main content

Memory Hierarchy & RAM — Overview

Overview​

No single memory technology is simultaneously fast, large, and cheap. Computers instead use a hierarchy: a small amount of extremely fast memory (registers, caches) backed by progressively larger, slower, cheaper tiers (RAM, then disk). Programs benefit from this transparently because most real workloads exhibit locality of reference — they tend to reuse recently accessed data (temporal locality) and access nearby addresses (spatial locality) — which lets small, fast caches absorb most memory traffic.

Core Concepts​

A pyramid of memory tiers: processor registers at the narrow top, then cache, RAM, flash, hard drives, and tape backup at the wide base
The shape is the argument: capacity grows downward, speed and cost-per-bit grow upward. Nobody chose this — it is what you get when fast storage is expensive. Wikimedia Commons, Public domain

The pyramid also splits along a second line the drawing marks as power on versus power off: everything from RAM upward loses its contents when the machine loses power, and everything below it survives. That boundary is why filesystems and database durability are hard problems rather than bookkeeping.

TierApprox. sizeApprox. latencyVolatile?
CPU registersBytes< 1 cycleYes
L1/L2/L3 cacheKB-MB4-40 cyclesYes
DRAM (RAM)GB~100-300 cyclesYes
SSD/NVMeGB-TB~10,000-100,000 cyclesNo
HDDTB~10,000,000+ cyclesNo

Each tier down is roughly 10-1000x larger and slower than the one above it — the difference between an L1 cache hit and a disk access is comparable to the difference between one second and several weeks.

In This Section​

  • RAM Fundamentals — SRAM vs. DRAM, why DRAM needs refresh, and DDR generations and channel bandwidth.
  • CPU Caches — L1/L2/L3 organization, associativity, hit/miss handling, and cache coherence (MESI).
  • Virtual Memory & Paging — why virtual memory exists, page tables, the MMU, the TLB, and page faults.

See Operating Systems for how the OS manages memory at the process level (page replacement, swapping) on top of the hardware mechanisms covered here.

Why It Matters​

  • CPU & Processor Architecture: a pipeline stall waiting on a cache miss is one of the most common real-world causes of "CPU-bound but slow" code.
  • Multicore & Parallelism: cache coherence and false sharing are memory-hierarchy problems that directly limit multicore scaling.
  • Storage: understanding that RAM is volatile and orders of magnitude faster than storage explains why databases and OSes cache aggressively.