CPU cache
A tiny, very fast memory inside the processor that keeps the data it is about to need.
A CPU can do billions of operations per second, but fetching data from RAM takes far longer than one operation: the processor would spend most of its time waiting. The CPU cache fixes that. It is a small amount of extremely fast memory built into the chip itself, holding copies of the data and instructions the processor used recently or is likely to use next.
Caches come in levels. L1 cache sits right next to each core: tiny (tens of kilobytes) and almost instant. L2 cache is bigger and a bit slower. L3 cache is larger still, often several megabytes or more, and usually shared by all the cores. When the CPU needs something it checks L1, then L2, then L3, and only goes to RAM if none of them has it. Finding it is a hit; going all the way out is a miss.
Caches work because programs are predictable: they tend to reuse the same data and to read memory in order. The chip loads whole neighboring chunks at once and keeps what was used lately. That is why two programs doing the same math can run at very different speeds depending on how they lay out their data, and why chipmakers advertise cache size next to cores and clock speed.
Keep in mind
- Cache is small, very fast memory on the CPU chip itself.
- L1 is the smallest and fastest, L3 the largest and usually shared.
- A hit is served from cache; a miss has to wait for RAM.
- Software that reads memory in order makes much better use of the cache.
