JVM Internals & Memory

Hardware & the Memory Hierarchy

A cache miss to main memory costs ~100 ns — time for hundreds of instructions. Caches, prefetching, and out-of-order execution mean memory access patterns, not instruction counts, dominate modern performance. Java's abstractions sit directly on this reality.

Latency ladder: L1 ~1 ns → L2 ~4 ns → L3 ~15 ns → RAM ~100 ns → SSD ~100 µs → network ~ms
Cache lines are 64 bytes: sequential access is nearly free (prefetch); pointer-chasing is a miss per hop
Store buffers and out-of-order execution are why the Java Memory Model exists
False sharing: two threads writing different fields on one cache line ping-pong it between cores
TLB, NUMA, branch prediction — each occasionally surfaces in Java performance work
Mechanical sympathy: arrays beat linked structures for traversal, always

Every programmer's latency numbers (Optimizing Java ch. 3)
Operation	Cost	Scaled (1 cycle = 1 s)
L1 cache hit	~1 ns	seconds
L3 cache hit	~15 ns	~1 minute
Main memory	~100 ns	~6 minutes
NVMe read	~100 µs	~3 days
Datacenter round trip	~500 µs	~2 weeks
Disk seek (HDD)	~10 ms	~1 year

The CPU-DRAM gap grew for decades; hardware papered over it with cache hierarchies, prefetchers, speculative and out-of-order execution, and per-core store buffers. Those buffers mean a write is not globally visible when the instruction retires — cores can disagree about memory order. Memory fences (which volatile and locks emit) drain them. The JMM is the language-level treaty over exactly this machinery.

Cache effects you can measure

// Row-major vs column-major traversal of int[4096][4096]:
for (i...) for (j...) sum += a[i][j];   // sequential: prefetcher streams it
for (j...) for (i...) sum += a[i][j];   // strided: cache miss per access — 5–10× slower

// False sharing: pad or @jdk.internal.vm.annotation.Contended
class Counters { volatile long a; /* 56 bytes padding */ volatile long b; }

False sharing is the classic multicore surprise: independent counters on one 64-byte line serialize both writers through cache-coherency traffic (MESI). LongAdder and JDK internals pad with @Contended; your hot per-thread structs may need manual padding. Diagnosis: perf counters (perf stat, JFR's cache-miss events) — never guesswork (Profiling).