Performance & Optimization

Microbenchmarking & JMH

Naive timing loops measure the JIT's ability to delete your benchmark, not your code. JMH (Java Microbenchmark Harness) exists because warm-up, dead-code elimination, constant folding, and coordinated omission defeat hand-rolled measurement.

Never System.nanoTime() around a loop — warm-up, DCE, and OSR artifacts dominate
JMH: @Benchmark methods, forked JVMs, warm-up iterations, statistical output
Return values or sink them into Blackhole — otherwise the JIT deletes the computation
Beware constant folding: inputs must come from @State fields, not literals
Microbenchmarks answer micro questions; validate at system level before believing them
Run on quiet hardware; mind turbo/thermal effects; compare distributions, not single runs

A correct JMH benchmark

@State(Scope.Benchmark)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Warmup(iterations = 5, time = 1)
@Measurement(iterations = 5, time = 1)
@Fork(3)
public class ConcatBench {
    @Param({"10", "1000"}) int size;
    List<String> words;

    @Setup
    public void setup() { words = randomWords(size); }

    @Benchmark
    public String builder() {
        StringBuilder sb = new StringBuilder();
        for (String w : words) sb.append(w);
        return sb.toString();               // RETURNED — can't be dead-code eliminated
    }
}

What JMH automates (Optimizing Java ch. 5): forked JVMs isolate profile pollution between benchmarks; warm-up iterations let tiered compilation settle before measurement; Blackhole consumes results with minimal, JIT-opaque cost; @State defeats constant folding; and the statistics engine reports mean ± error across iterations. Every one of these corresponds to a way naive benchmarks silently lie.

The deeper caveat (both performance books): a microbenchmark answers "how fast is this method in isolation, fully warm, on this data". Production asks "does this change move p99 under mixed load". Use microbenchmarks to compare implementations of a proven hot spot (Profiling finds those), and re-validate at system level. Most engineers need system benchmarks weekly and microbenchmarks yearly.