What Happens After You Hit Enter
When you press a key on your keyboard, the instruction to display that character travels through one of the most complex pipelines in all of engineering. A modern CPU doesn't just execute instructions one at a time — it processes dozens simultaneously, predicts what you'll ask for next, and speculatively executes work before you've even asked. All in under a nanosecond per instruction.
The microarchitecture — the actual hardware implementation of the instruction set — determines how efficiently this happens. Two CPUs can implement the same instruction set (like x86-64 or ARMv9) with wildly different performance characteristics depending on their microarchitectural choices.
The Pipeline
Modern CPUs break instruction execution into stages, processing different instructions at different stages simultaneously (pipelining). A simplified view of a typical high-performance pipeline:
Fetch and Decode
The front end pulls instructions from the instruction cache (L1I, typically 32-64KB) and decodes them into micro-operations (µops). For x86 CPUs, this decoding step is significant — complex x86 instructions get broken into simpler RISC-like µops that the execution engine processes. ARM and RISC-V instructions are simpler to decode since they're already closer to µop format.
Modern front ends can decode 4-8 instructions per cycle. Intel's Golden Cove (used in Alder Lake) decodes 6 instructions per cycle. Apple's Firestorm cores in the M1/M2 decode 8 instructions per cycle — one of the widest front ends in any production CPU.
For a related perspective, see Semiconductor IP Licensing: ARM, Synopsys, and the IP Block .
Branch Prediction
This is where it gets interesting. About 20% of all instructions are branches (if/else, loops, function calls). Without branch prediction, the pipeline would stall at every branch while waiting to figure out which way to go. Modern branch predictors are remarkably accurate — 97-99% prediction rates on typical code.
The dominant predictor architecture today is TAGE (TAgged GEometric) and its variants. TAGE uses multiple tables indexed by different lengths of branch history, allowing it to capture both short and long patterns. AMD's Zen 4 uses a TAGE-based predictor with a 6,720-entry BTB (Branch Target Buffer). Apple's cores reportedly use even larger predictor structures, which partially explains their strong single-threaded performance on branch-heavy code.
When the predictor guesses wrong, all speculatively executed work gets thrown away — a pipeline flush costing 15-25 cycles depending on the pipeline depth. This is why branch misprediction rate matters so much for performance.
Out-of-Order Execution
Instructions rarely execute in program order. An out-of-order (OoO) engine reorders instructions to maximize execution unit utilization.
For a related perspective, see Chip Design Startups: Funding, Time-to-Tapeout, and the RISC.
The Reorder Buffer
The reorder buffer (ROB) tracks all in-flight instructions and ensures they retire (commit results) in program order even though they executed out of order. A larger ROB allows the CPU to look further ahead for independent work. Apple's M4 performance cores have a ROB of around 600+ entries. Intel's Golden Cove has 512 entries. AMD's Zen 4 has 320 entries.
ROB size has a direct relationship with how well a core handles cache misses. When a load instruction misses in all caches, it might take 200+ cycles for the data to arrive from main memory. A large ROB can keep executing other instructions during that wait, hiding the memory latency. This is one reason Apple's cores perform so well — their massive ROB and instruction window lets them find useful work to do while waiting for memory.
Execution Units
The back end contains multiple execution units handling different operation types: integer ALUs, floating-point/SIMD units, load/store units, branch units. A wide core like Apple Firestorm has 6 integer ALUs, 4 FP/SIMD units, and 3 load + 2 store units. Intel and AMD have somewhat narrower execution engines but compensate with higher clock speeds.
The Memory Hierarchy Connection
No discussion of microarchitecture is complete without cache behavior. The L1 data cache (typically 32-48KB, 4-5 cycle latency) is tightly integrated with the load/store units. An L1 miss goes to L2 (256KB-1MB, 12-15 cycles). L2 miss goes to L3 (shared across cores, 30-50 cycles). L3 miss goes to main memory (150-300+ cycles).
This connects to the ideas in CXL Memory Expansion: Disaggregated Memory Pools and Compute.
Modern cores use hardware prefetchers that detect access patterns and fetch data before it's requested. Stride prefetchers catch sequential accesses, and more sophisticated prefetchers can handle linked list traversals and irregular patterns. The prefetcher's accuracy has a huge impact on real-world performance — a good prefetcher can hide most memory latency for predictable workloads.
The Power-Performance Tradeoff
Everything I've described — wide decode, large ROB, many execution units, aggressive prefetching — costs power. That's why every major CPU now uses heterogeneous core designs: big high-performance cores for demanding work and small efficient cores for background tasks. Intel calls them P-cores and E-cores. ARM calls them Cortex-X (big) and Cortex-A (little). The operating system scheduler decides which core runs each thread based on the workload's demands.
I'd argue that microarchitecture design is the most intellectually demanding engineering discipline in computing. The design space is enormous, the simulation and verification requirements are staggering, and the performance implications of individual design choices compound in ways that are almost impossible to predict analytically. It's equal parts science, engineering, and art.