HBM4: The Memory That AI Can't Live Without
High Bandwidth Memory has gone from a niche technology for high-end GPUs to the single most supply-constrained component in the entire semiconductor industry. HBM4 represents the next generation, and it's arriving at a moment when every major AI company is desperate for more memory bandwidth. Here's what's changing and why it matters.
From HBM3e to HBM4: What's Actually Different
HBM3e, the current generation shipping in products like NVIDIA's H200 and AMD's MI300X, delivers up to 1.15 GHz effective speed per pin and maxes out at 1.18 TB/s bandwidth per stack. A single 8-Hi stack provides 24 GB of capacity. Those are impressive numbers, but AI models keep growing and the bandwidth wall is real.
HBM4 makes several architectural changes:
- Interface width doubles from 1024 bits to 2048 bits per stack — this is the biggest single change
- Expected bandwidth per stack: 2+ TB/s, roughly double HBM3e
- Capacity per stack targeting 32-48 GB with 12-Hi and 16-Hi configurations
- New base die architecture that may integrate logic (compute-near-memory capabilities)
- TSV (through-silicon via) density increases to support the wider interface
That doubled interface width is significant because it means the memory controller on the processor side needs a complete redesign. You can't just drop HBM4 into an existing HBM3 socket — the physical interface is fundamentally different.
The Manufacturing Challenge
Stacking DRAM dies and connecting them with through-silicon vias sounds straightforward on paper. In practice, it's one of the most demanding manufacturing processes in the semiconductor industry.
See also: Photonic Computing Explained: Silicon Photonics, Optical Int.
Each HBM4 stack requires:
- Individual DRAM dies thinned to approximately 30-40 micrometers (for context, a human hair is about 70 micrometers)
- Thousands of TSVs etched through each die with precise alignment across all layers
- Microbump bonding between each layer with sub-10-micrometer pitch
- A base logic die that handles all the I/O and potentially some compute functions
- Thermal management across a stack that can generate 15-20 watts in a small area
Going from 8 layers to 12 or 16 compounds every challenge. Thermal dissipation from the center dies becomes the limiting factor — the dies in the middle of the stack have no direct path to a heatsink. SK Hynix and Samsung are both developing advanced thermal solutions, including hybrid bonding that improves thermal conductivity between layers.
SK Hynix vs. Samsung vs. Micron
SK Hynix currently dominates HBM production with roughly 50% market share and a technology lead. They were first to ship HBM3e in volume, and they've secured the lion's share of NVIDIA's supply contracts. Their Icheon and Cheongju fabs are running near capacity.
Samsung is playing catch-up after quality issues delayed their HBM3e qualification with NVIDIA. They've reportedly passed qualification as of late 2025, but the reputational damage cost them early HBM4 design wins. Samsung's approach for HBM4 leans heavily on their hybrid bonding expertise, which could give them a density advantage if it works at scale.
See also: AI Chip 2026: NVIDIA B200 vs AMD MI455X vs Gaudi 3.
Micron is the third player. Smaller market share, but they've been competitive on bandwidth-per-watt metrics. Their Boise and Hiroshima fabs are allocated for HBM production, and they've landed supply agreements with several AI chip makers outside the NVIDIA ecosystem.
The Logic Base Die: A Major Advancement
One of HBM4's most interesting features is the potential for an active logic base die. Previous HBM generations used a relatively simple base die primarily for I/O buffering. HBM4's specification allows for integrating actual compute logic into the base die.
This opens the door to processing-near-memory (PNM) architectures. Instead of moving all data to the processor, you do some computation right at the memory stack. For operations like attention score computation in transformers, where you're reading massive amounts of data to perform relatively simple math, PNM could reduce data movement energy by 10-100× for specific operations.
Samsung has been particularly vocal about PNM capabilities in their HBM4 roadmap, though how much logic the first-generation HBM4 base dies will actually contain remains to be seen.
Related reading: Memory Testing and Reliability: DRAM Retention, RowHammer, a.
Pricing and Availability
HBM is expensive. An HBM3e stack costs roughly $100-150 per unit in volume, compared to $5-10 for a standard DDR5 module with similar capacity. When an H100 needs five HBM3 stacks, that's $500-750 just in memory — a significant chunk of a chip that sells for $25,000-40,000.
HBM4 will cost more, at least initially. The wider interface, denser TSVs, and higher layer counts all increase manufacturing cost. Industry estimates suggest 40-60% price premium over HBM3e at launch, gradually declining as yields improve and production scales.
Supply remains the constraint. Total HBM production capacity across all three vendors was roughly 500,000 wafer starts per month in 2025, and it's not growing fast enough to meet demand. AI companies are signing multi-year prepaid supply agreements — essentially paying upfront for memory that won't ship for 12-18 months.
Impact on AI System Design
HBM4's doubled bandwidth changes the math for AI accelerator architects. With 2+ TB/s per stack and 4-6 stacks per chip, a single accelerator could have 8-12 TB/s of memory bandwidth. That's enough to keep even the largest transformer models fed without becoming memory-bound — at least at current model sizes.
The capacity increase matters too. A 48 GB stack × 6 stacks = 288 GB per accelerator. That's enough to hold a 70B parameter model in FP8 on a single chip. For inference workloads, this dramatically simplifies deployment — no need for tensor parallelism across multiple cards for models that are currently too large for a single GPU.
I'd argue HBM4's real impact won't be felt in the headline specs. It'll be in enabling new model architectures that were previously impractical because they required more memory bandwidth than any single accelerator could provide. Mixture-of-experts models, longer context windows, and multi-modal architectures all stand to benefit disproportionately.