Emerging Memory Technologies: MRAM, ReRAM, and Processing-in-Memory

Memory-Centric Computing Architectures

The semiconductor industry has a dirty secret that we don't talk about enough: for most modern workloads, the processor sits idle waiting for data more than it actually computes. This isn't a new observation — the "memory wall" was identified in 1995 — but it's gotten dramatically worse as compute scaling has outpaced memory bandwidth and latency improvements by orders of magnitude. Memory-centric computing tries to flip this equation, and I think it's one of the most important architectural shifts happening right now.

The Memory Wall in Numbers

Let me give you some concrete figures to show why this matters. An AMD EPYC 9754 (Bergamo) processor can execute about 3 trillion integer operations per second. Its 12-channel DDR5-4800 memory system delivers about 460 GB/s of bandwidth. That means the processor has roughly 6.5 operations per byte of memory bandwidth.

A dense matrix multiply of 16-bit values needs 2 bytes loaded per multiply-accumulate operation (in the simplest case). So you need 6+ TB/s of bandwidth to keep the compute units fed, but you only have 0.46 TB/s. The compute is 13x faster than the memory can feed it.

For sparse workloads like graph analytics, recommendation systems, or database operations, the situation is even worse. These workloads have unpredictable, pointer-chasing access patterns that defeat caching and prefetching. Effective bandwidth utilization drops to maybe 10-20% of peak because of random access latency (around 80-100ns for DDR5). Your multi-billion-transistor processor might achieve only 2-5% of its theoretical throughput on graph traversal.

Processing-in-Memory: Putting Compute Where the Data Lives

The most direct solution to the memory wall is to move computation into the memory itself. Instead of shipping data from DRAM to a distant processor, you execute operations inside or very close to the memory arrays.

Samsung's HBM-PIM (Processing-in-Memory): Samsung added a small SIMD processing unit to each bank within an HBM2E stack. Each bank's PIM unit can execute simple arithmetic (add, multiply, ReLU) on data as it's read from the DRAM array, before it ever crosses the memory interface. Samsung demonstrated a 2.5x performance improvement on LSTM inference workloads by offloading element-wise operations to HBM-PIM. The key insight is that these aren't complex operations — they're simple per-element transforms that are bottlenecked by data movement, not compute.

See also: Galaxy S26 & Exynos 2600: Chip 2nm Có Đáng Tin? 2026.

UPMEM's PIM-DRAM: UPMEM takes a more aggressive approach, embedding full programmable processors (DPUs — DRAM Processing Units) within each DRAM chip. Each DPU is a 32-bit RISC core clocked at 350-500 MHz with its own instruction memory and a 64MB MRAM (DRAM) bank. A standard UPMEM DIMM has 128 DPUs, giving you 128 independent processing threads operating directly on in-memory data.

I've seen UPMEM's technology benchmarked on genomics (DNA sequence alignment) and database operations, showing 10-20x speedups over CPU-only solutions for data-intensive workloads. The programming model is unusual — you write code for the DPU cores in a subset of C, and the host CPU orchestrates task distribution — but it works.

Near-Memory Computing: The Practical Compromise

If processing-in-memory is too radical a change, near-memory computing puts accelerators physically close to the memory (on the same package or interposer) without modifying the DRAM itself.

HBM with logic dies: In a standard HBM stack, the bottom die is a "logic die" (or base die) that handles I/O and inter-stack communication. Companies are exploring adding compute capabilities to this logic die. SK Hynix's AiM (Accelerator-in-Memory) technology adds programmable compute units to the HBM logic die, enabling simple operations at the full internal bandwidth of the HBM stack — which is much higher than the external TSV bandwidth. Internal bandwidth per HBM stack can be 2-4 TB/s, versus 256-512 GB/s on the external interface.

Samsung's AxDIMM and CXL-PNM: Samsung has demonstrated an "accelerator DIMM" that puts FPGA-based processing logic on a DDR5 DIMM, allowing in-DIMM execution of operations like database filtering and recommendation model inference. Their CXL-based Processing-Near-Memory approach uses the CXL 2.0 Type-2 device protocol to present both memory and compute capabilities to the host.

This connects to the ideas in Satellite and Space-Grade Chips: Radiation Hardening and Ext.

CXL: The Enabler for Memory-Centric Architectures

Compute Express Link (CXL) is, I think, the most important infrastructure technology for memory-centric computing. CXL 3.0 (ratified in 2023) provides three protocols:

• CXL.io: PCIe-based device discovery and management
• CXL.cache: Allows devices to cache host memory with hardware coherence
• CXL.mem: Allows the host to access device-attached memory as if it were local

CXL.mem is the key protocol. It allows you to attach disaggregated memory pools — potentially with compute capabilities — that appear as addressable memory to the host processor. A CXL memory expander from Samsung (CMM-D) or Micron (CZ120) plugs into a PCIe/CXL slot and adds memory capacity beyond what the CPU's native DDR5 channels support.

The latency overhead of CXL.mem over local DDR5 is roughly 80-150ns additional, depending on the implementation. That's significant for latency-sensitive workloads, but for bandwidth-limited and capacity-limited workloads (large in-memory databases, graph analytics, AI model serving), the additional capacity is worth the latency penalty.

Where this gets interesting for memory-centric computing is CXL Type-2 devices — accelerators with their own attached memory that the host can also directly access. This enables a shared memory space between the host CPU and the accelerator, eliminating explicit data copies. For workloads like recommendation inference (which Meta runs at massive scale), the memory access patterns are so irregular that shared address space between compute and data is extremely valuable.

Analog In-Memory Computing

The most radical approach — and the one I find most fascinating — is performing computation directly in the memory array using analog physics. The idea: if you store neural network weights as conductance values in a resistive memory array (ReRAM, PCM, or even DRAM), you can perform matrix-vector multiply in O(1) time by applying voltages to the rows and reading currents on the columns. Kirchhoff's current law gives you the dot product for free.

Related reading: Semiconductor Testing and Packaging Outsourcing: OSAT Indust.

IBM's analog AI chip (published in Nature, 2023) demonstrated a 35-million-parameter neural network running on 34 analog-compute tiles with PCM-based crossbar arrays. Each tile performs 256x256 matrix-vector multiply in a single analog operation. The energy efficiency was reported at 14 TOPS/W for INT8-equivalent operations — competitive with digital accelerators but at a fraction of the area.

The challenges are real though. Analog computation has noise and drift problems. PCM conductance values drift over time (crystallization physics), requiring periodic refresh and calibration. ADC (analog-to-digital converter) precision limits the effective bit-width of computation. And programming the memory cells to precise conductance values is an iterative process with write-verify cycles that can be slow.

Startups like Mythic (which ran into funding difficulties), Syntiant (focused on always-on audio), and Rain Neuromorphics are all exploring variations of analog in-memory computing. I think this approach will find its niche — probably in ultra-low-power edge inference where 4-bit precision is acceptable — but it won't replace digital computing for general-purpose workloads anytime soon.

Where This Is Heading

If I had to bet on the memory-centric computing technologies that will actually reach mainstream production in the next 5 years:

1. CXL memory pooling and expansion — will be the first to achieve broad adoption. It's a standards-based, backward-compatible approach that doesn't require new DRAM technology. Intel Sapphire Rapids and AMD Genoa already support CXL 1.1, and EPYC Turin supports CXL 2.0.
2. HBM with compute on the logic die — will appear in AI accelerators from major vendors. NVIDIA, AMD, and likely Google are all looking at this for next-gen products. The bandwidth inside an HBM stack is too valuable to leave on the table.
3. PIM-DRAM — (UPMEM-style) will grow in niche applications — genomics, database acceleration, and certain HPC workloads — but won't go mainstream until a major DRAM vendor (Samsung, SK Hynix, or Micron) productizes it at scale.
4. Analog in-memory compute — will remain a research topic and niche product for ultra-low-power edge applications. The precision and reliability challenges are too fundamental for near-term mainstream adoption.

The memory wall isn't going away. If anything, it's getting worse as AI workloads demand ever more data bandwidth. The question isn't whether memory-centric computing will happen — it's which specific approaches will win.