Part 1. The Evolution of AI and AI Chips – and the HBM That Powers Them

What happened

Samsung published Part 1 of a series on the evolution of AI and AI chips and the HBM that powers them; the content is technical background. It explains the division of roles among AI chips (GPUs, AI-specific ASICs, CPUs), notes that during decode at smaller batch sizes memory bandwidth can become a key performance constraint, and describes KV cache data storage needs that grow as context lengths and concurrent requests increase. The article recounts that GPUs traditionally used GDDR for high memory bandwidth, but because GDDR devices are placed around the GPU on the board, adding memory requires additional space and routing, and raising per-pin data rates increases challenges of power consumption, heat, and signal integrity. HBM addresses this by combining vertical stacking with a wide I/O interface, connecting multiple stacked DRAM dies through Through Silicon Vias (TSVs), delivering high bandwidth, higher capacity, greater energy efficiency, and high-density packaging in a smaller footprint. It also outlines HBM's generational evolution, including stack counts rising from 8-high to 12-high and 16-high, higher-density DRAM dies, and advanced logic-process base dies. Samsung's zHBM is currently under development and directly stacks memory on top of the compute device via 3D integration to shorten data paths. The content is informational only, with Samsung providing no warranty as to its accuracy or completeness.

Impact and signal

Affected components and suppliers

Source and time

Source: Samsung Semiconductor Newsroom · 10/28/2026 · View original source