[AI Ecosystem] The semiconductor paradigm shifts toward memory
This is Part 4 of SK hynix's AI Ecosystem article series, published October 21, 2026, written by Raja Koduri, founder and CEO of OXMIQ Labs, who previously led GPU and AI accelerator architecture development at Apple, AMD, and Intel; the theme is that the semiconductor paradigm is shifting toward memory. The article argues that AI performance is shifting from pure compute to data movement and placement, with memory bandwidth and hierarchy dominating costs; as transformers grow, data reuse declines and systems repeatedly read from DRAM or HBM, and in low-latency, small-batch inference memory bandwidth can limit performance. It states that HBM vertically stacks multiple DRAM dies connected via through-silicon vias and an interposer, reducing the compute-to-memory distance from centimeters to millimeters; the author says he was involved in developing an HBM-based GPU at AMD in 2015 and later led development of Intel's Ponte Vecchio, which integrated 47 chiplets and four HBM2e stacks into a single package. The article also covers hybrid bonding (fine-pitch direct copper-to-copper bonding replacing microbumps) and processing-in-memory (PIM), noting PIM can reduce energy consumption by 10 to 100 times depending on the workload, while facing challenges such as thermal management. This is an opinion/background analysis piece, not a report of an actual price, capacity, inventory, or supply-demand event.
- Related manufacturers
- SK hynix, OXMIQ Labs
- Part numbers
- Ponte Vecchio
- Impact score
- 10 / 100, low impact