HPE puts storage to work on AI’s expensive memory problem

What happened

HPE is giving its Alletra Storage MP X10000 another job: holding inference state that would otherwise be evicted from expensive GPU memory and recalculated later. In testing validated by Kamiwaza using NVIDIA H200 GPUs, the Kamiwaza AI platform, and HPE Alletra MP X10000, the solution achieved up to 20× faster time to first token and 17× higher effective inference throughput versus running without KV cache offload. The Kamiwaza-validated testing used NVIDIA Nemotron 70B and more than 30 concurrent agents under large-context, multi-agent inference. The article stresses that these headline gains are workload-specific, with context length, concurrency, cache hit rates and transfer overhead determining the return, and that twenty times faster is a test result, not a universal entitlement. HPE Alletra MP X10000 provides a shared tier for offloaded cache, while RDMA for S3 accelerates the storage data path. This is partner content contributed by HPE.

Impact and signal

Affected components and suppliers

Source and time

Source: The Register · 10/29/2026 · View original source