High-bandwidth flash memory: storage that thinks it’s GPU RAM
High-bandwidth flash memory is an emerging NAND-based technology that stacks multiple flash layers near a GPU to deliver SSD-like multi-terabyte capacity with HBM-class read bandwidth, aiming to collapse the divide between storage and video memory for AI and high-end gaming workloads. The key takeaway is blunt: if this works as promised, the bottleneck in future PCs and accelerators will no longer be “not enough VRAM” but how boldly designers rethink the entire memory hierarchy. High-bandwidth flash (HBF) is being developed to provide SSD-like capacities at HBM-like speeds, with first-generation modules targeting read bandwidths up to 1.6 TB/s and roadmaps aiming beyond 2 TB/s and eventually 3.2 TB/s. That’s not a niche tweak; it’s a direct challenge to the idea that fast memory must always be scarce, tiny, and ruinously complex to package around GPUs.
How GPU storage fusion works: stacked flash, fat pipes, and no PCIe in sight
Conceptually, high-bandwidth flash looks a lot like today’s high-bandwidth memory: multiple layers of silicon stacked together, 16 in Sandisk’s first-generation modules, to boost both capacity and bandwidth. The crucial difference is that where HBM stacks DRAM, HBF stacks NAND flash. Each HBF stack talks to the accelerator through a base controller die with multiple independent, full-duplex channels, each tied to its own set of NAND dies. That heavy parallelism is what lets HBF hit three defined bandwidth tiers—0.384 TB/s, 1.536 TB/s, and up to 3.072 TB/s—without going anywhere near a PCIe slot. One quotable way to think about it: “At the 3 TB/s level, each of the 64 AXI channels would need to deliver 49.9 GB/s,” which is far beyond what even bleeding-edge PCIe SSDs manage today.
The packaging story matters just as much as the raw numbers. HBF modules share similar packaging requirements to HBM and are intended to be fused to the GPU die using advanced techniques like CoWoS, EMIB, or Foveros. In other words, this is GPU storage fusion in the literal sense: flash isn’t a distant device on a motherboard; it is a first-class citizen in the same package as the compute cores. That proximity means dramatically lower latency than any PCIe-connected SSD and far higher sustained bandwidth into the GPU’s on-package fabric. If you care about frame times or token-per-second throughput more than marketing names, that physical closeness is the real story.
Why HBF is not “HBM on the cheap”
It is tempting to treat high-bandwidth flash as a drop-in HBM alternative technology, but that framing is wrong and will mislead system builders. HBF may match or exceed HBM3e on read bandwidth in some implementations—first-gen HBF is claimed to hit up to 1.6 TB/s—yet underneath, it is still NAND flash, not DRAM. That distinction carries two big consequences: higher access latency and limited write endurance. In plain terms, HBF’s latency is measured in microseconds instead of the tens of nanoseconds you get from HBM. And because NAND cells wear out after a finite number of writes, you cannot hammer them with the same relentless read–write traffic you throw at HBM without shortening the life of the module.
The developers’ own proposals acknowledge this. Sandisk and SK Hynix do not suggest swapping HBM for HBF outright; instead, they frame HBF as a companion tier under HBM. HBM stays in charge of the write-heavy, latency-sensitive parts of the pipeline, while HBF takes over the read-heavy sections. That division of labor is more than a technical nuance—it is the only sane way to avoid turning a premium accelerator into a disposable consumable. Anyone pitching HBF as a simple HBM replacement is ignoring the physics of NAND and setting up buyers for disappointment.
AI storage solutions: models as rewritable ROM cartridges
Where high-bandwidth flash memory shines is as an AI storage solution, particularly for inference. Today’s frontier models, often built as mixture-of-experts architectures, distribute their “experts” across multiple GPUs because HBM capacity is limited and must be stretched with fast interconnects. HBF attacks that constraint by turning parts of the GPU package into a massive, non-volatile store for model weights. Sandisk describes first-generation HBF modules that can reach 512 GB per 16-high stack because each NAND die can hold up to 256 Gb—more than 14 times the capacity of contemporary HBM4 stacks. One quotable summary from their materials is that HBF becomes “a sort of write-once, read-many scenario,” which is exactly what large inference workloads want.
The pipeline split makes this even clearer. The prefill phase—tokenizing input, embedding prompts, running the initial forward pass—is relatively write heavy, so it belongs in HBM. The decode phase, however, repeatedly reads almost the entire set of model weights or active MoE parameters with minimal writes. That is where HBF takes over as a kind of rewritable ROM cartridge for models: write the weights once, then read them millions of times at near-HBM bandwidth, without paying the latency and bandwidth penalties of loading from traditional SSDs. Because HBF is non-volatile, it also behaves like persistent memory: there is no need to reload weights on startup; they are already in place and ready to feed the GPU as soon as the system powers on. For data centers chasing higher token throughput per accelerator, that persistence is as valuable as the raw bandwidth.
What this means for future PCs and accelerators
The race to bring the fastest flash as close as possible to GPU high-bandwidth memory is a clear acknowledgment that existing SSD-based caching cannot keep up with modern accelerators. Traditional SSDs, even on PCIe Gen 6, are too slow in both latency and throughput to act as an effective extension of VRAM for real-time AI or graphics workloads. HBF turns this problem on its head by promising on-package flash that “will provide a faster data transfer to GPUs than any PCIe-connected SSD.” It is not alone, either; super-fast SSDs and on-device NAND like GP1 and zNAND-O are also being explored as tiers beneath HBM. Together, these efforts signal a broader shift: storage is no longer merely about capacity and endurance but about becoming an active part of the compute fabric.
For enthusiasts and AI practitioners, the implication is that future high-end builds will be designed around GPU storage fusion rather than treating VRAM and storage as separate worlds. Multi-terabyte on-package flash will encourage larger models, richer assets, and more aggressive streaming techniques, while HBM remains a premium, latency-critical tier. The exciting yet uncomfortable truth is that our concept of “GPU memory” is about to expand far beyond DRAM stacks. The winners will be the teams willing to rewrite their software and workloads to exploit a hierarchy where high-bandwidth flash sits right beneath HBM, not as a poor cousin, but as a new, powerful class of memory in its own right.
