Memory is cyclical. AI may be big enough to break the reflex.

Memory Analyst essay, updated June 27, 2026

Start with the reflex

"Memory is cyclical, everyone knows that, and the recent run up in memory names is an obvious bubble."

That is the easy view. It might even be right for a trade. Memory stocks have always been dangerous when they look cheap. Supply catches up, pricing rolls over, margins collapse, and the market remembers why it never wanted to pay a growth multiple for commodity bits.

But I think that reflex may be blinding people to the simple scale of what AI is doing to memory demand.

The old cycle was driven by PCs, phones, servers, inventory, and capital discipline. The AI cycle is different because the demand unit is not just "more devices." It is more accelerators, more memory per accelerator, more memory bandwidth per accelerator, larger resident models, longer context, and more inference turns per completed task.

The first clue was system memory attach

The first clue that there might be more to the story came when reporting around NVIDIA's next-generation Rubin platform started to show just how much memory the system would need. Citi-linked industry reporting said Rubin could require roughly 16 TB of NAND per GPU, or 1,152 TB per NVL72 rack. NVIDIA's own Rubin materials point in the same direction on HBM: the next generation is built around HBM4, up to 288 GB of HBM per GPU, and up to 22 TB/s of HBM bandwidth.

That is where the memory story starts to look different. It is not just "AI needs more GPUs." It is "known GPU roadmaps are pulling a much larger and much faster memory stack behind them."

This is the old memory wall problem with a new forcing function. Compute can keep improving, but if models need more state to be close to the accelerator, memory manufacturers do not just need more bits. They need faster bits, stacked tighter, packaged better, qualified into frontier systems, and delivered into a supply chain that is already constrained.

Inference is where the math gets non-linear

The first AI infrastructure debate was mostly about training clusters and raw compute. The next debate is about inference systems that serve persistent, agentic, long-context workloads. Those systems are not just doing math. They are holding state.

That state shows up in two buckets. The first is resident model weights: the parameters and experts that need to be loaded across serving replicas. The second is KV cache: the memory used to keep previous tokens available so the model can continue generating without recomputing the whole conversation or task history.

Agents make this much more extreme. A normal chat can be short. A useful agent may pull a codebase, Slack history, contract set, research corpus, or spreadsheet model into context, then iterate over the task many times. Longer context means more memory per active workload. More turns means more workload per finished output.

Goldman Sachs frames token consumption as rising 24x to 120 quadrillion tokens per month by 2030. Micron SVP Jeremy Werner has reportedly framed current context growth as roughly 30x per year. I do not assume that context keeps compounding anywhere near that rate. I do think it tells you which way the pressure points.

Show the work

The simple version is that data center memory demand is driven by four things: token demand, model size, context, and optimization. But those variables are not all multiplied together in one naive index.

Model size primarily pressures the resident-weight bucket. Context primarily pressures the KV-cache bucket. Token growth increases the number of workloads served. Agent turns increase the number of inference passes per useful output. Optimization divides against all of it through routing, quantization, compression, paging, offload, reuse, better kernels, and better scheduling.

This is less explosive than multiplying token growth by model growth by context growth. It is also more honest. But the additive model can still produce a very large number because the KV bucket gets big quickly once token demand, active sessions, and average useful context all rise at the same time.

What the calculator says

In the current base case, token demand rises from 5 quadrillion tokens per month in 2026 to 120 quadrillion by 2030. The model assumes routing and efficiency matter, so resident weights only rise to 5.2 EB of grossed HBM demand by 2030. The problem is that KV cache rises to 20.8 EB. The base case is not a model-size story. It is mostly a state story.

The assumptions are the product

The thesis should live or die on visible assumptions, not vibes. Here is the current base case in plain English:

None of that is meant to be sacred. If you think routing takes most token growth away from frontier systems, use the routing-bear case. If you think KV cache optimization is spectacular, use the context-disciplined case. If you think context becomes the core workload of the AI economy, use the memory-bull or stress cases. The calculator is there because the debate is the sensitivity.

The range is the point

The calculator produces very different outcomes depending on what you believe about routing, context, and supply. That is not a bug. It is the thesis in miniature.

The bear case is not dumb. In the routing-bear case, memory demand collapses relative to supply because most traffic never touches the most HBM-heavy systems and efficiency compounds. The bull case is that the world does not stay that clean. Agents, coding, research, and enterprise workflow are exactly the use cases that want state, context, and iteration.

The supply side is not generic DRAM

The supply side is not just "more HBM exists." Usable HBM for frontier AI depends on capacity, yields, packaging, qualification, customer allocation, and whether the memory is close enough in performance to substitute for modern HBM in real clusters.

TrendForce says AI infrastructure should sustain strong HBM demand through 2026 and 2027, with HBM capacity per AI chip moving from 96/192 GB to 216/288 GB, Rubin Ultra expected at 384 GB per GPU, and HBM wafer input among the top three suppliers rising toward roughly 30% of total DRAM wafer input by the end of 2027. That is not a normal memory cycle. That is a capacity reallocation problem.

This is why China matters, but not in a simple way. China can add gross memory supply. The relevant question for frontier AI is how much of that supply is modern-equivalent HBM that can actually relieve NVIDIA, ASIC, and hyperscaler clusters. The calculator splits rest-of-world modern HBM from China gross HBM for that reason.

What would make this wrong?

The honest bear case is not "memory is cyclical." That is a label, not an argument. The real bear case is that AI demand routes around HBM faster than HBM demand compounds.

Those are real risks. They are also measurable. If the base case is wrong, it should show up in the dials: lower HBM-served token share, lower context bucket, higher KV efficiency, faster supply, or more modern-equivalent China supply.

Why the rerating question is fair

Everyone knows memory stocks are cyclical, and they always look cheap right before the bubble bursts. But what if the structurally important product is no longer generic memory? What if the relevant product is frontier HBM, qualified into AI systems, attached to accelerators, and constrained by packaging, yields, bandwidth, and customer allocation?

We have already seen one traditionally cyclical semiconductor company rerate into a growth story because AI turned its product into the bottleneck. It is now the most valuable company in the world.

The point is not that memory becomes NVIDIA. It is that the old commodity prior may be too lazy. If AI compute keeps scaling, and if useful AI work becomes more stateful, then memory may become less cyclical, more strategic, and more margin-rich than investors are used to believing.

Source notes

• Goldman Sachs Research frames agentic AI token consumption as potentially rising 24x to 120 quadrillion tokens per month by 2030.
• NVIDIA's H100 architecture note says H100 SXM5 supports 80 GB of HBM3 and over 3 TB/s of memory bandwidth.
• NVIDIA's Vera Rubin platform note describes up to 288 GB of HBM4 and up to 22 TB/s of memory bandwidth per Rubin GPU.
• Wccftech and TechTarget summarize Citi-linked reporting on 16 TB NAND per Rubin GPU / 1,152 TB per NVL72 rack. I treat that as reported industry data, not audited NVIDIA guidance.
• NVIDIA Dynamo, vLLM PagedAttention, and LMCache research are the source spine for the KV-cache and context mechanics.
• TrendForce is the source for the HBM supply, wafer allocation, pricing power, and 2026-2027 capacity framing.
• Michael Dell's market framing is useful intuition for memory-per-accelerator x accelerator-count growth, but I do not use it as a bottom-up forecast.
• The AI and Memory Wall is useful background for why memory bandwidth and locality can gate AI system performance.

Use the calculator

The calculator is the living version of this thesis. Change token growth, context pressure, model weights, supply, and China modern-equivalent assumptions, then watch which bucket dominates. The point is not one exact forecast. The point is seeing which assumptions carry the argument.

Open the Memory Analyst calculator