Why Google's Frozen v2 Chip Could Speed Up Gemini Workloads

Picture a server chip that doesn’t just run a model, but subtly follows its playbook. Engineers at Google are quietly building exactly that: a server accelerator codenamed Frozen v2 that bakes parts of the Gemini architecture into silicon, with an eye on radically higher efficiency and lower latency.

The information comes from a report in The Information citing two insiders. Their claim is striking: Frozen v2 could process six to ten times more tokens per watt than Google’s latest tensor processing units. The target date? 2028. The motivation is blunt and familiar to anyone who has watched cloud demand outstrip capacity—Google Cloud reportedly had to turn away some customer deals because of an AI processing crunch.

Embedding architecture instead of rules

Traditional TPUs and GPUs are generalists. You load a model, and the hardware makes a lot of runtime decisions—how to move data, which operations to schedule, when to fetch memory. That flexibility is powerful. But it comes with overhead. Frozen v2 takes a different tack: it fixes certain Gemini-specific decisions into the transistor-level logic. The result is fewer execution steps and less data shuffled per request. Less movement. Less overhead. Lower latency. New real-time uses become plausible.

There was an even bolder idea on the table. Early Frozen designs led by Jeff Dean reportedly tried to hardwire model weights directly into silicon. That would be a radical optimization. It would also age quickly—chips tied to a single model version have a very short useful life. Google split the difference. Frozen v2 would embed the architecture skeleton while keeping weights updateable, so the same chip can serve multiple Gemini releases as long as they follow the same architectural patterns.

This approach could cut response times enough to unlock new classes of interactive and low-latency AI applications.

Google does not intend to replace its TPU fleet with Frozen v2. Think of it as an experimental lane running alongside TPU production—a testbed for more specialized silicon while model designs stabilize. The TPU product roadmap itself recently separated training and inference needs with eighth-generation chips announced at the Cloud Next event in April. The Frozen v2 effort is smaller in scale; Google isn’t planning mass-volume production on the same level as TPUs.

Frozen v2’s anticipated arrival also lines up with other industry moves. Google has reportedly contracted Intel to package more than three million TPU chips by 2028. Meanwhile, startups and rivals are already exploring model-aware silicon. Toronto-based Taalas launched its HC1 chip paired with Llama 3.1 8B and says it hits 17,000 tokens per second without on-package HBM. The startup raised about €186 million in funding. And last year Nvidia signed a technology license deal with Groq valued at about €18.6 billion—evidence that the market sees big value in integrating model behavior and hardware.

There are trade-offs. Embedding architecture reduces some kinds of flexibility. It favors use cases where model families remain consistent over time. It also shifts the product lifecycle: silicon designed around a model architecture must be justified by predictable design stability and deployment scale. That explains Google’s cautious stance. Frozen v2 is experimental; not every experiment becomes a product.

For now, Google has not publicly confirmed Frozen v2. A company spokesperson reminded The Information that many internal projects never reach production. Still, the basic idea—move some model logic into silicon to save energy and shave latency—has moved from academic curiosity to competitive strategy. Expect more firms to test similar blends of model-aware hardware and updateable weights as demand for efficient, low-latency AI continues to climb.