// TOM'S HARDWARE US — HARDWARE & GADGET
Google reworks TPU silicon into specialized systems for agents and security
Google has yet to share specs, prices or ship dates for either new system.
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.
Google's eighth-generation TPU family has now expanded further. Back in April, the company announced TPU 8t for training AI models and TPU 8i for general-purpose inference workloads. Google has now introduced TPU 8tx, built on the same silicon as the previously announced 8t, for confidential, hardware-isolated inference aimed at regulated enterprises and governments, and TPU 8i Max, which combines two 8i chips on one package for long-context agentic work.
TPU 8tx swaps the Arm-based Axion hosts of the 8t and 8i for a 5th Gen AMD EPYC CPU, adopting a Boardfly network, and the 8i Max doubles the compute, SRAM, and HBM bandwidth of the original 8i. The original split, the TPU 8t and 8i, made natural sense. The 8t targets large-scale training, while the 8i leans toward low-latency inference and reasoning. TPUs have become a fast-growing business, with the company claiming nearly 100 customers, including Meta’s recent move to TPUs. The more modular approach is flexible and can deliver specialized systems “in a matter of months rather than years,” helping that popularity along.
Secure, confidential inference; small-scale training
For the new parts, the TPU 8i Max isn't innovative on the surface, as it’s just two TPU 8i chips combined into a single package. The result is twice the theoretical compute, but more importantly, it fits twice the silicon within the “existing rack footprint,” and Google claims more compute per square foot and per watt.
Doubling both SRAM and HBM bandwidth helps keep things scaling, which would put the 8i Max at about 20.2 FP4 PFLOPS, 768 MB SRAM, and about 17.2 TB/s HBM bandwidth. This compares to Nvidia’s Rubin R200 GPU at a rated 50 FP4 PFLOPS for inference and 22 TB/s of HBM4 per socket, or AMD’s MI455X with 40 FP4 PFLOPS. The compute gap is closing.
The memory bandwidth doubling is also potentially one way around moving to HBM4. The memory wall, as Google calls it, or the bandwidth bottleneck, is a real concern “when dozens of AI agents work together across an entire software codebase.” Larger context windows — with larger models targeting 1 million tokens — mean larger KV caches, partly held in SRAM, and each new token has to read that cache back from memory, which is where the extra HBM bandwidth helps. It’s a different route from Nvidia’s LPX: more bandwidth in one package, rather than a second chip type for decoding and a storage tier for context.
Google also provides useful software for agents to get the most out of its hardware. TPU Sync is an open-source “model-state layer for TPU clusters” that moves KV caches off the chip’s HBM, “automatically tiering massive KV caches off the chip in milliseconds” down to host DRAM and, experimentally, shared storage. Google says this can cut KV transfer latency by “more than 90%,” with time-to-first-token gains that vary by workload. The company has also announced the Agent Compute Engine, a Google Cloud service that pauses idle agent sessions and moves their working memory aside until they’re needed again, another memory-saving effort.
Google credits its vLLM-TorchTPU stack with “state-of-the-art price-performance marks” on InferenceX and AgentX, which test realistic agentic coding and long-context work. However, the only published third-party numbers so far are for the last-generation Ironwood TPU. In SemiAnalysis’s InferenceX tests in September, running Qwen3.5 397B in FP8, Ironwood delivered 50.4% more tokens per dollar than Nvidia’s B200 and 96.0% more than the B300 at 20 tokens per second per user. One issue with that, however, is that Ironwood lacks native FP4, although newer November results show its FP8 beating Nvidia’s FP4 GB300 NVL72 on the same model, at 1.67 times the performance per dollar. There is also no eighth-gen result yet to compare how the new generation