Google TPU 8 debuts in two variants at Cloud Next 2026 — the 9,600-chip training pod and an inference chip delivering 80% better performance per dollar, challenging Nvidia's data-center dominance.

Google TPU 8 Split: 9,600-Chip Pod, 80% Cheaper Inference

by
💡 Key Takeaways

  • At Cloud Next 2026 on April 22, Google split TPU 8 into TPU 8t (training) and TPU 8i (inference) — the first time the TPU program has separated training and inference silicon in its 10-year history.
  • TPU 8t scales to 9,600 chips and 121 ExaFlops per superpod for ~3x Ironwood compute, while TPU 8i delivers 80% better performance-per-dollar, taking direct aim at Nvidia’s inference margin pool.
  • Anthropic anchors both generations with a 3.5 GW capacity deal for 2027; Google simultaneously inked a Vera Rubin partnership with Nvidia, framing the play as multi-silicon “AI Hypercomputer” rather than head-on replacement.

Nine thousand six hundred chips in one pod. One hundred and twenty-one exaflops in a single training cluster. Google TPU 8 debuted at Cloud Next 2026 on April 22, splitting into two purpose-built dies — TPU 8t for training and TPU 8i for inference — in the first such bifurcation across a decade of TPU silicon.

The justification is the agentic era. As workloads shift from one-off model training to persistent AI agent deployment, a single chip can no longer optimize both throughput at training scale and unit economics at inference scale. Google Cloud CEO Thomas Kurian called the split “a natural evolution,” reframing the architecture as a deliberate response to the way enterprises actually consume AI compute.

Google TPU 8 split training inference superpod illustration (AI-generated)
The 8th-gen TPU separates training (8t) and inference (8i) silicon, scaling 9,600 chips into one 121-exaflop pod — Generated by OpenAI DALL-E 3 (AI-generated)

Why Google TPU 8 Splits Now: From Single Die to Dual Track

Sundar Pichai, Thomas Kurian, and Amin Vahdat used the Cloud Next stage to position the dual-track architecture as the defining bet of Google’s eighth-generation silicon program. Through Ironwood (TPU 7), one chip carried both training and inference workloads. With the eighth generation, codename Sunfish handles training as TPU 8t and codename Zebrafish takes inference as TPU 8i.

What both chips share is just as telling. They run on Google’s Axion ARM-based CPU host and are fabricated on TSMC’s 2nm node, allowing Google to optimize across the chip, the package, and the host stack at the same time. General availability is targeted for “later this year,” meaning the silicon will start booking revenue in the second half of 2026.

The split is not branding. Training is bottlenecked by superpod throughput and shared memory bandwidth, while inference is bottlenecked by per-chip cost and latency. Trying to maximize both KPIs on the same die forces compromises on each — the very compromises Anthropic, OpenAI, and DeepMind have publicly complained about during their training cycles.

TPU 8t (Sunfish): The Numbers Behind 9,600-Chip Pods

TPU 8t is designed by Broadcom. A single superpod scales to 9,600 chips and 2 PB of shared HBM, hitting 121 ExaFlops in aggregate compute. That math translates to roughly 3x the per-pod compute of Ironwood and 2x better performance-per-watt across the same superpod boundary.

At the chip level, the spec sheet is dense. Each accelerator delivers 12.6 petaFLOPS of FP4 compute, 216 GB of HBM at 6.5 TB/s of bandwidth, 128 MB of on-chip SRAM, and 19.2 Tbps of chip-to-chip bandwidth. The physical interconnect uses Google’s Virgo Network fabric with optical-circuit switching, and goodput — the share of training time that is actually productive rather than lost to failures or stalls — clocks in at 97%.

For the same dollar, Google claims 2.8x faster training versus Ironwood. That is the most consequential number for anyone building frontier models. If the 80% performance-per-dollar story on TPU 8i is the headline for cloud finance teams, the 2.8x on TPU 8t is the headline for research leaders timing the next training run; together they shorten the cadence at which Anthropic, DeepMind, and any future tenant can iterate on a new flagship model.

TPU 8t silicon close-up — 12.6 PF FP4 and 216 GB HBM
TPU 8t pairs 12.6 PF FP4 with 216 GB HBM and 19.2 Tbps ICI to feed 9,600-chip superpods — Photo by Laura Ockel on Unsplash

TPU 8i (Zebrafish): Where the 80% Inference Discount Comes From

TPU 8i is designed by MediaTek. The chip carries 288 GB of HBM at 8.6 TB/s and 384 MB of on-chip SRAM — three times Ironwood’s SRAM budget — so that large-language-model KV caches stay on-chip during inference instead of spilling out to slower memory tiers. The new Boardfly topology directly connects 1,152 TPUs in a single inference pod, and the dedicated Collectives Acceleration Engine cuts on-chip communication latency by up to five-fold.

The headline number is 80% better performance-per-dollar versus Ironwood. In practical terms, Google says the same dollar buys roughly 2x the customer volume — and with 2x performance-per-watt layered on top, datacenter electricity per token drops in lockstep. For the agentic workloads Pichai keeps hammering on stage, where one user prompt can fan out into a dozen tool calls, that compounding effect is precisely the unit economics the chip was designed to win.

Boardfly itself is the under-discussed piece. By collapsing pod diameter and embedding collectives in hardware, it removes the network-stack tax that has historically limited inference scaling beyond a few hundred chips. That is what makes the 80% claim plausible at the kind of multi-tenant scale Google needs for Workspace, Search, and the new Gemini Enterprise Agent Platform.

Key point: TPU 8t pushes superpod throughput (2.8x training perf-per-dollar), TPU 8i pushes per-chip economics (80% inference perf-per-dollar). The split lets Google optimize each KPI without compromising the other.

Anthropic’s 3.5 GW Anchor and the Broadcom $46B Contract

Right after Anthropic dethroned OpenAI on the Stanford AI Index 2026 leaderboard, the lab emerged as the anchor customer for both TPU 7 (Ironwood) and TPU 8. Through the Broadcom-Google partnership, Anthropic gains 3.5 GW of new capacity starting in 2027, on top of roughly 1 GW — about one million TPU chips — already coming online in 2026.

Mizuho analyst Vijay Rakesh estimates that Anthropic-related Broadcom AI revenue will reach $21 billion in 2026 and $42 billion in 2027. Broadcom and Google have separately locked in a $46 billion AI silicon supply contract running through 2031. Add Marvell’s memory processing unit, Intel’s CPU collaboration, and MediaTek’s inference die, and Google has assembled a multi-vendor custom-silicon stack that locks in compute economics for nearly a decade.

Anthropic’s own revenue trajectory underwrites the deal. The company’s annualized run rate has crossed $30 billion, up from roughly $9 billion at the end of 2025 — a roughly 3x jump in a year. That growth materially reduces the near-term risk attached to the SEC filing language requiring Anthropic’s “continued commercial success” before the full 3.5 GW capacity converts into recognized Broadcom revenue.

3.5 GW Anthropic anchor — distributed datacenter optical fabric
Anthropic’s 3.5 GW capacity will land as a distributed cluster footprint, not a single megacampus — Photo by Adrien on Unsplash

Vera Rubin and “AI Hypercomputer”: Why Nvidia Wins Too

On the same day Google unveiled TPU 8, it also announced a Nvidia partnership. Nvidia Vera Rubin chips will deploy on Google Cloud A5X next-generation AI infrastructure, and GOOGL stock climbed roughly 1.5% on the announcement.

The deal looks contradictory only at first glance. Google already runs Nvidia GB300 systems internally for its own engineering workloads. The “AI Hypercomputer” framing — TPUs, Nvidia GPUs, and Axion CPUs running on a shared infrastructure layer — codifies what was already happening: not a winner-take-all silicon battle, but workload-aware routing that hands each model to the chip with the best unit economics for it.

The financial backdrop sharpens the strategic logic. TPU 7 Ironwood went generally available at the same event, contributing 4.6 PF per chip, 192 GB of HBM3e, and 9,216-chip superpods totaling 42.5 ExaFlops. Google Cloud Q4 revenue jumped 48% year-over-year to $17.7 billion, and the cloud backlog doubled to $240 billion — the demand reservoir Google needs to fill with both TPU and Nvidia silicon at once.

💡 Related Posts
$25B AI Chip IPO: Cerebras Files to Break NVIDIA’s Inference Monopoly
Google Cloud A5X infrastructure — Vera Rubin and TPU 8 coexistence
Google’s A5X racks will host Vera Rubin GPUs, TPU 8 ASICs, and Axion CPUs on the same infrastructure layer — Photo by Ivan N on Unsplash

Inference Cost Is the New Margin Battle for Nvidia

Implicator.ai put it bluntly. TPU 8i is “attacking inference cost specifically,” and alongside Cerebras’ inference-focused IPO play and the broader pivot to persistent AI agent deployment, the industry’s center of gravity is shifting from one-off model training to always-on inference. Inference unit cost is what now decides GPU margin.

Google’s own TPU shipment forecast hints at the magnitude of the shift: 4.3 million chips in 2026, 10 million in 2027, 35 million-plus by 2028. Each block of inference workload that migrates from GPUs to TPUs erodes Nvidia’s datacenter revenue pool, and the 35 million chip mark in 2028 is the first plausible inflection point at which custom silicon could reset the inference margin envelope industry-wide.

The hyperscaler pattern is converging. AWS already runs the Trainium and Inferentia split, Marvell is pushing its memory processing unit toward inference workloads, and Broadcom’s training-side dominance through Sunfish completes the picture. Nvidia keeps its lead on training, where its GPU programmability and CUDA moat still matter most, but inference is becoming the price-discipline layer where custom ASICs win on dollars per token.

In simpler terms: Training rewards programmability (Nvidia’s strength). Inference rewards cost-per-token (custom ASIC strength). TPU 8i’s 80% perf-per-dollar puts a soft cap on Nvidia’s inference revenue pool.
💡 Related Posts
1,503 Elo: Anthropic Dethrones OpenAI After 3-Year Reign

Three Checkpoints Investors and Engineers Should Track Next

First, the TPU 8 general availability date. Google’s own “later this year” guidance has to translate into actual shipment and billing, and the first independent benchmark — most likely MLPerf v6 Inference — is the gate at which the 80% perf-per-dollar claim either holds or quietly gets reframed.

Second, Google’s 2026 capex guidance. The current $175 to $185 billion range is roughly double the $91.4 billion booked in 2025, and it will be revised at every quarterly print. Each revision is effectively a real-time signal about whether the 4.3-million-chip TPU shipment forecast is tracking, accelerating, or slipping.

Third, the Anthropic conditional clause. The 3.5 GW capacity is contractually tied to Anthropic’s “continued commercial success” — meaning the moment Anthropic ARR growth flattens or Claude revenue softens, Broadcom and Google will start renegotiating the capacity allocation. That is the single quietest risk in the whole 2027 buildout, and the one most likely to surface inside a single quarter.

References

Stay Ahead of the Curve

Get the latest insights on AI, tech, and markets — delivered in both Korean and English.

Explore More Posts