Google Adds TPU 8tx and 8i Max: Private AI and 2x Compute, No Price Yet
In short
- Google announced two new TPU systems on November 9: TPU 8tx for private, confidential AI work and TPU 8i Max for huge context windows.
- Google claims up to 2x lower latency for 8tx versus TPU 8t, and 2x compute, SRAM and memory bandwidth for 8i Max versus 8i.
- The money angle: Google says TPU 8i already gives up to 80% better inference price-performance than the previous generation. No prices or dates for the new chips yet.
Google just added two more chips to its AI line-up. You will not buy one. But the price of the AI you rent from Google Cloud depends on chips like these, so it matters for your bill.
What happened
Amin Vahdat, Google’s chief technologist for AI and infrastructure, published the news on November 9. Google calls this the first year it has launched four TPU systems. TPU 8t and TPU 8i came in April. Now come two more.
TPU 8tx uses the same silicon as TPU 8t, paired with 5th generation AMD EPYC host CPUs. Its point is confidential compute: your data and model weights stay isolated while they are being processed. Google says you can fine-tune on private data, distill models and run reinforcement learning on one system without moving sensitive data between clusters. It claims up to 2x lower latency than TPU 8t with a near-zero performance penalty.
TPU 8i Max puts two TPU 8i chips on one package. Google says that gives 2x compute, 2x SRAM and 2x HBM bandwidth versus TPU 8i, in the same rack footprint. It is aimed at very large context windows and the highest compute density.
What Google did not say: no absolute specs, no release date, no regions and no pricing for either system. The post also repeats earlier claims: TPU 8t up to 2.7x better training performance than the last generation, TPU 8i up to 80% better inference price-performance. These are Google’s own numbers.
How to use it
For most readers, directly: you do not. Nothing here is available to order yet. But there are things you can do with the news.
- If you build products on Gemini or run models on Google Cloud, watch your price per token over the next months. Cheaper chips can mean cheaper tokens, but only if Google passes the savings on.
- If you handle customer data and wanted to fine-tune a model, confidential compute is the feature to track. It answers the question “where does my data go while the model trains?”
- If your use case needs very long context, ask Google Cloud when 8i Max will be available.
Do the math: say you pay €1,000 a month for AI inference on today’s hardware. If a new generation really gives 80% better price-performance, the same work costs €1,000 / 1.8, about €556. That is €444 a month, €5,328 a year. But that only happens if the savings reach your invoice. Google gave no prices, so treat this as a best case, not a forecast.
What it means
Google says TPUs now serve nearly 100 customers, including frontier labs, trading firms, retailers and universities. It is selling TPUs as a real alternative to Nvidia GPUs, and four systems in one year shows how hard it is pushing. The split also tells you where the market is going: one chip for training, one for fast inference, one for private work, one for huge context. AI hardware is getting specialised.
Google also says it open-sourced TPU Sync, which it claims cuts KV transfer latency and time to first token by more than 90%, and made GKE dynamic slicing generally available, with claims of up to 5x faster job startup. Again, those are claims.
Ivan’s Take
Big claims, no numbers you can check. No specs, no price, no date. I would not change any plan because of this post. I would only keep an eye on the price per token at Google Cloud, because that is where cheaper chips either show up or they do not.
Source: Google
If it doesn’t save you time or money, you won’t see it here
- Facts, not hypeEvery price and number is checked against the official source
- Always the real priceYou see what it costs before you click
- I say when it’s weakEven when it’s OpenAI or Google
