Google Introduces TPU 8tx for Confidential Inference and TPU 8i Max

An inference team choosing accelerators may be solving a security problem, a memory problem, or both.

Google Cloud announced TPU 8tx and TPU 8i Max on November 9. The 8tx combines 8t silicon with AMD hosts and adds protections for data in use. The 8i Max combines two 8i chips in one package. Google reports twice the compute, on-chip SRAM, and high-bandwidth memory bandwidth of a single 8i.

Match the hardware to the constraint

For a workload with confidential inputs, start with the threat model. List the data your service handles, when it needs to be readable, who can administer the host, and what your customer needs to verify about the environment. Ask which paths the 8tx protects. Check what happens to logs and outputs after they leave that boundary.

For a memory-limited service, test the 8i Max against the longest realistic requests. Measure time to first token and output speed at the same request volume. Include the memory occupied by the attention cache as sequences grow. The chip specifications describe hardware resources. Application throughput also depends on the serving software and how requests use those resources.

Check the software and hardware timelines separately

Google describes its TPU agent suite as a preview. It lists GKE Pathways and dynamic slicing as generally available. Confirm hardware regions, access, and prices for your own deployment before scheduling a benchmark. If you only call a hosted model, the Gemini 4 Carbon API coverage addresses the model service and its pricing. Those figures are separate from the cost of provisioning TPU hardware.

For a fair benchmark, keep one set of requests out of the tuning process. Run those requests after the configuration is fixed. Compare cost and latency at the same quality target. That gives the team a result tied to its workload and a way to compare it after the serving stack changes.