Google's TPU Chips Now Serve AI Workloads in Production at Meta

Google's custom tensor processing units (TPUs) are now serving AI in production at Meta, according to an announcement on Google's developer blog. The deployment runs through a new PyTorch-native version of the SGLang inference engine. Google also announced a matching open-source version of vLLM, another widely used inference engine.

Until now, both engines relied on Google's own JAX framework to run on TPUs. The new versions make the hardware a native option for PyTorch, the framework Meta created, and in most cases do so without requiring code to be rewritten.

Zongwei Zhou, Meta's PyTorch-TPU lead, said "SGLang-TorchTPU is running in production at Meta today" following "a close three-way collaboration between Meta, Google, and the SGLang team." The SGLang team works at RadixArk, a startup founded by the engine's creators. Zhou said the result gives Meta "a remarkably mature, PyTorch-native serving stack."

Jana van Greunen, Meta's director of PyTorch engineering, said in Google's TorchTPU post that Meta "cannot be constrained by a single hardware architecture." She added that TorchTPU "enables us to deploy our PyTorch workloads to Google TPUs." Google said it co-developed TorchTPU with Meta's PyTorch team.

Meta has tried TPUs before. Google describes PyTorch/XLA, the older bridge between PyTorch and TPUs, as a predecessor to TorchTPU. SemiAnalysis reported last year that Meta research teams used TPUs through that bridge from 2020 to 2023, before Meta canceled the contracts. Google's push to support PyTorch on TPUs was aimed mainly at Meta, which did not want to move to JAX.

It is unclear where Meta's production TPUs are located. Talks last year reportedly covered Google Cloud TPU rentals in 2026 and purchases in 2027, and Meta later reportedly signed a multi-year rental agreement.

The announcement came a day after Amazon said it was open-sourcing TorchNeuron, a native PyTorch backend for its Trainium2 and Trainium3 chips, at the same PyTorch Conference. Both moves appear aimed at loosening Nvidia's hold through software.

On performance, a tuned vLLM setup serving Moonshot AI's open-weight, 2.8-trillion-parameter Kimi K3 model reportedly exceeds 700 tokens per second for a single user on the seventh-generation Ironwood TPU. The figure comes from Inferact, the startup founded by vLLM's creators, which helped Google build vLLM-TorchTPU. The test used 16 Ironwood chips with speculative decoding, a technique that speeds up large language model inference using a small draft model. It was compared with vLLM's published setup on 16 Nvidia GB200s, which reached 452 tokens per second. Inferact's kernels for Kimi K3 and Qwen 3.8 27B also deliver roughly 1.4 to 2 times the GB200 baseline's throughput at small batch sizes without speculative decoding.

According to RadixArk, SGLang-TorchTPU will keep adding models, starting with DeepSeek V4.1 Flash and Reflection Beam. The team is also working toward Day 0 TPU support for new open models, meaning support on release day, by the first quarter of 2027. Google, Meta and RadixArk plan to share more at SGLang Summit 2026, to be held Nov. 12-13 in San Francisco.