Google announces next-generation TPU chip "Trillium" with performance expected to increase by 4.7 times

 It's tradition at I/O to announce next-generation TPUs, even though the chips won't launch until later this year. However, Pichai said that when the new generation of TPU arrives, the computing performance of each chip will be 4.7 times higher than that of the fifth generation.

Google accomplished this, in part, by enlarging the chip's matrix multiplication unit (MXU) and increasing overall clock speeds. In addition, Google has doubled the memory bandwidth of the Trillium chips.

What's more, Trillium features third-generation SparseCore, which Google describes as a "dedicated accelerator that handles very large embeddings common in advanced ranking and recommendation workloads," which the company believes will allow Trillium TPUs to train faster models and serve them with lower latency.

Pichai also described the new chip as Google's "most energy-efficient" TPU to date, which is especially important as demand for artificial intelligence chips continues to grow exponentially. He said: "In the past six years, the industry's demand for ML computing has increased by a million times, approximately 10 times per year. If we do not invest in reducing the power consumption requirements of these chips, this situation will be unsustainable. Google promises, The new TPU is 67% more energy efficient than fifth-generation chips. "

Google's TPUs tend to come in multiple variants these days. So far, Google has not provided more details about the new chips or how much it will cost to use them in the Google Cloud.

Earlier this year, Google also announced that it would be the first cloud computing provider to provide NVIDIA's next-generation Blackwell processor. However, this still means developers will have to wait until early 2025 to get access to these chips.

"We will continue to invest in the infrastructure to power our AI advancements, and we will continue to break new ground," Pichai said.