How Nvidia adapted its chips to stay ahead of an AI industry shift

livemint.com:

Subscribe to enjoy similar stories. Nvidia faced a growing threat early last year: The artificial-intelligence world was shifting in a way that invited competition. As millions of people started using AI tools, actually operating the underlying models to respond to their multitude of queries was becoming more important relative to the computing-intensive work of training those models—which had propelled Nvidia to the top of the AI boom.

Many felt that shift could give competitors including Advanced Micro Devices an opening to pry away market share. But Nvidia was already preparing to adapt and stay at the forefront of the AI race despite the shift away from creating models and toward operating them, a process known in the industry as “inference." Its latest AI chips, called Blackwell, are larger in size, have more computer memory and use less-precise numbers in AI computations. They can also be linked together in large numbers with superfast networking, which Dylan Patel, the founder of industry research firm SemiAnalysis, said led to “breakthrough gains" in inference.

“Nvidia’s performance gains for Blackwell are much larger in inference than they are in training," he said. Nvidia’s latest quarterly earnings report on Wednesday partly reflected its success in adapting to the industry’s shift. It included sales and profits that exceeded analysts’ forecasts, coupled with an optimistic forecast for the company’s current quarter.