IBM’s new AI chip offers 22X speedup with ‘mind-blowing’ energy efficiency

cointelegraph.com:

IBM recently debuted a new prototype artificial intelligence (AI) chip purported to be both faster and far more energy efficient than any chip currently available. 

According to research published in Science Magazine on Oct. 19, the new chip, dubbed NorthPole, “achieves a 25 times higher energy metric” on a relevant benchmark, “and a 22 times lower time metric of latency.”

Ostensibly, this translates to the potential for post-GPU performance at a fraction of the cost in energy requirements.

Damien Querlioz, a nanoelectronics researcher at the University of Paris-Saclay in Palaiseau, described NorthPole’s energy efficiency as “mind-blowing,” in an article published on Nature.

Per the IBM Research team’s paper:

One of the major impediments to improving AI processing is called the “von Neumann bottleneck.” Using currently available architecture, AI chips tend to have faster processing capabilities than the memory they require to run processes. As a result, latency is introduced whenever information is sent between the processing unit and random access memory.

This is especially true at “the edge,” where chips and data are stored together. Removing this bottleneck has long been considered by many experts to be the key to running powerful neural networks locally on devices.

Publishing this week in @ScienceMagazine, IBM Research's newest prototype AI chip, NorthPole, could help us move toward more energy-efficient AI. Learn more: https://t.co/qyKIUGg5ld pic.twitter.com/eFklLNSqkW

According to IBM Research, the new prototype chip built in the company’s Alamaden, California laboratory bypasses the von Neumann bottleneck by, essentially, integrating the memory component onto the processing chip itself.

As the chip’s lead developer,