The complex architecture underlying large language models such as ChatGPT, which contains billions of complex compute nodes, has long suffered from the enormous power consumption caused by multiple memory accesses and complex interconnections.

One of the promising approaches to solving this problem is the integration of memory and computing units into separate microcircuits. Both IBM and Intel dared to go in that direction, creating chips. Each neuron receives a special memory to perform its functions. An alternative tactic involves executing operations directly in memory; this is illustrated using the example of phase shift memory.

IBM’s revolutionary solution

IBM becomes a leader in this field by introducing revolutionary chip-based technology phase memoryThis marks a tangible step towards a functional AI processor. The developed hardware can perform speech recognition tasks with great accuracy while operating with significantly less power consumption.

This unique behavior turned out to be particularly beneficial for the complexity of neural networks. In these networks, individual nodes interpret incoming signals and modulate the degree of retransmission of these signals based on their natural state. The internal features of phase memory allow this modulation to be represented as a discrete memory bit operating in analog mode.

Traditionally, data storage relies on two basic situations: activated And closed – carefully designed to minimize data storage errors. However, the innovative aspect here is the adaptability of this memory to the intermediate range between the “on” and “off” binary states, which effectively reflects analog behavior.

This analogy with analog behavior parallels the volume control of a sound, where each rating corresponds to a continuous spectrum of potential values. These intermediate values ​​can be used to represent different strengths or degrees of importance of connections in neural networks.

While IBM’s innovative steps in this direction are notable, the latest version of the chip represents a leap towards a practical processor. Equipped with all the necessary components to facilitate communication between individual nodes, the chip has been calibrated to work with advanced language models.

The basis of the new chip is a “component called”tile“is a collection of crosses that form a grid of 512 by 2,048 individual bits of phase shift memory. A chip contains 34 such tiles, an impressive total of about 35 million bits of phase shift memory.

The flexibility of the processor allows you to dynamically change the strength of the link, determined by a different number of bits. In addition, the processor supports communication between microcircuits, which facilitates the delegation of complex tasks between several microcircuits.

In a real demonstration, the researchers used the processor’s capabilities to solve speech recognition tasks. The processor performed exceptionally well, achieving the following results: 12.4 trillion operations per watt of power consumption at peakSignificantly superior to conventional processors performing similar tasks.

However, it is important to emphasize that the processor optimization is tailored to a particular type of neural network. It is also less suitable for AI training tasks that require specific changes in the neural network training process. While the processor isn’t a one-size-fits-all solution for AI, it promises a noticeable reduction in power consumption, paving a greener path for future AI advancements.