Broadcom studies optical connection for GPUs and AI accelerators

Broadcom revealed at a recent event that it is working on a solution to integrate optical connections directly into GPUs and other AI accelerators. The idea is to take advantage of the data transfer rate of this type of transmission, which is much higher than the copper traditionally used.

The company aims to improve the interconnection capacity between different processing units. To do this, it is using co-packaged optics (CPO), a solution that integrates silicon and optical components directly on the “packaged” substrate.

The technology was shown at the Hot Chips convention, where Broadcom promised an interconnect bandwidth of up to 1.6TB/s with its solution. That would be the equivalent of 6.4TB/s, or 800GB/s in each direction.

The big highlight of this new feature is the energy savings that the use of CPOs can offer. Optical connections generally require more energy, and Nvidia even estimates that a system with NVL72 could increase its consumption by up to 20kW if it started using the technology in its interconnection. Standard consumption is already 120kW.

However, Broadcom says that its “optical engine” alleviates the problem of extra power consumption by integrating CPOs directly into the components, offering better efficiency than current solutions. Furthermore, compared to the NVL72 that can connect up to 72 GPUs working at the same time, the company promises that its technology can connect up to 512 GPUs.

Another advantage highlighted by the company for optical interconnection of GPUs and AI accelerators would be the distance at which these components can be placed. According to the Broadcom spokesperson, copper connections can start to degrade data after only 5 meters, while optical does not have this problem.

Related news:

The company, however, has not yet been able to showcase its optical engine on a commercially available GPU or chip. Instead, Broadcom used a test chip designed to emulate an A100 or MI250X. This could be taken as an indication that we are still some way off from seeing a finished product with the company’s OPC solution.

Fonte: The Register
Via: TechSpot