Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_22
Broadcom is one of many companies developing custom AI accelerators. So it's a big deal that the company showed off an AI computing ASIC with optical connectivity at Hot Chips 2024. This was likely done for a customer project. Broadcom makes custom AI accelerators for hyperscalers. The company also showed off co-packaged optics and silicon photonics, which is something we'd like to see.
We're live on Hot Chips 2024 this week so please excuse any typos.
Broadcom AI Computing ASIC with Optical Attachment Details Revealed at Hot Chips 2024
Co-packaged optics has been a hot topic for a long time.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_02
One of the big challenges in interconnect and networking is the electrical I/O reach through the PCB. While some designs use cables externally from the switch chip to the optical cage, the end goal still seems to be optics. However, the NVIDIA NVLink NVL72 is a great example of a copper connection rather than an optical connection.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_03
Silicon photonics enables optical modules with a reduced component count.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_04
Broadcom now uses silicon photonics and packaged optics not just for switches but also for scale-up computing.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_05
In case you were wondering, the switch in the photo is a Broadcom Tomahawk 5 Bailly.
Broadcom Tomahawk 5 Bailly Co Package Optical Sample
Here's the schematic for CPO. The key difference between Intel's old Silicon Photonics and Broadcom is that Broadcom uses pluggable lasers. Lasers fail, so having pluggable lasers allows for better maintainability. Similarly,
Broadcom AI ASIC with optically connected hot chip 2024_Page_07
Broadcom has a version of the CPO (co-packaged optics) component in the Tomahawk 5 slide shown earlier in this presentation.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_08
The Tomahawk 4 “Humboldt” was a first-generation system.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_10
This is very similar to Broadcom's OCP 2022 CPO demo.
Broadcom Packaged Optics Silicon Photonics OCP Summit 2022 5
This is the Tomahawk 4 implementation from Silicon Photonics.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_11
The new version is the Tomahawk 5 Bailly, a 51.2T Ethernet switch in the same class as the Marvell Teralynx 10 51.2T 64-port 800GbE switch we introduced recently.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_12
In the new version, Broadcom has improved the packaging, resulting in a more stable platform for optical interconnects, as FOWLP is used in mobile chips and more.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_13
Here are the steps to create a chip with an engine:
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_14
Here's a cross-section of the optical engine: Broadcom says this is a more scalable way to manufacture and integrate the optical engine.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_15
A few months ago, Broadcom pitched Bially the idea of having a 128-port 400G optical module. These optical modules include a light source, so it would have been more accurate for Broadcom to have included the light source.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_16
According to Broadcom, 800G modules consume 13-15W of power. With CPO and removing DSP complexity etc, this power drops to less than 4.8W.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_17
One of the key challenges is not only achieving the optical connection but making it work without error.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_18
Now, Broadcom says it is further optimizing its solution, a step to show vendors that it is on par with the quality they get with traditional pluggable optics.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_19
In a 51.2T switch, the optical network plug consumes a lot of power. Using co-packaged optics reduces the total power. The result is approximately 30% power savings.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_20
The next step will be to use similar techniques to combine compute ASICs with co-packaged optics. Here you can see CoWoS with HBM, compute ASIC, and optical chiplets packaged together.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_22
The GPU Attach looks a bit more advanced than the chips mentioned above, with more HBM and compute tiles, but still, the idea is that you can get 64 links out of the chip and plug them directly into a switch.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_23
Keeping the optics away from the xPU is also important because it keeps the optical engine away from hot compute. This may not seem like a big deal, but it is important to ensure the optics work reliably.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_24
Currently we have 64x 100G devices and 2 per 12.8T. What's insane about this is that the density goes from 12.8T to 102.4T, which is an almost insane amount of bandwidth.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_25
Broadcom is showcasing its use of bidirectional optical technology for high-radix networks, where transmission and reception occur at different wavelengths.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_27
This bidirectional optical approach is attractive because it lowers fiber costs.
Broadcom AI ASIC Optical Connection Hot Chip 2024_Page_28
There are plenty here.
Final Words
Optical connectivity has to come at some point. Intel announced its Silicon Photonics Connector in 2022, but still uses copper interconnects on its chips, even though it had plans to replace its HBM stack with optical connectivity via Lightbender next year in 2025 (Intel has shut down Lightbender). Broadcom does a lot of optical networking thanks to its networking business, and now ships packaged optical switches. The move to optical I/O will be a big deal for the AI space as companies look to build larger packages.
This was definitely one of the most amazing presentations this year.
