This new 3D chip could break AI’s biggest bottleneck
Researchers have created a new kind of 3D computer chip that stacks memory and computing elements vertically, dramatically speeding up how data moves inside the chip. Unlike traditional flat designs, this approach avoids the traffic jams that limit today’s AI hardware. The prototype already beats comparable chips by several times, with future versions expected to go much further. Just as important, it was manufactured entirely in a U.S. foundry, showing the technology is ready for real-world production.
Engineers from Stanford University, Carnegie Mellon University, University of Pennsylvania, and the Massachusetts Institute of Technology worked with SkyWater Technology, the largest exclusively U.S. based pure play semiconductor foundry, to create a new multilayer computer chip. The team says its architecture could mark a major shift in AI hardware and strengthen domestic semiconductor innovation.
Unlike most of today's chips, which are mostly flat and 2D, this prototype is built to rise upward. Ultra thin parts are stacked like floors in a tall building, and vertical wiring works like many fast elevators that move huge amounts of data quickly. With a record setting number of vertical connections and a tightly woven layout that places memory and computing units close together, the design avoids slowdowns that have limited progress in flat chips. In hardware tests and simulations, the 3D chip beats 2D chips by roughly an order of magnitude.
Researchers have made experimental 3D chips in academic labs before, but the team says this is the first time one has delivered clear performance improvements and been produced in a commercial foundry. "This opens the door to a new era of chip production and innovation," said Subhasish Mitra, the William E. Ayer Professor in Electrical Engineering and professor of computer science at Stanford University, and principal investigator of a new paper describing the chip presented at the 71st Annual IEEE International Electron Devices Meeting (IEDM). "Breakthroughs like this are how we get to the 1,000-fold hardware performance improvements future AI systems will demand."
Why Flat Chips Struggle With Modern AI
Large AI models such as ChatGPT and Claude constantly shuttle enormous volumes of data between memory, which holds information, and the computing units that process it.
On conventional 2D chips, everything sits on one surface and memory is limited and spread out, so data is forced through a small number of long, crowded paths. The computing parts can run far faster than data can be delivered, and the chip cannot keep enough memory nearby. The result is frequent waiting. Engineers call this problem the "memory wall," where processing speed outruns the chip's ability to feed it data.
For years, chipmakers pushed back against the memory wall by shrinking transistors, the tiny switches that handle computations and store data, and packing more of them onto each chip. But researchers say that approach is nearing hard physical limits, known as the "miniaturization wall."
The new design aims to get past both limits by building upward. "By integrating memory and computation vertically, we can move a lot more information much quicker, just as the elevator banks in a high-rise let many residents travel between floors at once," said Tathagata Srimani, assistant professor of electrical and computer engineering at Carnegie Mellon University, the paper's senior author, who began the work as a postdoctoral fellow advised by Mitra.