AI’s Next Great Bottleneck Isn’t the Chip—It’s Everything Between the Chips

For years, the artificial-intelligence industry has behaved as though the future could be secured simply by manufacturing a more powerful chip.

More transistors. More GPU cores. More high-bandwidth memory. More performance per watt. More enormous numbers printed across keynote slides while an executive in a leather jacket explains that the latest processor is approximately seventeen civilizations more advanced than the one released six months ago.

I understand the obsession. Chips are tangible. They can be photographed, benchmarked and given names that sound like rejected villains from a superhero movie. They also provide investors with a comfortably simple story: Company A has the fastest chip, so Company A wins.

Unfortunately, computers do not run on keynote slides.

They run as systems, and those systems must continually move staggering amounts of data among processors, memory, switches, racks and data centers. A GPU can perform calculations at astonishing speed, but it cannot calculate data it has not received. If the information arrives slowly, inefficiently or at enormous electrical cost, then the world’s most advanced processor becomes a very expensive machine waiting for someone to hand it the next assignment.

That is why I believe SK hynix’s newly published roadmap for co-packaged optics deserves more attention than the usual parade of semiconductor announcements. The company and researchers from several leading universities have outlined a path toward replacing more of AI infrastructure’s electrical connections with optical ones. Their work, published in Nature Electronics, proposes a progression from two-dimensional optical packaging through 2.5D interposer-based integration and eventually three-dimensional heterogeneous stacking.

The basic idea is simple enough: AI systems need to move more data, more quickly and with less energy. Copper is beginning to struggle with that assignment. Light may be better suited to it.

The engineering required to make that transition, naturally, is anything but simple.

The AI Industry Has Discovered Traffic

The easiest way I can explain the problem is to imagine a city that has spent billions constructing spectacular skyscrapers while treating every road, bridge and rail line as a minor administrative detail.

Each new building is taller and more sophisticated than the last. Every architect boasts about the number of offices it contains. The brochures show marble lobbies, rooftop gardens and elevators capable of reaching the top floor before passengers have finished regretting their career choices.

Then eight o’clock arrives.

The buildings may be magnificent, but nobody can get to them.

That is increasingly the predicament facing large AI systems. The processors are the skyscrapers. The interconnects are the transportation network. The industry has been remarkably successful at increasing computational capacity, but the volume of information that must travel through the system is growing so quickly that data movement is becoming a defining limitation.

High-bandwidth memory, or HBM, helped address one portion of this problem by placing extremely fast memory close to AI processors. SK hynix has become one of the most important companies in the AI supply chain largely because of its strength in HBM. Yet even HBM does not make the rest of the system disappear.

Modern AI clusters can contain thousands of accelerators. Those processors must communicate across boards, servers, racks and sometimes entire facilities. Training a large model involves coordinating work across this enormous collection of hardware. Every accelerator needs rapid access to the right data at the right moment.

As the cluster grows, the challenge changes. The question is no longer merely, “How fast is the chip?” It becomes, “How efficiently can the entire system keep all these chips working together?”

That is the difference between owning a collection of talented musicians and having an orchestra.

According to the SK hynix roadmap announcement, traditional electrical interconnects encounter greater power consumption, latency and signal complexity as speeds and distances increase. The accompanying paper in Nature Electronics identifies resistive losses, capacitive loading and frequency-dependent distortion as fundamental constraints on electrical links.

In less ceremonial language, electricity becomes increasingly difficult to push through copper at the required speeds without consuming more power, generating more heat and introducing more opportunities for the signal to deteriorate.

The AI industry has therefore reached an awkward moment. It spent years demanding faster chips, received faster chips and now must confront the fact that those chips need to talk to one another.

Who could have predicted that communication would become a problem?

Aside from anyone who has attended a staff meeting, I mean.

What Co-Packaged Optics Actually Means

Co-packaged optics, commonly shortened to CPO, brings optical communication components much closer to the processor or network switch they serve.

In conventional systems, high-speed electrical signals often travel from a chip across a circuit board to a separate optical transceiver. That transceiver converts the electrical signal into light, which can then travel through optical fiber.

The arrangement works, but the electrical journey between the processor and the transceiver becomes increasingly troublesome as bandwidth climbs. Those short copper traces may not look dramatic, but at extreme data rates they consume power and complicate signal integrity.

CPO attempts to shorten that electrical path by integrating the optical transceivers directly into, or very near, the processor package. Information can then be converted to light sooner and transmitted optically across the distances for which light is better suited.

Think of it as moving the train station next to the factory instead of transporting every product across town by truck before loading it onto a train.

The factory still needs trucks for the final few yards. It simply stops pretending trucks are the ideal way to move everything across the country.

Optical links offer several compelling advantages. They can carry tremendous amounts of information over distance with relatively low signal loss. Multiple wavelengths of light can travel through the same fiber, allowing enormous aggregate bandwidth. Optical communication can also reduce the energy required to move each bit, an increasingly important consideration when AI data centers already consume enough electricity to make utility planners stare silently into the middle distance.

The roadmap published in Nature Electronics establishes ambitious system targets: more than 100 terabits per second of bandwidth per node, energy consumption below one picojoule per bit and chip-to-chip latency below 10 nanoseconds, according to industry coverage of the work by TrendForce.

Those numbers represent more than an effort to make an individual component faster. They describe the communications foundation required to keep future AI systems supplied with data.

That distinction matters.

The next phase of AI competition may not be won by the company with the most impressive processor in isolation. It may be won by the company—or collection of companies—that can make processors, memory, networking, packaging, software, power delivery and cooling behave like one coordinated machine.

The glamorous chip must learn to live with roommates.

The Roadmap: 2D, 2.5D and 3D

SK hynix and its research partners describe a staged development path.

The first stage involves two-dimensional co-packaged optics. Optical components and computing devices sit alongside one another on the same general packaging plane. This approach can shorten electrical links and bring photonics closer to processors without immediately attempting the most difficult possible form of integration.

It is the sensible opening move: improve the neighborhood before trying to stack the entire city vertically.

The next stage uses 2.5D integration. In this architecture, processors, memory and photonic components can be mounted on a shared interposer. An interposer is essentially a sophisticated platform containing dense connections among the chips placed upon it.

The “half dimension” in 2.5D sounds like something a physicist invented to prevent ordinary people from becoming too confident. In practice, it means multiple advanced components can be tightly integrated side by side without being fabricated as one gigantic monolithic chip.

This is already a crucial concept in high-performance computing. HBM stacks are commonly placed close to processors using advanced packaging and interposer technologies. Extending that environment to include photonics could create exceptionally dense, high-bandwidth communication among compute, memory and optical interfaces.

The longer-term destination is three-dimensional heterogeneous integration. Instead of placing every component beside every other component, engineers stack different technologies vertically.

That could mean combining logic, memory and photonic layers in a compact structure with extremely short internal connections. It offers the possibility of extraordinary bandwidth density, but it also introduces a splendid collection of manufacturing and thermal problems.

Every technological roadmap eventually reaches the stage where the diagram becomes beautifully simple and the factory becomes deeply unhappy.

A 3D package may need to combine materials and manufacturing processes that were not originally designed to coexist. Logic chips run hot. Optical components can be sensitive to temperature. Lasers, modulators, detectors, memory and electrical circuitry must be manufactured, aligned, connected and tested with extremely high precision.

Then the finished package must achieve acceptable yields at a cost customers will tolerate.

It is possible to build many remarkable things in a laboratory. The semiconductor business becomes interesting when someone asks whether millions of them can be manufactured reliably on a deadline.

Bringing Light All the Way to Memory

The most strategically significant portion of SK hynix’s vision is not merely using optics around network switches. It is the possibility of extending optical communication toward the memory interface itself.

That is a much larger ambition.

Today, optical networking commonly connects systems across greater distances, while electrical connections continue to dominate inside packages and between processors and nearby memory. SK hynix’s proposed optics-centric architecture would use a photonic interposer to connect pools of AI accelerators with pools of memory.

In other words, memory may become less tightly confined to one processor package and more accessible across an optical fabric.

If successful, this could allow multiple processors to share a large pool of memory more efficiently. That matters because the size and complexity of AI models continue to rise. A processor may be extremely capable yet remain constrained by how much memory is physically attached to it, how quickly that memory can be accessed and how efficiently data can move among processors.

Shared memory pools could make AI infrastructure more flexible. Instead of stranding memory capacity beside a processor that does not currently need it, the system could potentially allocate resources more dynamically.

I would compare it to replacing hundreds of private office filing cabinets with a central archive that everyone can access instantly—except this archive must answer billions or trillions of requests with almost no delay, consume very little energy and never tell an executive that his document was accidentally placed in a folder labeled “miscellaneous.”

This is where SK hynix’s involvement becomes particularly interesting.

The company is not merely observing the optical revolution from a safe distance while continuing to sell memory stacks. It is positioning memory as an active part of a broader AI architecture.

That is an important strategic evolution. HBM has already demonstrated that memory is not a commodity footnote in modern AI. Memory bandwidth can determine how effectively an accelerator performs. Packaging can determine how closely memory and logic cooperate. Interconnects can determine how well the entire cluster scales.

SK hynix appears to understand that the future value of memory will depend not only on the specifications of the memory device, but also on the architecture surrounding it.

The company wants to help design the traffic system, not merely sell exceptionally fast warehouses beside the highway.

From Component Supplier to Systems Partner

SK hynix’s Seunghoon Hong described the changing role of memory companies as moving beyond product supply toward helping customers build competitive systems. That statement may sound like standard corporate ambition, but I think it reveals the real strategic logic behind the roadmap.

AI hardware is becoming a co-design business.

The processor, memory and network can no longer be optimized independently without consequences. Increasing the performance of one component may simply shift the bottleneck elsewhere. A faster accelerator may demand more memory bandwidth. More memory bandwidth may require more advanced packaging. A larger cluster may require better optical networking. Better networking may create new cooling, power and software requirements.

The entire machine is interconnected, both literally and commercially.

This is why companies traditionally identified with one part of the stack are expanding outward. Nvidia has moved far beyond selling GPUs. It offers networking equipment, systems, software libraries and complete AI infrastructure. AMD acquired photonic specialist Enosemi to strengthen its co-packaged-optics capabilities, as Reuters reported in 2025. Hyperscale cloud companies increasingly design custom accelerators and influence memory configurations, networking standards and packaging.

Everyone has discovered that controlling one excellent component is useful. Influencing the architecture is better.

SK hynix does not need to transform itself into another Nvidia. It does, however, need to ensure that future AI systems are designed in ways that preserve memory’s strategic importance—and preferably in ways that favor technologies SK hynix can manufacture, integrate or help define.

Publishing an architectural roadmap in a respected journal serves several purposes. It demonstrates technical credibility. It encourages industry discussion. It helps align universities, suppliers, customers and standards organizations. It can also influence which problems the broader ecosystem decides are worth solving.

A roadmap is not merely a prediction about the future. It is frequently an invitation for everyone else to help build the future its authors would prefer.

The Heat Problem Has Entered the Conversation

If optical interconnects are so attractive, a reasonable person might ask why the industry has not already stuffed them into every processor package.

The answer is that photons may be elegant, but the hardware surrounding them remains stubbornly physical.

Thermal management is one of the largest challenges. AI processors produce substantial heat. Placing temperature-sensitive optical components close to those processors creates an obvious conflict. The entire purpose of co-packaging is proximity, yet proximity makes the thermal environment harsher.

Engineers must ensure that lasers, modulators and detectors operate reliably while sitting near devices that behave like tiny industrial furnaces.

External laser sources may help by moving some heat-generating or failure-prone components away from the main package. But that decision introduces trade-offs involving coupling, packaging complexity, maintenance and efficiency.

Manufacturing is another obstacle. Semiconductor fabs are extraordinary at producing electronic devices with microscopic precision. Photonic components bring different materials, geometries and tolerances. Combining the two in high volume requires new processes, new equipment and new ways to test whether the finished device works.

Yield could become especially painful. In a complicated package containing several expensive components, one defective element can diminish the value of the entire assembly. The more ambitious the integration, the more opportunities the manufacturing line has to transform a costly engineering masterpiece into an equally costly paperweight.

Then there is standardization.

AI infrastructure is built from products supplied by many companies. Optical interfaces, packaging specifications, control mechanisms and software must interoperate. Without common standards, the industry risks creating a collection of impressive proprietary islands.

Technology companies adore open ecosystems, provided the ecosystem opens directly into their own gift shop.

Repairability and serviceability also matter. Pluggable optical modules can be replaced individually. Co-packaged optics may offer better performance, but a failed optical component inside an integrated package could be much harder and more expensive to replace.

These are not reasons to dismiss CPO. They are reasons to resist confusing a roadmap with a finished highway.

The Bandwidth Wall Is Also an Energy Wall

The AI infrastructure debate often focuses on raw speed, but energy efficiency may ultimately exert even more pressure.

Moving data consumes energy. As systems scale, data movement can account for a significant portion of total power consumption. The farther and faster electrical signals must travel, the harder engineers must work to preserve them.

That usually means more circuitry, more power and more heat.

Heat then requires cooling. Cooling requires additional power. Additional power demands larger electrical infrastructure. Larger facilities place more pressure on grids, water resources and local communities.

At some point, “just add more GPUs” stops being a technical strategy and begins sounding like the operating philosophy of someone who has never received an electricity bill.

Optical interconnects will not make AI free or environmentally weightless. Lasers consume power. Optical signals must still be generated, modulated, detected and converted. The manufacturing process has its own costs.

But reducing the energy required to move each bit could have an enormous system-level effect when multiplied across the astronomical volumes of data traveling through an AI cluster.

The target of less than one picojoule per bit is therefore not an ornamental engineering milestone. It reflects the reality that future systems cannot scale indefinitely if communications power grows alongside bandwidth without meaningful efficiency gains.

The industry is learning that performance per watt is not merely a responsible metric. It is a survival metric.

Why I Think This Roadmap Matters

I do not view SK hynix’s announcement as proof that optical memory fabrics will dominate AI systems tomorrow. Roadmaps are not purchase orders, and scientific publication is not mass production.

I view it as evidence that the center of AI competition is moving.

The first era of generative AI hardware was defined by access to accelerators. Companies scrambled to secure GPUs. Investors treated every mention of HBM as though someone had discovered a new element. Data-center operators rushed to install enough hardware to satisfy demand.

The next era will be defined by utilization, efficiency and scale.

It will not be enough to own processors. Operators must keep them busy. They must feed them data without wasting intolerable amounts of power. They must coordinate thousands—or eventually far more—accelerators as if they were one machine. They must manage memory as a system resource rather than an isolated attachment.

That requires networking, packaging, optics, memory architecture and software to advance together.

SK hynix’s roadmap matters because it places a leading memory manufacturer directly inside that systems conversation. The company is saying, in effect, that the future of memory cannot be separated from the future of interconnects.

I think that conclusion is correct.

The most important component in a computer is often the one currently preventing every other component from reaching its potential. Yesterday that was compute. Then it was memory bandwidth. Tomorrow it may be optical connectivity, power delivery, cooling or software orchestration.

There is no permanent hero in system architecture. There is only the next bottleneck.

My Cautious Conclusion

I am enthusiastic about the direction of co-packaged optics, but I am not ready to declare copper deceased and begin arranging the memorial service.

Electrical interconnects will remain useful across many distances and applications. Engineers will continue improving signaling, packaging and circuit design. Optical technology will enter the system gradually, where its benefits justify the added manufacturing complexity and cost.

The transition will probably be uneven. Networking switches may adopt CPO differently from accelerators. Rack-scale optical links may mature before direct optical memory interfaces. Some architectures may favor external lasers; others may integrate more functions into the package. Standards will evolve, companies will defend proprietary advantages and at least one supposedly universal interface will become universally supported by three vendors.

That is how technology develops in the real world: not as a clean replacement, but as an extended argument among physics, economics and corporate strategy.

Still, the direction is difficult to ignore.

AI models are growing. Clusters are expanding. The volume of data in motion is climbing. Electrical links face increasing pressure from bandwidth, distance, energy and signal-integrity requirements. Optical communication offers a credible way to relieve those constraints.

SK hynix’s roadmap does not solve every problem, but it identifies the battlefield with unusual clarity.

The AI race is shifting from chips to systems.

That means the winners will not necessarily be the companies capable of producing the fastest individual component. They will be the companies that understand how computation, memory and communication must work together—and that can manufacture those relationships at scale.

For a long time, the technology industry invited us to stare reverently at the processor. It was the brain, the engine and the star of the show. Everything else was supporting equipment.

Now the supporting equipment is quietly informing the star that the performance cannot continue until the wiring, memory, power and cooling have been reconsidered.

I find that deeply human.

We create an extraordinary mind and then discover it is limited by communication, energy, memory and its inability to cooperate efficiently with others.

Apparently, even artificial intelligence must eventually learn the lessons the rest of us have been avoiding.