By Paul Momtahan, Infinera

Learn about the components inside a coherent optical engine, what they do, and how they use modulation to send and receive data.

Optical communications over metro, long-haul, and submarine networks once used simple direct-detect technology. That’s no longer the case. Demand for higher-speed optical connections has brought on complex modulations and better use of fiber-optic cables. Coherent optics, which use phase, amplitude, and the two polarizations of light, enable huge increases in wavelength speeds and fiber capacity.

Fiber-optic communications rely on lasers, photodetectors, and electronics packed into optical engines. The optical transmitters and receivers reside in embedded high-performance coherent modules or pluggable optical modules that go into network switches and servers. Here, we’ll look at the modulation used and the construction of an optical engine.

Simple modulation techniques, such as on-off keying or non-return to zero (NRZ), transmit a single bit (0 or 1) at a time, as shown in Figure 1. With NRZ, there was no need to tune the receiver to a specific frequency, hence the name “direct detect.” Direct detection worked well, initially for 2.5 Gb/sec wavelengths and later for 10 Gb/sec signals, but then ran into scaling challenges because the only way to increase data rates was to transmit more symbols (0 or 1) per second, that is, to increase the baud rate. Scaling the baud rate created multiple challenges due to optical impairments such as chromatic dispersion and polarization-mode dispersion. That’s because doubling the baud rate increases the sensitivity to chromatic dispersion by a factor of four and doubles the sensitivity to polarization-mode dispersion.

Since its emergence in the late 2000s, coherent optical technology has changed optical transport over the metro, long-haul, and submarine networks, enabling significant increases in wavelength speed, spectral efficiency, and fiber capacity. Coherent optics changed the optical transmission paradigm. It doubled the number of bits per symbol by transmitting independently on the two polarizations of light. Using polarization-multiplexed quadrature phase-shift keying (PM-QPSK) modulation, coherent optics doubled the number of bits per symbol again. Phase detection required the transceiver to tune to the wavelength’s frequency, hence the term “coherent.”

This quadrupling of bits per symbol with polarization and phase enabled 100 Gb/sec wavelengths by increasing the baud rate by 2.5 instead of a factor of ten that would have been required with direct-detect techniques.

The other significant change in coherent optical transmission came from using DSP to compensate for optical impairments such as chromatic dispersion and polarization-mode dispersion. In the direct-detect era, chromatic dispersion was compensated for in the photonic domain by placing dispersion-compensating modules (DCMs) on each fiber. These modules added cost and latency and required complex planning. Compensating for polarization-mode dispersion was impossible in the photonic domain and was a key limiting factor for transmission distances.

Because coherent technology uses X and Y polarizations, it doubles the number of bits per symbol. Data rates evolved from 100 Gb/sec based on PM-QPSK, with more complex modulation that leverages both phase and amplitude. From the four bits per symbol of PM-QPSK, PM-8QAM enables six bits per symbol, PM-16QAM enables eight bits per symbol and PM-64QAM enables 12 bits per symbol, as shown in Figure 2. Advanced modulation schemes such as probabilistic constellation shaping have provided increased granularity of bits per symbol and better tolerance to optical impairments. Symbol rates have also increased, from ~30 Gbaud (100 Gb/sec) to ~50-60 Gbaud (400 Gb/s) to ~60-70 Gbaud (600 Gb/sec) to ~90-100 Gbaud (800 Gb/sec). Next will come ~130-140 Gbaud (1.2 Tb/sec).

What’s inside a coherent optical engine?

At a high level, a coherent optical engine comprises three basic building blocks: a digital ASIC, analog electronics, and photonics (Figure 3), with the analog electronics and photonics often packaged together as a transmit-receive optical sub-assembly (TROSA). Together with the RF interconnects and the packaging, these three building blocks constitute a coherent engine. Each block consists of multiple functions.

The digital ASIC, often called “the DSP,” includes DSP functions for receiving and transmitting directions. Common transmit DSP functions include modulation encoding, spectral shaping, and pre-compensation. Common receive DSP functions include chromatic-dispersion compensation, clock recovery, polarization recovery, polarization-mode dispersion and polarization-dependent loss compensation, carrier recovery, and modulation decoding. Non-DSP functions commonly integrated into the same digital ASIC include the digital-to-analog converter (DAC), the analog-to-digital converter (ADC), forward error correction (FEC), framing, multiplexing, encryption, and performance monitoring. The DAC converts the digital signal from the transmit DSP to an analog voltage to drive the photonics. The ADC takes the analog signal from the analog electronics and converts it to digital for the received DSP.

Between the digital ASIC and the photonics, analog electronics plays a critical role. It consists of drivers and transimpedance amplifiers (TIAs), with four drivers and four TIAs required for a single coherent interface. In the transmit direction, the drivers take the low voltages from the DAC and convert them to the higher voltages required by the modulator. In the receive direction, the TIAs take the currents from the photodetectors and convert them to voltages required by the ADC.

Inside the photonics, key transmit functions include the laser and the modulator. The laser generates light with the required frequency and power. As with all dense wavelength-division multiplexing (DWDM) lasers, it is made from indium phosphide. The modulator then takes the light from the laser and encodes the data by changing the phase and amplitude. It uses an electric field to change the material’s refractive index as the light passes through. This material is indium phosphide, silicon (i.e., silicon photonics), or potentially lithium niobate. The photonic functions are typically packaged as a photonic integrated circuit (PIC). The transmit and receive functions, including the laser, modulator, photodetectors, and amplifiers, often reside in a single indium phosphide PIC for a single wavelength or multiple wavelengths.

High-performance embedded and compact pluggables

Silicon’s CMOS process-node evolution and diverse market requirements have led the coherent optical engine market to bifurcate into two distinct segments: high-performance embedded optical engines and compact coherent pluggables, as shown in Figure 4.

For the pluggable market, coherent DSP designers have built ASICs optimized for low power consumption and small footprint, with the 7-nm CMOS generation enabling 400 Gb/sec in QSFP-DD, OSFP, and CFP2 pluggable form factors. Meanwhile, high-performance engines leverage larger, more powerful, and power-hungry digital ASICs that deliver the highest possible baud rates and advanced features that maximize wavelength capacity reach and spectral efficiency. These high-performance engines are embedded in transponders and are the form factor of choice for long-haul and submarine applications.

Reduced cost, power, and footprint

Because the digital ASIC/DSP is made with silicon, it is subject to the same CMOS process node improvement cycle, made famous by Moore’s Law, which drives the entire chip industry. The silicon industry has a roadmap through 2034 to enable increased DSP baud rates. Expect to see 2-nm DSP ASICs in 2026/2027 and a further reduction (1.5 nm or 1.4 nm line width) in the future. In terms of modulator materials, silicon photonics are limited to approximately 140 Gbaud, and indium phosphide has a path to baud rates well beyond 200 Gbaud.

Alternative high-baud-rate modulators, including those based on thin-film lithium niobate and plasmonics, are at various stages of research and development but are currently largely unproven in large quantities. At industry conferences such as OFC and ECOC, novel materials, including plasmonic modulators and graphene photodetectors, offer a possible road to terabaud (1,000 Gbaud) coherent optics.

Higher baud rates increase wavelength capacity reach by leveraging lower-order modulation to achieve the same data rate. Lower-order modulation benefits from greater Euclidean distance between points in a constellation diagram, making them easier to distinguish in the presence of noise.

The increased wavelength capacity-reach of higher baud rates minimizes cost, power, and rack units per Gb/sec for a given reach-path requirement. For these reasons, the coherent optical industry is evolving embedded engines to 1.2 Tb/s and ~130-140 Gbaud leveraging 5-nm CMOS DSPs and 1.6+ Tb/s and 200+ Gbaud leveraging 3-nm CMOS DSPs, with even higher data rates and baud rates expected to follow. Regarding pluggables, the current generation of 400 Gb/sec 7-nm coherent pluggables is evolving to 800Gb/s, leveraging ~120-130 Gbaud. These 800 Gb/sec pluggables (800ZR/ZR+) leveraging 5-nm, 4-nm, or 3-nm CMOS should hit the market around 2025. Looking ahead, 1.6 Tb/s pluggables (1600ZR/ZR+) leveraging ~200-260 Gbaud and 3-nm or better CMOS should appear around 2027.

Increasing fiber capacity requires more spectrum

Higher baud rates have a downside. The amount of spectrum the wavelength uses is proportional to the baud rate. For example, doubling the baud rate doubles the wavelength’s spectrum. So, a higher baud rate won’t increase spectral efficiency or fiber capacity. Spectral efficiency is limited by Shannon’s law. This law/theorem limits the amount of information communicated over a channel with a given bandwidth and amount of noise. This is described by the famous equation:

Where C is the maximum achievable channel capacity, B is the bandwidth, and SNR is the ratio of signal power to noise power.

At higher SNR values, where we can ignore the 1 in the equation, increasing spectral efficiency by adding 1 bit per symbol to the spectral efficiency requires us to double (+3 dB) the required SNR, thus halving the reach. Therefore, going from PM-64QAM (12 bits per symbol), which already has very limited reach (typically ~100-200 km), to PM-128QAM (13 bits per symbol) would halve the reach (~50-100 km), and 256QAM (14 bits per symbol) would quarter the reach (~25-50 km), while at the same time driving a significant increase in DSP complexity and power consumption.

This law/theorem also puts an upper bound on the maximum spectral efficiency achieved at longer distances. Today’s high-performance embedded optical engines typically operate between 1 dB and 2 dB from the Shannon limit. The maximum possible scope for improved spectral efficiency is in the 30% to 40% range, with the next generations of high-performance optical engines leveraging 5-nm or 3-nm CMOS DSPs typically targeting spectral efficiency improvements in the 10% to 20% range. The gains from each subsequent generation of high-performance optical engines are likely to be even more incremental.

How can we increase fiber capacity? DWDM optical-line systems (ROADMs, amplifiers, etc.) are currently limited to 4.8 THz in the C-band (Figure 5). One option to increase capacity is to leverage the additional 4.8 THz of the L-band, giving a total of 9.6 THz by upgrading the optical-line system to support C+L. The optical engine requires a laser that can tune in the 1569-nm to 1610-nm range of the L-band rather than the 1529-nm to 1567-nm range of the C-band.

An alternative approach to increase the data capacity of the C-band by up to 30% is Super C, which provides 6.1 THz of spectrum without the need to effectively double the number of line system components, as is the case with C+L. This also requires more widely tunable lasers than a conventional C-band optical engine. For the ultimate capacity, Super C plus Super L can provide 11.6 THz, enabling over 100 Tb/s on a single fiber pair.

Summary

Coherent optical engines comprise a digital ASIC/DSP, analog electronics, and photonics and leverage polarization, phase, and amplitude to massively increase wavelength data rates relative to the previous direct-detect technology. More recently, coherent engines have split into a high-performance embedded segment (800 Gb/sec evolving to 1.2+ Tb/sec) and a compact pluggable segment (400 Gb/sec evolving to 800 Gb/sec, then to 1.6 Tb/sec). Future coherent evolution will further leverage higher baud rates to drive down cost, power, and footprint. As we reach the Shannon limit, we will need more spectrum in C+L, Super C, or Super C + Super L to scale fiber capacity.