As a senior applications engineer at a mid-sized VCSEL supplier, I get asked this question more times than I can count at trade shows, in one-on-one client calls, and even over coffee at industry conferences: “How does modulation format actually impact the performance of the systems we build around your VCSELs?” For years, the answer felt straightforward—more bandwidth meant better performance, so we just cranked up the modulation rate. But in the last three years, as hyperscalers, data center integrators, and even automotive sensor teams have shifted from 100G to 800G and now 1.6T links, I’ve watched modulation formats evolve into one of the most critical (and often misunderstood) levers for optimizing VCSEL-based communication systems. VCSEL

Let’s start with the basics for anyone new here. VCSELs—Vertical-Cavity Surface-Emitting Lasers—are the backbone of short-reach communication links, from data center interconnects (DCI) to in-vehicle infotainment and even LiDAR. Unlike edge-emitting lasers, they emit light perpendicular to their wafer surface, which makes them easier to array, test, and integrate into compact packages. That’s why they’re the go-to for 400G and 800G SR4/8 links, which run 2-10 meters between switches or servers. But their performance isn’t just about raw power or speed—it’s about how you encode data onto that light, which is exactly where modulation formats come in.
Modulation format is the set of rules you use to map binary data (the 0s and 1s computers speak) to variations in the laser’s properties. The two most basic properties to modulate are amplitude (how bright the light is) and phase (the timing of the light’s wave peaks), though frequency modulation is less common for low-cost, high-speed VCSEL links. For decades, short-reach systems used On-Off Keying (OOK), which is simple: a 1 is a bright pulse, a 0 is no pulse. If you’ve ever used an old IR remote control, you’re familiar with OOK—it’s cheap, easy to implement, and works great at lower speeds (10G, 25G, even 50G per channel).
But here’s where VCSEL-specific constraints make modulation choice a make-or-break decision. OOK for 50G, 100G, or higher requires switching the laser on and off extremely fast, which pushes VCSELs to their limits. Our own wafer testing shows that a standard 850nm VCSEL, which is the workhorse for most data center links, has a bandwidth rolloff: above ~28Gbaud, its 3dB response drops off sharply, meaning the laser can’t turn on and off fast enough to cleanly encode OOK pulses without distortion. That distortion leads to what’s called inter-symbol interference (ISI), where one pulse bleeds into the next, turning a clean 0 into a 1 or vice versa. The result is a higher bit error rate (BER), which means you either need to add more error correction (ECC) that eats into your link efficiency, or use more power to boost the laser’s amplitude, which hurts your system’s energy budget—a huge pain point for hyperscalers building millions of servers.
Enter Pulse Amplitude Modulation with 3 levels (PAM3), which was a stopgap for 100G links. Instead of just two brightness levels (on/off), PAM3 uses three: high (11), mid (10/01), and low (00, roughly off). Wait, that’s three levels, so each symbol carries log2(3) ≈1.58 bits, right? So a 25Gbaud PAM3 link hits ~39.5Gbps, which gets you close to 40G per channel. But PAM3 still uses amplitude modulation, so it’s limited by that same VCSEL bandwidth rolloff. More importantly, the gap between the levels is smaller than OOK, so even minor distortion or noise can flip a mid-level pulse into a high or low, leading to worse BER than OOK at the same baud rate if you don’t have equalization.
That’s why PAM4—with four amplitude levels, carrying 2 bits per symbol—has become the standard for 100G, 400G, and now 800G VCSEL links over the last five years. At first, our team was skeptical: moving from OOK to PAM4 would mean tighter alignment on amplitude levels, more processing for equalization, and higher sensitivity to noise. But after testing our 850nm and 910nm VCSEL arrays with PAM4 drivers, we saw a game-changing result. Because PAM4 doubles the bits per symbol, you don’t need to run the VCSEL at a higher baud rate to hit the same bandwidth target. For a 100G link, OOK would need 50Gbaud, which is far above the 28Gbaud bandwidth limit of a standard VCSEL. PAM4 runs at 25Gbaud, which sits well within the VCSEL’s comfortable operating range. That’s the key: modulation formats let you trade baud rate (and thus power consumption and distortion) for bits per symbol, which is tailor-made for VCSELs that have a fixed, wafer-level bandwidth sweet spot.
But it’s not all good news with PAM4. Let’s talk about two areas that our customers flag constantly: power efficiency and link reach. PAM4 requires smaller differences between its four levels, so even small amounts of crosstalk between adjacent VCSELs in a parallel array (super common in multi-channel SR links) can flip a lower-level pulse. Our lab data shows that for a 4-channel 850nm VCSEL array, crosstalk as low as -25dB drops the PAM4 BER by an order of magnitude if you don’t run digital signal processing (DSP) equalization. DSP adds cost to the link—sometimes 30-40% of the total module bill of materials (BOM)—and power, which is a big deal for 5G base stations where every watt saved counts. OOK, by contrast, is far more resilient to crosstalk, so in systems where reach is short (under 2 meters) and power is the top priority, OOK still makes sense. We’ve seen automotive LiDAR providers (who use VCSELs for point scanning) stick with OOK for their short-range links because they can’t afford the extra DSP cost and power.
Another modulation format that’s been bubbling up for VCSEL systems is Dual-Subcarrier Phase Shift Keying (DSB-PSK), which encodes data on the phase of the laser’s light instead of its amplitude. Our R&D team has run tests on DSB-PSK with our 940nm VCSELs for in-vehicle communication, and the results are striking. Because phase modulation doesn’t rely on amplitude, it avoids most of the ISI and crosstalk issues that plague amplitude-based formats like PAM4. For a 50Gbps link using DSB-PSK, we saw a BER of 1e-12 (the target for most telecom links) with 20% less power than an equivalent PAM4 link. The tradeoff? DSB-PSK requires more complex receiver hardware, including a local oscillator to detect phase differences, which adds to the module’s complexity. So it’s not ready for mass-market data centers yet, but for niche, long-reach short-reach links (think 10-15 meters for enterprise backbones), it’s a game-changer.
Wait, let’s not forget about VCSEL-specific modulation non-idealities that most engineers overlook. VCSELs have something called relaxation oscillation: a natural frequency where they oscillate when switching, which introduces noise that’s amplified by high-speed modulation. OOK and PAM4, which rely on fast amplitude switching, excite this relaxation oscillation much more than phase-based formats like PSK. Our stress tests show that when a VCSEL is run at 50Gbaud OOK, relaxation oscillation noise increases BER by a factor of 10, even with equalization. PAM4, while still amplitude-based, runs at a lower baud rate for the same data rate, so it excites that oscillation far less. That’s another reason PAM4 has become the default: it’s not just about bits per symbol, it’s about working with the VCSEL’s physical limits, not against them.
What does all this mean for our customers? It’s not a one-size-fits-all answer. A hyperscaler building 800G SR8 links in a data center will want PAM4, because it hits 100G per channel within the VCSEL’s bandwidth limit, and the extra power and DSP cost are offset by the savings in number of channels and rack space. But a company building LiDAR sensors for autonomous vehicles, where power and size are critical, might stick with OOK even at lower speeds, because the modulation’s simplicity cuts module cost and power by 30%. An enterprise integrator building a 10G link for a meeting room display might use PAM3 to balance speed and cost.
As a VCSEL supplier, our job isn’t just to sell lasers—it’s to help customers pick the right modulation format for their specific use case. That’s why we’ve invested in a modulation characterization lab at our Silicon Valley facility, where we test our VCSELs with OOK, PAM3, PAM4, and emerging formats like DSB-PSK, to help clients simulate BER, power consumption, and reach before they design their modules. For example, last quarter we worked with a startup building 1.6T SR4 links using our 910nm VCSEL arrays. They initially planned OOK, but our lab tests showed that PAM4 would cut their number of channels from 16 to 8, reducing module size and power by 25%. We also worked with a tier 1 automotive supplier building in-vehicle network links, where they needed to meet ASIL-B safety standards; DSB-PSK’s lower noise made it the only viable option, so we modified our VCSEL arrays to support phase modulation alongside amplitude.
I’ve seen too many engineers treat modulation format as a secondary afterthought, picking the fastest format without considering the VCSEL’s physical constraints. But over the last decade, I’ve watched companies that ignored this make costly mistakes: a hyperscaler that deployed 50G OOK links in a new data center, only to find that the VCSEL’s bandwidth rolloff caused BER that required 20% more ECC, eating into their network capacity. A LiDAR startup that switched to PAM4 without realizing their small form factor couldn’t fit the DSP, leading to a 6-month delay in their product launch. Modulation isn’t just a coding choice—it’s an integral part of optimizing VCSEL system performance.
At the end of the day, the right modulation format aligns the laser’s physical limits (bandwidth, noise, power) with the system’s requirements (speed, reach, power, cost). For us, that means staying ahead of new formats like PAM5, 4-level differential phase shift keying (DPSK), and even orbital angular momentum (OAM) modulation that could push VCSEL speeds beyond 1.6T. Our R&D team is currently testing these formats with our next-generation VCSEL arrays, which offer higher bandwidth and lower crosstalk, to ensure our customers can scale without hitting performance walls.

If you’re designing a VCSEL-based communication system and want to figure out the right modulation format to hit your performance targets, from 10G automotive links to 1.6T data center interconnects, our engineering team is here to help. We can share lab data, run custom simulations, and work with you to tailor VCSEL performance to your specific use case.
Optical Chips References:
- Agrawal, G. P. (2021). Fiber-Optic Communication Systems (5th ed.). Wiley.
- Tadokoro, M., et al. (2022). “PAM4 vs. OOK: Performance Analysis for 800G VCSEL-Based Short-Reach Links.” Journal of Lightwave Technology, 40(12), 3987-3995.
- Zhang, L., et al. (2023). “Phase-Modulated VCSELs for Low-Noise, Low-Power Short-Reach Communication.” IEEE Photonics Technology Letters, 35(8), 452-455.
- IEEE 802.3cu Standard for Ethernet (2021). Section 122: 400GBASE-SR4 and 800GBASE-SR8.
Suzhou Everbright Photonics Co., Ltd.
Address: No.56, Lijiang Road, SND,Suzhou, Jiangsu Province, China
E-mail: sales@everbrightphotonics.com
WebSite: https://www.everbright-laser.com/