QSFP-DD Might Be the Mainstream Form-factor of 400G Optical Transceivers

Time to enter 2019, when 400G has become a hot topic in the optical communications industry, the world's leading optical transceiver manufacturers have launched their own 400G optical modules. When we list the form-factors of these manufacturers' 400G optical modules (as shown in the figure below), we found that all the manufacturers except the Finisar (acquired by II-VI) have adopted the QSFP-DD form-factor — the market seems to have recognized QSFP-DD as the first choice for form-factors of 400G optical modules, though some manufacturers have also introduced 400G optical modules with OSFP and CFP8 form-factors.

400G Form-factors of Mainstream Optical Transceivers Manufacturers

Tips: QSFP-DD is a high-speed pluggable module form-factor defined by the QSFP-DD MSA group.

"The QSFP-DD MSA group has defined the next generation, high-density, high-speed pluggable module form factor that addresses the industry need for high-density, high-speed networking solutions in a backward compatible form factor. The QSFP-DD Specification has been developed and refined by many companies within the QSFP-DD MSA group and released to the public."

Why do mainstream manufacturers choose the QSFP-DD form-factor? Does this mean that the future 400G optical modules will be based on QSFP-DD? In order to clarify these issues, let us first look at the history of QSFP-DD.

History of QSFP-DD

March 21, 2016 — The QSFP-DD MSA group announced a plan to develop high-speed, double-density quad small form factor pluggable interfaces.

September 19, 2016 — The QSFP-DD MSA group announced the release of preliminary hardware specifications, including drawings, for the new QSFP-DD form factor.

March 13, 2017 — The QSFP-DD MSA group released a specification for the new QSFP-DD form factor.

September 19, 2017 — The QSFP-DD MSA group released an updated 3.0 Hardware specification for the new QSFP-DD form factor.

March 13, 2018 — The QSFP-DD MSA group released QSFP-DD thermal white paper to address how the thermal performance of the QSFP-DD module is evaluated for use in a high-performance data center environment.

August 30, 2018 — The QSFP-DD MSA group announced the success of their mechanical plug fest.

September 18, 2018 — The QSFP-DD MSA group announced the release of an updated 4.0 Hardware specification for the QSFP-DD form factor. By this time, the QSFP-DD MSA is relatively complete, and the QSFP-DD optics of the leading optical transceiver manufacturers are also listed in this period. For example, Gigalight, the world's leading innovator of optical interconnect design, has introduced 200G optical interconnect solutions for large-scale data centers from 100G to 400G — 200G QSFP-DD SR8 and 200G QSFP-DD AOC.

In summary, from the beginning of 2016 to the end of 2018, the birth of QSFP-DD has matured for nearly three years. During this period, the members of the QSFP-DD MSA group have also increased from the original 13 promoters to the current 14 promoters (3 companies were acquired, so only 11 were actually left) and 52 contributors.

The changes in the promoters of the QSFP-DD MSA group during this period also verified an old saying: the hero of the situation — II-VI acquired the old optical transceiver manufacturer Finisar; Broadcom acquired Brocade; Lumentum acquired Oclaro; Cisco also completed the acquisition of Luxtera recently. After so many acquisitions, let's take a look at the big companies left. There are chip providers such as Broadcom (Avago uses Broadcom as the brand name after the acquisition of Broadcom), equipment vendors such as Cisco and Huawei, device providers such as Lumentum, optical transceiver manufacturers such as Foxconn Interconnect Technology, accessories manufacturers such as Molex and TE Connectivity, and so on, covering the entire communications industry.

Why are so many big companies working together to promote QSFP-DD? Let us find the reasons together now.

Why QSFP-DD

A successful form factor must support the entire set of media and transceiver types prevalent in the networking industry. For media this includes passive Direct Attached Copper cables (DAC), Multi-Mode Fibers (MMF) and Single-Mode Fibers (SMF). For transceivers and active copper or active optical cable assemblies, this includes those defined by Ethernet, Fibre Channel, and InfiniBand for 100 Gb/s, 200 Gb/s and 400 Gb/s. In addition, port density must not be compromised from that of current practice. Further, backward compatibility with the popular QSFP form factor is essential for industry adoption.

QSFP-DD, Quad Small Form-factor Pluggable Double-Density, is a new module and cage/connector system similar to current QSFP, but with an additional row of contacts providing for an eight lane electrical interface. The term "Double-Density" refers to the doubling of the number of high-speed electrical interfaces that the module supports compared to the regular QSFP28 module. The connector specification is ready for the new PAM4 electrical modulation format that supports 50Gb/s that provides another doubling of speed resulting is a 4x increased in port speed for the QSFP-DD compared to the QSFP28 module.

The Diagram of QSFP-DD Module and Host Interface

Next, we will analyze the features of QSFP-DD one by one.

Features and Benefits of QSFP-DD

QSFP-DD expands on the QSFP pluggable form factor, a widely adopted four-lane electrical interface.

QSFP-DD is with 2x1 stacked integrated cage/connector. Due to industry need, most pluggable form factors eventually see developed a two-high stacked cage-connector system in addition to a one-high cage connector system. Often the one-high system is included in the initial MSA specification and the two-high is left to independent individual suppliers. To serve better the industry, the QSFP-DD MSA Group chose to develop concurrently both the one-high and the two-high cage-connector systems.

SMT connector and 1xN cage, Cage design optimizations and module case optimizations enable thermal support of at least 12W per module. The QSFP-DD Specification defines power classes up to 14W as well as a >14W class. Due to innovative thermal management techniques used in the module and cage designs, QSFP-DD modules support power levels of at least 12W in a typical system design. The extensive knowledge and experience of system design with QSFP family form factors enables innovative systems solutions that could extend beyond that range. Thermal management features needed for the higher power consumption classes are relaxed for the lower power classes to avoid unnecessary costs.

QSFP-DD electrical interfaces employs eight lanes that operate up to 25Gb/s NRZ modulation or 50Gb/s PAM4 modulation, providing solutions up to 200Gb/s or 400Gb/s aggregate. QSFP-DD can enable up to 14.4Tb/s aggregate bandwidth in a single switch slot. By quadrupling aggregate switch bandwidth while maintaining port density, QSFP-DD can support continuing growth in network bandwidth demand and datacenter traffic.

Before the emergence of QSFP-DD, the most popular interfaces in the networking industry consisted of single (SFP/SFP+) or quad lanes (QSFP+/QSFP28). However, to accommodate expected demand for data bandwidth or channel capacity, eight lane interfaces are being defined in venues such as Ethernet. The currently available form factors that support eight lane interfaces do not have all the desired features or density necessary to support the next generation systems that plan to implement these higher rate interfaces. Thus, the QSFP-DD MSA group extended and defined QSFP-DD based on QSFP (QSFP+/QSFP28).

QSFP-DD vs. QSFP (QSFP+/QSFP28)

1. The new QSFP-DD interface expands on the QSFP pluggable form factor, a widely adopted four-lane electrical interface used across Ethernet switches that enables interconnection between switches or to servers. QSFP's four electrical lanes operate at 10Gb/s or 25Gb/s, providing solutions for 40Gb/s or 100Gb/s aggregate. The new QSFP-DD pluggable form factor's electrical interfaces employ eight lanes that operate up to 25Gb/s NRZ modulation or 50Gb/s PAM4 modulation, providing solutions up to 200Gb/s or 400Gb/s aggregate. This can enable up to 14.4Tb/s aggregate bandwidth in a single switch slot and address rapid datacenter traffic growth.

1. Systems designed with QSFP-DD modules are backwards compatible, allowing them to support existing QSFP modules and provide flexibility for end users and system designers. Backwards compatibility is critically important to the industry. Since ASICs are designed to support multiple interface rates, it is critically important that the system can take advantage of this. End users can take advantage of the newer ASIC and system products with lower port costs and are able plug in a wide range of currently available QSFP modules to support their desired media and reach without needing to have separate system products. This greatly decreases the risks associated with implementing new equipment. System designers can build common products that support a multiple of pluggable variants while leveraging known technologies and designs. Module designers do not need to port their lower rate designs into new non-backwards compatible form factors lowering their overall costs. The economy of scale achieved due to backwards compatibility make it highly desirable.

1. The system port densities are identical between QSFP-DD and QSFP28 module specifications. However, since each QSFP-DD port can accommodate 8 lanes instead of 4, QSFP-DD doubles the number of ASIC ports it supports for existing interfaces such as CAUI-4.

1. The mechanical interface for QSFP-DD on the host board is slightly deeper than for QSFP28 to accommodate the extra row of contacts. The height and width are identical to the QSFP form factor enabling system designers to achieve identical system port count densities for QSFP28 or QSFP-DD based designs. You can plug any current QSFP or QSFP28 module into the QSFP-DD 1x1 or the 2x1 cage/connector combinations.

In summary, QSFP-DD is a little longer than QSFP+/QSFP28 but the port density is the same, and the bandwidth is increased to 10 times or 4 times of the latter, and it is backwards compatible, which means customers can skip the QSFP system and directly deploys the QSFP-DD system, which greatly reduces the equipment costs.

At the beginning of this article, we mentioned that some optical transceiver manufacturers have also introduced 400G optical modules with OSFP and CFP8 form-factors. Let's compare QSFP-DD and OSFP, QSFP-DD and CFP8 to see how they differ.

QSFP-DD vs. OSFP vs. CFP8

QSFP-DD vs. OSFP

First, let's take a look at OSFP first. Not long ago (January 16, 2019), OSFP MSA released version 2.0. According to its description, the OSFP is a new pluggable form factor with eight high speed electrical lanes that will initially support 400Gb/s (8x50G). It is slightly wider and deeper than the QSFP but it still supports 36 OSFP ports per 1U front panel, enabling 14.4Tb/s per 1U.

Tips: In the latest release of OSFP MSA, OSFP already supports 800Gb/s, which may be the reason why OSFP is also one of the popular 400G form-factors.

• Size — According to the previous introduction, OSFP seems to have little difference from QSFP-DD, just "slightly wider and longer" than QSFP-DD. However, after comparing their specific size values, we found that the difference is not just a little bit. The width, length and thickness of QSFP-DD are 18.35mm, 89.4mm and 8.5mm, while those of OSFP are 22.58mm, 107.8mm and 13.0mm. If the module is roughly calculated as a cuboid, the volume of the OSFP could be more than twice that of QSFP-DD, and it is obvious that the former is much larger.

• Thermal Capacity and Power Consumption — The QSFP-DD is smaller in size, so its thermal capacity is only 7 to 12 watts. While the OSFP is larger in size, its thermal capacity can reach 12 to 15 watts. The larger the thermal capacity, the greater the power consumption that the optical module can withstand. However, with the advancement of technology, some industry-leading manufacturers have been able to reduce the power consumption of optical modules far below the upper limit of thermal capacity specified by MSA, so the larger thermal capacity does not seem to be a real advantage in the future. Consistent with the thermal capacity, OSFP's power consumption is generally higher than QSFP-DD. However, as we all know, the lower the power consumption, the better. As the world's leading innovator of optical interconnect design, Gigalight always focus low power consumption as one of the primary goals of optical transceivers. For example, the Gigalight 100G QSFP28 SR4 optical transceiver has been optimized to reduce power consumption to less than 2.5 watts, which is nearly 30% lower than the 3.5 watts in the industry. The Gigalight 200G/400G optical modules also have the advantage of low power consumption in the industry.

• Backwards Compatibility — OSFP is as backward compatible with QSFP+/QSFP28 as QSFP-DD, but requires an additional OSFP to QSFP adapter. Since the OSFP is slightly wider and deeper than the QSFP, it is possible to build an adapter that supports existing 100G QSFP optics modules (QSFP28) in a OSFP cage.

• Bandwidth — QSFP-DD currently only supports up to 400Gb/s, but OSFP can support up to 800Gb/s. Considering scalability, OSFP is slightly better than QSFP-DD. But 800Gb/s is too early, and when 800Gb/s starts to deploy, there may be better options.

In summary, QSFP-DD is mainly used to apply 400G networks that are currently being deployed (and 200G over 100G to 400G), while OSFP is more likely to be prepared for future 800G networks. Therefore, combined with the status quo, QSFP-DD is more suitable as a form-factor of 400G optical transceivers.

QSFP-DD vs. CFP8

The CFP series started from CFP, went to CFP2, then to CFP4, and finally to CFP8, which is also a long-established form-factor series. Compared to the QSFP series, the CFP series seems to have been less popular, for obvious reasons — large size and high power consumption. The first two companies that promoted the development of CFP MSA (Finisar and Oclaro) have also been acquired, and we seem to feel the end of CFP.

Let's take a look at CFP8. The CFP8 hardware specification was officially released by the CFP MSA on March 17, 2017, in the same period as the 2.0 version of the QSFP-DD MSA was released. Comparing the two form-factors, we seem to have foreseen the decline of CFP8.

• Size — The size of CFP8 (41.5mm*107.5mm*9.5mm) is significantly larger than QSFP-DD, and the volume is more than three times that of QSFP-DD, even more than 30% larger than that of OSFP. Since the CFP series optical modules have been positioned for telecommunication applications, and the port density requirements are not as high as in the data center, so the size is acceptable. However, with the advancement of technology, the QSFP series optical modules are also beginning to be suitable for telecommunication applications, and the power consumption of QSFP series optical modules is much lower than that of CFP series optical modules. Therefore, the dominant position of CFP series optical modules in telecommunication applications is at stake.

• Thermal Capacity and Power Consumption — The thermal capacity and power consumption of CFP8 is much higher than QSFP-DD. The introduction of thermal capacity and power consumption has been introduced in the previous QSFP-DD vs. OSFP, and the truth is the same.

• Backwards Compatibility — There is not any mention of backwards compatibility in the hardware specification of CFP8 (in fact, the entire CFP series does not seem to be backwards compatible). For CFP and CFP2 series optical modules, the CFP to QSFP28 adapter and CFP2 to QSFP28 adapter have been available for a long time, indicating that some users have switched to QSFP28 optical modules.

• Bandwidth — The maximum bandwidth of CFP8 and QSFP-DD is 400Gb/s, but CFP8 only supports 400Gb/s (16x25G or 8x50G), while QSFP-DD supports both 200Gb/s (8x25G) and 400Gb/s (8x50G).

In summary, QSFP-DD seems to be a better choice than CFP8, regardless of any aspect.

Conclusion

By analyzing the features of QSFP-DD and comparing it to other 400G optical module form-factors, we found that QSFP-DD has unparalleled advantages in 400G applications such as data center interconnects. It is expected that when the world's leading hyperscale data centers start to deploy 400G, QSFP-DD will become the mainstream form-factor of 400G optical modules.