Putting 10-GbE PHY options into perspective

As the 10-G market matures, several choices have evolved for physical layer connection

BY BUD NOREN
Fulcrum Microsystems
Calabasas, CA
http://www.fulcrummicro.com

The 10-Gbit Ethernet ecosystem is rapidly maturing, with multiple vendors offering network interface adapters with offload capabilities and high-density low-latency switches. For many high-speed applications, 10 GbE is becoming an increasingly attractive option to traditional Fiber Channel and InfiniBand interconnects. As the market matures, several choices have evolved for physical layer connection and, as with the 1-Gbit generation, both optical and copper are options. Each physical layer alternative has advantages in terms of distance, cost, latency, and physical media, which must be weighed together in relation to the system’s application.

Board, backplane interfaces

At the board level, XAUI (10-Gbit attachment unit interface) is replacing XGMII (10-Gbit media-independent interface) due to reduced pin count and much longer allowed trace lengths. XAUI is already the de facto standard for 10 GbE in the backplane, providing a highly efficient, low-cost interface between chassis blades with low design risk. With pre-emphasis and equalization, signal distance can be extended beyond 20 in. XAUI also offers support for multiple connectors, requires only a few layers and has lane reversal.

The IEEE Standards Association recently approved 802.3ap for backplane Ethernet, specifying 10-Gbit operations over a distance of up to 40 in. on a standard FR4 PCB. The 10GBASE-KX standard provides for two different implementations: 10GBASE-KX4, and 10GBASE-KR. The 10GBASE-KX4 standard specifies four lanes (similar to XAUI), while the 10GBASE-KR implementation uses one lane with 64/66B encoding. For backplanes with a high total bandwidth requirement or where trace congestion is an issue, the 10GBASE-KR solution can now be used via SerDes chips from several vendors (see Table 1 ).

Table 1. The 802.3ap specification yields three options for backplane connectivity.

Connecting outside the box

Copper interconnects have thus far dominated data center interconnectivity applications at 1-Gbit rates and lower because they have been the most cost-effective choice for interconnecting servers over the relatively short distances spanned. Copper interconnects using unshielded twisted pair cable (10GBASE-T) are expected to eventually also dominate 10-Gbit Ethernet data center applications, and first-generation solutions are available now.

These early solutions have established the feasibility of transporting 10-Gbit data over 100 m of UTP cable, but it will require another generation or two to produce solutions with the potential for widespread adoption. In the meantime, other interconnect technologies, both optical and copper, should be considered.

Copper PHYs

10GBASE-CX4 . Ideal for high-performance datacenters, CX4 offers low cost and zero added latency over short distances. The relatively bulky twinax cable, similar to that used for InfiniBand, transports XAUI signals up to 15 m, or further with added equalization, and offers the lowest per port cost at the expense of range. The CX4 connector can also provide power through one of its pins. Optical cables are available that have CX4 connectors and that provide the electrical-to-optical and optical-to-electrical conversion in the cables themselves, allowing reaches well beyond 15 m.

10GBASE-T. This newly released standard uses a familiar and compact RJ-45 connector and inexpensive Cat6 cable to transmit up to 55 meters, and supports auto-negotiation between Gbit and 10 Gbit. Using the new partitioned, augmented Cat6 or “6a” cable specification, which is still in draft form and is designed to reduce crosstalk between UTP cables, 10GBASE-T signals can be transported up to 100 meters. Implementations of 10GBASE-T have been demonstrated by multiple vendors; however, it does have relatively high power consumption and several µs of latency, both of which are likely to be improved as components mature. This may be a good choice for enterprise Gbit Ethernet aggregation products.

Optical PHYs

Optical fiber links represent an elegant solution for interconnectivity in the data center due to the small size and weight of the fiber, ease of cable management, long reach, low susceptibility to EMI, and low latency. Cost reductions have begun to make then a more competitive choice compared to copper (see Table 2 ).

Table 2. Copper and fiber PHYs provide four options for the datacenter.

Parallel optics. A multi-fiber ribbon comprising four transmit and four receive fibers provides a lightweight, flexible cable with a CX4 connector, with about a 100-m reach. A relatively low-cost option when using 850 nm VCSEL optics, parallel optics offers low (but non-negligible) power consumption and negligible latency.

10GBASE-SR. The “SR” designates “short range.” This standard specifies distances from 26 m over legacy 62.5 µm core MM fiber, to 86 meters on standard 50-µm core MM fiber, to 300 m using high-quality laser-optimized (OM3) MM fiber using 850-nm VCSEL technology. It is a low-cost option with less than 1 µs of latency.

10GBASE-LRM . This newly ratified standard supports distances up to 220 m on legacy FDDI-grade multimode cable at 1,310 nm. Electronic dispersion compensation is required at the receiver and components are still expensive but it allows systems to use installed FDDI fiber with a low latency of 650 ns.

Transceiver modules

Multi-Source Agreement (MSA) groups formed by industry players have established physical form factors for optical and copper transceivers, while standards bodies such as Optical Internetworking Forum (OIF) have established standards for electrical interfaces for 10-Gbit transceiver modules. Module form factors have migrated from the initial 300-pin MSA to the 70-pin XENPAK, which has since given way to the smaller XPAK and X2. By moving some components outside the module, the 30-pin XFP is even more compact, while SFP+ is the smallest form factor yet—although it doesn’t support copper connections.

To some extent, these modules share architecture. For the optical I/O they use similar transmit and receive optical subassemblies (TOSA and ROSA), while on the electrical side they use similar components such as transimpedance amplifiers (TIA), laser drivers, and modulators, CDR circuits, and SerDes components. There is good availability of fiber reaches among most module types and CX4 connectors are available on the larger modules: XENPAK, XPAK, and X2 (see Table 3 ).

Table 3. MSA transceiver standards provide five choices

The small XFP and SFP+ modules differ in their architectures, contributing to their compact footprint. Besides moving the SerDes function outside the module, the XFP is smaller due to its use of a serial 10G signal as the electrical side I/O rather than the four-lane XAUI interface. The SFP+ module further shrinks the module size and power by also removing the CDR and electronic dispersion compensation (EDC) functions out of the module.

Figure 1 shows that the module interfaces have evolved away from XAUI and toward 10 Gbit serial interfaces. However, there are chips that perform the SerDes function outside the modules, providing a XAUI interface to upstream devices.

Fig. 1. Relative sizes of optical modules and the functionality pushed to the circuit board from generation to generation. The 10GBASE-T solutions that exist today also take XAUI inputs. This implies that Layer 2 devices such as MAC, NIC and switch chips, such as Fulcrum’s FocalPoint series of 10 Gbit Ethernet switches, that provide XAUI interfaces allow the most flexibility in choice of optical modules.

FAJH_Fulcrum_5Sep2007

Fig. 2. Fulcrum’s FocalPoint chip is a good example of current 10-Gbit Ethernet switches.

Today, X2 modules are most common in volume shipping applications, but most new designs now use XFP modules, with its smaller form factor enabling higher port densities. Products incorporating XFP modules will soon be the most widely shipped form factor; however, SPF+ promises even higher port densities, lower cost, and lower power, so once volume quantities of these modules become widely available, transition to SPF+ is likely to occur quickly. ■

For more on 10 GbE systems, visit www.electronicproducts.com/digital.asp.