Serial RapidIO vs. 10 Gigabit Ethernet

Understanding the differences, disadvantages, and benefits of each

BY KASHIF HASNI
Integrated Device Technology, San Jose, CA
www.idt.com

The explosion of multimedia content for mobile applications has spawned an evolution in wireless networks. High-performance systems must be developed to keep up with the demands of increased voice and data traffic. In order to stay ahead in a fiercely competitive market, system designers must choose a scalable interconnect standard that provides the best performance, cost, power, and reliability. Serial RapidIO (S-RIO) is the only protocol that addresses the specific needs of wireless base-band processing applications, although 10 gigabit Ethernet (10GE) is sometimes considered as an alternate solution.

Fig. 1. Efficient communication within a cluster of high performance processing elements is a key consideration for wireless base band systems.

In order to understand the differences between 10GE and S-RIO, one must look at the history of the development of the two standards. Ethernet was designed for large LAN or WAN type networks, with the assumption that powerful processors would be available to augment the software protocol stack. The nature of transactions in these types of networks typically includes large streams of data traffic with hefty payload sizes. However, wireless base band processing has different requirements. In a wireless system, high-performance processing elements carrying both control and data plane traffic need to communicate with each other quickly and reliably. Serial RapidIO was designed to support these types of requirements.

Performance matters:

3G+ and 4G wireless protocols and Advanced Antenna System (AAS) architectures require strong and predictable system performance. To address these requirements, latency and protocol efficiency have been given special attention in RapidIO. To minimize latency, RapidIO was designed with a small packet header and a simple routing architecture based on Destination ID’s (DestID). This simple transport layer function enables sub-100-ns cut-through switch latencies. RapidIO provides the ability to “stomp” packets in flight, allowing a switch device to forward a packet before the entire packet is received (stomp is a control symbol used in the RapidIO protocol to cancel a partially transmitted packet in the event of an error). This benefits overall system performance without compromising reliability. If an error is detected on a packet that is being forwarded, a RapidIO port can simply stomp a packet in transmission with no software intervention required.

In contrast, 10GE packets have much larger headers, resulting in higher latency when communicating with other processing elements. For LAN and WAN type networks, large packet sizes can compensate for the large header size and associated overhead. However, traffic in wireless applications is typically smaller in size and more frequent in number than traffic associated with a LAN/WAN model. For this type of traffic with small payloads, 10GE’s large packet headers negatively impact the efficiency of each transaction and degrade overall system performance. 10GE layer 2 switch latencies are also much larger than latencies seen with S-RIO, well above 200ns in best case scenarios. Ethernet cannot stomp packets in flight. Instead, it passes on packets as “best effort” and relies on software at layer 3 or 4 to verify the integrity of the packet contents.

Achieving reliability without hampering performance:

Reliability is where RapidIO shines. The acknowledgement scheme that is built into the hardware prevents dropped packets and guarantees delivery. The mechanism for reliable packet delivery is handled at the physical layer and performed purely in hardware. Handling packet delivery in hardware has a significant positive impact on system performance. Since reliable delivery is achieved without software intervention, transactions are not delayed by software routines. With the 10GE standard, the physical layer only manages packet delivery at a best-effort level (that is, packets may be dropped). Guaranteed packet delivery requires traversal to the TCP layer where reliable delivery mechanisms may be implemented in software.

Accessing the software layer takes up valuable time, consumes processing resources, and hampers system performance. This software intervention can result in system latencies of more than ten microseconds for 10GE designs. In contrast, S-RIO system latencies are only about one microsecond. Packet retransmissions are even more complex in 10GE systems and can consume tens of microseconds. For S-RIO designs however, packet re-transmissions are transparent, handled completely in hardware, and take less than a microsecond to complete. Also, reliance on software for guaranteed packet delivery results in non-deterministic system latencies. Depending on what software routines are running at the time of packet arrival, the time required to complete packet delivery operations is somewhat unpredictable. 10GE’s reliance on software for guaranteed packet delivery makes it a poor choice for systems that value low and deterministic latency.

It’s about system cost, not just device cost:

10GE requires the availability of a processor in order to run software that implements the protocol stack. This management of the software stack adds processing overhead and reduces system efficiency. Ethernet can consume anywhere from 15-30% of processing performance for software stack management. For a $100 processor, that can amount to unseen system interconnect costs of $15-$30 per processor. In a RapidIO system, where the protocol minimizes the reliance on software to reduce the burden on the processor, those savings could be used towards faster processors resulting in higher performance systems that can be designed at lower costs.

System power:

The reduced processor burden required to handle the RapidIO protocol also results in lower system power. Since the use of multi-gigahertz processors is lessened for protocol management, system power consumption is kept to a minimum. This also shrinks thermal management costs, and reduces system complexity as a result. In 10GE systems, increased usage of processing resources can result in higher system power.

Scalability provides a competitive advantage:

The wireless infrastructure market is highly competitive. As vendors contend with each other to offer more subscribers per line card, building scalable systems is of paramount importance. System designers must design an architecture that can be easily modified to adhere to changes in performance. A RapidIO system can easily scale from its lowest speed to its highest speed using the same register set. Available port speeds are 1, 2, 2.5, 4, 5, 8, 10, 16 and 20 Gbits/s. In contrast, Ethernet requires a new set of registers and exhaustive changes to the system software in order to go from one gigabit speeds to 10 gigabit bandwidth. Another important missing component to the 10GE ecosystem is the availability of small port count switches. Most devices address large port count backplane or aggregation devices with many 1 GE ports and only two 10GE ports.

The benefits of S-RIO include low and deterministic latency, reduced burden on system processors, high reliability, and loose coupling between processors and software for protocol management. These benefits have made S-RIO the default protocol of choice for wireless applications. Due to its disadvantages, 10GE is typically discarded as a viable protocol for wireless baseband applications. With RapidIO, designers can build scalable systems that maximize performance while minimizing power and cost, and can allow them to finish ahead in a highly competitive market. ■

Table 1. Summary of differences between 10GE and S-RIO