Off-Chip Communications

Low-Speed Interfaces 

These communication schemes are intended for control purposes or for low- data-intensive applications, where only small amounts of data are exchanged. They mostly rely on serial interfaces with explicit low-speed clocks for easy data transfer and with cost-effective hardware, both on the transmitter and receiver sides. The amount of bits to transfer is limited, normally in the case of fixed-length transactions, and in some cases, addressing of external devices allows connecting more than one device on the same interface. These inter- faces are best suited to controlling external devices, such as smart sensors or similar, where the amount of data is less. For this reason, they are also intended for controlling external devices, providing a mechanism to access internal registers to set operation modes, and collecting either small amounts of data or status flags. 

I 2 C and SPI are the most commonly used interfaces in today’s world of standardized devices, controllers, smart sensors, ADCs, DACs, etc. They offer standard wiring for many off-the-shelf devices, providing a common method to interface with many of them. 

There are, in some cases, specific needs, such as very low latency, which produce yet standardized interfaces with particular requirements. It is the case of the CAN bus, extended in automotive and similar sectors, where access to the bus is granted based on the priorities of the messages to be sent. 

FPGAs and FPGA design tools are ready to use such communication interfaces. SoPC devices contain—normally more than one—hardwired interfaces attached to their internal processors. Also, tools for SoPC design, even for pure FPGAs with just configurable logic, contain libraries with the required modules, accessible from embedded processors by means of stan- dard interfaces to very rapidly customize the design and add a variety of such standardized communication modules. The design of these types of systems is quite simple since, for instance, there are numerous tutorials pro- vided as examples to build systems with I 2 C or SPI interfaces. 

High-Speed Interfaces 

Contrary to what one might think, there is a tendency to solve point to point fast I/O with optimized serial interfaces more than with parallel ones. For maximum throughput, several of these serial I/O interfaces may be con- nected in parallel to achieve really high throughput. An example of this is the PCIe interface described in Section 2.4.4, which may use, according to the specification, different number of serial lines. Details of PCIe as well as other standards that are normally used in FPGAs are discussed in Section 2.4.4. In this section, however, the criteria on how to use them and conditions of operation and precautions when such structures are used are given. 

High-speed interfaces are packet-based synchronous communication interfaces with clock recovery at the receiver end since it is not possible to transmit the clock as an additional line (as opposed to low-speed inter- faces). The clock is recovered with the aid of DPLL logic or similar struc- tures. Transceivers with the required specialized interfaces are equipped in many FPGAs, and there are families or specific devices that are equipped with many of such interfaces in order to achieve a very high aggregated data bandwidth. These specialized FPGAs are intended to be used in commu- nication devices, such as high-speed switches and routers, where process- ing lots of serial I/O efficiently and at high speed can only be done through hardware, not with software. 

These high-speed serial I/O interfaces are also useful in the domain of HPC, where high data throughput between several FPGAs, forming an FPGA cluster, is required. Except for specific applications with a precise connectivity, such as for pipeline-streamed applications, these clusters very often serve a general purpose, so connectivity between all FPGAs in the cluster is required. Figure 7.1 shows an example of a backplane that connects five circuits, for example, FPGAs, on a fully connected topology (all to all). 

FIGURE 7.1 All-to-all backplane connection, suitable for serial high-speed I/O interfaces.

Each of these links in the figure can be built using high-speed serial I/O interfaces. These rely on differential separate pairs for transmission and recep- tion, so, in essence, each of these connections represents four signals. Therefore, the densest part of the backplane, with six connections, would represent just 24 signals. If every pair of transmitting lines is designed to offer around 6 Gbps each, or even 50 Gbps each, which is possible in some technologies, the aggregated throughputs of the cluster would be 120 Gbps to 1000 Gbps. 

The design of these serial differential I/O lines at PCB level is, however, very critical. Pins in the FPGA need to be as close as possible, and the paths along the PCB must be almost identical, with sufficient grounding around them to avoid susceptibility to noise, since they operate at low voltages, to allow for such high switching activity at pin level. 

While these point-to-point connections are intended for PCB use, or for very short distance interconnections, there are cases where longer distances are required. This is the case of high-speed Ethernet connections, where internal resources in the FPGA are not sufficient to achieve the required physical layer communication requirements. In these cases, the transceivers to correctly modulate and adapt the signals to transmit through longer distances must be held outside the FPGA. For Ethernet, these transceivers are normally called the PHY chips since they provide the functionality of the physical layer. They are placed in between the FPGA and the Ethernet connector. 

The way these types of communication protocols are used with FPGAs is always based on the same hardware structure: 

• The medium access control layer is placed inside the FPGA by using either a specific hardwired block for most SoPC devices or a module placed in the reconfigurable fabric for all other cases. 

• The physical layer is performed in the PHY chip, which requires some additional resources such as clock oscillators (dependent on the chip and Ethernet standard to be used) or passive components. 

• The Ethernet connector to provide the attachment to the transmis- sion media and galvanic isolation is placed close to the PHY device. 

The connection between the FPGA and the PHY is well standardized, but there are several options depending on the chip and the transmission stan- dard to follow—mostly dependent on the maximum transmission speed. Some of these standards are media-independent interface (MII); reduced MII (RMII), with smaller connectivity and reduced performance (up to 100 Mbps); GMII (for gigabit standards); or variations of them. Care must be taken to choose the right standard since not all media access blocks in the FPGA and all PHY chips support all of them. PHY devices also contain some low-speed control interface (such as SPI) for accessing configuration and sta- tus registers in the device from the FPGA. 

Support of software-based communication layers in the FPGA is also main- tained by manufacturers, providing versions of TCP/IP or UDP/IP stacks adapted for the internal soft or hard processors and OSs or bare-metal applications. 

It is advised to follow some of the numerous tutorials provided as examples that accompany the SoPC tools in order to fully understand and properly design all elements in the communication stack. There are many example tools that pro- vide simple web servers, or similar applications, in FPGA-based SoPC designs, so the time used to implement the final customized solutions is reduced. For a discussion and a practical and more detailed explanation on how fast serial I/O works on Xilinx devices, see the book by Athavale and Christensen (2005).