image
Services
image
image
image

image


Notes the Xilinx Hard IP PCI Express Core

The PCI Express hard IP block in Xilinx Virtex-5 and later families provides a Transaction Layer Packet (TLP) interface for the user (FPGA fabric) side. The TLP interface has a 64-bit data path and runs at a frequency dependent upon the number of PCIe lanes: 62.5, 125, or 250 MHz. There are separate receiver and transmitter TLP interfaces and these have strobes that are used to validate packet data on the interfaces. These strobes are similar to those of the parallel PCI bus in that they allow for wait states to be generated on either end of the interface, the core side or the user side.

  • Beware of Glitches!


  • The strobes mentioned above consist of Start Of Frame (SOF), End Of Frame (EOF), and ready strobes for both user and core sides. A transaction is not valid unless the ready strobes from both sides are asserted. The besides validating the 64-bit data bus, the SOF and EOF strobes are also qualified by the ready strobes, and these can actually glitch in between operations on the bus.

  • TLP Buffering


  • On the Virtex-5 implementation, there is buffering for a maximum of 8 transmit TLPs for each type: posted, non-posted, and completion. Note that this amount cannot be exceeded, but can be reduced depending upon the TLP size. There is a maximum of three 512 byte or seven 256 byte TLPs for either posted or completion, and a max of eight non-posted TLPs.

  • Receiver Completion Streaming


  • With a one lane link, completion streaming is not necessary. Therefore, the signal TRN_RCPL_STREAMING_N should be permanently deasserted. However, if throttling of the receiver interface is performed (by deasserting TRN_RDST_RDY_N), then the user interface should ensure that no more than 8 non-posted requests are pending resulting in more than 8 completions. The user interface is responsible for tracking completions by using the TAG field to match outstanding requests with completions.

  • PCIe Error Reporting


  • In addition, the user interface is responsible for completion timeouts. This should be indicated by asserting the completion timeout error flag (CFG_ERR_CPL_TIMEOUT_N). The following other errors must be detected and indicated:

    • End-to-end CRC ECRC Error

    • Unsupported Request Error

    • Completion Timeout Error

    • Unexpected Completion Error

    • Completer Abort Error

    • Correctable Error

    For the case of multiple errors in one packet, only one should be reported.


  • Completion Packets


  • As in all headers, the user is responsible for building the completion header to transmit completions for read requests. These must contain the latched TAG and requester ID from the request, as well as the ID assigned to the endpoint from configuration space. This is made available to the user from the configuration interface.

    It should be noted that the core provides completions to all configuration requests.


  • Legacy Interrupts


  • The Xilinx PCIe IP core supports both MSI and legacy interrupts. This note describes the implementation of the latter - legacy interrupts. Interrupts on the PCIe interface are very different than on the parallel PCI bus. Essentially a message must be sent to the root complex when the interrupt is to be asserted, and then another message must be sent when the interrupt is to be negated. This is accomplished using three strobes: two generated from the user and one acknowledge which comes back from the root complex. The signal CFG_INTERRUPT_N is asserted to generate an interrupt request. Note that this can be a request to assert the interrupt or a request to negate the interrupt. The signal CFG_INTERRUPT_ASSERT_N is driven low during the request to assert the interrupt and driven high to negate it. Once the request is generated with these two strobes it must remain active until the signal CFG_INTERRUPT_RDY_N is received low from the root complex. The following timing diagram illustrates this (from the Endpoint Block Plus User's Guide):

    PCI Expess Legacy Interrupt Timing Diagram


    The following code fragment shows one method of providing this function (must be in a clocked process):

    PCI Expess Legacy Interrupt Code Fragment


  • PCIe Byte Order


  • In the parallel PCI bus if one designed to 32-bit quantities, the byte order did not matter. This is not so with the PCIe interface; depending upon your target "endianess", a byte swap may have to be performed on the data payload as it arrives from the receiver or is sent to the transmitter.

  • PCIe Simulation Test


  • Xilinx provides a PCI Express simulation model. This model actually appears to be based upon an instance of the hard IP. Out of the box, this model has a number of inadequacies. First off, since it models a serial interface, it is very slow, in simulation speed. Additionally one must run for a long time in the beginning of a simulation to complete link training in the core. Second, there isn't any way to test the TLP wait states on both sides of the TLP interface. This is a major drawback since this is essentially the complex part of the TLP interface to the core. Third, there are other features which can't easily be tested since the core prevents these features (such as the Expansion ROM Base Address Register response, FIFO flow control, and others). And lastly, it isn't a true pseudo-code model in that read data is not returned to the model (our version has actually been modified to provide this). This means that from the stimulus code when a read is performed, you can't test the code and act on the value.

    At Verien, we generated our own TLP pseudo-code behavioral model to test our PCI Express designs. This has the advantage that it generates cycles from the TLP interface - not the serial interface, and is therefore about an order of magnitude faster (and without link training). It also allows wait states to be tested for both the receiver and transmitter, either for a fixed amount or randomly applied. Since read data is returned to the model, the test vectors can be written much like a diagnostic.




    Please contact us if you have any questions on this, or to provide feedback. Thank you!






    image