1

GBT Interface Card for a Linux Computer

Carson Teale

2

Introduction• All detectors at the LHC need to be sent:

– Timing information about beam crossings through 40.08 MHz LHC reference clock

– The first-level trigger-accept decisions– Commands properly deskewed in time

• Data is sent and received through counting room electronics– Any data from the detector electronics is then distributed to the

network• Current system of electronics at detectors needs to be

replaced when luminosity of LHC increases• Gigabit Transceiver(GBT)

– New radiation tolerant ASIC for bidirectional 4.8 Gb/s optical links

3

My Project

• To study the design of a custom board to receive GBT signals and transfer them directly to a PC– Used commercial FPGA development board– Goals:• Establish communication link between GBT and PC

running Linux• Test speed and correctness of connection

4

FPGA• Counting room electronics don’t need to be radiation hard• Implement transceiver using commercial FPGA

– FPGA used for more efficient parallelized processing of incoming data• High Tech Global board with Altera Stratix IV FPGA and two

SFP+ transceivers used to establish link to GBT.

Stratix IV FPGA

SFP+ transceivers

5

GBT Module

HTG Dev board

Stratix IV FPGA

SFP+ SFP+

PC running Linux

Intel Core 2 Duo

RAM2GB DDR2 800MHz

DMA Controller

PCIe 2 8x32 Gb/s

GBT 4.8Gb/s

Diagram of Communication Link

6

Establishing the Communication Link

• Configure PCI express hard IP core on FPGA• Write Linux device driver for DMA to allow

reading/writing to memory• Implement GBT transceiver code on FPGA• Connect GBT module to FPGA and send data• Analyze data recorded on memory of

computer

7

Configuring PCIe hard IP core• Quartus II software used to develop and compile code for FPGA• Megawizard allows for configuration of number of lanes, clock speed, number and size of BARs(base address registers)• BARs tell device where its address mapping is located in the memory mapped I/O space of the system• Megawizard generates PCI modules along with test DMA module

8

Pin Assignments

Quartus II Pin Planner

HTG board schematics

9

Linux Device Driver

• Purpose: translate instructions between an application and a hardware device

• Provides interface between user space and kernel space– Done through functions to read and write to and

from a file since devices are represented as files• Written as a module to be dynamically loaded

into the kernel at runtime

10

GBT Serialization-Deserialization Procedure

Transmitting

Receiving

11

What I’ve done since last time

• I’ve been modifying a linux driver – Changing configurations in driver to match those

of the PCI IP core in the FPGA– Updating code for newer kernel

• Testing GBT interfacing code through internal loopback– Make sure process of receiving and transmitting

data using GBT protocol is working on FPGA end– Debugging using LEDs and logic analyzer

12

What Still Needs to be Done

• Fix problems with the linux DMA driver and modify it to allow board to read/write to memory through PCIe bus

• Determine error in GBT code running through internal loopback on FPGA

• Hook up physical GBT module to FPGA using SFP+ transceiver

• Run tests to verify speed and accuracy of GBT module

13

What I Learned

• More about how to program an FPGA• A lot about Quartus II• Some VHDL• The basics of Linux device drivers

14