Transcript Data rate improvements

UniBoard Testing
UniBoard Meeting, October12-13th
2010
Jonathan Hargreaves, JIVE
Eric Kooistra, ASTRON
Contract no. 227290
UniBoard Meeting,October 12-13th
2010
The Tests
DDR3 memory
10 GbE
FPGA
with Nios II uP
and
JTAG
ADC
Transceivers
JTAG UART
clock, pps, wdi
ID, test IO
I2C, SPI, MDIO
10/100/
1GbE
Configuration flash
Terminal
Test each interface separately to verify the board
Then test everything at once
UniBoard Meeting,October 12-13th 2010
DDR Memory Testing
•
•
•
•
Write and read back pseudo random patterns in sequential and data mask modes
Test exercises each address line
Test can be run in single shot or continuous modes
Tested 1GB and 4GB modules at 800MT/s and 1066MT/s (+20%)
Test can fail immediately because the module was not initialised correctly
• Module was not seated correctly or wrong module inserted
• Power supply failure (1.5V)
Test starts but one or more read/write accesses gives an error
• Can be due to a single data, address or control line
• Further testing by toggling the outputs and using JTAG revealed 4 bad connections
on the board
UniBoard Meeting,October 12-13th
2010
Testing the FN 10Gbps ports
EPCS was switched on
FPGA
Altera
GX230
Tested 1m and 10m optical cable and SFP+
modules from JDSU and Finisar
XAUI
MDIO
XAUI
MDIO
XAUI
Port 3 not tested due to RX-TX swap
MDIO
ARP and UDP packets transmitted while
monitoring error counts in the PHY chip
FPGA
Also test 2m and 5m passive copper
(Direct Attach) and 2m active copper
cables
Altera
GX230
VSC848611
PHY
PHY
VSC848611
PHY
MDIO
VSC848611
XAUI
MDIO
XAUI
XAUI
MDIO
PHY
VSC848611
PHY
VSC848611
PHY
VSC848611
XAUI
PHY
MDIO
VSC848611
UniBoard Meeting,October 12-13th
2010
SFP+
Cage
TYCO
2007637
-1
VSC848611
XAUI
MDIO
XAUI and MDIO connections between
FPGA and PHY chip were OK
PHY
SFP+
Cage
TYCO
2007637
-1
Cable Under
Test
Testing the FN 10Gbps ports
All 3 ports worked well with 10m
optical
Optical measurements, 1m & 10m multimode cables, rx equalization setting x1,
Extended PCS on
Port
Port 2 struggled with 1m optical
0
Cable
Link
E-PCS
corrected/uncorrected
errors per second
(typical)
1m
FN0 -> FN1
0/0
15
0
0
0
0
FN1 -> FN0
0/0
15
0
0
0
0
FN0 -> FN1
0/0
16.2
0
0
0
0
FN1 -> FN0
0/0
16.2
0
0
0
0
FN2 -> FN3
1/0
15.7
0
0
0
0
FN3 -> FN2
1/0
15.7
0
0
0
0
FN2 -> FN3
0/0
15.8
0
0
0
0
FN3 -> FN2
0/0
15.8
0
0
0
0
FN0 -> FN1
0/0
15
0
0
0
0
FN1 -> FN0
0/0
15
0
0
0
0
FN0 -> FN1
1/0
15.8
0
0
0
0
FN1 -> FN0
0/0
15.8
0
0
0
0
FN2 -> FN3
1/0
15.7
0
0
0
0
FN3 -> FN2
0/0
15.7
0
0
0
0
FN2 -> FN3
0/0
15.6
0
0
0
0
FN3 -> FN2
0/0
15.6
0
0
0
0
FN0 -> FN1
0/0
16.2
0
0
0
0
FN1 -> FN0
0x748/0
16.2
255
1553
FN0 -> FN1
0/0
63.5
0
0
0
0
FN1 -> FN0
0 to 3/0
63.5
0
0
0
0
FN2 -> FN3
0x234/0
15.7
255
1898
1
3.3581E-12
FN3 -> FN2
0x144/0
15.7
255
283
FN2 -> FN3
0x3 to 0xb/0
15.5
0
0
0
0
FN3 -> FN2
0/0
15.5
0
0
0
0
10m
1m
Active copper cable gave high errors
2m passive copper was OK on ports
0 and 1 but not 2
10m
1
1m
10m
1m
10m
SFP+ is essentially analogue end to
end
2
1m
10m
May need to optimise TX gain, preemphasis, de-emphasis and RX
equalisation each link
1m
10m
UniBoard Meeting,October 12-13th
2010
Length
of run
(hrs)
Block
errors
Character Sequenc BER based
errors
e errors on char err
13 2.66289E-12
0 5.00708E-13
Testing the BN 10Gbps ports
The BN XAUI connections were tested using the XGB board and CX4 cables
between 1m and 5m in length
The status of the transmit and receive PLLs was checked to make sure they stayed
locked during the test
Ports 0 1 and 2 transmitted ARP and UDP packets to their neighbours
UniBoard Meeting,October 12-13th 2010
The Tests - continued
DDR3 memory
10 GbE
FPGA
with Nios II uP
and
JTAG
ADC
Transceivers
JTAG UART
clock, pps, wdi
ID, test IO
I2C, SPI, MDIO
10/100/
1GbE
Configuration flash
Terminal
UniBoard Meeting,October 12-13th
2010
On board transceivers FN-BN mesh
• Tested the 12 full featured transceivers
• The 4 CMU transceivers have not been tested, because the 12 suffice
• 16 hours test for 8 FPGA with 12 TR each at 6.250 Gbps went OK
TR mesh
FN0
BN0
FN1
BN1
FN2
BN2
FN3
BN3
4 * 3 full featured TR
On board transceivers FN-BN mesh – 8 terminals
BN-BI transceiver port test
•
•
•
•
Uses the 12 full featured transceivers
The 4 CMU transceivers have not been tested, because the 12 suffice
15 hours test at 5 Gbps for all BN with 6 CX4 cables upto 5 m went OK
A test using XAUI on all TR including the CMU TR also went OK.
3 * 4 full featured TR
BN0
NC
BN1
NC
XGB
BN2
NC
BN3
NC
BN-ADC port test
• Test using 4 LOFAR receiver units
per BN
• Each receiver unit delivers 8 bit
counter data at 200 Msps via
LVDS
• All 4 BN received the counter data
from the receiver units OK.
• The I2C connections between BN
and receiver units function OK.
• Higher sample rate tests will be
done when the APERTIF ADC
unit is available.
1 Gb Ethernet Test
• Test whereby the 8 nodes transmit > 1000 frames to each other via
the on board 1 GbE switch went OK.
• Communication between 8 nodes and the 4 RJ45 connectors still
under test.
Auxiliary tests
• WDI (watchdog interrupt) works OK.
• INTA, INTB pull up lines between FPGAs work OK.
• I2C to on board sensors
- FPGA temperature sensors for all 8 nodes: OK
- 1GbE switch temperature sensor via BN3: ?
- Hot swap controller voltage sensor via BN3: ?
Good to know (see UniBoard SVN for more)
• Development scripts:
- unb_sopc <design_name>  generate SOPC system files
- unb_app <design_name> <app=main_name>  compile SW
- unb_qcomp <design_name>  synthesize the design
- unb_sof <design_name>  load image on FPGA(s)
- unb_rld <design_name>  compile SW and download to FPGA
• Unb_common:
- FPGA pinning TCL files
- unb_node_ctrl.vhd  FPGA clock, reset, watchdog
- unb_system_info.vhd  FPGA ID, version, g_sim
- SW function PIO debug wave  track SW in Wave window
• Generic g_sim to distinguish between target and simulation without
editing the VHDL file
• FPGA node reservation web page
UniBoard reservation web page
To do: integrated test design
• Test all interfaces in one design for efficient functional verification of
the next 8 UniBoards
• Full load test using dummy logic, RAM and DSP to heat the FPGA
and verify the power supplies