Mixed SpaceWire - SpaceFibre Networks

2-4 November 2008 Martin Suess (1 , Steve Parkes (2 (1

European Space Agency,

(2

University of Dundee E-mail: martin.suess at esa.int, sparkes at computing.dundee.ac.uk

2 nd International SpaceWire Conference in Nara, Japan Slide : 1

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Overview

• Introduction • SpaceWire – SpaceFibre comparison • SpaceFibre Virtual Channels • Priorities and Group Adaptive Routing • Mixed Network Examples • SpaceFibre Outlook • Conclusion

2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 2

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Introduction

•

SpaceWire

supports bi-directional traffic of up to 200Mbit/sec over a distance up to 10m.

•

SpaceFibre

shall be capable to improve to both figures by at least a factor of 10: – Data rates ≥ 2.5Gbit/sec – Distance ≥ 100m – Provide additional features like galvanic isolation • This requires a number of modifications at different levels of the protocol stack.

• The aim is to maintain compatibility between SpaceWire and SpaceFibre on Packet and Network level.

• In the following the solutions implemented in the SpaceFibre breadboard are presented 2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 3

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Physical & Signal Level - Optical

•

SpaceWire

uses cables with 4 twisted pairs with nine-pin micro miniature D-type connectors.

•

SpaceFibre

uses two optical fibres as medium: – The Draka MaxCap 300 radhard graded-index multimode fibre has been selected after testing – A cable protecting the fibre was designed based on expanded polytetrafluoethylene – Diamond AVIM connectors where already qualified for space • Electro optical transceivers produce 850-nm laser light with a power of 3dBm peak 2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 4

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Diamond AVIM connectors and electro optical transceiver breadboard

2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 5

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Physical & Signal Level - Electrical

• In addition an electrical version for short distances forseen.

• Prototype used 4 coaxial cables with SMA connectors.

• The electrical interface of the transceivers use Current Mode Logic (CML).

• CML is directly used as signal level in the electrical version.

• Tolerance to common mode voltage differences can be improved by blocking capacitors.

• More investigations are needed before physical & signal level definition of electrical SpaceFibre.

2-4 November 2008

AC coupling of a CML transmitter and receiver

2nd International SpaceWire Conference in Nara, Japan Slide : 6

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Character Level - 8B10B Encoding

• •

SpaceWire

defines data and 4 control characters – FCT, EOP, EEP, ESC – The combination of ESC with FCT and Data Characters defines the Null control code and the Time-Codes

SpaceFibre

characters are based on 8B10B encoding – DC balanced data signal plus 12 special characters for control functions – Three of these special characters are comma characters – Comma characters contain a unique sequence of ones and zeroes that are used for character alignment 2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 7

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Character Level - Ordered Sets

• Ordered Sets are a sequence of 4 characters starting with a comma character (K28.5) • The second character defines the type of ordered set • The last two characters can carry additional information • Ordered Sets greatly extend the possibility to embed control information in the data steam • Several types of ordered sets are defined for SpaceFibre: – Link-level, power management, reset, flow control, faming and user ordered sets – User ordered sets are used to propagate time codes and interrupts though the network 2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 8

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Exchange Level - Flow Control & Framing

•

SpaceWire

uses flow control to prevent receive buffer over flow.

– Each FCT indicates 8 Bytes of free buffer space.

•

SpaceFibre

maintains the concept of flow control.

– Granularity of flow control is increased due to higher bandwidth.

– Each FCT ordered set controls the flow of one frame of maximum 255 data words.

• A frame starts with a Start of Frame ordered set and ends with an End of Frame ordered set.

• The End of Frame ordered set contains the 16 bit CRC of the frame for error detection.

• SpaceWire packets are segmented into frames and reassembled at link level.

2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 9

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Virtual Channels in SpaceFibre

• Flow control and start of frame ordered sets contain the virtual channel number.

• With separate frame buffers the virtual channel data flow is logically separated while using the same medium.

• Congestion in one virtual channel does not influence the traffic in the other virtual channels.

• A SpaceWire packet in one virtual channel can pass a packet in another virtual channel.

• Priorities can be used to control the access of a virtual channel to the physical medium so that the higher priority channel has always direct access.

2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 10

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Virtual Channels in SpaceFibre

VC1 Buffer VC2 Buffer VC3 Buffer VC4 Buffer

Virtual Link 1 Virtual Link 2

SpaceFibre

Virtual Link 3 Virtual Link 4

VC1 Buffer VC2 Buffer VC3 Buffer VC4 Buffer 2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 11

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Number of Virtual Channels in a SpaceFibre Link

• Maximum number of virtual channels is 256.

• In practice less will be used to limit the number of buffers needed.

• The individual virtual channels must be accessible without blockage or bottle neck.

• In a SpaceWire router the total number of ports for path addressing is limited to 31.

• The SpaceWire standard allows to use two consecutive address bytes for path addressing in large routers.

– The first path address byte indicates the SpaceFibre link.

– The second path address byte indicates the virtual channel number.

• Beyond this some of the virtual channels could be accessed by logical addressing only.

2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 12

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

SpaceWire - SpaceFibre Router

SpaceFibre Port 1 Internal Configuration Port 0 SpW SpW SpW SpW SpW SpW SpW SpaceWire Port 4 SpaceWire Port 5 SpaceWire Port 6 SpaceWire Port 7 SpaceWire Port 8 SpaceWire Port 9 SpaceWire Port 10 4 5 6 7 8 9 10

Crossbar Switch

2,1 2,2 2,3 2,4 2,5 2,6 VC 1 VC 2 VC 3 VC 4 VC 5 VC 6 SpaceFibre • • •

Router example with: 3 SpaceFibre links with 6 virtual channels each 7 SpaceWire links 2 External ports

SpaceFibre •

Non blocking crossbar switch provides direct access to every virtual channel

2-4 November 2008 External Port 11 External Port 12 3 ort P re ib eF ac Sp 2nd International SpaceWire Conference in Nara, Japan SpaceFibre Slide : 13

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Virtual Channel Priorities & Group Adaptive Routing

• Each virtual channel can provide the full bandwidth of the SpaceFibre link.

• If two or more virtual channels request a bandwidth higher than the full bandwidth of the link some arbitration is required.

• Priorities can be assigned to virtual channels: – The virtual channel with higher priority is allowed to send the next frame.

– Round robin arbitration is performed between virtual channels of the same priority.

• User ordered sets for time-code and interrupt distribution have priority and are sent in the middle of the frame currently transmitted.

• SpaceWire-RT protocol should be used to provide Quality of Service beyond priorities.

2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 14

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Group Adaptive Routing

• If packets are routed through the same virtual channel the access is arbitrated by the router.

• Routers can provide group adaptive routing among virtual channels with the same priority: – Two packets to the same logical address can use parallel virtual channels.

– The receiving side can then decide which should be processed first.

– Available overall bandwidth is not increased.

• Routers can provide group adaptive routing among several SpaceFibre links.

– This can be used to increase the available bandwidth.

2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 15

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Network Example – Single SpaceFibre Link

High Data Rate Instrument Electronics redundant Spa Sp ceF ibr ibre e SpaceFibre Mass Memory redundancy Modulator & Downlink Unit redundant 2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 16

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

SpaceFibre as Backbone

p S W p S W p S W p S W p S W SpaceWire, SpaceFibre Routing Switch SpaceFibre SpaceWire, SpaceFibre Routing Switch SpaceFibre SpaceWire, SpaceFibre Routing Switch p S W 2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 17

Mixed Networks

SpW SpW MIXED SPACEWIRE - SPACEFIBRE NETWORKS SpW High Data Rate Instrument Electronics redundand SpaceFibre SpaceWire, SpaceFibre Routing Switch SpaceFibre SpaceWire, SpaceFibre Routing Switch SpaceFibre Mass Memory redundancy 2-4 November 2008 Modulator & Downlink Unit 2nd International SpaceWire Conference in Nara, Japan Slide : 18

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

SpaceFibre Breadboarding

2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 19

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

SpaceFibre Breadboarding

2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 20

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

SpaceFibre Outlook

• A first SpaceFibre prototype covering all levels has been implemented and tested.

• A first outline specification has been published and discussed in the frame of the SpaceFibre working group.

• The experience gained will be consolidated and used for the development of a SpaceFibre demonstrator.

• The SpaceFibre Demonstrator activity will target: – Development of a SpaceFibre IP core, – Test and validation of IP core using existing prototype, – SpaceFibre Demonstrator implementation based on Actel FPGA and Wizard Link SerDes, – Preparation of a SpaceFibre specification as input for the standardisation process.

2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 21

MIXED SPACEWIRE - SPACEFIBRE NETWORKS

Conclusion

• The different levels of SpaceFibre have been compared with SpaceWire.

• SpaceWire and SpaceFibre is intended to be fully compatible on Packet and Network level.

• This allows the easy implementation of mixed SpaceWire – SpaceFibre networks.

• Some examples of those networks have been presented.

• This compatibility is seen as essential feature of SpaceFibre.

• Experience has been gained with a prototype implementation.

• As next step a demonstrator will be developed based on space qualified components.

• Standardisation in ECSS is envisioned in corporation with the other space agencies.

2-4 November 2008 2nd International SpaceWire Conference in Nara, Japan Slide : 22