Transcript Aperture Array Station Processing Andrew Faulkner SKADS Project Engineer September 2009 Station Processing – Cape Town AJF.

Aperture Array
Station Processing
Andrew Faulkner
SKADS Project Engineer
September 2009
Station Processing – Cape Town
AJF
SKA Overall Structure
Beam Data
0.3-1.2 GHz
Wide FoV
Dense AA
Tile &
Station
Processing
Central Processing Facility - CPF
To 250 AA Stations
...
...
Data
..
70-450 MHz
Wide FoV
..
Time
Control
Sparse AA
...
0.8-10 GHz
Single
Pixel or
Focal plane
array 12-15m
Dishes
Post Processor
DSP
Control Processors
& User interface
80 Gb/s
DSP
...
To 2400 Dishes
September 2009
Mass Storage
Correlator – AA & Dish
16 Tb/s
Station Processing – Cape Town
Time
Standard
User interface
via Internet
AJF
Mid-Frequency Aperture Array
~60m
Tile
Support
Bunker
• Freq range 0.3  1.2 GHz
• Dense array (Nyquist sampled)
• ~75,000 Receiver chains
September 2009
Station Processing – Cape Town
AJF
EMBRACE - SKADS
August 2009
Station Processor
AJF
The AA Station
Next Proc. Bunker
n x Optical
fibres per 2nd
stage processor
>1016 MACs
Station
Processor 1
1st Stage Processors
AA-hi
To Correlator
Mid P1
Mid P2
Station
Processor 2
.
…
Mid Py
0.3-1.0GHz Analog links
Internal
Digital links
.
.
Box
Low P1
O-E?
Phase
Standard &
Distribution
...
..
.
.
Low Pz
Station
Processor X
O-E?
Control
processors
500MHz Analog links
AA-lo
Prev. Proc. Bunker
September 2009
Phase transfer
over fibre
Station Processing – Cape Town
To Central
Control
10Gb Digital
fibre links
AJF
1-D Beamforming
Incoming signal
Elements
Electronic Delay
+ + + + + + + + + + + + + + + + + + + + + + + +
Beam
Delay
t
0
September 2009
Element #
Station Processing – Cape Town
AJF
1-D Beamforming
Incoming signal
Elements
Spectral Separation
+ Electronic Delay
+ + + + + + + + + + + + + + + + + + + + + + + +
Beam
C0I0+ C1I1+ C2I2+ C3I3+ C4I4+ C5I5 + C6I6+ C7I7+ C8I8+ C9I9+C10I10 +C11I11+C12I12+C13I13+C14I14+C15I15+C16I16+C17I17+C18I18+C19I19+C20I20+C21I21+C22I22+C23I23
C0I0+ C1I1+ C2I2+ C3I3+ C4I4+ C5I5+......
Using Phase delay
approximation in narrow
frequency bands
September 2009
Station Processing – Cape Town
AJF
Hierarchical structure
Incoming signal
Elements
Electronic Delay
+ + + + + + + + + + + + + + + + + + + + + + +
Tiles
Station
processor
September 2009
+
+
+
+
Station Processing – Cape Town
+
Beam
AJF
AA Station Implementation
Tile beam
data
TH_n
TH_1
Ae
Ae
Tile
Processor
- hi
…..
…….
TH_0
Station
Processor
To Correlator
Ae
Station
beam data
September 2009
Station Processing – Cape Town
AJF
Survey Speed – AA’s forte!
At 1 GHz with 250 deg2 FoV:
• AA station is big: say 60m dia.
 small beams
 ~0.28º width
• So, many, many station beams: >3000 !
• But, Tile beams are larger,
~5º width
• So, # of Tile beams required:
10
Easy....?
August 2009
Station Processor
AJF
Visually..
Station beams
Tile beam
Great!
Constrains data rate of
Tile  Station processor
September 2009
Station Processing – Cape Town
AJF
Ah, but...
Station beams
Tile beam
Only perfect for the central
beam on Tile beam
This is what we’ve done.....
September 2009
Station Processing – Cape Town
AJF
Tile Beamforming
Incoming signal
Elements
Electronic Delay
+ + + + + + + + + + + + + + + + + + + + + + +
Tiles
Station
processor
+
+
+
+
+
Beam
Delay
t
Tile Beam
Discontinuities will give high
sidelobes and variable forward gain:
Dynamic range badly affected
0
September 2009
Element #
Station Processing – Cape Town
AJF
Some Consequences
• Need ‘extra’ Tile beams over minimum calculated.
– Probably also interpolate between Tile beams for more precision
• Performance can still be arbitrarily good: dynamic range
etc. Maybe on limited FoV.
• Bandwidth from Tiles to Station processor determined on
quality of beams as well as FoV: still programmable.
• Concentrate beamforming centrally as much as possible
• Calculating “allowable error”, hence Tile  Station Proc.
Communications requirment is “work-in-progress”
Hierarchical Beamforming looks best option
September 2009
Station Processing – Cape Town
AJF
AA Station Processor Reqts.
1.
Beamform the output of all the Tile processors into Station beams
2.
Send “correlator ready” data over fibre to the central processor
3.
Part of the station calibration scheme
4.
Flexibly handle the data from the AA-hi and AA-lo arrays
5.
Maybe provide the ‘F’ part of the FX correlator
6.
Maybe provide station data for local processing
7.
Possibly hold ‘observation history’ data to post process
8.
Be readily scaleable for:
August 2009
a.
Number of tiles
c.
Internal station data rates
b.
Data bandwidth to correlator
d.
Data length to correlator
Station Processor
AJF
Outline spec. of Processing Chip
Inputs
Outputs
0
Each Stream:
12 x 10Gb/s
1
0
Processing
Device:
2
1
2
PChip
3
4
3
1013 MAC
or 10 TMAC
5
September 2009
4
5
Station Processing – Cape Town
AJF
PChip B’forming requirements
Assuming: All 6 outputs at full speed using all 6 inputs
2 reals per complex sample and 4 real MACs per complex MAC.
1.
6 inputs, I0 – I5 and 6 outputs O0 – O5.
2.
Beamforming for one output stream:
a.
On = C0I0+ C1I1+ C2I2+ C3I3+ C4I4+ C5I5
b.
The input data rate per stream is 120Gb/s raw:
96Gb/s actual
c.
Each complex sample is 2 x 4 bit reals so:
~12.5GS/s per stream.
d.
Processing per output stream is: 4*6*12.5 =
300 GMACs
e.
Total processing for 6 output streams =
1,800 GMACs.
3.
This is well within the PChip capability of 10TMACs
4.
Maybe some pre-processing required on each sample.
September 2009
Station Processing – Cape Town
AJF
Station Comms
• 120 Gb/s total
• 50m range
• Power:
– Tx: 2.4 watts
– Rx: 2.0 watts
• 12-channel fibre
– 10Gb/s channels
• VCSEL technology
• Pluggable
• 19mm pitch
August 2009
Station Processor
AJF
AA Station Data rates
TH_n
12 fibre lanes
@10Gb/s each
TH_1
Ae
Tile
Processor
e/o
- hi
e/o
e/o
…..
…….
Ae
TL_1
TL_m
September 2009
Station Processor n
Notes:
1. No control network shown
2. Up to 4 station processor systems can
be implemented in parallel
3. Data shown are raw, typ. get 80% data
Station Processing – Cape Town
Local Processing
e.g. Cal; pulsars
…..
1.0 GHz
analogue
Max 4 Station
Processors
Long distance drivers
TL_0
10Gb/s fibre
To Correlator
Tile
e/o
Processor
- lo
300
45
345
42Tb/s
....
e/o
Typical:
AA-hi tiles:
AA-lo tiles:
Total:
I/p data rate:
…..
e/o
…..
…….
e/o
Inputs #:
1296
Channel rate: 120Gb/s
(raw)
Total i/p rate: 1.5 Pb/s
Long distance drivers
1.0-1.2GHz
analogue
e/o
e/o
e/o
e/o
………...
…..
…..
o/e
o/e
o/e
o/e
o/e
o/e
o/e
o/e
e/o
Station
Processor
0
e/
o
e/
o
e/
o
e/
o
…..
o/e
o/e
o/e
o/e
o/e
o/e
TH_0
Long distance drivers
Ae
12 fibre lanes
@10Gb/s each
AJF
0
1
2
3
4
5
o/e
o/e
o/e
o/e
o/e
o/e
12-channel Rx
module. e.g Avago
AFBR-820BXXZ
o/e
o/e
o/e
o/e
o/e
o/e
To 1st stage
Processors
or Primary
Station
Processors
o/e
o/e
o/e
o/e
o/e
o/e
Each link is 12 fibre
lanes@10Gb/s
Total Raw input
data rate: 4.32Tb/s
o/e
o/e
o/e
o/e
o/e
o/e
o/e
o/e
o/e
o/e
o/e
o/e
30
31
32
33
34
35
o/e
o/e
o/e
o/e
o/e
o/e
Each link is 12 diff.
copper lanes@10Gb/s
PChip
PChip
PChip
PChip
PChip
“All to All”
Connections
Station Processor
12-channel Tx
module. e.g Avago
AFBR-810BXXZ
e/o
e/o
e/o
e/o
e/o
e/o
To Secondary
Station
Processors or
long distance
fibre drivers
e/o
e/o
e/o
e/o
e/o
e/o
PChip
e/o
e/o
e/o
e/o
e/o
e/o
PChip
e/o
e/o
e/o
e/o
e/o
e/o
Control
Processor
Each link is 12 fibre
lanes@10Gb/s
Total Raw output data
rate: 4.32Tb/s max
Line
Tx/Rx
August 2009
Station
Processor Board
PChip
e/o
e/o
e/o
e/o
e/o
e/o
PChip
PChip
PChip
PChip
e/o
e/o
e/o
e/o
e/o
e/o
Station
Control
AJF
Station Processor system (120Gb/s per Tile)
Station Processor
Max no. of Tiles
…..
AA-hi
Long distance drivers
To
Correlator
…..
Station Processing – Cape Town
Long distance drivers
Each link is 12 fibre
lanes@10Gb/s
…..
Secondary
Station
Processor
Board
(max 35)
…..
“All to All”
Connections
…..
AA-lo
0
1
2
…..
…..
September 2009
…..
Each link is 12 fibre
lanes@10Gb/s
Primary
Station
Processor
Board
(max 35)
…..
Secondary
Station
Processor
Board
1
…..
…..
…..
To each
1st stage
Processor
Primary
Station
Processor
Board
1
…..
35
…..
1296
Secondary
Station
Processor
Board
0
…..
is
…..
…..
AA-hi+AA-lo
Primary
Station
Processor
Board
0
Long distance drivers
0
1
2
Each correlator channel
is10Gb/s (maybe colour
multiplexed together)
AJF
Long distance drives
12-channel Rx
module. e.g Avago
AFBR-820BXXZ
o/e
o/e
o/e
o/e
o/e
o/e
PChip
Each link is
fibre10Gb/s raw
…..
To Secondary
Station
Processors
0
1
2
3
4
5
Block 0
e/o
e/o
e/o
e/o
e/o
e/o
e/o
e/o
e/o
e/o
e/o
e/o
To
Correlator
Long distance 10km
Tx module.
Total Raw input
data rate: 720Gb/s
Block 5
e/o
e/o
e/o
e/o
e/o
e/o
e/o
e/o
e/o
e/o
e/o
e/o
Each link is 12 fibre
lanes@10Gb/s
August 2009
Station Processor
Line
Tx/Rx
Control
Processor
Total Raw output data
rate: 720Gb/s max
Station
Control
AJF
Estimated Costs
(120Gb/s per Tile)
Total: €480k
August 2009
Station Processor
AJF
Conclusions
1.
2.
3.
4.
5.
Beamform to 1 chip depth at the Tile
Station Processor data rate is much higher than output
Processing performance is unlikely to be an issue
Comms costs and performance critical
Focus on making wide-area comms cheaper!
AA Station
Beamforming can be
done!
August 2009
Station Processor
AJF