Transcript Recent e-EVN Developments Arpad Szomoru, JIVE Shanghai Observatory, November 2006, A. Szomoru, JIVE.

Recent e-EVN Developments
Arpad Szomoru, JIVE
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Outline
•
•
•
•
The Proof-of-Concept project
EXPReS: current status
Satellite tracking: Huygens and SMART-1
FABRIC: a distributed software correlator
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Price recording media ($/GB)
Shanghai Observatory, November 2006, A. Szomoru, JIVE
January 2002: Proof-of-Concept
e-VLBI over GÉANT
NL
DE
SE
CH
HU
IT
FR
GR
CZ
BE
AT
UK
PT
0
GEANT
GÉANT
ES
SI
PL
IE
HR
LU
RO
EVN
telescope
LV
BG
Shanghai Observatory, November 2006, A. Szomoru, JIVE
CY
LT
IL
SK
EE
e-VLBI Milestones
20
num_badorder
25000
No LBE
num_lost
18
16
20000
14
12
15000
10
8
10000
6
4
5000
2
0
0
Shanghai Observatory, November 2006, A. Szomoru, JIVE
20
40
60
80
100
120
Transfer number
140
160
180
0
200
No. Lost
No. Out of order
• September 2002:
• 2 X 1 Gbit Ethernet links to
JIVE
• Demonstrations at
igrid2002 and ER2002
• UDP data rates over 600
Mbit/s
e-VLBI Milestones: 2003
Onsala
Sweden
Chalmers
University of
Technology,
Gothenburg
• May 2003: First use of
FTP for VLBI session
fringe checks.
• November 2003:
Cambridge –
Westerbork fringes
detected, only 15
minutes after
September: e-VLBI
July:10 Gbit•access
• October 2003:observations
first light on were
data transfer
GEANT-Surfnet
Westerbork – JIVE
made.1 Gb/s
between
Bologna
• 6 X 1 Gbit links
to JIVE
• November 2003: Onsala
and JIVE – connection
300Mb/s • 64Mb/s, with
disk Observatory
Space
buffering at JIVE
only. connected at
(Sweden)
1Gb/s.
Shanghai Observatory, November 2006, A. Szomoru, JIVE
e-VLBI Milestones: 2004
800
Mbps
600
400
200
On-Dw
Dw-On
Tr-Dw
Dw-Tr
Bo-Dw
Dw-Bo
• 2004:
December 20 2004:
•
September
in
ax
of JBO
m
m
• January 2004: Disk
First e-EVN connection
science
to Manchester
at 2 x
September
2004:
buffered
e-VLBI
• March
2004: first real- • June 2004:• network
session (Ar, Cm,
Tr,
stress
• April 2004: ThreeFour telescope
real1 Gb/s
WestfordOn,
Wb)
test
(iperf)
involving
• On, Wb,time
Cmfringes
at 128Mb/s
telescope,• real-time
Torun time e-VLBI (Ar,
Haystack June 2004:
Bologna,
Torun, Onsala• Spectral line
• e-VLBI test with Tr,
for first GGAO
e-VLBIto
image
fringes at 64Mb/s
connected at 1Gb/s.
Cm,
Tr,
Wb)
and JIVE
and Jb
Intercontinental
observations On
at 32
Wb).
• On –• Wb
fringes(On,
at Jb, real• First fringes to
Ar at • Jb - Tr fringes at
Wf -On,
Mb/s
256Mb/stime •fringes,
First
real-time
EVN
Shanghai
32 Mb/s
32Observatory,
Mb/s November 2006, A. Szomoru, JIVE
256Mb/s
image at 32Mb/s
0
e-VLBI Milestones: 2005
• March 2005: e-VLBI science
session
• Spring 2006: Metsahovi
• January 2005: Huygens • First continuum science connected at 10Gb/s
February
2005: network
observations at 128 and 64
descent tracking,
salvage
transfer test (BWCTL)
Mb/s, involving
6 radio2005: trench
• Summer
of Doppler experiment
employing various
network (Wb,
telescopes
Jb,mile”
Cm,
forAr,
“last
• Use of dedicated
monitoring tools On,
involving
Tr)
connection to Medicina
lightpath Australia-JIVE,
Jb, Cm, On, Tr, Bologna
dug
data transferred
at
~450
and JIVE
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Mb/s
Shanghai Observatory, November 2006, A. Szomoru, JIVE
POC results
• Demonstration of feasibility
• Identification of problems
• Has led to closer ties with networking community and
generated political interest
• Has laid the foundation for the next step forward
(EXPReS)
Shanghai Observatory, November 2006, A. Szomoru, JIVE
e-VLBI & EXPReS
• I3 proposal to the EC (Communication & Network
Development Call)
• Ranked first out of 43 proposals; nearly fully funded to an
amount of 3.9 MEuro.
EXPReS = EXpress
Production
Real-time
e-VLBI
Service
Shanghai Observatory, November 2006, A. Szomoru, JIVE
EXPReS aims
• Upgrade EVN to e-EVN
•
•
•
•
Help solve last mile problem at telescopes
16 * 1 Gbps into Dwingeloo to JIVE correlator
Software in field and correlator to become ‘real’ real-time
Inclusion of e-MERLIN telescopes in e-EVN (and vice-versa)
• And look beyond 1 Gb/s
• More capacity on digital sampling, more bandwidth
• As being implemented for e-MERLIN in UK
• Hardware (PC-based) and protocols for transport
• Correlator with more capacity: distributed correlation
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Why bother?
• Target of Opportunity - unscheduled observations triggered
by sudden astronomical events. This capability will become
much more important when LOFAR comes online
• Adaptive Observing - Use e-VLBI as a finder experiment
• Or, e-VLBI sessions a few days apart, adapt schedules for
later observations based on results (rapid results on large
sample, focus in detail on best candidates)
• Automatic Observing - small number of telescopes
observing for extended periods doing spectral line
observations of large galactic samples
• Interface with other real-time arrays – e-MERLIN,
LOFAR, SKA.. Also function as SKA-pathfinder
• Bandwidth no longer limited by magnetic media: 10Gbps
technology already becoming mainstream
• Because we can…
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Expanding the e-VLBI Network
Shanghai Observatory, November 2006, A. Szomoru, JIVE
And adding new antennas
40m antenna at Yebes, Spain
Radio and mm frequencies
Sardinia Radio Telescope
64m; radio to millimeter
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Recent developments
• Regular science/test sessions throughout the year
• First open calls for e-VLBI science proposals
• First science run completely lost, but, first ever real-time
fringes to Mc (128 Mbps)
• Second and third science runs: 24 hours at 128 Mbps.
• Fourth run: 16 hours at 256 Mbps. However, nearly 25% of
time lost to technical problems..
Shanghai Observatory, November 2006, A. Szomoru, JIVE
First e-EVN Astronomy Publications
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Current status
• Highest data rates:
• 6-station fringes at 256 Mbps
• 3-station 512 Mbps fringes (Cm, Wb, On, August 21)
• Current connectivity:
• Ar: 64 Mbps in the past, but <32 Mbps this year
• European telescopes: 128 Mbps always, 256 Mbps often, 512 Mbps
to Wb, Jb and On
• Australia:
• Telescopes connected
• PCEVN-Mk5 interface needed
• China:
• Shanghai Observatory connected at 2.5 (?) Gbps
• Connection via TEIN (622Mbps), ORIENT?, lightpath Hong Kong
– Netherlight?
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Hybrid networks in the Netherlands..
Shanghai Observatory, November 2006, A. Szomoru, JIVE
..and across Europe: GÉANT2 network upgrade
Outline
Shanghai Observatory, November 2006, A. Szomoru, JIVE
1 Gbps
10 Gbps
155 Mbps
EVN Symposium 2004, A. Szomoru, JIVE
2.5 Gbps
Network testing
400
100
k
em
0
is
-m
em
di
sk
• UDP
m
Maximal reliability
Not really required
Sensitive to congestion
Tuning necessary
Dw-Bo
Bo-Dw
Dw-Bo*
Bo-Dw*
2d
•
•
•
•
200
et
• TCP
300
2n
• Use existing protocols on currently
available hardware
• Connectionless
• Unaccountable
• Internet weather
• Hard to quantify
• Hard to pinpoint bottlenecks
Shanghai Observatory, November 2006, A. Szomoru, JIVE
800
600
Mbps
• Tailor made protocols?
• Lightpaths
Dw-On
400
Dw-Tr
200
Dw-Bo
0
m
ax
m
in
e-VLBI transfer tests
• February 2005: network
transfer test (BWCTL)
employing various network
monitoring tools involving
Jb, Cm, On, Tr, Bologna
and JIVE
Shanghai Observatory, November 2006, A. Szomoru, JIVE
e-VLBI in practice: control interface
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Interface to station Mk5s
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Integrating fringe display
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Data status monitor
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Streamlining of post processing
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Web-based Post-processing
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Spacecraft
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Huygens descent tracking
• Detection during descent
• Salvage of Doppler
experiment
• Building up experience with
spacecraft tracking
• Special purpose, narrow band
software correlator
Shanghai Observatory, November 2006, A. Szomoru, JIVE
f 
1
2
W
I
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Mopra
•
Huygens VLBI data – Parkes & Mopra
(CSIRO)  Sydney.
•
Dedicated light path:
– Sydney  Seattle  JIVE (user
controlled light path)
–
2 x 13 minutes scans transferred at
data rates of 450 Mbps
–
Calibrator fringes <12 hrs after
observations made.
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Parkes
Cessna
e-VLBI to South America? SMART-1
SMART-1 factsheet
Testing solar-electric propulsion and other deep-space technologies
Name SMART stands for Small Missions for Advanced Research in
Technology.
Description SMART-1 is the first of ESA’s Small Missions for Advanced
Research in Technology. It travelled to the Moon using solar-electric
propulsion and carrying a battery of miniaturised instruments.
As well as testing new technology, SMART-1 is making the first
comprehensive inventory of key chemical elements in the lunar surface. It
is also investigating the theory that the Moon was formed following the
violent collision of a smaller planet with Earth, four and a half thousand
million years ago.
Launched 27 September 2003
Status Arrived in lunar orbit, 15 November 2004. Conducting lunar orbit
science operations.
Notes SMART-1 is the first European spacecraft to travel to and orbit
around the Moon.
This is only the second time that ion propulsion has been used as a
mission's primary propulsion system (the first was NASA's Deep Space 1
probe launched in October 1998).
SMART-1 is looking for water (in the form of ice) on the Moon.
To save precious xenon fuel, SMART-1 uses 'celestial mechanics', that is,
techniques such as making use of 'lunar resonances' and fly-bys.
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Spacecraft Spectrum
Coherent Harmonics
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Carrier
SMART-1: Occultation by Moon
SMART-1 carrier wave voltage, power and
phase as detected by Medicina station
Post-egress “classical” diffraction pattern
and zoom on pre-egress features, like those
seen around seconds 5 and 8-10
For comparison: power (red) and phase (blue)
patterns for diffraction on a flat circular screen
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Signals (SMART-1 Impact on
Moon)
http://sci.esa.int
Hobart 26m
SMART-1 last
light @ Hobart:
05:42:22.394060(5)
03 Sep 2006 UTC
Shanghai Observatory, November 2006, A. Szomoru, JIVE
TIGO
Concepcion
e-VLBI to South America
Shanghai Observatory, November 2006, A. Szomoru, JIVE
FABRIC
Future Array of Broadband Radio-telescopes on Internet Computing
• Will need a new correlator for > 1 Gb/s
• Current EVN Mk4 based on 16x16x1Gb/s
• Implemented on 1024 special chips
• Next generation will possibly use standard CPUs
•
•
•
•
Current EVN Mk4 processor equivalent 40 T-ops (2bit)
LOFAR BlueGene ≈ 27 Tflops
32 station x 4 Gb/s ⇒ 640 T-ops
And requirement to route 32x32 4 Gb/s input stream
• Possible solution: distribute the computing
• Use the Internet as the cross-switch
• SETI@home gets 59 Tflops
• Proposed to do pilot on the Grid
Shanghai Observatory, November 2006, A. Szomoru, JIVE
•
Input data:
•
•
•
•
Input data:
•
No. of telescopes (N) 16
Array Observing frequencies: 329 MHz –
22 GHz
Data bandwidth: 128 MHz per
polarization (Right and Left-hand circular)
divided into 8 bands. Channel bandwidths
0.5 MHz to 16 MHz).
•
•
•
Data input rate: 1 Gbps per telescope
•
•
Data encoding: 1 and 2-bit
representation of data samples
•
Data input rate: at least 32 Gbps per
telescope.
Flexible data encoding: 2, 4 and 8-bit
representation of data samples
•
•
Output data:
•
•
•
Integration time ¼ seconds (will become
1/32s with PCInt)
2048 spectral channels per baseline
Data Output rate: 6 MB/s (will become 80
MB/s with PCInt)
Shanghai Observatory, November 2006, A. Szomoru, JIVE
No. of telescopes (N) ~ at least 20 - 60 (easily
expandable architecture)
Array Observing frequencies: 300 MHz – 115
GHz
Data bandwidth: At least 4 GHz per
polarization (Right and Left-hand circular)
divided into possibly 8 (or more) bands. Ability
to deal with range of channel bandwidths (1
MHz to 1 GHz).
•
(representation chosen depends on observing
frequency e.g. 2-bit sampling at high
frequency, 8 bit-sampling at very low
frequencies/low bandwidth [LOFAR case]).
Output data:
•
Integration time ~ 10 milliseconds
•
•
100000 spectral channels per baseline
Data Output rate: ~ 1000 (=N*(N-1)/2)
baselines x 100000 x 100 times per second x
10 bytes i.e. 0.1 TBytes per second.
FABRIC components
observing schedule
in VEX format
DBBC
VSI
PC-EVN
#2
field system
controls antenna
and acquisition
VSIe??
Shanghai Observatory, November 2006, A. Szomoru, JIVE
GRID
resources data
user correlator
parameters
earth orientation
parameters
resource allocation
and routing
correlator control
including model
calculation
FABRIC
=
The GRID
output
data
Central astronomical scheduling
Resource allocation from central control
Transported data may be
time/frequency slices
Telescopes will do
multicast transmission
Output data stored in central archive,
proprietary rights for users
Compute nodes will be cluster size
Shanghai Observatory, November 2006, A. Szomoru, JIVE
e-EVN: the future
• Aim: 16 * 1 Gbps production e-EVN network
• Lightpaths across GÉANT: point-to-point connections
between JIVE and telescopes.
• Guaranteed bandwidth, no need to worry about
congestion..
• Depending on connectivity of stations, choice of
configurations with specific data rates
• Towards a true connected-element interferometer; we’re
well on our way
Shanghai Observatory, November 2006, A. Szomoru, JIVE
Shanghai Observatory, November 2006, A. Szomoru, JIVE