EXPReS and the e-EVN Arpad Szomoru, JIVE 7th International e-VLBI Workshop, Shanghai, June 2008, A.
Download ReportTranscript EXPReS and the e-EVN Arpad Szomoru, JIVE 7th International e-VLBI Workshop, Shanghai, June 2008, A.
EXPReS and the e-EVN Arpad Szomoru, JIVE 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE Outline • A short overview • Results • Next 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE 2002: first contact with DANTE and NRENs: EVN as application of GÉANT backbone NL DE SE CH HU IT FR GR CZ BE AT UK PT 0 GEANT ES SI PL IE HR LU RO LV EVN telescope 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE BG CY LT IL SK EE From proof-of-concept to EXPReS • SURFnet provides 2 * 1 Gbps Amsterdam - JIVE • Demo at INGRID 2002, ER2002 • PoC runs until 2005, when EXPReS is funded • EXPReS kicks off March 2006 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE 2004 2003 2006 EVN Symposium 2004, A. Szomoru, JIVE 2008 2007 New networks... 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE the EXPReS network upgrade 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE and new network monitoring tools 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE Hardware upgrades • Control computers, data acquisition platform, routers, Mark5 motherboards, CPUs and power supplies, local network, correlator interface boards, optical serial links, Mark5B upgrades, dedicated network monitoring platform, 10 G interfaces.... 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE New software tools 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE Control code improvements • Correlator: • General speedup of correlator control code • Re-connecting of individual stations without job re-start • Significant increase of robustness and reliability • Has made e-VLBI a one-person operation at correlator • Led to record length unattended correlation jobs (both e and non-e) 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE QuickTime™ and a Cinepak decompressor are needed to see this picture. Control code improvements (2) • Mark5: • Re-write of code with rigorous thread control (with help of UMan colleagues) • Much improved stability of units • New features, such as packet dropping/padding, channel dropping, simultaneous recording and transmitting of data on Mark5A • Enabled using available bandwidth to the limit, both on dedicated short- and shared long-haul links • Simplicity pays off: trivial to add EVN stations to e-EVN 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE Data rate improvements Aggregate data rate 7000 Number of telescopes @ data rate Data rate (Mbps) 6000 1280 3000 2000 0 5 Ap r/0 Se 4 p/ 0 M 4 ay /0 Se 5 p/ 05 O ct /0 N 5 ov /0 M 5 ay /0 Au 6 g/ 0 D 6 ec /0 Fe 6 b/ 0 M 7 ar /0 7 Ju n/ 07 O ct /0 D 7 ec /0 7 1024 6 896 Data rate (Mbps) 4000 1000 1152 768 640 512 6 1 2 384 256 4 128 0 Sep/02 -128 5000 7 Jan/04 May/05 Oct/06 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE Feb/08 Jul/09 Sharply reduced time to publication Test case: black holes in outburst: •Data processing took 1-2 weeks with first images within 48 hours 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE •Publication took less than 2 months Rapid response science 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE Demonstrations • The APAN demo: • Opened up Siberia connection • Excellent PR • Longest (e) baseline ever 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE And more demos.. • EXPRES-OZ • Three Australian telescopes (Mopra, ATCA, Parkes) • 12 hours at 512 Mbps without a hitch • Beautiful detection of SN1987a 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE And even more... • TERENA 2008: • 4-continent e-VLBI • 4-continent fringes! • Mixed-configuration observation 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE Demonstrations and visibility • Fantastic way to focus effort • But can be really disruptive and time-consuming... QuickTime™ and a Cinepak decompressor are needed to see this picture. 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE Adaptive observing • Fast/adaptive (re-)scheduling • Transform EVN into a truly flexible instrument • Based on automated download and execution of modified observing schedules possibly on-the-fly • Reactions from stations vary from “excellent idea” to “over our dead bodies” • But will further improve reliability and responsiveness of EVN • All bits tested and in place, full operational test is planned 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE Optimal use of bandwidth, flexibility • Data rate adjustments: • Packet dropping: • Drop packets at sending side (while preserving headers!) • Pad datastream at receiving side with dummy data • Channel dropping: • Selectively drop frequency channels at receiving side • High demand on cpu, need hyperthreading, right kernels, dual cpu’s • Mixed configurations: • Use different configurations at different stations (limited to 32, 64, 128 etc Mbps) • Mixed-mode operations: • Enable all possible combinations of recording (at stations and/or at correlator), real-time and delayed data transfer • Make best use of complete EVN network, including stations that have less-than optimal connectivity 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE Beyond 1 Gbps • Beyond 1 Gbps • 1024 Mbps will simply not fit on 1 Gbps... • Westerbork: use of DWDM equipment, dual 1-Gbps connection, channel bonding • Could be used to Jodrell Bank as well (dual 1-Gbps lightpaths to JIVE) • Onsala, Effelsberg: share 10-Gbps connections with eLOFAR, via VLAN (nearly in place) • Next, Torun? • Most sensitive telescopes in Europe soon at a full 1024 Mbps • And tests ongoing at 4 Gbps, from Onsala 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE And soon to come... • Yebes first fringes, via ftp (!), first e-light soon • Hartebeesthoek: demo last month at 64 Mbps, S-African universities expect to be connected to Europe at 10 Gbps in 2009 • Arecibo: recent runs at ~512 Mbps • St Croix? Nançay? Urumqi? Ventspils • Japan: considering joint JVN-EVN observations at 6.7 GHz, possible e-demo involving Yamaguchi • Bandwidth-on-demand/software correlator demo at next SC • Full 1024 Mbps from Wb, Ef, On, Jb • 4 Gbps next.... 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE Further in the future: more spacecraft tracking • LAPLACE and TANDEM • accepted by ESA for study for 2015-2025 • Earlier projects may include Bepi-Colombo LAPLACE: a mission to Jupiter and Europa: • VLBI on Europa landers/orbiter • Radio astronomy experiments Jovian orbiter 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE TANDEM – Titan and Enceladus mission: • VLBI on Titan probes/balloons • Radio astronomy exps Enceladus orbiter Long term perspective e-VLBI • e-VLBI effort vital for keeping VLBI on forefront • • • • Reach new science topics Growth path to much more sensitivity Easier accessible user facility Seamless integration with other networks, new instruments • Science case EVN2015 has recently been completed • Calls for next generation correlator • And could lead to data streams of ~100 Gbps per telescope • So much more to do... • Take advantage of 10 Gbps infrastructure • Requires work on digital equipment • And explore bandwidth on demand • Invoke some buffer mechanism • To operate real-time e-VLBI network continuously 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE Unexpected side-effects of EXPReS.. Before.... 7th International e-VLBI Workshop, Shanghai, June 2008, A. Szomoru, JIVE After!