FABRIC first year progress report Huib van Langevelde, JIVE EXPReS is made possible through the support of the European Commission (DG-INFSO), Sixth Framework Programme, Contract.
Download ReportTranscript FABRIC first year progress report Huib van Langevelde, JIVE EXPReS is made possible through the support of the European Commission (DG-INFSO), Sixth Framework Programme, Contract.
FABRIC first year progress report Huib van Langevelde, JIVE EXPReS is made possible through the support of the European Commission (DG-INFSO), Sixth Framework Programme, Contract #026642 Future Arrays of Broadband Radio-telescopes on Internet Computing • Broadband WP 1 • Data acquisition to work with 10Gbps Internet • Requires new filtering and sampling hardware • Optimize protocol • Calibration issues and control software interface to telescope • Interfaces from public domain transport to “private” • LOFAR • e-MERLIN • Prototype and test • Onsala telescope on e-MERLIN 11/6/2015 EXPReS midterm review • Internet computing WP 2 • Use Grid for calculating correlation function • Based on virtual lab projects • Workflow management • Load balancing • Deployable software correlator • Based on previous Huygens machine • Suitable for Grid nodes and parallel computing • Data quality control, display and merging • 5 station demonstrator • With new science capabilities Slide #: 2 Data acquisition WP 1.1.1 MRO • Requirements formulated • • • • • Quite complex decision to be made Recording: Mk5 and PCEVN systems available Sampling: dBBC and DBE under development Compatibility with correlators And e-MERLIN in/out • Decision reached • iBOB: • Internet BreakOut Boards • University of Berkeley • FPGA and 2 x 10 Gbps • ADC converter port • R+D into radically different solution 11/6/2015 EXPReS midterm review Slide #: 3 e-MERLIN interface • Connect 4Gbps streams (Onsala) WP 1.2.2 UniMan • To feed into e-MERLIN correlator • Compatible to MERLIN channelization • Same architecture as MERLIN out (SA1) • And as data-acquisition • Design started • Hardware ordered • Personnel found • Joint with SA1 11/6/2015 EXPReS midterm review Slide #: 4 Protocol research • Various testbeds WP 1.2.1 UniMan • with range of controls in e.g. dropout • Protocols adressed • • • • TCP certainly not optimal UDP more promising VSI-E needs more tests DCCP not robust yet • Currently contributed effort by UniMan • Engineer available for FABRIC work later this year • Will bring more specific focus • Strategic discussion document delivered 11/6/2015 EXPReS midterm review Slide #: 5 Testing Onsala/7 month demo WP 1.2.3 OSO • Onsala will do 4Gbps demo • Joining the e-MERLIN array • Bandwidth upgrade under negotiation • Did PC-EVN test • Transfer at high rate • Using Tsunami • Direct transport • To JIVE • Or Metsähovi • Correlation at JIVE • Very useful exercise 11/6/2015 EXPReS midterm review Slide #: 6 LOFAR connectivity • Similar to e-MERLIN • E-LOFAR stations on public network? • Interface issues • And timing • Also for clock distribution • Discussions started 11/6/2015 EXPReS midterm review GbE switch BG/L Rack BG/L Rack Cluster of servers 10 TB RAID per node Infiniband interconnect GbE switch BG/L Rack GbE switch GbE switch BG/L Rack GbE switch Cluster of servers 4BG RAM/node Infiniband interconnect 10 GbE switch 10 GbE switch WP 1.2.4 ASTRON 10 GbE switch 10 GbE switch 10 GbE switch • strategic doc delivered Cluster of servers general purpose nodes Infiniband interconnect Slide #: 7 Cluster of servers general purpose nodes Infiniband interconnect Distributed correlation • Correlation currently in dedicated silicon • • • • Special purpose computer With 2 bit operations At 100 T-ops Future needs at 20 P-ops • Can be done on custom computers with 1op=1flop • Interesting science demo would take a few cluster computers • Software correlation can outperform 2 bit operation 11/6/2015 EXPReS midterm review Slide #: 8 eVLBI-Grid collaboration WP 2.1.1 PSNC • Connect telescopes and correlator nodes • Need to be deployed on Grid • Based on virtual laboratory • Required a lot of design • PSNC group new to VLBI • JIVE group new to Grid • 2.9 documents produced • Challenge: resource allocation • Issues with data from Mk5 system • Reached next stage • Workflow manager • Integrating correlator core 11/6/2015 midterm review Slide #: 9 WP 2.1.2 PSNC 11/6/2015 EXPReS midterm review Slide #: 10 Software correlator • Deployable software correlator WP 2.2.1 JIVE • Cluster computer enabled • Grid enabled • Starts with special purpose algorithm for Huygens mission • Extreme high spectral resolution • Keep for future applications • Can have better signal to noise performance than 2-bit • Interesting astronomy applications • Code transferred to standard environment • Runs on cluster computers • But not yet optimized 11/6/2015 EXPReS midterm review Slide #: 11 Correlator core 11/6/2015 EXPReS midterm review WP 2.2.2 JIVE Slide #: 12 Management • Distributed project with >30 participants • In 7 institutes and >4 disciplines • Communication • Meetings: kick-off meeting in Dwingeloo 22 Mar 2006 business meeting Poznan 27 Sep 2006 few teleconferences • Splinter meetings & visits: correlator, protocols, iBOB • Wiki for sharing documents • E-mail exploder with some external parties • Original work package structure quite good 11/6/2015 EXPReS midterm review Slide #: 13 Challenges • Design hurdle for data acquisition • iBOB rather drastic new development route • Large investment in existing technologies • Decisions took some time, purchases even more • Personnel acquisition dependent as well • Bridge the gap between eVLBI and Grid • Took several iterations and meetings • Teams with variety of expertise • And strong opinions 11/6/2015 EXPReS midterm review Slide #: 14 Personnel issues • Kick-off in March 2006 • • • • Direct start at Poznan and partially at JIVE Other post-doc level people JIVE took 6 & 9 months At JBO digital engineer is shared with SA1 Protocol person only available in 2007 • Lot of the work done by ‘matching’ effort • OK for the design phase • Project manager is now also coordinator • Definitely delayed the integrated design document 11/6/2015 EXPReS midterm review Slide #: 15 Milestones number name DJ1.1 Data acquisition requirements document 2 3 report MRO DJ1.2 Protocols strategic document 2 2 report JBO DJ1.3 Visualization software 4 software JIVE DJ1.4 Correlator design specification 5 report JIVE DJ1.5 Overall design document 6 report JIVE DJ1.6 eVLBI-Grid design document 6 9 report PNSC DJ1.7 eVLBI fringes PC-EVN 7 7 demo OSO DJ1.8 LOFAR connection strategic document 7 11 report ASTRON DJ1.9 Data acquisition design document 8 11 report MRO DJ1.10 eVLBI-Grid interface document 10 10 report PNSC 11/6/2015 original EXPReS midterm review actual type Slide #: 16 responsible