Transcript Your Presentation Title

CICADA Project
John Ford
August 5, 2008
The NRAO is operated for the
National Science Foundation (NSF)
by Associated Universities, Inc.
(AUI), under a cooperative
agreement.
GBT PTCS Conceptual Design Review
April 8/9, 2003 Green Bank
Outline
• CICADA
– Program organization
– Assets
– GUPPI
• Future directions
2
CICADA
• Configurable Instrument Collection for
Agile Data Acquisition
– FPGA based data acquisition and
processing
– Uses CASPER tools and hardware
• Umbrella program for organizing FPGA
projects
– Purchase/obtain boards, software, development systems
– 2 BEE2, 5 iBOB, 6 ADC, 10 GbE switches, servers, etc.
3
CICADA Projects
• Spectrometers
– KFPA backend
– Replacement for GBT Spectrometer
• Event capture
• Pulsar machines
– Incoherent machines
• Spigot replacement
– Coherent dedispersion machines
4
Basic KFPA Spectrometer
2 GHz bandwidth, 32K channels
• CASPER Hardware
– 14 ROACH/ROACH-II boards, ADC's
– Needs to be in receiver room, fiber to lab
• EVLA Station boards plus CASPER software
– Uses standard EVLA hardware
– Directly compatible with EVLA digitizers
– External supplier
5
GUPPI History
• GBT Future Instrumentation Workshop, September
2006
• University of Cincinnati Group worked on it until May,
2007, Produced report and basic design
• 2 WVU summer students and Glen Langston built
“event capture” device over the summer
• Scott Ransom yells at us to “stop planning and get to
work” in August, 2007
• October 29th, 2007, Held workshop to brainstorm the
project and get started on detailed design and
implementation
• April 4th, first pulsar observation (under test conditions
with 43m telescope)
6
Another pulsar backend? We already
have 5!
BCPM: 4-bit, 48-96MHz, 96 chan, public
Spectral Proc: 6-bit, <40MHz, 1024 chan, public
Spigot: 3-level, 50/800MHz, 1024/2048 chan, public
GASP: 8-bit, <100MHz, coherent dedisp, private
CGSR2: 2-bit, <100MHz, coherent dedisp, private
The only machines to give Full Stokes are GASP, CGSR2,
and the Spectral Processor
GUPPI: 8-bit, 800MHz, 4096 chan, Full Stokes, and
coherent dedisp
GUPPI's Advantages
Searching: 800MHz of BW, 4096 chan, and RFI-resistance
(from polyphase filterbank) will make GUPPI a “SuperSpigot” that will be unbeatable for searches from 1-5 GHz
(previously the BCPM and Spigot)
High Dynamic-Range Studies: 8-bit sampling, high spectral
resolution and Full Stokes will make GUPPI perfect for
scintillation studies, HI absorption, Faraday rotation
measurements, polarization studies, and singlepulse
studies (previously the Spectral Processor)
Ultra-High Precision Timing: 8-bit sampling, 800MHz of
BW, and RFI resistance will allow unprecedented timing
precision from 1-3 GHz for millisecond pulsars (i.e. the
NANOGrav project and the search for nHz gravitational
waves; previously GASP/CGSR2)
GUPPI Team
• NRAO- CV
– Paul Demorest, Ron Duplain, Rich Lacasse, Scott Ransom
• NRAO-GB
– Patrick Brandt, Glen Langston, Randy McCullough, Jason Ray
• Others
– Casperites
– Glenn Jones
Original UC design
10
GUPPI BEE2 signal processing
chain
• Each 1.6 GS/s stream uses 1 FPGA for signal
processing, 1 IBOB for data acquisition
• 4096 point PFB/FFT, with data streaming in and out,
diagnostics in Block Ram
• Combines signals from each polarization, calculates
stokes parameters, accumulates, and packages data,
transmits over 10 Gb Ethernet to host.
• Minimum 50 microsecond accumulations
• ~100 - 200 MB/sec data rate
11
GUPPI iBOB Design
• Uses 1 ADC module and 2 10 Gbit XAUI links to
digitize and transmit data streams to BEE2
• Has room for diagnostics, digital downconverter
12
Sampler
GUPPI BEE2 signal processing
design
• Each 1.6 GS/s stream uses 1 FPGA for signal
processing
• 4096 point PFB/FFT, with data streaming in and out,
diagnostics in Block Ram
14
Signal Processing
Xaui Alignment Block
GUPPI BEE2 signal combining
FPGA
• Combines signals from each polarization, calculates
stokes parameters, accumulates, and packages data,
transmits over 10 Gb Ethernet to host.
• Minimum 50 microsecond accumulations
• ~100 MB/sec data rate
17
Signal combining/output
Software Connections
Client
Server
• “external” interface
• Controller access
Controller
Data Acquisition
• core logic
• data storage
• parameter exposure
• data quick look
• Data Acquisition
interface
• data status
Demux
• common parameter
access
• fully qualify
parameter names
The dashed box contains
those modules which will
run on GUPPI host,
“beef.”
IBOB Interface
BEE2 Interface
• parameter access
• parameter access
• (TinySH client)
• (client-server)
(“Runs” on IBOB.)
(Runs on BEE2.)
User client module functions
•
•
•
•
Simple command line interface
Allows users to set and get all parameter values
Allows users to start and stop FPGA processes
Python based, extensible with standard Python
functionality
• Users can write/run their own scripts to control
observations
• Client modules can be seamlessly integrated into GBT
M&C system, or any system that can open a
connection to a Python SimpleXMLRPCServer
20
Hardware Parameters
• hardware design
excerpt (right)
• software register:
• BRAM:
GUPPI Command Prompt
• simple command prompt wrapped around
Python interpreter
• tab completion for functions and parameter names
• four basic functions:
– get and set for parameters
– load and unload for FPGA profiles
Just add Python
• write simple scripts to build more functions
e.g. use Python execfile
• import Python modules for extensibility
• e.g. matplotlib (pylab) for plotting
Data Acquisition Software
•
•
•
•
•
•
Multi-threaded shared memory architecture
C program
Connects to 10 Gb Ethernet using UDP
Buffers data, provides quick-look functions
Streams data to disk in PSRFITS format
Handles 300 MB/S data stream with Myricom 10 Gb
Ethernet card, Tyan Motherboard, Opteron Processors
and AMCC hardware RAID
• Interfaces to Python based controller through shared
memory command buffer
24
Portability and Extensibility
• Controller written in Python
• Data Acquisition software written in C
• Host is 64 bit Linux system, BEE2 runs 32 bit Linux
system
• All connections to hardware are portable to
newer/different interfaces
• All code written by NRAO staff GPL licensed
25
First Light
26
GUPPI Future Directions
• Add more diagnostics
• Add other configurations, narrow bandwidths, more
channels, wider/narrower outputs, etc.
• Add coherent dedispersion modes
– Long FFT's needed to implement inverse ISM filter
– Possibly brute force better? Using convolution instead of FFT>Multiply->IFFT
– Maybe better to stream out to computers for calculations.
• Make it robust enough to release for everyday use
– Integrate with GBT observing system
27
Conclusions
• Reconfigurable Computing platforms make for quick
hardware development
– GUPPI started in earnest in October, 2007. First light was in
April 2008.
• Standard software interfaces make for quick control
interface development
– The BEE2 and IBOB platforms use a common shell interface
to the FPGA parameter space, allowing for easy portability
between all hardware subsystems
– Python for the development language allowed the power of
the Python interpreter to be used to provide complex functions
easily
28