Transcript Addressing and Control Logic

PanSTARRS Gigapixel Array Controller
System
Outline
• Overview
• Present Status
– CPU board
– FPGA board
– DAQ board
• Staffing
• Infrastructure
• Schedule
PanSTARRS Internal Spec Review 8/5/2003
Illustration of Bar Mounting Scheme - 3
OTA flexcables attach to internal
motherboard with amplifiers. A single
controller module is shown for reference
(1 controller for 4 OTAs). Note: hermetic
feedthrough deleted for clarity.
PanSTARRS Internal Spec Review 8/5/2003
Gigapixel Camera Controller Electronics
• Controllers are mounted
behind cryostat.
• 16 controllers per Gigapixel
camera.
PanSTARRS Internal Spec Review 8/5/2003
Baseline Controller Design
IOTA3U
IOTA3U
IOTA3U
IOTA3U
IOTA3U
IOTA3U
IOTA3U
OTA
OTA
OTA OTA
OTA OTA
OTA
OTA
OTA
OTA
OTA OTA
OTA OTA
OTA
OTA
OTA
OTA
OTA OTA
OTA OTA
OTA
OTA
OTA
OTA
OTA OTA
OTA OTA
OTA
OTA
IOTA3U
2-row OTA Controller “Rack”
•
•
•
IOTA3U controller operates 4 OTAs
For a 8 x 8 OTA focal plane, 2 rows of 1 x 8 OTA’s
Each row has 8 IOTA3U controllers.
LAN Switch
1Gb ethernet
Pixel Server
1Gb ethernet
Fiber isolated
Pixel Server
Pixel Server
Pixel Server
OTA
OTA
OTA OTA
OTA OTA
Pixel Server
OTA
OTA
OTA
OTA
OTA OTA
OTA OTA
Pixel Server
OTA
OTA
OTA
OTA
OTA OTA
OTA OTA
Pixel Server
OTA
OTA
OTA
OTA
OTA OTA
OTA OTA
OTA
OTA
Pixel Server
IOTA3U
IOTA3U
IOTA3U
IOTA3U
IOTA3U
IOTA3U
IOTA3U
IOTA3U
Pixel Server
Pixel Server
Pixel Server
Pixel Server
Pixel Server
Power Supply
Pixel Server
Pixel Server
Pixel Server
PanSTARRS Internal Spec Review 8/5/2003
Data Storage Advances
•
Storage Area Networks and Network Attached Storage
–
–
•
Pixel Servers also have built in RAIDs
–

Cost effective high performance
Connectivity to “second tier” processors if needed
Lowest cost but performance not benchmarked

Fiber Broadcast

Back-end
LAN+SAN
SAN
SWITCH
PanSTARRS Internal Spec Review 8/5/2003
Second
Tier
Processing
1 x 4 IOTA3U Assembly
OTA
OTA
OTA
OTA
OTA
Flexprint PCB
Flexprint PCB
Cryostat wall
HDC
HDC
Interface bd
CPCI backplane
•
CPCI backplane
•
•
Signal chains and ADCs
PanSTARRS Internal Spec Review 8/5/2003
Signal chains and ADCs
Front Panel
FPGA
Chassis
COTS CPU
Module wall
Signal chains and ADCs
Signal chains and ADCs
FPGA
COTS 1G ethernet
COTS CPU board
Cooling
Interface Board
Cooling
Green areas are custom
designed
Blue areas are COTS
3U hardware could include
connectors, front panels, guide
rails, cooling components,
chassis parts, etc.
Mechanical Check 1 x 4
Green crosshatch shows 1 x 4 OTA shadow
4 sets of 2mm metric connectors shown
2 are CompactPCI bus standard
2 for OTA signal processing
Standard 3U front panel shown in maroon.
PanSTARRS Internal Spec Review 8/5/2003
IOTA3U Boards
•
COTS CPU board
–
–
Interface bd
CPCI backplane
•
DAQ3U Signal chains and ADCs
DAQ3U Signal chains and ADCs
FPGA3U
COTS 1G ethernet
COTS CPU board
•
•
–
Baseline LINUX based PowerPC
1 or 2 channel 1G Ethernet fiber PMC
daughterboard
CPCI bus interface
FPGA3U board
–
–
–
–
CPCI interface
FPGA +1Ge??
Buffer memory
Data interface to DAQ3U
DAQ3U board
–
–
–
–
Data interface
FPGA
Buffer memory
Signal Chain and ADCs
Backplanes
–
–
CPCI backplane
Interface backplane
Signal fanout
Filtering
Connectors
PanSTARRS Internal Spec Review 8/5/2003
COTS CPU – Status
•
Colleagues at CFHT have evaluated an
SBS RL4
– PowerPC 7400/750 CPU at 400 to
500MHz
– Fast 10/100Mbit Ethernet
– PMC extension slot
– Conduction cooled option
•
LINUX support not good
•
IfA has purchased and received a
Menmicro F1N
•
– PowerPC MPC8245/300MHz
– 1 Gbit Ethernet daughterboard
– LINUX ElinOS support (RTAI also)
STATUS – Under Test
PanSTARRS Internal Spec Review 8/5/2003
FPGA3U Board Design
•
Virtex2pro FPGA possible centroid/shift
calculator
JTAG
Support
CPLD
Power
section
FPGA
PROM
Local
Xtal
Xilinx Virtex 2Pro
Fiber coupled 100BT*
/100BT /10 BT
1Gb/100BT
PHY
FPGA
405 PPC
128Mx32
SDRAM
Compact PCI Bus
Trigger signals
Optocoupled
Trigger I/O
LDVS
Test Conn
To DAQ3U
PanSTARRS Internal Spec Review 8/5/2003
32Kx32 Dual
Port RAM Buffer
Xilinx ML300 Development Board
JTAG
System
ACE
Power
section
FPGA
PROM
Local
Xtal
Xilinx Virtex 2Pro
Fiber coupled
1G ethernet
1G
PHY
FPGA
128MB DDR
SDRAM
405 PPC
•PMC PCI Bus
1394 PHY
LDVS
PCMCIA
RS232
LCD Touch
screen
Display
To DAQ3U
Parallel
port
Keyboard
port
PanSTARRS Internal Spec Review 8/5/2003
DAQ3U Board Design
• Virtex2pro FPGA clocking and pixel data
aggregation (possible to incorporate into
FPGA board Virtex2pro
Power
section
Support
CPLD
FPGA
PROM
LDVS
JTAG
Analog
Devices
9826
Analog
Devices
TBD
RAM
9826
Analog
Devices
Fiber
coupled
100BT
100BT
PHY
Xilinx Virtex 2Pro
FPGA
405
PPC
Trigger signals
9826
Analog
Devices
9826
Analog
Devices
Optocoupled
Trigger I/O
Test
Conn
32 bit
pixel read
bus
9826
Analog
Devices
Analog Devices
5379
PanSTARRS Internal Spec Review 8/5/2003
Voltage
range
buffers
9826
Analog
Analog
switch
Analog
switch
Analog
switch
Analog
switch
Analog
Analog
switches
switchesAnalog
switches
switches
3 CCD
outputs
3 CCD
outputs
3 CCD
outputs
3 CCD
outputs
3 CCD
outputs
3 CCD
outputs
Mechanical Check DAQ3U
PanSTARRS Internal Spec Review 8/5/2003
ADC selection
Baseline AD9826 due to small size and speed but need to test
for INL and ENOB (&/or oversample)
16BIT ADC
Analogic ADC4322
Datel ADS-932MC
Datel ADS-935MC
Linear Tech LT1604
Linear Tech LT1620
Analog Devices 7671
Analog Devices 7677
Analog Devices 7676
Analog Devices 9260
Analog Devices 9826
AKM semi
SPT8100 (13ENOB)
Maxim MAX1200
999999-14
XYZ 7/7/2015
Freq
Ana Input Input R Tconvert
2Mhz
+/-2.5/5/10 750/1.5k
300ns
2Mhz
+/-2.75
500
pipeline
5Mhz
+/-2.75
400
3pipeline
333Khz
+/-2.55
?
?
500Khz
+/-2.5
?
380ns
1-0.8Msps +/-2.5/5/10 948/1.63K no pipe
1-0.8Msps -.1 to 3VDE ?
no pipe
500Ksps
+/-2.5
?
no pipe
15MSPS 3CHNL 0 t0 4V*
5Mhz
1Mhz
3pipeline
0 to +5V 5.5K ohms 200ns X6 cyc
0 t0 4V* 55K ohms 4us(pipeline)
S/N 100khz
86dBmin
81dBmin
83dBmin
88dBtyp
90dBmin
90dB typ
94dB 20Khz
94dB 20Khz
S/N 900khz THD 100Khz
75dBmin -86dBmax
80dBmin -88dBmax
75dBmin -86dBmax
-100dBmax
-88dBtyp
TBD
n/a
-110dB
n/a 94dB 20Hz
TOTAL 3LSB
78dBmin
83min
80dB typ
-84max
-82max
ref stab Integral nonlin Diff nonIin
15ppm/C
+/- .003%
+/- .75LSB
+/-30ppm/C
+/- 1LSB
+1LSB
+/-30ppm/C
+/- 1LSB
+1LSB
+/-15ppm/C
+/-15ppm/C
+/- 2LSB
external
+/- 1LSB
external
+/- 1LSB
+/- 1LSB
external
+/- 1LSB
ENOB
Size
14.29s/n 2.4"x1.5"
13.45s/n 2.1"x1.11"
13.78s/n 2.1"x1.11"
36 SSOP
36 SSOP
external 48 LQFP
external 48 LQFP
external 48 LQFP
int?
+/- 16LSB
+/- .5LSB
13s/n
?
external
+/- 1.25LSB
+/- 3.5LSB
+/- .5LSB
+/- .6LSB
?
external
28 SSOP
Power
2.3W
1.85W
2.85W
0.2W
0.42W
.15W
.13W
.15W
400mW
44 LQFP 0.465mW
44MQFP
AD9826 ADC Preliminary Testing 1
•
•
•
Fabricated small test PCB with same pinout as
Analogic AD4322 (2MHz 16 bit) .
2 PCBs tested in earlier version array controller.
Quick tests showed ~X3 gain vs AD4322s, but 1112 ADUs Standard Deviation (range 9-12) in 1
channel Sample and Hold mode.
– Data sheet says “3LSB” PGA=1
– PGA set =1, sampling freq ~5.6Mhz
– Running Raytheon 206 multiplexer (3 ADUs STD
nominal)
•
Tested with all 8 channels and image with
Raytheon 206 mux to confirm signal gain and
S/N.
PanSTARRS Internal Spec Review 8/5/2003
AD9826 ADC Preliminary Testing 2
•
•
Bench tested with precision voltage source, logic analyzer
3 channel CDS mode has 0-1 ADUs STD.
–
–
–
–
•
PGA set =1, sampling freq ~1.9Mhz, AC coupled.
Device only wants Vreset more positive than Vsignal.
Test not valid if grounded input = 0 volts into ADC.
First/few? pixels bad.
NOAO results:
– CDS and S&H modes have 7-8 ADUs noise @ 1.6usec
– Removed 2nd stage premap stage = removed “capacitive charge effect”.
None seen at IfA
– “Note, apparently the AFE devices were designed to be used in a free
running mode so that the capacitive effects relating to charging the AC
coupling cap to the bias clamp voltage are eliminated.” Will investigate,
IfA setup had different offset/clamp circuit.
•
•
Will retest next week to confirm results.
Samples of TI ADCs received.
PanSTARRS Internal Spec Review 8/5/2003
Clocking
Will build on previous (Redstar3) system VHDL design
• 16,000 gate Altera FPGA EP6016
• 128kx16 Sync SRAM
• Clocking FPGA
–
–
–
–
–
patterns
7 x 1/48Mhz = 145.83nsec min pattern time
1/48Mhz = 20.8333nsec pattern time extension
10 bit pattern run length extension = 21.313usec
rows and frames
10 bit rep count = 1024 times
– integration timer
– 25usec time increments
– 28 bit timer = 3.728 hrs max ITIME
PanSTARRS Internal Spec Review 8/5/2003
Pixel server = Thin Server 1U rackmount PCs
• Compaq Proliant DL360 G3 under test
• Supermicro dual Xeon under test
•Pixel servers in use on previous design with good results
(faster than dual 32 bit 40MHz DSP system).
PanSTARRS Internal Spec Review 8/5/2003
OTA Controller Power Supplies
•
•
•
•
•
•
•
•
HP 66000 System Power Supply, 8 modules/chassis
Each module has over voltage and over current protection
– Simultaneous shutdown
Configured for simultaneous turn on and off
Proven low noise and safety performance on Mauna Kea (IRTF, SUBARU)
– UPS run down and HELCO black/brownouts
Mechanical Specifications 19 inch rackmount
Width: 426 mm (16.75 in)
Height: 178 mm (7 in)
Depth: 678 mm (26.7 in)
PanSTARRS Internal Spec Review 8/5/2003
Staffing Estimates
•
Needed: 5 FTEs estimated, plus use of Contract Manufacturers
2 Electronic engineers (Onaka,TBD)
2 Software engineers (Lockhart, Isani *)
1 Electronic Tech (Ching)
•
Presently: 3+ FTEs on staff
– 1 new EEI causal hire => direct hire position hire started
– + 1 SE interviewing*
– 1 EEII (with data comm, ISCSI ASIC/IC experience) in discussion
•
Contract Manufacturers (CM) for board assembly
– Ching site visited 3 CMs.
– IfA H85 AO 10 board run being sent through one CM as first run.
PanSTARRS Internal Spec Review 8/5/2003
Production Plan
•
Two phase plan
– Prototype phase
Use Contract Manufacturer (CM) for high density parts (i.e. BGA).
But obtain in house ability to rework - requires some infrastructure investment
at IfA.
Contract out parts of electronics design that is “industry standard” (i.e. FPGA
CompactPCI VHDL).
–
Production Phase
Use Contract Manufacturer (CM) to get quantity discounts.
Contract out design of automated test electronics. JTAG for digital?
Build multiple copies of low cost test station.
Have a conservative (high) spares count.
PanSTARRS Internal Spec Review 8/5/2003
Lab Support/Infrastructure
•
In-house capability to do BGA rework.
– BGA inspection microscope received training done.
– BGA Xray microscope received training 8/8/03.
•
•
Previous Redstar2 and Redline controllers available for
comparative measurements.
Switching CAD software to leverage new hire experience.
–
Protel to Cadence (for SPECCTRA router)
Cadence university donation request sent (with UH Physics).
Software download done, waiting for license.
•
Xilinx FPGA requires additional tools
– Symplicity VHDL synthesizer received.
– Mentor Modelsim simulation models on order (not received).
PanSTARRS Internal Spec Review 8/5/2003
Development Task Breakdown
Prototype Phase
Basic Clock
FPGA code
Proto
Chassis
Clock Down
load
Backplane design
Component
selection
Proto
Interface Bd
Trigger
Centroid and shift
Single
OTA
test
FPGA3U
Buy Dev
Xilinx Board
Proto FPGA3U
Interface to
DAQ3U
ADC read
code
Analog
testing
Dual OTA
test
DAQ3U
Proto DAQ3U
Backplane
interface
Interface to
FPGA3U
Trigger
design
Analog
Testing
COTS CPU + 1G ethernet
Dev
environment
1G
ethernet
tests
Centroid and
shift
Multi OTA
code
Pixel Server Workstation
1G
ethernet
tests
PanSTARRS Internal Spec Review 8/5/2003
Centroid and
shift
Proto User
Interface
Data Storage
testing
Data Export testing
Schedule
Phase I End Goal: 4X1 OTA controller finished by Dec 2003
Hardware
–
–
–
–
Components selected Mid Aug.
PCB layout done = Oct.
Flex cabling finished Nov.
Controller prototype Dec-Jan.
Software
–
–
–
–
–
1 G ethernet throughput tested – Aug.
CompactPCI throughput tested – Aug.
Centroid/shift/load – Aug/Sept.
Pixelserver code – Nov.
OTA control development during Integration Oct-Dec.
PanSTARRS Internal Spec Review 8/5/2003
Draft Workpackage Description
Prototype Phase 1.0:Predesign 1.0.X
• Workpkg 1.0.1
–
–
–
–
Title: Location of OTA centroid/clocking generation.
Type: Study+test+decision
Description:
Combination study and test code generation for embedded
PowerPC on FPGA vs. individual Pixelserver vs. top level
Pixelserver.
– Metrics: Appropriate centroid algorithm, speed of calculation
vs collection, clocking pattern FPGA download and swtch
time.
– Estimate: 60hrs Software Engineer, 30hrs EEII.
– Dependency: FPGA clocking VHDL code at testable level on
ML300.
PanSTARRS Internal Spec Review 8/5/2003
Draft Workpackage Description 2
Prototype Design workpackages: 1.1.X
• Workpkg 1.1.1
– Title: Basic OTCCD clocking control
– Type: Design+code+test
– Description:
Software code of clocking, test of download. Hardware design on
ML300, schematic and VHDL for CAD input for PCB.
– Metrics: OTCCD Sequenced readout functionality, speed.
– Estimate: 60Hrs SE, 120hrs EEII, 80hrs SE*.
– Dependency: none.
PanSTARRS Internal Spec Review 8/5/2003
Workpackages
Prototype Phase 1.0:Predesign 1.0.X
Workpkg 1.0.1
Title: Location of OTA centroid/clocking generation.
Type: Study+test+decision
Description:
Combination study and test code generation for embedded PowerPC
on FPGA vs. individual Pixelserver vs. top level Pixelserver.
Metrics: Appropriate centroid algorithm, speed of calculation vs
collection, clocking pattern FPGA download and swtch time.
Estimate: 60hrs Software Engineer, 30hrs EEII.
Dependency: FPGA clocking VHDL code at testable level on
ML300.
Workpkg 1.0.2
Title: Number of FPGAs per controller.
Type: Study+decision
Description:
Study of #FPGA I/Os + interboard connector vs. added power
complexity of 2nd FPGA per controller.
Metrics: Power consumption, PCB area available, connectors
available, analog signal chain integrity.
Estimate: 40hrs EEII.
Dependency: none.
Workpkg 1.0.3
Title: 1G ethernet from FPGA or embedded COTS CPU?
Type: code+test+decision
Description:
Combination lab test of ML300 dev platform 1Ge throughput
vs. PCI+COTS CPU (MENmicro).
Metrics: Sustained throughput > = 64MBytes/sec
Estimate: 60hrs Software Engineer,40hrs EEII.
Dependency: none.
PanSTARRS Internal Spec Review 8/5/2003
Workpackages 2
Workpkg 1.0.4
Title: Real CCD test of AD9826 ADC on Leach system
Type: Design+fabrication+test
Description:
Design, fab and test AD9826 ADC on actual CCD to determine
noise and speed performance acceptability.
Metrics: Read noise spec = CCD and = data sheet at 1MSPS
Estimate: 20hrs EEII remaining (ET hrs used also).
Dependency: none.
Workpkg 1.0.5
Title: Release 1.0 of requirements for controller
Type: Design
Description:
Hardware and software requirements document.
Metrics: Completeness, science and control goals.
Estimate: 20Hrs SE, 20hrs EEII.
Dependency: LAN/WAN interface, Telescope scheduler reqs.
------------------------------------------------------------------Prototype Design workpackages: 1.1.X
Workpkg 1.1.1
Title: Basic OTCCD clocking control
Type: Design+code+test
Description:
Software code of clocking, test of download. Hardware design on
ML300, schematic and VHDL for CAD input for PCB.
Metrics: OTCCD Sequenced readout functionality, speed.
Estimate: 60Hrs SE, 120hrs EEII, 80hrs SE*.
Dependency: none.
PanSTARRS Internal Spec Review 8/5/2003
Workpackages 3 see draft plan
Workpkg 1.1.2
Title: FPGA PCB
Type: Design+code+CAD+fabricate+test
Description:
Production of FPGA printed circuit board.
Metrics: Power dissipation, size, speed of interfaces.
Estimate: 40Hrs EEI, 60hrs EEII, ET estimate TBD*.
Dependency: Predesign phase.
Workpkg 1.1.3
Title: DAQ3U PCB
Type: Design+code+CAD+fabricate+test
Description:
Production of DAQ3U printed circuit board.
Metrics: noise performance at speed, size, power dissipation.
Estimate: 40Hrs EEI, 60hrs EEII, ET estimate TBD*.
Dependency: Workpkg 1.0.4.
Workpkg 1.1.4
Title: Cryo flex/PCB
Type: Design+CAD+fabricate+test
Description:
Production of rigid-flex cable assembly. Cyrotest integrity.
Metrics: noise performance,size, vacuum performance, durability.
Estimate: 40Hrs EEI, 30hrs EEII, ET estimate TBD*.
Dependency: OTCCD pinouts final.
PanSTARRS Internal Spec Review 8/5/2003