Transcript Mini AerCam Project Presented By Jeff Cotner Independent Consultant to NASA JSC through LMSO/Seat Aer Cam Project 11/6/2015

Mini AerCam Project
Presented By Jeff Cotner
Independent Consultant to
NASA JSC through LMSO/Seat
Aer Cam Project
1
11/6/2015
My background
• 15 years of VHDL
•
•
•
•
( The tools have certainly improved)
7 years of FPGA
(The tools have certainly improved)
17 years of consulting
(The rates have certainly not improved)
VHDL instructor
Communications, Avionics, Power Systems
Aer Cam Project
2
11/6/2015
Introduction
• MiniAerCam – What is it?
• Very simply an inspection vehicle

•
•
Contains Video Imagers, Avionics,
Propulsion, GPS, Gyros, Illumination,
C&T System, Power System
Intended for use on Shuttle and Station
JSC Robotics in charge of development
Aer Cam Project
3
11/6/2015
Mini-AERCam APB Architecture
PowerPC 750FX
500MHz
32bit / 100MHz
256 MB
Flash
TBD MB
TBD MBps/TBD MBps (R/W)
115.2 kbps?
Gyros
UART
57.6 kbps
Console
I2C
200 kbps
Instrumentation
UART
100 kbps
IrDA
Ethernet
Chip
Main FPGA (Virtex II 3000)
MPEG2 for
Video,
uncompressed
for LDRI
GPS
UART
Loading/Heart Beat
Illuminator
Aer Cam Project
Memory
Controller
Thruster (x12)
Iso (x2)
& EM I/F
Thrusters
Iso Valves
Electro Mag
Reset
Controller
PPS
Video 2
Color
I2C or SPI
LDRI
?
Video 1
Mono
? Mbps
32bit/100MHz
19.2 kbps
? Mbps
SDRAM
UART
SDRAM
/EDAC
Controller
? Mbps
Comm
? Mbps
4
11/6/2015
Radiation Mitigation
Requirements
-SEU will not cause destructive latch-up
-SEU or effect will be detected
-SEU can cause undesired consequences
(This is not so much a requirement as a fact)
-Single point failure can cause reboot
-Reboot times allow for fast recovery from
SEU induced failures or double bit hits
NOTE: AerCam is not redundant, loose a part, can’t
fly. Too many SEUs reboot.

Aer Cam Project
5
11/6/2015
Radiation Mitigation

Rad Tolerant “Hard Core”
– Actel based Reset Controller


–
–
–
–

Incorporates watchdog timer in VHDL
Latch-up immune, extremely low SEU rate, TMR in
hardware
Battery
DC-DC power converter
Oscillator
Remaining elements on APB can be power cycled to
clear errors
Rad Hardened main FPGA
– Partial reconfiguration to detect and correct
configuration errors due to SEUs
Aer Cam Project
6
11/6/2015
Radiation Mitigation
Definitions Main FPGA
-Interfaces Program FLASH, SDRAM, I/O to
Processor
-Provides sanity signals to Hard Core
-EDAC SDRAM
-CRC Program Flash
Aer Cam Project
7
11/6/2015
Radiation Mitigation

Definitions Reset Controller (Hard Core)
– -Configures Main FPGA and also Partial
Reconfiguration (No scrubbing just
rewrite configuration memory)
-Controls Power to Vehicle Systems
-Determines Sanity of Processor and Main
FPGA (Cycle Power or Reset if Sanity
lost)
Aer Cam Project
8
11/6/2015
Radiation Mitigation



Multiple copies of configuration data in Flash (In Same Device)
Multiple copies of Application code in Flash (In Same Device)
PowerPC 750FX processor
– SOI process, no latch-ups
– L1 cache parity protected, L2 cache ECC protected

Thrusters
–
–
–
–


Separate arm and fire registers
Hardware controlled duration
Individual thruster watchdogs
Iso valve can shutdown all thrusters
Main memory ECC protected
Individual subsystems can be power cycled to clear errors
Aer Cam Project
9
11/6/2015
APB
Color
Imager
LDOs
Thruster,
Iso Valve,
EM
Power
Supply
Aer Cam Project
LDRI
LDO
LDO
Comm
LEDs
DC-DC
DC-DC
GPS
I2C Bus
?
LDO
Mono
Imager
LDO
IrDA
LDO
Thruster
Drivers
Raw Battery
+6V (TBR) Converted Power
+3V (TBR) Converted Power
On/Off Control
?
Gyros
LDO
Mini-AERCam Power Distribution Architecture
Battery
(12V-16V)
24V DC-DC
10
11/6/2015
Why we think this is acceptable
• AerCam travels at very slow velocities




•
A stuck thruster will just spin the vehicle
-Multiple stuck thrusters are a problem
Thrusters have multiple guards against sticking
Can reboot fairly quickly to regain control
If communications are lost, Shuttle can
pull away from AerCam
These justifications are from Robotics at JSC
If you happen to be an astronaut and have an issue, please take this
up with Robotics. I’m just a lowly Logic Designer.
Aer Cam Project
11
11/6/2015
SDRAM Controller

Requirements
-Standard SDRAM controller
-EDAC
-Multi device access (Virtex PPC 405)
-Autonomous Rewrite cycle for EDAC
-As robust as possible given no TMR &
possible reactions to SEU
-Design reuse a possible by product
-125 MHz clock speed
Aer Cam Project
12
11/6/2015
Icing on the Cake

Extra things we did on this controller
-Programmable scrub range
Why scrub memory you aren’t using
-Scrub cycle complete signal
Nice to know how long a scrub pass takes
-Track SDRAM hits by sector
Conceivably could tell when a portion
of a SDRAM has failed.
Aer Cam Project
13
11/6/2015
Design Approach
VHDL structured design
 As bullet proof a state machine as possible
 Verification through

-Functional Simulation
-Post Synthesis Simulation
-Post Lay Out Simulation
-Use of complex VHDL stimulus models
Aer Cam Project
14
11/6/2015
Design Organization
Aer Cam Project
15
11/6/2015
The Bullet Proof State Machine
Ours isn’t perfect either, but not too bad
 Our wish list

-Want a pretty safe state machine
-SEU disruptions to a minimum
-Easy checking for illegal states
-Fast clock speed
Aer Cam Project
16
11/6/2015
How we think we got there
Barrel Shifter provides the safety
-Initial state and a linear progression
through possible states returning to Idle
 Barrel Shifter provides an easy method to
check for illegal states

More than one bit set and you’ve got problems

Overlay desired SDRAM cycle over linear
progression of states
(Burst R/W, Precharge, Mode Write, Refresh)
Aer Cam Project
17
11/6/2015
More About The State Machine
Barrel Shifter
‘0’
‘0’
‘0’
‘1’
‘0’
Single One Detector
State
Error Error
State Machine
S0
S1
State Machine Outputs
S2
S3
S4
Decoded
States
(Registered)
State Machine Inputs
This example denotes S3 since a one is in the S3 position
Aer Cam Project
18
11/6/2015
More About The State Machine

More detail about the previous slide
– Initial State is S0 “0001” (Also Idle)
– State Error exists when more than one bit is set
– Inputs to state decoder determine what sequence of
outputs are generated
i.e. for this SDRAM controller there are a maximum of 15 states.
If you look at the SDRAM data sheet you can overlay each type
of operation (Read,Write,Mode,Refresh) over these 15 states.
The nice thing about SDRAM cycles, they are started and run to
completion with interruption, that’s why this State Machine
works
Aer Cam Project
19
11/6/2015
Maybe Some Code will help
A clocked process to do the barrel shifter
State_gen : Process (Return_Idle,clk)
begin
if (Return_Idle=‘1’) then state <= “0001”;
elsif (clk’event and clk=‘1’) then
if ((gen-c1=‘1’ or gen-c2=‘1’) and state=“0001”) then
state <= “0010”
else
state <= state (1 to 3) & ‘0’;
end if;
end if;
end process State_gen;
Assume state : std_logic_vector (0 to 3); four states
Aer Cam Project
20
11/6/2015
More code please

State Decoder stuff
State_decode : process (Reset_L,clk)
Begin
If (Reset_L=‘0’) then
All_Outputs <= ‘0’; Return_Idle<=‘1’; Done<=‘0’;
elsif (clk’event and clk=‘1’) then
case state is
when “0001” =>
if (gen-c1=‘1’) then
modify outputs for gen-c1 event in S0;
Aer Cam Project
21
11/6/2015
Return_Idle <= ‘0’;
elsif (gen-c2=‘1’) then
modify outputs for gen-c2 event in S0;
Return_Idle <= ‘0’;
end if;
Done <= ‘0’;
when “0010” =>
if (gen-c1=‘1’) then
modify outputs for gen-c1 event in S1;
Return_Idle <= ‘0’; Done <= ‘0’;
elsif (gen-c2=‘1’) then
modify outputs for gen-c2 event in S1;
Return_Idle <= ‘1’; (Return to Idle truncated cycle)
Done <= ‘1’;
end if;
(This should illustrate the concept)
Aer Cam Project
22
11/6/2015
Some additional information
-In the code above All Outputs registers are
reloaded during every state.
-gen-c2 doesn’t need all provided states so it
terminates early using Return_Idle signal
(i.e. SDRAM modewrite & precharge
don’t need as many clocks as Read/Writes)
-SDRAM Ras,Cas,WE can be overlaid on
the linear progression of states based on
the needed cycle (Read,Write,Refresh,etc)
Aer Cam Project
23
11/6/2015
More additional Information
-In states other than idle, Return_Idle must be set to
zero in at least one state, otherwise the state
register will eventually become all zeros. If you
detect all zeros in the state register you can
protect against this error also.
-Nested if statements gives priority but beware, too
much nesting causes excessive timing delays.
-Requests to state machine are held high until Done
signal is asserted indicating requested cycle is
complete.
Aer Cam Project
24
11/6/2015
Isn’t that what one hot is?

Not quite the same
-One hot can jump from any state to any
other state. Barrel shifter always forces
linear state transitions.
-Barrel shifter has basically two options
Hold in initial state or progress through
states. State decoder can truncate the
cycle of states and return to Idle.
Aer Cam Project
25
11/6/2015
Additional State Machine Safety

Refresh all outputs every state
-Extra logic but minimizes undesired output due to SEU
(Some designers only code an output change in states during which
a given output should change. You generate extra logic when
you code logic for every given output in every state)
-Keeps SEU induced output transitions to one clock
This is true as long as the SEU occurs in an output
register. When the inputs get clobbered all bets
are off.
Aer Cam Project
26
11/6/2015
Where is the design currently

Very much a work in progress
-Functional simulation done
-Design synthesized
-Needs Post Synthesis and Post Layout
verification and timing checks
-Other FPGA peripheral cells need to be integrated into
design and simulated

What’s the holdup
-Funding issues and time
Aer Cam Project
27
11/6/2015
Performance and Size

•
Design much smaller than anticipated
-Place and Route tools indicate entire
SDRAM controller occupies approx.
1% of a Virtex II 3000
How fast will it run
-Estimates are SDRAM controller will
operate at clock speeds up to 115MHz
Final requirement was 100MHz
Aer Cam Project
28
11/6/2015
Design Tweaks

We’ve got some time before flight
-Since design turned out so small might
consider some TMR
-Perform more exhaustive verification
exercise
-Investigate periodic SDRAM mode
register refresh
-Test PPC 405 path to SDRAM
Aer Cam Project
29
11/6/2015
If I Only Could Of…….
• Some things that might be done differently
-Had better requirements from the onset
-Understood SDRAM SEU failures better
-Added some additional protection when
processor crashes i.e. Maybe some cumulative
vector logic so we could null motion in Reset
Controller (Hard Logic) when sanity is lost.
-Hard Proximity Detector & RF Beacon
AerCam starts screaming if it gets too close to vehicle.
Aer Cam Project
30
11/6/2015
VHDL Pointers for Safe Logic

Avoid too much clever stuff
–
Stick to cloud of logic with output register
• Plug all the holes
-This can really help out in verification
-The beauty of the barrel shifter
-This can be very hard with conventional state
machines. What does “When others” really
mean at the end of the case statement?
Aer Cam Project
31
11/6/2015
VHDL Pointers for Safe Logic
• When you want to be sure Instantiate
(if you can) this issues dogs me with XST
(ADD8 is a good example. XST won’t take
an instantiation you must infer, Synplify will)
-Use this all the time with DLLs and Buffers
IOBs, etc. in Virtex
-Keep in mind if you instantiate everything you
might be better off doing things with a schematic
Aer Cam Project
32
11/6/2015
VHDL Pointers for Safe Logic

Be wary of synthesis tools
-Optimization can be a real pain
Warning messages often tell as much as
looking at Logic generated by tools.
Warnings often tell you something isn’t coded right.
-Take care that registers didn’t get
inferred as latches.
-Use acceptable coding practices
Standard inference rules produce uniform results.
Good coding practices reduce the need to sift
through synthesizer logic outputs.
Aer Cam Project
33
11/6/2015
VHDL Pointers for Safe Logic
• Get to know your synthesis tool
•
•
•
User groups can be a great thing too!
Tool vendors go to great lengths to ensure
they meet standards
Lots of examples exist to help produce
uniform synthesis results
You might not believe it, but someone else
has already had most of your problems and
it’s probably in a database you can get to.
Aer Cam Project
34
11/6/2015
VHDL Pointers for Safe Logic
• Be wary of the great sales pitch
-Still have concerns about C code to logic
conversion tools. The demos look great but they
seem to be optimized to the tools. For some real
world applications I have seen them generate extra
clocking stages (design won’t make timing
objectives) or synthesis tools optimize out some
desired logic.
-Haven’t looked at these recently so it could be better.
Aer Cam Project
35
11/6/2015
VHDL Pointers for Safe Logic
• The boss wants one of the software guys to
do VHDL for a FPGA
-My little tale from VHDL class
-I have seen many programmer types generate
great looking code, made functional simulation
work but wouldn’t synthesize and/or make
timing.
-Best results generated when a designer
understands logic design as well as HDL
Aer Cam Project
36
11/6/2015
General Logic Issues

Understand what your part is connected to
-Not a good idea to generate invalid data to
some connected component.
-Your part may continue to work but everything
may come to a grinding halt and/or smoke!
-Man invented data sheets for a reason
Aer Cam Project
37
11/6/2015
General Logic Issues

Hiccups will happen
-SEUs can wreak havoc on a design.
-TMR if you can
-Otherwise view unprotected logic blocks
in system context.
-Detect problems somewhere and recover
(sometimes processors and/or
software can do the job)
Aer Cam Project
38
11/6/2015
General Logic Issues

Hiccups continued
-If you must glitch try to isolate or minimize the
effect on the next block.
i.e. the notion of a latch refresh every clock
This is true even with TMR. The longer a
latch sits uncorrected, the greater the chance a
redundant latch can get hit and TMR breaks.
Aer Cam Project
39
11/6/2015
The Holy Grail of Logic Design

Verification
-Perhaps the most daunting task facing logic
designers today.
-My best advice is to break down design into
smaller blocks that you can verify
independently.
-Try to constrain the outputs of a block even
when SEU strikes. If you can characterize the
output of a block, even in error or SEU
conditions it allows a more sane verification of
the design as a whole
Aer Cam Project
40
11/6/2015
Verification Continued
• The old days of running every possible test
vector into a complex design are over.
 The new verification tools are probably
very rich (both in features and cost)
 I plan to check out Verification tools the
next few days.
Aer Cam Project
41
11/6/2015
HDLs And Critical Systems
Verification is most significant issue
(From my Perspective)
– How do you verify the design to confidence
– If you can’t verify to 100% how do you
mitigate potential problems
– Not talking SEUs but logic problems.
– SEUs need their own examination
Aer Cam Project
42
11/6/2015
HDLs And Critical Systems
• Synthesis tools can be very predictable
-Use accepted inference rules
-Combinatorial Logic Inside Clocked Process
-Think about verification as a design requirement
-Understand that logic can fail. If it is critical in nature,
how is the failure dealt with.
-These issues (logic and SEU) were around before
Apollo but modern silicon densities and design
complexities compound problems.
Aer Cam Project
43
11/6/2015