Advanced Cubesat Imaging
Download
Report
Transcript Advanced Cubesat Imaging
Nathan Bossart, Joe Mayer, Bob Urberger
RASCAL ACIP
Team Introduction
http://acip.us
Facilitator: Bob Urberger
Computer Engineering majors
Space Systems Research Lab
RASCAL Mission
“Rascal is a two-spacecraft
mission to demonstrate
key technologies for
proximity operations…”
“After the on-orbit
checkout, one 3U
spacecraft is released and
passively drifts away....
After a suitable distance,
the released spacecraft
will activate its propulsion
system and return to within
a few meters of the base.
The second spacecraft will
be released and the
process repeated...”
RASCAL ACIP
Imaging Payload
Awareness of Cubesat
Environment
Computer Vision
Low-Level Processing
Object Detection
Distance Determination
High-Level Data
Navigation
Thruster Control
RASCAL ACIP
Functional Breakdown
Unique Face Identifier
Known
Pattern
Capture Images
Raw
Data
Transfer Data
Structured
Data
Process Images
High-Level
Data
Output/Store Control Data
RASCAL ACIP
Modules
LEDs
Unique Face Identification
Camera
Capture Images
Transfer Data
Computational
Hardware
Process Images
Output/Store Control Data
RASCAL ACIP
LEDs
Required to perform
classification
Not enough detail
visible for other features
Fails when face partially
visible
Three approaches
Unique pattern of LEDs
for each face
Unique combination of
colors for each face
Both unique patterns
and colors
RASCAL ACIP
Unique pattern works
regardless of camera
spectrum
Color combinations
only work with visible
spectrum cameras
Can classify cube
corners as well as faces
with well chosen color
patterns
Camera
Potential Camera Choices:
FLIR Tau 640
640x480, 14-bit
PCB integration
Control signaling simple
Low resolution,
Visible spectrum image
sensor
monochromatic
16.1 MB/s input data rate @
30Hz
2-5MP - 16 to 24 bit color
Parallel data output from
cameras
Component Requirements:
FLIR
Visible spectrum
Requires lens fixture
Complex control signaling
High resolution, wide color
range
2MP with 24 bit color:
○ 57.6 MB/s input data rate
@ 30Hz
RASCAL ACIP
Processing Hardware
Processing blocks in hardware
Caching and system control managed in software
Timing and Gate Consumption
Alternatives:
Pure software implementation
Pure hardware implementation
RASCAL ACIP
Imaging Functions
Image Processing
Distance
Data
Distance
Detection
Image
Data
Structure
PreProcessing
Objects
In Frame
Object
Detection
Object
Classification
Image
Edges
RASCAL ACIP
Preprocessing
Noise suppression
Color Conversion
Object enhancement
Image segmentation
Conversion and downsampling
RASCAL ACIP
Distance Detection
Identify depth from a
single image
Monocular Cues
Relative size
Structured geometry
identified should be
easy to identify
scale
Comparison of
regular structure
imaged objects to
known shape scale at
particular depth
RASCAL ACIP
Square or equilateral
triangle
Distance Detection in RASCAL
Square LED pattern on spacecraft face
Critical point identification
Homography estimation
Projective transform
Point correspondence for scale
Hardware Domain
Parallel matrix multiplication
RASCAL ACIP
Object Detection
Identifying Objects
in an Image
Region or Contour
Based
Edge Detection
Relies heavily on Pre-
processing
RASCAL ACIP
(Columbia University)
Object Detection in RASCAL
Hardware Domain
Canny/Deriche
Sobel Operator
Constraints
Cubesat size
Environmental
Resolution
(Columbia University)
RASCAL ACIP
Objection Classification
Post-object
detection / image
segmentation
Support vector
machine (metric
space classification)
Assign a class
based upon preprogrammed control
data
RASCAL ACIP
Object Classification in RASCAL
RASCAL ACIP
Completed with baremetal software
ARM Assembly / C
Minimum distance
principle (efficient)
Multi-tiered and/or multidimensional space from
attributes given
Determine a number of
attributes with significant
differences between
faces
Testing: expect a very
high level (>95%) of
correct classifications
Constraints of Object
Classification
Must work with a
variety of
backgrounds (Earth,
Moon, Sun, Space,
etc.)
Ideally real time
(bounded) and low
latency
Updated at >=10 Hz
Must function with
different sizes
(patterns can vary
from a few pels to
larger than the frame)
Definitive
discrimination
functions with high
reliability
Alternative algorithm:
neural nets, fuzzy
logic
Output to Control
System will output calculated information about
placement, attitude, distance, etc.
In the future, a separate team will construct a
system to interpret data and convert to control
signals/data
Since this is out of the scope of our project, the
output format/setup is ultimately our choice
RASCAL ACIP
Functional Testing
Output Unique Pattern
Process Image
Software Verification
Hardware Verification
Capture Images
Stream Images
Verify Control Signals
Output/Store Data
Buffer
Transfer Data
Oscilloscope
Frame Buffer
RASCAL ACIP
System Testing
Camera integration
Hardware timing constraints
Block connectivity verification
Blocks signal each other as intended
Full pipeline simulation
Blocks interact as expected
Physical synthesis testing
Data produced from each frame
RASCAL ACIP
Timeline
10/28
Project Task
Data In and Out of System
Obtain Camera
Separate Processing into Blocks
Interface Camera with Hardware
Algorithm Verification in Software
Store Camera Data in Hardware
Preprocess Image
Achieve Block Functionalality
Merge Processing Blocks
Confirm Full Integration
Project Wrap-Up
RASCAL ACIP
11/17
12/7
12/27
Dates of Years 2013-2014
1/16
2/5
2/25
3/17
4/6
4/26
Estimated Costs
Designed for very low budget and small
amount of needed materials
Largely out of SSRL funding
Function
Part
Obtain vision data
Camera
Low Estimate High Estimate Notes
$4,000
$10,000 SSRL funding
Camera Specification
$0
$10,000 SSRL funding
Display patterns
LEDs
$1
$10 SSRL funding
Algorithm Resources
Books
$0
$0 library & creative commons
Development Tools
Zedboard
$0
$0 donation
Xilinx Vivado
$0
$0 donation
Oscilloscope
$0
$0 provided
Desktop PC
$0
$0 provided
RASCAL ACIP
TOTAL COST:
$4,001
$20,010
Future Work for Integration
Thruster control
system
Placement into
spacecraft
Radiation, vibration,
space-readiness
We will provide
thorough
documentation for
future groups
RASCAL ACIP
Bibliography
http://cubesat.slu.edu/AstroLab/SLU-03__Rascal.html
Jan Erik Solem, Programming Computer Vision with
Python. Creative Commons.
Dr. Ebel, Conversation
Dr. Fritts, Conversation
Dr. Mitchell, Conversation
Milan Sonka, Vaclav Hlavac, Roger Boyle, Image
Processing, Analysis, and Machine Vision. Cengage
Learning; 3rd edition.
http://www.cs.columbia.edu/~jebara/htmlpapers/UTHES
IS/node14.html
RASCAL ACIP
Questions?
RASCAL ACIP