Transcript Scrabble Tile Counter

Scrabble Tile Counter
CSC 375/475 Introduction to Robotics
DePaul University
Caden Howell
June 10, 2009
What is it?
• Attempt to read the letters of a set of
Scrabble tiles
• Report whether the set matches the
standard Scrabble tile distribution.
©2009 Caden Howell
Third party projects are the property of their respective owners.
How does it work?
• Use NXT hardware to physically move tiles over
the webcam
• Break the letter down into a grid of superpixels
• Train a neural network on the superpixel values
– Classified 2200 tiles by hand for training data.
– Input is 100 superpixel nodes set to 1 or 0
– Output of neural network is 6 bits representing a letter
• Use the network to classify tiles as letters
©2009 Caden Howell
Third party projects are the property of their respective owners.
Collecting training data
©2009 Caden Howell
Third party projects are the property of their respective owners.
Qwiknet
Illustration of
neural net back
end structure,
running in
QwikNet.
Neural net
used by the
Scrabble tile
project was a
different
implementation
written in
Python.
©2009 Caden Howell
Third party projects are the property of their respective owners.
ROBOTC program
• Used
ROBOTC
sample
Bluetooth
code
BtBasicMsg.C
void readMessages()
{
static int nLastMessage = 0;
while (true)
{
nMessage = message;
if (nMessage != 0)
{
//Keep a running count of the number of
messages successfully read
++nNearEndRead;
if (nMessage != (nLastMessage + 1))
++nReadOutOfSequence;
nLastMessage = nMessage;
nFarEndSent = messageParm[1];
nFarEndRead = messageParm[2];
ClearMessage();
nElapsedTime = nPgmTime;
dispenseTile(15);
}
else
++nRcxNoMsg;
wait1Msec(30);
}
(See C:\Program Files\Robotics
Academy\ROBOTC for
Mindstorms\Sample
Programs\NXT\Bluetooth)
}
©2009 Caden Howell
Third party projects are the property of their respective owners.
Python libraries
• wxPython for windowing
• OpenCV for computer vision
• PIL “Python Image Library”
• speech for Microsoft Speech SDK (TTS)
• jaraco.nxt for sending Bluetooth
commands to NXT (thanks Krista)
• bpnn.py is an open source
implementation of a simple single-layer
neural network
©2009 Caden Howell
Third party projects are the property of their respective owners.
External programs used
• Weka* to compare different classification
methods
• Qwiknet* to find good parameters for the
neural net
*both of these programs are introduced in
DePaul’s Machine Learning class
• Nconvert to preprocess images (thanks
Ben)
– nconvert -o nconvertout.bmp -brightness 0 -gauss 5 gamma 4 -contrast 40 -noise reduce histogram.bmp
©2009 Caden Howell
Third party projects are the property of their respective owners.
Accuracy
• Training data (tested
in Qwiknet) 99.6%
• Test data (401 points)
90.8%
Most miscategorized letters
(of 36 miscategorized letters)
blank
W
D
V
S
N
K
F
J
I
G
Y
X
Z
T
R
B
P
M
E
©2009 Caden Howell
Third party projects are the property of their respective owners.
Issues/Improvements
• Built-in webcam badly positioned
– Used external webcam
• Direct control Bluetooth commands not successful
– Moved motor commands to ROBOTC program
• Bad classification accuracy from neural network
– Expanded classes to include inverted tiles (52 instead of 26)
– Threw out outlier training data
• Poor/inconsistent lighting conditions
– Added bike light
• Increased grid size
– 10 x 10 instead of 8 x 10
– Had to throw out 1000 data points collected by hand
• Poor documentation for OpenCV
– Used nconvert instead
©2009 Caden Howell
Third party projects are the property of their respective owners.
Future Improvements
• Most tile sets are 100 tiles of a known
distribution; distribution of remaining tiles
should be factored in to tile classification
• Noise due to rotation and shifting could
have been reduced by reducing the width
and breadth of the hopper
©2009 Caden Howell
Third party projects are the property of their respective owners.