Markovito’s Team (INAOE, Puebla, Mexico) Team members Sabina: hardware platform • PeopleBot • Laser SICK LMS200 • PTZ system - video camera VCC5 • Two rings of sonars and.
Download ReportTranscript Markovito’s Team (INAOE, Puebla, Mexico) Team members Sabina: hardware platform • PeopleBot • Laser SICK LMS200 • PTZ system - video camera VCC5 • Two rings of sonars and.
Slide 1
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 2
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 3
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 4
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 5
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 6
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 7
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 8
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 9
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 10
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 11
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 12
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 2
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 3
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 4
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 5
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 6
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 7
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 8
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 9
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 10
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 11
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…
Slide 12
Markovito’s Team
(INAOE, Puebla, Mexico)
Team members
Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers
Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid
Visual features (SIFT) are integrated
to the map for improving localization
Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples
Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)
Face recognition
SIFT feature extraction
Localization and tracking
Face recognition
Video streaming
Results
Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models
Can identify people standing
or sitting, facing forward or
seen from the side
Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech
Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task
Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities
Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.
Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained
Architecture:
Modular, Layered, Distributed
Follow
Me
…
Who’s
who
Lost &
Found
Shared Memory
Voice
Naviga- LocaliGestures Faces
tion
zation
Sensors
Silhouettes
Actuators
Objects
…
Bye…