Slide 1

Markovito’s Team
(INAOE, Puebla, Mexico)

Team members

Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers

Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid

Visual features (SIFT) are integrated
to the map for improving localization

Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples

Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)

Face recognition

SIFT feature extraction
Localization and tracking

Face recognition
Video streaming

Results

Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models

Can identify people standing
or sitting, facing forward or
seen from the side

Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech

Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task

Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities

Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.

Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained

Architecture:
Modular, Layered, Distributed
Follow
Me

…

Who’s
who

Lost &
Found

Shared Memory

Voice

Naviga- LocaliGestures Faces
tion
zation

Sensors

Silhouettes

Actuators

Objects

…

Bye…


Slide 2

Markovito’s Team
(INAOE, Puebla, Mexico)

Team members

Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers

Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid

Visual features (SIFT) are integrated
to the map for improving localization

Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples

Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)

Face recognition

SIFT feature extraction
Localization and tracking

Face recognition
Video streaming

Results

Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models

Can identify people standing
or sitting, facing forward or
seen from the side

Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech

Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task

Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities

Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.

Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained

Architecture:
Modular, Layered, Distributed
Follow
Me

…

Who’s
who

Lost &
Found

Shared Memory

Voice

Naviga- LocaliGestures Faces
tion
zation

Sensors

Silhouettes

Actuators

Objects

…

Bye…


Slide 3

Markovito’s Team
(INAOE, Puebla, Mexico)

Team members

Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers

Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid

Visual features (SIFT) are integrated
to the map for improving localization

Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples

Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)

Face recognition

SIFT feature extraction
Localization and tracking

Face recognition
Video streaming

Results

Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models

Can identify people standing
or sitting, facing forward or
seen from the side

Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech

Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task

Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities

Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.

Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained

Architecture:
Modular, Layered, Distributed
Follow
Me

…

Who’s
who

Lost &
Found

Shared Memory

Voice

Naviga- LocaliGestures Faces
tion
zation

Sensors

Silhouettes

Actuators

Objects

…

Bye…


Slide 4

Markovito’s Team
(INAOE, Puebla, Mexico)

Team members

Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers

Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid

Visual features (SIFT) are integrated
to the map for improving localization

Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples

Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)

Face recognition

SIFT feature extraction
Localization and tracking

Face recognition
Video streaming

Results

Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models

Can identify people standing
or sitting, facing forward or
seen from the side

Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech

Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task

Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities

Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.

Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained

Architecture:
Modular, Layered, Distributed
Follow
Me

…

Who’s
who

Lost &
Found

Shared Memory

Voice

Naviga- LocaliGestures Faces
tion
zation

Sensors

Silhouettes

Actuators

Objects

…

Bye…


Slide 5

Markovito’s Team
(INAOE, Puebla, Mexico)

Team members

Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers

Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid

Visual features (SIFT) are integrated
to the map for improving localization

Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples

Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)

Face recognition

SIFT feature extraction
Localization and tracking

Face recognition
Video streaming

Results

Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models

Can identify people standing
or sitting, facing forward or
seen from the side

Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech

Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task

Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities

Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.

Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained

Architecture:
Modular, Layered, Distributed
Follow
Me

…

Who’s
who

Lost &
Found

Shared Memory

Voice

Naviga- LocaliGestures Faces
tion
zation

Sensors

Silhouettes

Actuators

Objects

…

Bye…


Slide 6

Markovito’s Team
(INAOE, Puebla, Mexico)

Team members

Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers

Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid

Visual features (SIFT) are integrated
to the map for improving localization

Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples

Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)

Face recognition

SIFT feature extraction
Localization and tracking

Face recognition
Video streaming

Results

Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models

Can identify people standing
or sitting, facing forward or
seen from the side

Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech

Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task

Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities

Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.

Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained

Architecture:
Modular, Layered, Distributed
Follow
Me

…

Who’s
who

Lost &
Found

Shared Memory

Voice

Naviga- LocaliGestures Faces
tion
zation

Sensors

Silhouettes

Actuators

Objects

…

Bye…


Slide 7

Markovito’s Team
(INAOE, Puebla, Mexico)

Team members

Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers

Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid

Visual features (SIFT) are integrated
to the map for improving localization

Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples

Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)

Face recognition

SIFT feature extraction
Localization and tracking

Face recognition
Video streaming

Results

Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models

Can identify people standing
or sitting, facing forward or
seen from the side

Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech

Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task

Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities

Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.

Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained

Architecture:
Modular, Layered, Distributed
Follow
Me

…

Who’s
who

Lost &
Found

Shared Memory

Voice

Naviga- LocaliGestures Faces
tion
zation

Sensors

Silhouettes

Actuators

Objects

…

Bye…


Slide 8

Markovito’s Team
(INAOE, Puebla, Mexico)

Team members

Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers

Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid

Visual features (SIFT) are integrated
to the map for improving localization

Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples

Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)

Face recognition

SIFT feature extraction
Localization and tracking

Face recognition
Video streaming

Results

Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models

Can identify people standing
or sitting, facing forward or
seen from the side

Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech

Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task

Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities

Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.

Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained

Architecture:
Modular, Layered, Distributed
Follow
Me

…

Who’s
who

Lost &
Found

Shared Memory

Voice

Naviga- LocaliGestures Faces
tion
zation

Sensors

Silhouettes

Actuators

Objects

…

Bye…


Slide 9

Markovito’s Team
(INAOE, Puebla, Mexico)

Team members

Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers

Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid

Visual features (SIFT) are integrated
to the map for improving localization

Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples

Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)

Face recognition

SIFT feature extraction
Localization and tracking

Face recognition
Video streaming

Results

Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models

Can identify people standing
or sitting, facing forward or
seen from the side

Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech

Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task

Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities

Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.

Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained

Architecture:
Modular, Layered, Distributed
Follow
Me

…

Who’s
who

Lost &
Found

Shared Memory

Voice

Naviga- LocaliGestures Faces
tion
zation

Sensors

Silhouettes

Actuators

Objects

…

Bye…


Slide 10

Markovito’s Team
(INAOE, Puebla, Mexico)

Team members

Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers

Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid

Visual features (SIFT) are integrated
to the map for improving localization

Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples

Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)

Face recognition

SIFT feature extraction
Localization and tracking

Face recognition
Video streaming

Results

Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models

Can identify people standing
or sitting, facing forward or
seen from the side

Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech

Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task

Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities

Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.

Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained

Architecture:
Modular, Layered, Distributed
Follow
Me

…

Who’s
who

Lost &
Found

Shared Memory

Voice

Naviga- LocaliGestures Faces
tion
zation

Sensors

Silhouettes

Actuators

Objects

…

Bye…


Slide 11

Markovito’s Team
(INAOE, Puebla, Mexico)

Team members

Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers

Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid

Visual features (SIFT) are integrated
to the map for improving localization

Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples

Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)

Face recognition

SIFT feature extraction
Localization and tracking

Face recognition
Video streaming

Results

Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models

Can identify people standing
or sitting, facing forward or
seen from the side

Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech

Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task

Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities

Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.

Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained

Architecture:
Modular, Layered, Distributed
Follow
Me

…

Who’s
who

Lost &
Found

Shared Memory

Voice

Naviga- LocaliGestures Faces
tion
zation

Sensors

Silhouettes

Actuators

Objects

…

Bye…


Slide 12

Markovito’s Team
(INAOE, Puebla, Mexico)

Team members

Sabina: hardware platform
• PeopleBot
• Laser SICK LMS200
• PTZ system - video
camera VCC5
• Two rings of sonars
and infrared sensors
• Stereo vision (videre)
• Directional microphone
and speakers
• Gripper 2 D.O.F and
bumpers

Map building
The initial map is built integrating laser and sonar scanners
with particle filters, represented as a probabilistic grid

Visual features (SIFT) are integrated
to the map for improving localization

Navigation and Localization
My flexible and robust navigation
algorithm combines an initial plan
based on PRMs with a reactive
navigator that uses TOPs learned
from examples

Global and local localization is based on natural landmarks:
corners and walls (laser), and SIFT (vision)

Face recognition

SIFT feature extraction
Localization and tracking

Face recognition
Video streaming

Results

Identification based on silhouettes
People identification uses stereo
vision and is based on distance
and silhouettes models

Can identify people standing
or sitting, facing forward or
seen from the side

Voice recognition, synthesis and
animated face
My animated face can
express different emotions
and it is synchronized with
my speech

Speech synthesis and recognition
uses standard tools combined with
text processing, directed by the
coordinator according to the task

Object Manipuation
The Katana arm provides to
Sabina with object
manipulation capabilities

Rapidly Exploring random trees were
implemented for motion planning in
order to reach a grasping
configuration.

Coordinator - MDP
The coordination of the
different modules to perform
certain task is based on a
Markov decision process
(MDPs)
According to each task in this
competition, the reward
function of the MDP is defined,
and by solving the MDP an
optimal policy is obtained

Architecture:
Modular, Layered, Distributed
Follow
Me

…

Who’s
who

Lost &
Found

Shared Memory

Voice

Naviga- LocaliGestures Faces
tion
zation

Sensors

Silhouettes

Actuators

Objects

…

Bye…