Transcript MAJORDOME

MAJORDOME
Gérard CHOLLET, Richard CROCE,
Laurence LIKFORMAN,
Dijana PETROVSKA-DELACRETAZ,
Pascal VAILLANT
(chollet,croce,lauli,petrovsk,vaillant)@tsi.enst.fr
ENST/CNRS-LTCI
46 rue Barrault
75634 PARIS cedex 13
http://www.tsi.enst.fr/~chollet/
Majordome Outline

What is it ?

What it does for you ?

Research and application topics:




The SIROCCO project
The EUREKA !2340 MAJORDOME project
VoIP, VoiceXML, Human-Computer Interaction
Perspectives
Majordome is a distributed
Personal Digital Assistant



It is your digital slave. It is personal. It remembers
everything that you told him.
It uses resources from you mobile (wireless) device,
from your home, from your office, from the Internet,
from the environment, …
You interact with him using voice, pen, graphics, …
Interactions with your Majordome






Majordome recognizes your identity, your voice, your
handwriting, ...
His speech recognizer is adapted to your voice,
His handwriting recognizer is adapted to your writing
style,
He can speak to you,
He can display information for you,
He can talk with other persons either locally or over
the phone.
What Majordome does for you ?






Answers your phone,
Receives and interpret your faxes, your emails, …
Supplements your memory (address book, agenda,
bookmarks, alarm clock, health record, bank account,
documentation, …)
Serves as an interface between you and the (digital)
world,
Searches the web, internet forums, …
Controls your home, your car, your children, your
parents, …
A framework: A L I S P
A utomatic
L anguage
I ndependent
S peech
P rocessing
with applications in
Speech Coding, Synthesis, Recognition,
Speaker Verification and Language Identification
SIROCCO project
Unlimited Vocabulary Speech Recognition
INRIA (IRISA et LORIA), LIA, IRIT, ENST-LTCI
http://www.irisa.fr/sirocco/
SIROCCO








Unlimited vocabulary speech recognition system
French lexicon (MathLex) with 64kwords (AUF task)
Feature extraction with Spro (G. Gravier)
Context-dependent HMM phone models
Word pronunciation graph
Uses CMU-Toolkit for Language modeling
Beam search for word hypothesis
Rescoring of word hypothesis by A*
Holistique
EDF
«MAJORDOME»
Unified Messaging System
Eureka Projet no 2340
D. Bahu-Leyser, G. Chollet, R. Croce, K. Hallouli , J. Kharroubi, D. Kofman,
L. Likforman, E. Matta-Sanchez, D. Petrovska, M. Sigelle, P. Vaillant, F. Yvon
Participants
•
speech : G. Chollet, R. Croce, J. Kharroubi, D. Petrovska
•
fax : K. Hallouli, L. Likforman, Marc Sigelle
•
language : P. Vaillant, F. Yvon
•
platform : D. Kofman, E. Matta-Sanchez, R. Croce
•
ergonomy : D. Bahu-Leyser
Majordome’s Functionalities
(
Voice
Fax
• Speaker
verification
• Dialogue
• Routing
• Updating the
agenda
E-mail
• Automatic
summary
Overview of Majordome


Background tasks (server-side only):
 sorting and filtering messages from different
sources (E-mail, voice, fax, SMS,…);
 extracting relevant information for reporting to
user (names of senders, subject,…).
Dialogue with the user: over phone or Web.
 The system presents the state of the mailbox, the
type of messages, their sender, subject, and may
sum them up or read them on request;
 The users access their mailbox, addressbook, time
schedule, or URIs (Web addresses).
Voice technology in Majordome


Server side background tasks:
continuous speech recognition applied to voice messages upon
reception
 Detection of sender’s name and subject
User interaction:


Identification of the speaker (and Verification if necessary)
Speech recognition (receiving users’ commands through
voice interaction)

Text-to-speech synthesis (reading text summaries, E-mails or
faxes)
Voice Over IP Platform
Network
192.168.222.0/11
Cisco Catalyst
6507
1Gbps (FO Interne)
Unisphere
ERX-700
Salle C-234
VTHD
Visio
conference
Intranet
Distance
Learning
Service
Renater
Salle C-234
Video
Server
(
GK
GW
IPVR
Salle C-234
Salle PBX
RTC/RNIS
PBX
ENST-Paris
‘Majordome’ partners
Majordome / NetCentrex project
PABX /Gateway ENST
-Call Control Server
-Application Server
Calling person
NetCentrex #
IP-VR NetCentrex
Recorder Machine
Usual #
No response
Is the called
person here ?
Usual user called
Vocal E-mail
NetCentrex user
called
Majordome / NetCentrex project
PABX /Gateway ENST
-Call Control Server
-Application Server
Calling person
NetCentrex #
Usual #
Voice Interactive call
No response
IP-VR NetCentrex
•
Speaker
verification
•
Dialogue
Usual user called
•Vocal e-mail
•
Routing
Updating the
agenda
•
• Automatic
summary
NetCentrex user
called
Perspectives

Add Vision, Hearing and Understanding to Mobile
Terminals (UMTS)

Multimedia for Distance Education and Conference
Indexing


Semantic Web,

‘Universal Networking Language’
‘Smart Home’, ‘Smart Car’, ‘Smart Office’
Perspectives



The application context of the Majordome
project could be of interest to COST-278.
The Majordome/NetCentrex platform could be
made available to interested partners.
HTK, ISIP and SIROCCO softwares are
available as freeware. One of them will be used
on the NetCentrex platform.