Transcript Diapositive 1 - JRC Science Hub

European Language Resources Association
ELRA/ELDA: The Importance of Sharing
Linguistic Resources
Victoria Arranz
ELRA/ELDA
55-57 Rue Brillat-Savarin, 75013 Paris, France
Tel. +33 1 43 13 33 33 -- Fax. +33 1 43 13 33 30
Email: [email protected]
http://www.elda.org/ or http://www.elra.info/
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Presentation Outline
• ELRA’s Foundation and Mission
• ELRA’s Strategic Technical Activities:
– Issues Concerning Archiving and Providing of
Language Resources for the HLT Community
– Administrative and Legal Issues Regarding
Language Resources
– Involvement in the Evaluation of Human
Language Technologies
– Promotion of the Language Technology Field
• Concluding Remarks
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
European Language Resource Association
An Improved Infrastructure for Data Sharing and HLT Evaluation
Centralized Non-profit organisation for the:
• Collection, distribution and validation of LRs and
tools,
• Production or commissioning of production of LRs
• Evaluation of Human Language Technologies
Operational agency  ELDA:
Evaluation & Language Resources
Distribution Agency
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Europe and the Multilingual Issue
More languages...Even within the same country
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
European Language Resource Association
An Improved Infrastructure for Data Sharing
An Association of users of Language Resources
A Repository Center:
Technical & Logistic issues
Commercial issues (prices, fees, royalties)
Legal issues (Licensing, IPR)
Information Dissemination
Infrastructure for the evaluation of Human Language Technologies
providing resources, tools, methodologies, logistics
Exit strategies / Capitalization on evaluation packages
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
LRs & Human Language Technologies
as seen by ELRA
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Contrastive Analysis
Public “role”- Streamlining public funds
ACQUILEX
ARS
GENELEX
ONOMASTICA
PLUS
POLYGLOT
REWARD
SUNDIAL
SUNSTAR
BEFORE ELRA:
No resources were made available
Duplication of efforts and funding
<== LRs NEVER DISTRIBUTED
AFTER CE Established ELRA
(LRs CURRENTLY DISTRIBUTED
See ELRA Catalogue)
ACCOR
COST-232
CRATER
MULTEXT
PAROLE
SPEECHDAT Family
TSNLP
(Euro)WordNet
Return on Investment
Rational Capitalization
"Public" Assets
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Language Resources Activities
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Language Resources Activities
• Issues concerning Archiving and Providing of
Language Resources for the HLT community:
– Maintenance of ELRA’s Catalogue
– Development of the Universal Catalogue and Identification
of Useful Resources
– Development of the Basic LAnguage Resource Kit
(BLARK)
– Development of the Extended LAnguage Resource Kit
(ELARK)
– Validation and Quality Assessment
– Production or Commissioning of the Production of LRs
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Language Resources Activities
• Administrative and Legal Issues regarding
Language Resources:
– Handling of Legal Issues related to the Availability
of LRs
– Distribution of LRs and Pricing Policy
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Language Resources Activities
• Involvement in the Evaluation of Human
Language Technologies:
– Evaluation projects within ELDA
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Language Resources Activities
• Promotion of the Language Technology
Field:
– Information Dissemination, Promotion and
Awareness, Market Watch and Analysis
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Issues Concerning Archiving and
Providing of Language Resources
for the HLT Community
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
The Universal Catalogue
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Basic LAnguage Resource Kit (BLARK)
• Compilation of necessary LRs that should be
available for all languages & regarding different
Language Technologies and Modules
• Defined by Steven Krauwer and first launched
through ELSNET with Dutch Initiative “Dutch
Human Language Technologies Platform”
• In the framework of ENABLER thematic network
(European National Activities for Basic Language
Resources): ELDA elaborated report
• Sample: work done for Arabic resources within
NEMLAR project: http://www.nemlar.org
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
ELRA’s BLARK
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Validation and Quality Assessment
• Users need info on LRs: technical specifications,
assurance of quality,...
• Validation is used by ELRA in reference to: checking the
adherence to standards, QC of LRs, suitability for market
• Definition of validation methodology required:
– Projects for the production of guidelines, standards and
specifications: EAGLES, PAROLE, SPEECHDAT,
INTERVAL, ...
– Establishment of a Validation Committee (VCom)
• SPEX (for speech resources): http://www.spex.nl/validationcentre/
• CST (for written resources): http://cst.dk/validation/index.html
– Cooperation with key players with relevant expertise to act as
validation units
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
ELRA’s Production of LRs
• Spoken Language
Resources:
–
–
–
–
–
–
–
–
CHIL
C-ORAL-ROM
OrienTel
Neologos
Speecon
TC-STAR
ESTER
MEDIA
EU Enlargement and Integration Workshop
Arona, September 2005
• Written LRs:
– Technolangue
projects:
• Lexitec
• Euradic
– Technolangue/Evalda
platform projects:
• Equer
• EASy
Victoria Arranz ‹#›
 ELRA/ELDA
Administrative and Legal Issues
Regarding Language Resources
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Handling of Legal Issues Regarding LRs
• Support of lawyers for basic principles on LRs licensing
• One of ELRA’s priorities: simplify rel. between providers & users
• Drafting of generic contracts (with responsibilities & obligations) to
encourage producers/providers to distribute
• Establish what usage is allowed for LRs: R&D or Research only?
• Protect providers and their LRs
• Are available on ELDA’s web site: http://www.elda.org/article69.html
• ELRA:
– not owner of LRs & sets fair price with owner, based on:
• Production costs
• Expected revenues
• ELRA’s distribution policy: try to offer discounted price for members
• ELRA’s effort to obtain low prices for R&D
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Involvement in the Evaluation of
Human Language Technologies
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Levels of Evaluation (ELSE)
-Basic Research Evaluation
(validate research direction)
-Technology Evaluation
(assessment of solution for
well defined problem)
-Usage Evaluation (endusers in the field)
-Impact Evaluation (socioeconomic consequences)
-Programme Evaluation
(funding agencies)
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Reusability of Resources
ELSE: Reusability of resources from one campaign to another
Producer/Consumer
BNT
CLIR
Broadcast News
Reuse
Reuse
Cross-Lingual Information Retrieval /
Extraction
TTS
SUMZ
LM
Reuse
Reuse
Reuse
Text To Speech Synthesis
Reuse
Text Summarisation
Reuse
Reuse
Reuse
Reuse
Language Model Evaluation
Reuse
Reuse
Reuse
Reuse
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
ELRA’s Activity on Evaluation
• Carrying out evaluation campaigns: further
involvement at:
– French (Technolangue/Evalda) and
– European (CLEF, TC-STAR, CHIL, etc.) levels.
• Distributing evaluation packages
• Producing or commissioning the production of
needed LRs for evaluation
• Conducted with the support of an Evaluation
Committee: ECom
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Evaluation Projects within ELDA
• CLEF 2004 campaign: http://clef.isti.cnr.it/
• TC-STAR: http://www.tc-star.org/
• CHIL: http://chil.server.de/servlet/is/101/
• Technolangue/Evalda: ARCADE II, CESART, CESTA,
EASY, ESTER, EQUER, EVASY, MEDIA
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Promotion of the Language
Technology Field
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Information Dissemination, Promotion and Awareness,
Market Watch and Analysis
• Considerable effort on the promotion and
dissemination of the LT field & its activities:
– Partner/coordinator in several conferences,
meetings, workshops:
• LREC Conference series (~900 participants)
• SCALLA 2004
• International Conference on Arabic Language
Resources and Tools 2004
• Speech and Language System for Human
Communication Conference 2004
• LangTech Conferences
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Information Dissemination, Promotion and Awareness,
Market Watch and Analysis
• Elaboration and distribution of the ELRA Newsletter,
quarterly publication dedicated to inform readers
about HLT events, projects, new LRs, etc.
• Maintenance and/or setting up of a number of web
sites:
–
–
–
–
–
–
–
ELRA: www.elra.info
ELDA: www.elda.org
LREC: www.lrec-conf.org
COCOSDA: www.cocosda.org
NEMLAR conference: www.nemlar-conf.org
LangTech conferences: http://www.lang-tech.org/
Technolangue: www.technolangue.net
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Information Dissemination, Promotion and Awareness,
Market Watch and Analysis
• Public announcements posted to mailing lists
• Sponsorship activities
• Carrying out of market analysis and survey
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
Concluding Remarks
• Overview on major activities at ELRA and ELDA
• Aim of ELRA/ELDA continue active role in
support of HLT community:
– Supporting researchers and developers in their needs of
LRs
– Evaluation guidelines
– Validation guidelines
– Development of standards
– Working on the legal, distribution or pricing issues
– We believe: we’re moving towards the future needs of LT
in an organised and productive manner
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA
LREC-2006…
www.lrec-conf.org
EU Enlargement and Integration Workshop
Arona, September 2005
Victoria Arranz ‹#›
 ELRA/ELDA