Transcript Diapositive 1

Using Several Ontologies for
Describing Audio-Visual
Documents:
A Case Study in the Medical Domain
Antoine Isaac1 & Raphaël Troncy2
Sunday 29th of May, 2005
Multimedia and the Semantic Web
Agenda
• Context and Aims
• Corpus
• Ontological Resources
– AV Ontology, Medical Ontology
•
•
•
•
Annotating the Videos
Querying and Reasoning
Performing SW-inspired Reasoning
Conclusion
2005/05/29
A. Isaac & R. Troncy - MSW'2005
1
Describe AV documents
• Various uses / Different granularities
– Identification, feature extraction, structural decomposition,
semantic description
• Description deep meaning cannot be accessed
and processed by classical systems
– Knowledge is often implicit: labels and comments in natural
language
– Formalisation for description syntax, not semantics
• Formal semantics should be interesting
– Reasoning with AV document descriptions
– Interoperability with formal domain-specific ontologies, allowing
to mix AV and domain-related reasoning
2005/05/29
A. Isaac & R. Troncy - MSW'2005
2
Objectives
• Settle an mini-experiment to show the benefits of
using semantic web technologies for annotating
audiovisual content
• Show that the use of:
– formal ontologies,
– annotation pattern,
– inference capabilities
… is highly desirable for a better access to AV
content
2005/05/29
A. Isaac & R. Troncy - MSW'2005
3
Corpus
• Medicine-related TV documentaries
– 30 documents, about 30 hours
– 50% deal with heart (surgery) theme
• Description point of view: how AV features
are used to popularize scientific notions
• Describe both the form and the content
– AV-oriented parts (documentary structure)
– Theme-oriented parts (medicine notions)
2005/05/29
A. Isaac & R. Troncy - MSW'2005
4
Ontological Resources
• Audio-Visual Core Ontology [Isaac & Troncy, 2004]
– Characterization of programs and sequences (AV genre)
– Decomposition of programs and sequences
– Ability to introduce description from the point of view of the
activities that constitute the context of AV documents
• roles of people involved, way production and broadcast are
achieved, etc.
• Dual Legitimacy
– Use: conceptualization grounded on observed purposes and
domain initiatives, study of 30 years of documentary practices
– Conception: articulation with an upper-level ontology: DOLCE
[Gangemi, 2002]
2005/05/29
A. Isaac & R. Troncy - MSW'2005
5
Ontological Resources
<owl:Class rdf:ID="DialogSequence">
<rdfs:subClassOf rdf:resource="#SpokenSequence"/>
<rdfs:subClassOf>
<owl:Restriction>
<owl:onProperty>
<owl:ObjectProperty rdf:about="#hasParticipant"/>
</owl:onProperty>
<owl:minCardinality rdf:datatype="&xsd;int">2</owl:minCardinality>
</owl:Restriction>
</rdfs:subClassOf>
</owl:Class>
2005/05/29
A. Isaac & R. Troncy - MSW'2005
6
Ontological Resources
• Extension of AV core with specific application
notions
– Exemplification, demonstration, etc.
• Re-use of Medical Ontologies
– Menelas: domain of coronary pathologies
• Concepts dealing with heart surgery
– Alternative choices are possible
• Galen (concepts dealing with surgical procedures)
• Articulation between the ontologies
– No use of automatic alignment methods or tools
– State by hand OWL axioms (equivalentClass)
2005/05/29
A. Isaac & R. Troncy - MSW'2005
7
Description Process
• Segmentation of the AV material
– Selection of relevant documentary items
• Knowledge-based AV annotation
– Documentary structure characterization
– Segment content description
2005/05/29
A. Isaac & R. Troncy - MSW'2005
8
Segmenting the Videos
2005/05/29
A. Isaac & R. Troncy - MSW'2005
9
Describing the Videos
• Documentary patterns
– Layered approach [Troncy, 2003]
– AV description language [Troncy & Carrive, 2004]
• Knowledge-based Annotation Process
– The structure is described at the knowledge
level
• Concepts and relations from the AV ontology are
manually introduced in the description
– Content description
• Link to external world themes and entities
2005/05/29
A. Isaac & R. Troncy - MSW'2005
10
Describing the Videos
• Relational Indexing Pattern
– Help for user: specify how concepts and relations
have to be used (annotation ‘design pattern’)
– Important for ontology conception and use (with
reasoning knowledge)
• Simple pattern that can lead to complex
descriptions
– Recursive relational structure
2005/05/29
A. Isaac & R. Troncy - MSW'2005
11
Describing the Videos
2005/05/29
A. Isaac & R. Troncy - MSW'2005
12
Querying and Reasoning
• Example:
« retrieve the programs that explain a disease and
show one of its causes »
• Need for the following inferences:
– Subsumption
CVDisease(x)  Disease(x)
– Composition
hasSubSequ ence(x,y)  explains(y, z)  explains(x, z)
hasSubSequ ence(x,y)  shows(y,z)  shows(x,z)
2005/05/29
A. Isaac & R. Troncy - MSW'2005
13
Querying and Reasoning
re p re s
Explicit statements
Inferred statements
re p re s
hasAVFeature
represents
t
Program:
JN_children
ipan
explains
h a sP
artic
s
ha
hasParticipant
explains
Person: prof1
CVDisease:
blueDisease
re s
en
ents ts
Animated:
animatedValue
hasSubSequence
ExpertInterview:
cvDiseasesItw
ents
re p
re
a
Fe
AV
e
tur
hasEditing
Insert
p
e x re s
pl en
ai ts
ns
re p r
es
expl ents
ains
causes
hasAVFeature
Drawing:
stenosisDrwg
explains
represents
represents
Heart: heart1
part
Stenosis:
paStenosis
PulmonaryArtery: pa
stateOf
role
2005/05/29
ExpertRole:
prof1Role
A. Isaac & R. Troncy - MSW'2005
14
Performing reasoning
• A layered complexity approach
– RDFS: subsumption
– OWL(DL): complex definitions + algebraic properties
– Rules: horn clauses
• Concrete implementation
– RDFS: Sesame Architecture [Broekstra, 2002]
– OWL DL: BOR Reasoner [Simov, 2002]
– OWL-DLP [Grosof, 2003] + Rules : Sesame custom
inference module
2005/05/29
A. Isaac & R. Troncy - MSW'2005
15
Examples
• DL definitions
ExpertInterview  ( Interview
som e hasParticipant ( Person  som e role ExpertRole) )
ExpertRole  ( ( Academ icRole  ProfessionalRole HospitalRole)
 ( InstitutionRole) )
• (Composition) rule
hasSubSequ
ence ( x, y)  represents( y, z)  represents( x, z)
2005/05/29
A. Isaac & R. Troncy - MSW'2005
16
Summary
Explicit
triples
Inferred
triples
RDF Model
All triples
129
AV Ontology
5231
10810
16041
Menelas
Ontology
10534
26637
37171
Instances
276
1507
1783
Total
16041
38954
54995
2005/05/29
A. Isaac & R. Troncy - MSW'2005
17
Conclusion
• This experimentation:
– Uses Semantic Web languages and tools for describing AV
contents
– Uses different ontologies to capture the structure and the content
of the documents
– Uses relational indexing pattern for the annotation
• Future work: thorough evaluation of those techniques
involving real users
• A problem that cannot be generally solved: fixing a tradeoff between expressivity and tractable computation
– Ad hoc, according to the needs of the application targeted
2005/05/29
A. Isaac & R. Troncy - MSW'2005
18