Semantic Integration and Retrieval of Multimedia Metadata Roberto García and Universitat de Lleida, Lleida, Spain Òscar Celma Universitat Pompeu Fabra, Barcelona, Spain Rhizomik.
Download ReportTranscript Semantic Integration and Retrieval of Multimedia Metadata Roberto García and Universitat de Lleida, Lleida, Spain Òscar Celma Universitat Pompeu Fabra, Barcelona, Spain Rhizomik.
Semantic Integration and Retrieval of Multimedia Metadata Roberto García and Universitat de Lleida, Lleida, Spain Òscar Celma Universitat Pompeu Fabra, Barcelona, Spain Rhizomik Contents Introduction Ontological framework Integration framework Integration & Retrieval Scenario Conclusions Future Work Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Contents Introduction Ontological framework Integration framework Integration & Retrieval Scenario Conclusions Future Work Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Introduction Digital Media requires Multimedia Metadata. Main requirements: Great amounts of metadata Low-level to user-friendly metadata Interoperability Proposal: semantic metadata and an ontological framework for integration. Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Contents Introduction Ontological framework Integration framework Integration & Retrieval Scenario Conclusions Future Work Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Ontological Framework Multimedia ontology. Profit from non-semantic initiative MPEG-7. XML Semantic Web: XSD2OWL: schema to ontology. XML2RDF: XML instance data to RDF instances. Complete MPEG-7 OWL Ontology Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Ontological Framework MPEG-7: Multimedia Content Description Standard content-based description for audiovisual information content searching, filtering and identification Descriptors: low-level features structure semantic models collections creation... Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Ontological Framework Other semantic multimedia metadata initiatives: Hunter02: not complete, RDF+DAML. Tsinaraki04: not complete, semantic part of MDS. Troncy03: not complete, from an ontology to MPEG-7. Direct benefits: XQuery vs. Semantic Query, example next… Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Ontological Framework SegmentType hierarchy. One XQuery for each type vs. One semantic query for supertype. Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Ontological Framework ReDeFer XSD2OWL Mappings: XML Schema OWL element|attribute rdf:Property owl:DatatypeProperty owl:ObjectProperty element@substitutionGroup rdfs:subPropertyOf element@type rdfs:range complexType|group|attributeGroup owl:Class complexType//element owl:Restriction extension@base|restriction@base rdfs:subClassOf @maxOccurs @minOccurs owl:maxCardinality owl:minCardinality sequence choice owl:intersectionOf owl:unionOf Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Contents Introduction Ontological framework Integration framework Integration & Retrieval Scenario Conclusions Future Work Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Integration Framework MPEG-7 OWL Ontology as integration base for: Semantic metadata: align ontologies with MPEG-7 ontology. MusicBrainz (http://musicbrainz.org) Simac (http://www.semanticaudio.org) Music Vocabulary (http://www.kanzaki.com/ns/music) Etc. XML2RDF: RDF from MPEG-7 XML instance metadata. Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Integration Framework. Architecture Context-based metadata RDF XML Audio Video Signal Processing Content-based metadata MPEG-7 XML MPEG-7 Integration XML2RDF Ontology XSD2OWL SWRL Engine XMLSchemas: MPEG-7... RDF DL Classifier Another Ontology Higher-level metadata RDFS / OWL: MusicBrainz... Semantic Integration and Retrieval of Multimedia Metadata Retrieval Rhizomik Integration Framework MusicBrainz to MPEG-7 OWL mappings: musicbrainz:Artist mpeg7:CreatorType musicbrainz:Album mpeg7:CollectionType musicbrainz:Track mpeg7:AudioSegmentType dc:author mpeg7:Creator dc:title mpeg7:Title musicbrainz:sortName mpeg7:Name musicbrainz:duration ≡ mpeg7:MediaDuration Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Integration Framework Simac to MPEG-7 OWL mappings: simac:Artist mpeg7:CreatorType simac:name ≡ mpeg7:GivenName simac:Track mpeg7:AudioSegmentType simac:title ≡ mpeg7:Title simac:duration ≡ mpeg7:MediaDuration simac:Descriptor ≡ mpeg7:AudioDSType simac:mode simac:key simac:tempo simac:meter ≡ ≡ ≡ ≡ mpeg7:Scale mpeg7:Key mpeg7:Beat mpeg7:Meter Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Integration Framework Music Vocabulary ontology to MPEG-7 OWL mappings: music:Music_Unit mpeg7:AudioSegmentType music:sections ≡ mpeg7:AudioSegment music:Artist mpeg7:CreatorType music:key ≡ mpeg7:Key music:meter ≡ mpeg7:Meter Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Integration Framework. Architecture Context-based metadata RDF XML Audio Video Signal Processing Content-based metadata MPEG-7 XML MPEG-7 Integration XML2RDF Ontology XSD2OWL SWRL Engine XMLSchemas: MPEG-7... RDF DL Classifier Another Ontology Higher-level metadata RDFS / OWL: MusicBrainz... Semantic Integration and Retrieval of Multimedia Metadata Retrieval Rhizomik Integration Framework XMLXML tree model model XML2RDF: tree RDF RDFgraph graph. Root Blank nodes elem elem elem elem Empty rdf:Properties elem elem attr Text Empty Text Text Text Deduce blank node types from XSD2OWL ontologies restrictions. Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Integration Framework • XML2RDF example mpeg7:Confidence mpeg7:CreationLocation mpeg7:DescriptionMetadataType 1.0 mpeg7:AdministrativeUnit mpeg7:CreationTime 2000-10-10T19:45:00+09:00 mpeg7:Creator mpeg7:CreatorType mpeg7:Instrument mpeg7:LastUpdate mpeg7:CreationToolType mpeg7:PrivateIdentifier mpeg7:Region mpeg7:PlaceType mpeg7:Agent mpeg7:Role Tokyo Shibata Yoshiaki jp mpeg7:Name mpeg7:AgentType, mpeg7:PersonType mpeg7:Name mpeg7:ControlledTermUseType mpeg7:href mpeg7:Setting mpeg7:Tool mpeg7:name Creator creatorCS sensitivity mpeg7:value 2001-09-20T03:20:25+09:00 0.5 mpeg7:PublicIdentifier mpeg7:TermUseType completeDescriptionExample mpeg7:Name Wizzo Extracto ver. 2 mpeg7:Rights 098f2470-bae0-11cd-b579-08002b30bfeb mpeg7:Version mpeg7:RightsType mpeg7:RightsID RID# 1.1 Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Contents Introduction Ontological framework Integration framework Integration & Retrieval Scenario Conclusions Future Work Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Integration & Retrieval Scenario Low-level descriptors from audio track signal: <http://example.org/track#1> a simac:Track; simac:tempo “122”; musicbrainz:trmid “e3c41bc1-4fdc-4ccd-a471-243a0596518f”. Track editorial metadata: <http://example.org/track#2> a musicbrainz:Track; dc:title “Blowin’ in the wind”; dc:author [musicbrainz:sortName “Bob Dylan”]; musicbrainz:trmid “e3c41bc1-4fdc-4ccd-a471-243a0596518f”. Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Integration & Retrieval Scenario Generic MPEG-7 based integration rules. Example: audio track with the same fingerprint are the same individual, their metadata is merged. mpeg7:AudioType(track1) mpeg7:AudioType(track2) musicbrainz:trmid(track1, trm1) musicbrainz:trmid(track2, trm2) (trm1 = trm2) owl:sameIndividualAs(track1, track2) Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Integration Framework Context-based metadata RDF XML Audio Video Signal Processing Content-based metadata MPEG-7 XML MPEG-7 Integration XML2RDF Ontology XSD2OWL SWRL Engine XMLSchemas: MPEG-7... RDF DL Classifier Another Ontology Higher-level metadata RDFS / OWL: MusicBrainz... Semantic Integration and Retrieval of Multimedia Metadata Retrieval Rhizomik Integration & Retrieval Scenario Transparent retrieval, MPEG-7 based queries. RDQL example: SELECT ?title WHERE (?track,<rdf:type>,mpg7:AudioSegmentType), (?track,<mpeg7:Title>,?title), (?track,<mpeg7:Creator>,?author), (?author,<mpeg7:Name>,“Bob Dylan”), (?track,<mpeg7:Beat>,?tempo) AND (?tempo >= 120) USING mpeg7 FOR <http://rhizomik.upf.edu/ontologies/2005/03/Mpeg7-2001.owl#> Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Contents Introduction Ontological framework Integration framework Integration & Retrieval Scenario Conclusions Future Work Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Integration Framework Context-based metadata RDF XML Audio Video Signal Processing Content-based metadata MPEG-7 XML MPEG-7 Integration XML2RDF Ontology XSD2OWL SWRL Engine XMLSchemas: MPEG-7... RDF DL Classifier Another Ontology Higher-level metadata RDFS / OWL: MusicBrainz... Semantic Integration and Retrieval of Multimedia Metadata Retrieval Rhizomik Conclusions MPEG-7 OWL ontology, an upper-level multimedia ontology for: Semantic metadata integration align ontologies to MPEG-7 OWL. XML metadata integration XML2RDF. Semantic queries for retrieval generic queries based on MPEG-7. Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Contents Introduction Ontological framework Integration framework Integration & Retrieval Scenario Conclusions Future Work Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Future Work Semantics-based recommendation systems http://foafing-the-music.iua.upf.edu http://rhizomik.net Infer user-friendly metadata from low-level descriptors. mpeg7:AudioType(track) mpeg7:Beat(track, t) (t > 120) mpeg7:Loudness(track, l) (l > 0.9) genres:Rock(track) danceability(track, “high”) Semantic Integration and Retrieval of Multimedia Metadata Rhizomik Thank you for your attention Contact: [email protected] [email protected] Rhizomik