Transcript Folie 1 - British Computer Society

Making Search Personal with
NEPOMUK and Refinder
Dr Leo Sauermann, CEO
getrefinder.com
Hello, my Name is
Can’t remember things
Realizing the Semantic Desktop since 2002
[email protected]
28.11.2012
2
Making Search Personal with NEPOMUK and
Brigitte
+49 631 20575 - 0
Secretary of Prof. Andreas
Dengel, DFKI KM Lab.
Loves Skiing.
personal search….
28.11.2012
4
ok
Making Search Personal with NEPOMUK and
search ….
ok
Brigitte
+49 631 20575 - 0
Secretary of Prof. Andreas
Dengel, DFKI KM Lab.
Loves Skiing.
28.11.2012
5
Making Search Personal with NEPOMUK and
Secretary of Prof. Andreas
Dengel, DFKI KM Lab.
Loves Skiing.
How are they related?
A
lps
Secretary of Prof. Andreas
Dengel, DFKI KM Lab.
Loves Skiing.
A
lps
A
lps
Leo Sauermann
http://leobard.net
prototype: “Gnowsis”, 2003
14
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Prof Stefan Decker
http://www.stefandecker.org/
15
Name:
“Semantic Desktop”
Idea:
EU project
28.11.2012
Making Search Personal with NEPOMUK and Refinder
DFKI
http://www.dfki.de
Ansgar Bernardi
Andreas Dengel
Team of artificial intelligence scientists
16
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Mandriva Linux
http://mandriva.com
Sebastian Trüg
Stéphane Laurière
KDE-Linux operating system
17
28.11.2012
Making Search Personal with NEPOMUK and Refinder
28.11.2012
18
Making Search Personal with NEPOMUK and Refinder
…to conduct research about the
future of knowledge work, how to
empower an individual knowledge
worker to better exploit their personal
information space and to maintain
fruitful communication with others.
19
28.11.2012
Making Search Personal with NEPOMUK and Refinder
28.11.2012
21
Making Search Personal with NEPOMUK and Refinder
Desktop includes
• documents,
images, music,
videos, address
book, personal
calendar, todo
lists, note-taking
tools, emails, …
+ future stuff people
may need
http://www.flickr.com/photos/uniqueo/2667162885/
22
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Design Goals
• Make information reusable,
Detached from the application
• Link items, organize items
• Compare items,
gather them in lists
• Use information in project/task context
• Automatic, based on artificial intelligence
• Cross-plattform on KDE-Linux, Windows, MacOsX
establish open source semantic desktop standard
23
28.11.2012
Making Search Personal with NEPOMUK and Refinder
organize things
using tags and stars
28.11.2012
24
Making Search Personal with NEPOMUK and Refinder
Example: Dolphin
Assign tags, search for tags
http://www2.mandriva.com/linux/overview/
25
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Example: Dolphin
Assign tags, search for tags
http://www2.mandriva.com/linux/overview/
26
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Gwenview
app
plugin
file explorer
browser extension
api
data
data
base service
service
data
data
28
28.11.2012
Making Search Personal with NEPOMUK and Refinder
NEPOMUK Dazzler Blueprint
29
28.11.2012
Making Search Personal with NEPOMUK and Refinder
28.11.2012
30
Making Search Personal with NEPOMUK and Refinder
app
plugin
file explorer
browser extension
api
data
data
base service
service
data
data
31
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Works to integrate
all data on the web.
32
28.11.2012
Making Search Personal with NEPOMUK and Refinder
NIE Ontologies for rich data index
Create a rich index to build flexible applications on top
332 class types:
Integrated & related
contacts, calendaring, task management,
downloads, file types, generic multimedia
stuff, exif information for images, ID3 Tags
for MP3s, messages and email, file sharing,
…
personal information management: projects,
tasks, organizations, places, ….
• “Contact” is used in emails, im
messages, author of documents,
…
Reusable
• Representation in RDF, XML,
KEY/VALUE possible
866 properties of these classes:
structured data, a lot for calendar and
multimedia formats.
Standards-Based
…
• adaption of EXIF, vcard, vcal,
MPEG
http://oscaf.sourceforge.net
33
28.11.2012
Making Search Personal with NEPOMUK and Refinder
RDF Database: Virtuoso
Stores all structured data and fulltext in a searchable index
http://virtuoso.openlinksw.com/
Lorem ipsum…
RDF Graph Database
Data Database
Fulltext Database
34
28.11.2012
C
S
P
O
A
file://x
size
512
A
file://x
author
nepomuk://hh-45
Lorem ipsum dolor…
Making Search Personal with NEPOMUK and Refinder
Index the Data: crawlers
Initial indexing of the data into the store
Strigi
C++, Linux
strigi.sourceforge.net
Aperture
Java
aperture.sourceforge.net
35
28.11.2012
Making Search Personal with NEPOMUK and Refinder
28.11.2012
37
Making Search Personal with NEPOMUK and Refinder
Improving the Indexer
Initial indexing took 100% CPU time, little additional benefit over predecessor
Alan Grimes,
annoyed by 100% CPU time
taken by indexer:
• Indexer takes less CPU
and memory now.
…What the fuck is this semantic bullshit?
Computers most notoriously cannot think.
They are purely storage and processing
systems...
http://www.mail-archive.com/[email protected]/msg00538.ht
ml
38
28.11.2012
Making Search Personal with NEPOMUK and Refinder
it’s the user experience,
stupid
• More end-user
applications.
39
28.11.2012
• Multimedia
and PIM use.
Making Search Personal with NEPOMUK and Refinder
Filter in the File Explorer
Multiple facets can be used
http://trueg.wordpress.com/2010/01/26/what-we-did-last-summer-and-the-rest-of-2009-a-look-back-ontothe-nepomuk-development-year-with-an-obscenely-long-title/
40
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Semnotes
http://smile.deri.ie/projects/semn/details
41
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Semnotes
42
28.11.2012
Making Search Personal with NEPOMUK and Refinder
28.11.2012
43
Making Search Personal with NEPOMUK and Refinder
28.11.2012
44
Making Search Personal with NEPOMUK and Refinder
establish a
semantic desktop
standard
community!
28.11.2012
45
Making Search Personal with NEPOMUK and Refinder
Filter Big Bang Theory Episodes
Season 3, The Vengeance Formulation
47
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Filter Music
ATB Albums, No Silence
48
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Open files by tag
http://trueg.wordpress.com/2009/08/28/gsoc-wrap-up-part-2/
49
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Save files with tag&description
http://trueg.wordpress.com/2009/08/28/gsoc-wrap-up-part-2/
50
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Multimedia Use Cases
Beyond TV Shows
I'm using Nepomuk extensively everyday,
…, to manage all my media, with the help
of Nepoogle, Drop2TagIcon and
Neposideckick, and results are amazing.
Things like locate duplicate files or
manage TVShows are only some
samples of the capabilities of Nepomuk
at desktop level.
…I have 2133 video files … 35 are
related to the actress 竹内結子 (Takeuchi
Yūko), 11 files released in year 2012, and
3 have in common actor 阿部寛 (Abe
Hiroshi).
• Synonyms,
multi-linguality
• Relations between
files and actors
• Actors and their relations are copied
from web-databases and stored on
semantic desktop to enhance local
queries
http://forum.kde.org/viewtopic.php?f=15&
t=98468&sid=f90bf929b2f1faae2540e053
bab6f904&start=15#p221521
51
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Zeitgeist
52
28.11.2012
http://zeitgeist-project.com/experience/
http://trueg.wordpress.com/2011/06/06/randa-and-ontologies-and-whatnot/
Plans to integrate Zeitgeist and NEPOMUK for KDE
Making Search Personal with NEPOMUK and Refinder
Finding duplicate files
• select distinct ?u1 ?u2 where { ?f1 a nexif:Photo .
?f2 a nexif:Photo . ?f1 nfo:hasHash ?h . ?f2
nfo:hasHash ?h . ?f1 nie:url ?u1 . ?f2 nie:url ?u2 .
filter(?f1!=?f2) . }
http://trueg.wordpress.com/2011/12/06/finding-duplicate-images-made-easy/
53
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Filter & Drill down on data
Virtuoso RDF Store
SELECT * WHERE {
?s nco:fullname ?name .
?name bif:contains
"'brigitte*'".
Brigitte
}
Structured queries
Fulltext queries
Inference Engine: rules, ontologies, generates new facts
54
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Answer complex questions
Through natural language processing, full-text analysis and the index of relations
“Whats the phone
number of the KM
Group Secretary?”
 Not possible to answer using fulltext search.
Only SIRI or Watson have the superpowers to…
55
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Map the query to the graph
The answer is found by spreading activation and partial queries
“phone number of the KM-Group secretary”
Property <phone_number>
Instance <KM-Group>
[1]
(no result)
Class <secretary>
[2]
instances:
<SecretaryAD>, …
(no result)
[3]
[4]
<KM-Group><unit><SecretaryAD>
<KM-Group><unit><SecretaryAD>
<SecretaryAD><phone_number> „0049 631 205 75 101“
[5]
Combining Metadata and Document Search with Spreading Activation for Semantic Desktop Search,
Schuhmacher, K.; Sintek, M. & Sauermann, L. ESWC 2008
56
28.11.2012
Making Search Personal with NEPOMUK and Refinder
PIMO to build a personal network
Project
• Concetps &
Associations
to organize
Person
Andreas
• Connected
with documents
• Inspired by
basic human
concepts
nepomuk
Location
Meeting
Kaiserslautern
Kickoff
Brigitte
Contacts
Andreas Dengel
Brigitte Selzer
Sven Schwarz
PIMO - a Framework for Representing Personal Information Models
http://www.dfki.uni-kl.de/~sauermann/papers/sauermann+2007b.pdf
Files
E-Mails
Documents
Papers
Projects
nepomuk
Research
Inbox
Todo
SAP
nepo
Karlsruhe
Supports Orienteering Behavior
“users go somewhere, look around,
know where they are, go on”
Filtering
Long-term users stopped
using web-search interface
keeping orientation
Teevan, J.; Alvarado, C.; Ackerman, M.S. & Karger, D.R.
The perfect search engine is not enough: a study of orienteering behavior in directed search
CHI 2004, pp.415-422
58
16.07.2015
Semantic Desktop for PIM
NOKIA
Search, on the phone
„Brings the power of computers
to mobile devices“
• N900 and N810 are Maemo Linux-powered
• Nokia M9 Phone, millions shipped
• Tracker indexing framework
maemo.nokia.com
www.oscaf.org/node/20
www.grancanariadesktopsummit.org/node/170
59
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Philip Van Hoof, 22.11.2012
Last week
inter-app information sharing on a high level
using a inter-app defined and shared ontology
with a query language suitable for the purpose,
SPARQL, combined with a efficient file system
meta data indexer. We delivered that and proof
that its high quality software exists in the fact
that a few million N9 users, yes a few million,
are using it right now and yet there are no
massive protests on forums about corrupted
data: Tracker's SPARQL endpoint is today
tested and robust high quality software. I salute
Jürg [Billeter] for that.
mail.gnome.org/archives/tracker-list/2012-November/msg00009.html
60
28.11.2012
Making Search Personal with NEPOMUK and Refinder
Nepomuk,
in the cloud
28.11.2012
61
Making Search Personal with NEPOMUK and Refinder
Connect and Crawl
Scanning of any cloud app.
Connectors:
+
62
November 2012
Dropbox
Evernote
Contacts
RSS
Twitter
Docs
Basecamp
Box.net
SkyDrive
Campfire
MindMeister
Calendar
Refinder Cloud Search
Scalable, Full-text, Sorting, Faceting, Filtering, Highlighting,
customizable Ranking.
Scalable full-text and
phrase search and/or
term conjunction
Sorting and faceting on
any item attribute
Customizable
relevancy ranking
Results including item
attributes, result
highlighting, previews,
and filter options
63
November 2012
Refinder Cloud Search
More cloud apps, growing data, small screenspace
Desktop Data  Personal Data
Desktop Apps  Cloud Apps
64
November 2012
Timeline of NEPOMUK
and adoption rates
developers
Refinder
Nokia maemo & MeeGo
Aperture
NEPOMUK & Gnome
NEPOMUK-KDE 4.0
EU-IST Research
2006
65
2007
28.11.2012
2008
2009
2010
2011
users
4 guys
3k
~100s
0.6 mio
~10
25k
~10
million
~10
millions
~60
100
2012
Making Search Personal with NEPOMUK and Refinder
Join!
nepomuk-kde mailinglist
[email protected]
https://mail.kde.org/mailman/listinfo/nepomuk
28.11.2012
66
Making Search Personal with NEPOMUK and Refinder
I am looking for a partner to
scale Refinder up
[email protected]
+4369914669747
nepomuk-kde mailinglist
28.11.2012
67
Making Search Personal with NEPOMUK and Refinder