Transcript Tracking context with usage and attention metadata in

Tracking
context
with
usage and attention metadata
in multilingual
Technology Enhanced Learning
Riina Vuorikari
European Schoolnet /
Open Univ. of the Netherlands
Bettina Berendt
KU Leuven
www.berendt.de
„Learning Object Repositories
foster re-use and improve
learning“
 Really?
 Ochoa (2008), Vuorikari & Koper (2009):
re-use ~ 20%
A basic challenge: context
Researching
for a
term paper
in ...
Biology
Political Science
Motivation
(application goals)
 Use case
 Technology Enhanced Learning / Learning Object Repositories
 teachers from different linguistic and country backgrounds
 Goal: foster re-use of resources
 Assumption: understanding user context can improve
system behaviour
 Approach:
 consider UAM as part of context measurement, specifically:
 Investigate different roles of social tagging
 Profit from parallel research in different areas
Motivation
(general - workshop)
 To exploit usage and attention metadata:
need to
 define what properties of usage to use and
why
 how to gather data on that property
 what to do with that information
Motivation
(general – our work)
 Automatically gathered indicators of context,
attention, ...
 What‘s common across fields?
 How can we communicate better?
 What‘s specific in T.E. learning?
 How can tags help learning & teaching?
 How to bridge (e.g. lingual) barriers?
 What are the best research methods for
answering all these questions?
Types of context
in TEL
What is context?
 “Context is any information that can be used to
characterize the situation of an entity. An entity is a
person, place, or object that is considered relevant
to the interaction between a user and an
application, including the user and applications
themselves.” (Dey, 2001)
 Any information not explicitly carried by the
“surface level“ of an interaction with a
computational system
 Surface level:
 user issues a query,
 user accesses a resource, ...
 We will distinguish between
 Macro-context
 Micro-context
Macro-context





educational level
formal and informal learning
delivery setting (distance, blended, ...)
intended users and user roles
...
Micro-context
 User
 trait
 state
 Interaction
 atomic vs. activity structure
 implicit vs. explicit interest indicators
 Background knowledge / „semantic
enrichment“; often w.r.t. material
(Berendt, 2007)
Tags and context
What‘s in a tag?
item
user
• material
• user
• interaction
• „writing“
• „reading“
tag
Background
knowledge
Tags as interest indicators (1) –
or: measuring and using
interaction-context
 Comparative search efficiency:
 Search option [A|B|...]  a target LO
 Click-through rate aka confidence (fraction of successful
searches)
 Path length
 In LOR: Search with ...
 explicit search (for resources w/o tags or ratings): 4.4 searches
/ resource
 community browsing (tagclouds, lists, pivot browsing): 3.9
searches / resources
 social information (community browsing or explict search with
interest indicators (tags, ratings)): 2.8 searches / resource
 Recommendation of search options?!
Tags provide context to better
utilize background knowledge
 Can tags be used for enriching existing metadata of
educational resources in a multilingual context?
 30% of tag applications matched with descriptors from
a multilingual thesaurus, which had also been used to
index these learning resources (Vuorikari et al., 2009)
 “Thesaurus tags”
 Thesaurus tags provide for
 adding properties to tags (e.g., relation to a concept in a
multilingual thesaurus, language)
 linking resources to multilingual thesaurus descriptors
 can support retrieval of resources in a multilingual context
 Cf. convergence and quality tendencies in tagging in
general (Bollen & Halpin, 2009; Hayes et al., 2007)
What is user context – the
case of language
 User models have many different variables
 Language (first language, further languages,
proficiency, preferences): an important
descriptor of users
 Language situation (interaction in a first or
second language): an important descriptor
 Linguistic trait and state context variables!
Measures / indicators of
user language
 self-profiling that explicitly addresses the question
(“your mother tongue?”)
 IP address
 browser settings
 language of the currently used interface
 language of search terms and tags
 known or inferred language of tagged resources
 NB: indicator of the language situation: compare user
language with interface/resource language (e.g.,
Berendt & Kralisch, 2009)
Tags as interest indicators (2) –
or: tags for measuring (re-)use
 bookmarking and personal collections of digital
learning resources as a proxy for the use and
re-use of resources
 When coupled with user and resource location
/ language:
 become proxy for (re-)use across national and
language boundaries
(e.g., Vuorikari & Koper, 2009)
Why support trans-lingual reuse? (1): Situation on the Web
 What would be an „ideal“ language situation on
the Web?
 One H0: ~ amount of materials proportional to
number of first-language speakers of a language
 Not the case – non-English languages are
severely under-represented on the Web due to




Resource-creation behaviour
Link-setting behaviour
Link-following behaviour
Attitudes towards resources in different languages
(Berendt & Kralisch, 2009)
Why support trans-lingual reuse? (2): User preferences
 Theoretical reasons (cognitive effort), supported by
implicit and explicit user preferences:
 Clear tendency to access materials in one‘s mother
tongue in a medical information system
 17% of users of a LOR had saved only content in their
native language in their Favourites
 LORs: pragmatic reasons (curricula)
 Search-engine give better results when queried with
linguistically correct (rather than Latinized) spellings
 But: individual differences depending on proficiency in
English!
(Berendt & Kralisch, 2009; Vuorikari & Koper, 2009; Blanco & Lioma,
2009)
Using context for improving
trans-lingual re-use: How? (1)
 better-organised search result lists
 first a ranked list in the preferred language,
 then a ranked list in a second language, etc.
 done: Google for general search results and
by LeMill (lemill.net) for learning resources.
 organise tag clouds by language or
country as in LRE (lreforschools.eun.iorg)
Using context for improving
trans-lingual re-use: How? (2)
 for users with weaker language preferences; for
„unknown“ users; ...“:
 Make exceptions to the foregoing by recommendations
using
 travel-well tags:
 terms with the same or similar spelling in most languages
 ex.: technical terms like “mathematics”, place and person names
 How to identify them?
 via multi-lingual thesaurus or similar
 tag has been assigned by users from different languages
 Tag has been assigned to resources in different languages
 travel-well resources:
 Have been bookmarked by people from different languages,
country contexts, …
(Vuorikari & Ochoa, 2009)
Using context for improving
trans-lingual re-use: How? (3)
 Direct recommendations 1 (see above):
 Recommend content or tags in a preferred language
 Indirect recommendations:
 E.g., recommend bookmark lists of other users with a similar
“language preference profile”
 Profile similarity: “extensional” and/or “intensional”: degree of
tolerance for mixed-language resources, results, etc.
 Direct recommendations 2:
 repositories could specifically encourage users who are
competent in “smaller” languages also to author content in
their language.
Outlook
Some questions
... and now yours! (Thx!)
 How to best couple „content“ and „architecture“
research
 How to bring together research from Web analytics / eCommerce, search-engine research, digital libraries,
TEL, ...
 How to complement the exploratory research with
experimental research effectively and efficiently
 How to best combine different methods (validate that
the dependent variables used mean something!)
 Which privacy/security questions arise?
 What is specific for TEL?
References
 Most references can be found in the paper, available at
www.cs.kuleuven.be/~berendt/Papers/TEL_context_attention_vuorikari_berendt.pdf
 In addition:
 Bollen, D., & Halpin, H. (2009). An Experimental Analysis of
Suggestions in Collaborative Tagging. In Proc. Of WI-IAT’09, Milan,
Italy, 15-18 Sept 2009. IEEE Computer Society Press.
 Hayes, C. et al. (2007). ... In Berendt, B., Hotho, A., Mladeni\v{c}, D., &
Semeraro, G. (Eds.) (2007). From Web to Social Web: Discovering
and deploying user and content profiles. Workshop on Web Mining,
WebMine 2006, Berlin, Germany, September 18, 2006, Revised
Selected and Invited Papers. LNAI 4737. Berlin etc.: Springer.