Transcript TITLE
Affect in games CIG 2012 tutorial Kostas Karpouzis National Technical University of Athens [email protected] what I’ll talk about • Concepts of affect/emotion/behaviour – (hopefully!) related to games • Engagement/attention/flow • How to detect/make sense of them • Modelling/detecting player experience and sattisfaction – and adapting to what I won’t talk about • Designing affective NPCs – which show happiness, sadness, contempt, etc. • Affect and in-game behaviour – other than attention/engagement – e.g. fatigue and game play, how happy/sad players perform, etc. • Particular affect detection techniques – But we can discuss those later yours truly • Post-doc in Humaine Network of Excellence • Since: – CALLAS (emotion in arts and entertainment) – Feelix Growing (affect-aware robots) • Currently: – Siren (serious games for conflict resolution in school environments) – ILearnRW (serious games for children with dyslexia and dysorthographia) • Also: – Humaine Games SIG (w/ G. Yannakakis), IEEE TAC SI (Affect in Games) – IEEE Aff. Comp TF, IEEE CIS Games TC TF on Education affective games • Toys provide our first human-“machine” interface! • Games are everywhere these days! – Computers, consoles/TV, mobile phones – Browser and Facebook/Google+ do not require installation – ‘Freemium’ business model attractive to both gamers and developers – Huge interest in industry and academia • Lots of funding from EU projects affective games • Constrained environment easier to track people • Novel modalities: gestures, body movement, speech (lexical and prosody) – Ideally associated with what happens in the game • (Possibly) novel interaction paradigms, i.e. not WIMP affective gamers or (if things go wrong)… affective gamers let’s make use of it a bit of theory theories of emotion terminology • Emotions, mood, personality • Can be distinguished by – time (short-term vs. long-term) – influence (unnoticed vs. dominant) – cause (specific vs. diffuse) • Affect classified by time – short-term: emotions (dominant, specific) – medium-term: moods (unnoticed, diffuse) – and long-term: personality (dominant) terminology • what we perceive is the expressed emotion at a given time – on top of a person’s current mood, which may change over time, but not drastically – and on top of their personality • usually considered a base line level • which may differ from what a person feels – e.g. we despise someone, but are forced to be polite terminology • Affect is an innately structured, non-cognitive evaluative sensation that may or may not register in consciousness • Feeling is defined as affect made conscious, possessing an evaluative capacity that is not only physiologically based, but that is often also psychologically oriented. • Emotion is psychosocially constructed, dramatized feeling how it all started • Charles Darwin, 1872 • Ekman et al. since the 60s • Mayer and Salovey, papers on emotional intelligence, 90s • Goleman’s book: Emotional Intelligence: Why It Can Matter More Than IQ • Picard’s book: Affective Computing, 1997 why emotions? • “Shallow” improvement of subjective experience • Reason about emotions of others – To improve usability – Get a handle on another aspect of the "human world" – Affective user modeling – Basis for adapting experience to users name that emotion • so, we know what we’re after – but we have to assign it a name – in which we all agree upon – and means the same thing for all (most?) of us • different emotion representations – different context – different applications – different conditions/environments emotion representations • most obvious: labels – people use them in everyday life – ‘happy’, ‘sad’, ‘ironic’, etc. – may be extended to include user states, e.g. ‘tired’, which are not emotions – CS people like them • good match for classification algorithms labels • but… – we have to agree on a finite set • if we don’t, we’ll have to change the structure of our neural nets with each new label – labels don’t work well with measurements • is ‘joy’ << ‘exhilaration’ and in what scale? • do scales mean the same to the expresser and all perceivers? labels • Ekman’s set is the most popular – ‘anger’, ‘disgust’, ‘fear’, ‘joy’, ‘sadness’, and ‘surprise’ – added ‘contempt’ in the process • Main difference to other sets of labels: – universally recognizable across cultures – when confronted with a smile, all people will recognize ‘joy’ from labels to machine learning • when reading the claim that ‘there are six facial expressions recognized universally across cultures’… • …CS people misunderstood, causing a whole lot of issues that still dominate the field strike #1 • ‘we can only recognize these six expressions’ • as a result, all video databases used to contain images of sad, angry, happy or fearful people • a while later, the same authors discussed ‘contempt’ as a possible universal, but CS people weren’t listening strike #2 • ‘only these six expressions exist in human expressivity’ • as a result, more sad, angry, happy or fearful people, even when data involved HCI – can you really be afraid when using your computer? strike #3 • ‘we can only recognize extreme emotions’ • now, happy people grin, sad people cry or are scared to death when afraid • however, extreme emotions are scarce in everyday life – so, subtle emotions and additional labels were out of the picture labels are good, but… • don’t cover subtle emotions and natural expressivity – more emotions are available in everyday life and usually masked – hence the need for alternative emotion representations • can’t approach dynamics • can’t approach magnitude – extreme joy is not defined other sets of labels • Plutchik – Acceptance, anger, anticipation, disgust, joy, fear, sadness, surprise – Relation to adaptive biological processes • Frijda – Desire, happiness, interest, surprise, wonder, sorrow – Forms of action readiness • Izard – Anger, contempt, disgust, distress, fear, guilt, interest, joy, shame, surprise going 2D • vertical: activation (active/passive) • horiz.: evaluation (negative/positive) going 2D • emotions correspond to points in 2D space • evidence that some vector operations are valid, e.g. ‘fear’ + ‘sadness’ = ‘despair’ going 2D • quadrants useful in some applications – e.g. detect extreme emotions during team comms in FPSs going 3D • Plutchik adds another dimension • vertical intensity, circle degrees of similarity – four pairs of opposites going 3D • Mehrabian considers pleasure, arousal and dominance • Again, emotions are points in space what about interaction? • these models describe the emotional state of the user • no insight as to what happened, why the user reacted and how the user will react – action selection • OCC (Ortony, Clore, Collins) • Scherer’s appraisal checks OCC (Ortony, Clore, Collins) • each event, agent and object has properties – used to predict the final outcome/expressed emotion/action summary on emotion • perceived emotions are usually short-lasting events across modalities • labels and dimensions are used to annotate perceived emotions – pros and cons for each • additional requirements for interactive applications a bit of theory the quest for Flow (and fun) a theory of fun • Raph Koster – lead designer of Ultima Online – creative director of Star Wars Galaxies – http://www.theoryoffun.com /theoryoffun.pdf – http://www.raphkoster.com a theory of fun a theory of fun • ‘We talk so much about emergent gameplay, non-linear storytelling, or about playerentered content. They’re all ways of increasing the possibility space, making self-refreshing puzzles’ • So, what is it that makes a game ‘fun’? – ever-lasting challenge • How can we keep challenges coming? the concept of Flow – a state of concentration or complete absorption with the activity at hand and the situation. It is a state in which people are so involved in an activity that nothing else seems to matter (Csikszentmihalyi,1990) – “Being completely involved in an activity for its own sake. The ego falls away. Time flies. Every action, movement, and thought follows inevitably from the previous one, like playing jazz. Your whole being is involved, and you're using your skills to the utmost” flow revisited • the ‘holy grail’ of game design • just the right amount of challenge • making a game very hardgamers quit • making a game very easygamers bored flow revisited • it’s not about the graphics • or the controller • or the franchise (e.g. sports games) • just ask Rovio – makers of Angry Birds – $80M/yr, 600M dl’s flow revisited • ‘smart’ games adapt to player skill and engagement • keeping them coming back for more • at the end of the day… back to the drawing board • what can we model? – and how? • definition of ‘affective computing’ – ‘affective computing is computing that relates to, arises from, or deliberately influences emotion or other affective phenomena’ -- Roz Picard, 1995 – ‘a set of observable manifestations of a subjectively experienced emotion’ -- MerriamWebster’s dictionary observable manifestations observable manifestations hypothesis • ‘shallow’ treatment – i.e. not as far as ‘personality’, sticking to ‘affect’ • identify/track user reactions – facial expressions and gestures, body movements and stance, hand and body expressivity (for whole-body interaction) • relate those to events in the game hypothesis • ideally, we could identify the players’ stress level (via the ‘observable manifestations’) and their skill level (via their performance) • and cluster those to identify player types – for the particular game genre! • or use them to adapt the game – make it easier for players ‘in distress’ – or harder for players in the verge of boredom hypothesis • why bother with both affect and performance? • why are players standing still? – is it flow (immersion) or boredom? • or why do they move around? – is it immersion (e.g. in a racing game) or lack of engagement? • remember: Flow skill AND engagement generating content • we want to make games harder or easier to match player skill • predefined levels (e.g. ‘easy’ / ‘expert’) • still we have to define what ‘easy’ means • data-driven activity! TGFKASM generating content • game level as a multiparameter function produce game content procedurally • e.g. number and size of gaps, number of opponents, etc. • Multimodal player affect/satisfaction challenge in forthcoming CIGs in a nutshell • games provide an ideal medium to induce and capture affective interactions • well-designed games bring out different (and valuable!) reactions from players • gaming is a multi-faceted activity – thus, player models are usually detailed • player affect tells us a lot about the game lessons learned the Siren project conflict resolution games • Siren aims to produce a conflict resolution serious game – for 10-14 y.o. children – in school environments conflict resolution games The life cycle of conflict (Swanstrom and Weissmann, 2005) conflict resolution games • during escalation, negative emotions are present • cannot use neg. emotions to indicate stress adaptation conflict resolution games • rather, use estimated emotion to identify where players are in this figure (which phase) conflict resolution games • and produce content to ‘push’ users towards de-escalation • learning objective of the game! conflict resolution games • sensed affect can be used to identify player performance – i.e. whether players actually ‘move’ towards resolving the conflict • but which emotions are relevant? • negative vs positive • is that enough for all game genres? in a nutshell • player affect is genre-dependent • reflects many qualities from the user model • many open research questions – single- vs multi-player • easy to find people to play games – yay! thank you! Kostas Karpouzis [email protected]