Charismatic Speech CS 4706

What is Charisma ?

• The ability to attract, and retain followers by virtue of personal characteristics -- not traditional or political office (Weber ‘47) – E.g. Gandhi, Hitler, Castro, Martin Luther King Jr.,..

–

Personalismo

• What makes an individual Tuppen ’74, Weber ‘47) charismatic ? (Bird ’93, Boss ’76, Dowis ’00, Marcus ’67, Touati ’93, – Their message?

– Their personality?

– Their speaking style?

What is Charismatic Speech?

• Circularly… – Speech that leads listeners to perceive the speaker as charismatic • What aspects of speech might contribute to the perception of a speaker as charismatic ?

– Content of the message?

– Lexico-syntactic features?

– Acoustic-prosodic features?

Why Study Charismatic Speech?

• It’s an interesting phenomenon • To identify potential charismatic leaders • To provide a feedback system for individuals who want to improve their speaking style -- politicians, professors, students… • To create a charismatic Text-to-Speech system

Our Approach

• Collect tokens of charismatic and non charismatic speech from a small set of speakers on a small set of topics • Ask listeners to rate the ‘ The speaker is charismatic ’ plus statements about a number of other attributes (e.g. The speaker is …

boring, charming, persuasive,…

) • Correlate listener ratings with lexico-syntactic and acoustic-prosodic features of the tokens to identify potential cues to perception of charisma

American English Perception Study

• Data: 45 2-30s speech segments, 5 each from 9 candidates for Democratic nomination for U.S. president in 2004 – 2 ‘ charismatic ’, 2 ‘

not

charismatic ’ – Topics: greeting, reasons for running, tax cuts, postwar Iraq, healthcare – 4 genres: stump speeches, debates, interviews, ads • 8 subjects rated each segment on a Likert scale (1 5) for 26 questions in a web survey • Duration: avg. 1.5 hrs, min 45m, max ~3hrs

Results: How Much Do Subjects Agree with Each Other?

• Over all statements?

– Using weighted kappa statistic with quadratic weighting, mean  = 0.207

• On the charismatic •  = 0.232 (8 th statement? most agreed upon statement) • By token?

– No significant differences across all tokens • By statement?

– Individual statements demonstrate significantly different agreements (most agreement:

The speaker is accusatory, angry, passionate, intense;

least agreement:

The speaker is trustworthy, believable, reasonable, trustworthy

)

Results: What Do Subjects Mean by Charismatic ?

• Which

other

statements are most closely correlated with the charismatic statement ? (determined by kappa): a

functional

definition The speaker is enthusiastic The speaker is persuasive The speaker is charming The speaker is passionate The speaker is boring The speaker is convincing 0.620

0.577

0.575

0.543

-0.513

0.499

Results: Does Whether a Subject Agrees with the Speaker or Finds the Speaker ‘Clear’ Affect Charisma Judgments

• Whether a subject agrees with a token does

not

correlate highly with charisma judgments (  = 0.30) • Whether a subject finds the token clear does

not

correlate highly with charisma judgments (  = 0.26)

Results: Does the Identity of the Speaker Affect Judgments of Charisma ?

• There is a significant difference between speakers (p=2.20e-2) • Most charismatic – Rep. John Edwards (mean 3.86) – Rev. Al Sharpton (3.56) – Gov. Howard Dean (3.40) • Least charismatic – Sen. Joseph Lieberman (2.42) – Rep. Dennis Kucinich (2.65) – Rep. Richard Gephardt (2.93)

Results: Does Recognizing a Speaker Affect Judgments of Charisma ?

• Subjects asked to identify which, if any, speakers they recognized at the end of the study.

• Mean number of speakers believed to have been recognized, 5.8

• Subjects rated ‘recognized’ speakers as significantly more charismatic than those they did not (mean 3.39 vs. mean 3.30).

Results: Does Genre or Topic Affect Judgments of Charisma ?

• Recall that tokens were taken from debates, interviews, stump speeches, and campaign ads – Genre

does

influence charisma ratings (p=.0004) – Stump speeches were the most charismatic (3.38) – Interviews were the least (2.96) • Topic

does

affect ratings of charisma significantly (p=.0517) – Healthcare > post-war Iraq > reasons for running neutral > taxes

What makes Speech Charismatic ?

Features Examined

• • • Duration (secs, words, syls)

Charismatic speech is personal

: Pronoun density

Charismatic speech is contentful

: Function/content word ratio •

Charismatic speech is simple

: Complexity: mean syllables/word (Dowis) • Disfluencies • Repeated words • Min, max, mean, stdev F0 (Boss, Tuppen) – Raw and normalized by speaker • Min, max, mean, stdev intensity • Speaking rate (syls/sec) • Intonational features: – Pitch accents – Phrasal tones – Contours

Results: Lexico-Syntactic Correlates of Charisma

• Length: Greater number of words

positively

correlates with charisma (r=.13; p=.002) • Personal pronouns: – Density of first person plural and third person singular pronouns

positively

correlates with charisma (r=.16, p=0; r=.16, p=0) – Third person plural pronoun density correlates

negatively

with charisma (r=-.19,p=0) • Content: Ratio of adjectives/all words

negatively

correlates with charisma (r=-.12,p=.008) • Complexity: Higher mean syllables/word

positively

correlates with charisma (p=.034)

• Disfluency: greater %

negatively

correlates with charisma (r=-.18, p=0) • Repetition: Proportion of repeated words

positively

correlates with charisma (r=.12, p=.004)

Results: Acoustic-Prosodic Correlates of Charisma

• Pitch: – Higher F0 (mean, min, mean HiF0, over male speakers)

positively

correlates with charisma (r=.24,p=0;r=.14 p=0;r=.20,p=0) • Loudness: Mean rms and sdev of mean rms

positively

correlates with charisma (r=.21,p=0;r=.21,p=0) • Speaking Rate: – Faster overall rate (voice/unvoiced frames)

positively

correlates with charisma (r=.16,p=0)

• Duration: Longer duration correlates positively with charisma (r=.09,p=.037) • Length of pause: sdev

negatively

correlates with charisma (r=-.09,p=.004)

Results: Intonational Correlates of Charisma (Hand-Annotated Features)

• Pitch Accent Type: –

Positive

correlation with !H* and L+H* accents (r=.09,p=0;r=.09,p=.034) –

Negative

correlation with L*, H* and L*+H accents (r=-.13,p=.002;r=-.11,p=.014;r=-.08,p=.052) • Phrasal Types –

Negative

correlation with !H-L% and !H- endings (r= .11,p=.015;r=-.10,p=.026)

Summary for American English

• In Standard American English, charismatic speakers tend to be those also highly rated for enthusiasm, charm, persuasiveness, passionateness and convincingness – they are

not

thought to be boring • Charismatic utterances tend to be

longer

than others, to contain a

lower ratio of adjectives to all words

, a

higher density of first person plural and third person singular pronouns and fewer third person plurals

,

fewer disfluencies

, a

larger percentage of repeated words

, and

more complex words

than non-charismatic utterances

• Charismatic utterances are

higher in pitch

(mean, min) with

more regularity in pause length

,

louder

with more

variation in intensity

,

faster

, and with

more !H* and L+H* accents

and

fewer L*, H*, and L*+H accents and fewer !H- and !H-L% phrasal endings

Replication of Perception Study from Text Alone

• Lower statement agreement, much less on charismatic statement , different speakers most/least charismatic • `Agreement with speaker’, genre and topic had stronger correlations • Lexico-syntactic features show weaker correlations – 1 st person pronoun density complexity not at all

negatively

correlated and – Similar to speech experiment for duration, function/content, disfluencies, repeated words

Hypothesis: Charisma is a Culture-Dependent Phenomenon

• People of different languages and cultures perceive charisma differently • In particular, they perceive charisma in speech differently – Do Arabic listeners respond to American politicians the same way Americans do?

– Do Americans hear Swedish professors the same way Swedish students do?

Charismatic Speech in Palestinian Arabic

• Are these tokens charismatic ?: • Are these?:

Palestinian Arabic Perception Study

• Same paradigm as for SAE • Materials: – 44 speech tokens from 22 male native-Palestinian Arabic speakers taken from Al-Jazeera TV talk shows – Two speech segments extracted for each speaker from the same topic (one we thought not) charismatic and one • Web form with statements to be rated translated into Arabic • Subjects: 12 native speakers of Palestinian Arabic

How Does Charisma Differ in Arabic?

• Subjects agree on judgments a bit more (κ=.225) than for English (κ=.207) but still low – Agree most on clarity of msg, enthusiasm, charisma, intensity – all differing from Americans – Agree least on desperation (as Amer) , friendliness , ordinariness , spontaneity of speaker – Charisma statement correlates (

positively

) most strongly with speaker toughness , powerfulness , persuasiveness , charm , and enthusiasm and

negatively

with boringness

• Role of speaker identity important in judgments of charisma in Arabic as in English – Most charismatic speakers: Ibrahim Hamami (4.75), Azmi Bishara (4.42), Mustafa Barghouti (4.33) – Least: Shafiq Al-Hoot (3.10), Mohammed Al-Tamini (3.42), Azzam Al-Ahmad (3.33) – Raters claimed to recognize only .55 (of 22) speakers on average, perhaps because the speakers were less well known than the Americans • Topic important in charisma ratings (r=0,p=.043) Israeli separation wall > assassination of Hamas leader > debates among Palestinian groups > the Palestinian Authority and calls for reform > the Intifada and resistance

Lexical Cues to Charisma

• Length in words

positively

charisma , as for Americans correlates with • Disfluency rate Americans

negatively

correlates, as for • Repeated words

positively

charisma , as for Americans correlates with • Presence of Arabic ‘ dialect markers ’ (words, pronunciations) charisma

negatively

correlates with • Density of third person plural pronouns correlates w/ charisma

positively

– differing from Americans

Acoustic/Prosodic Cues to Charisma

• Duration

positively

for Americans correlated with charisma , as • Speaking rate

approaches negative

opposite from American correlation – – But rate of the fastest intonational phrase in the token

positively

correlated for both languages – Sdev of rate across intonational phrases

positively

correlated for charisma in Arabic • Pauses – #pauses/words ratio

positively

but not for Americans correlated with charisma

– Sdev of length of pause

positively

correlated in Arabic but negatively for Americans • Pitch: – Mean pitch

positively

correlates (as for Americans) but also F0 max and sdev – Min pitch

negatively

correlates (opposite from Americans) • Intensity: Sdev

positively

correlates w/ charisma

How Are Perceptions of Charisma Similar Across Cultures?

• Level of subject agreement on statements • Role of speaker ID, topic in charisma judgments • Positive correlations with charisma – Mean pitch and range – Duration, repeated words – Speaking rate of fastest IP • Negative correlations with charisma – Disfluencies

How Do Charisma Judgments Differ Across Cultures?

• Statements most and least agreed upon • For Arabic vs. English: – Positive correlations with charisma • Sdev of speaking rate, pause/word ratio, sdev of pause length, F0 max and sdev, sdev intensity – Negative correlations with charisma • Dialect, density of third person plural pronouns • Speaking rate, min F0

Future Work

• Machine learning experiments -- automatic detection of charisma • Cross-cultural perception experiments: American raters/Arabic speech, Palestinian raters/English speech, Swedish raters/English speech – Do native and non-native raters differ on mean scores per token? ( Yes , for Eng/Swe rating Eng and Eng/Pal rating Arabic) – Do mean scores correlate per token? ( Yes , for all)

• Amer and Swe rating English: – paired t-test betw means per token: p-value = 0.03064

– cor between means of rater-normalized ratings: r = 0.60

, p-value = 1.170e-05 • Amer and Pal rating English: – paired t-test betw means: p-value = 0.1048

– cor between means of rater-normalized ratings: r = 0.47

, p-value = 0.0009849 • Amer and Pal rating Arabic: – paired t-test betw means: p-value = 0.00164

– cor between means of rater-normalized ratings: r = 0.72

, p-value = 3.049e-08 • Swe and Pal rating English: – paired t-test betw means: p-value = 0.8479 (not normalized) – cor between means of rater-normalized ratings: (rater normalization) r = 0.55

, p-value = 9.467e-05

Arabic Prosodic Phenomena MSA vs. Dialect

• A word is considered dialectal if: – It does not exist in the standard Arabic lexicon – It does not satisfy the MSA morphotactic constraints – Phonetically different (e.g.,

ya?kul

vs.

ywkil

) • In corpus of tokens – 8% of the words are dialect.

– 80% of the dialect words are accented.

Next

• Summing up and preview of the take-home final

Arabic Prosody: Accentuation

• 70% of words are accented • 60% of the de-accented words are function words or disfluent items – Based on automatic POS analysis (MADA) – 12% of content words are deaccented • Distribution of accent types: – H* or !H* pitch accent, 73% – L+H* or L+!H*, 20% – L*, 5% – H+!H*, 2%

Arabic Prosody: Phrasing

• Mean of 1.6 intermediate phrases per intonational phrase • Intermediate phrases contain 2.4 words on average • Distribution of phrase accent/boundary tone combinations – L-L% – H-L% – L-H% – H-L% – H-H% 59% 26% 8% 6% 1%

Arabic Prosody – most common contours

H* L H* H L+H* L H* H* L H* !H* L L* L L+H* !H* L H* H* H H* !H* !H* L L+H* H 21.9

13.4

9.7

7.6

4.1

4.1

3 3 2.3

2.1

Arabic Prosody – Disfluency

• In addition to standard disfluency: – Hesitations – filled pauses – self-repairs • In Arabic, speakers could produce a sequence of all of the above. (see praat: file: 1036 and 2016) • Disfluency may disconnect prepositions and conjunctions from the content word: – يتأت ...

ينعي ...

ـل ...

و > = يتأتلو – w- l- uh- yEny uh- t?ty instead of wlt?ty