Transcript A socio-cognitive framework

Nükte Durhan
METU, Northern Cyprus Campus, School of Foreign Languages
(Ankara, 30 May 2012)






General look at socio-cognitive framework
Cognitive validity (theory-based) and routine
skills
Needs analysis results of the Academic
Speaking Skills (211) Course
Description of the speaking tasks in the
course and evaluation from cognitive validity
aspects
Context validity: considering its relevant
aspects and the speaking tasks
Areas that need work: scoring validity
Performance tests are characterised by the
relationship of the test task to the world
beyond the test. Ideally tasks are…
 direct and
 authentic
- situationally authentic
- interactionally authentic
But a test can never fully represent reality.

We want candidates to…
 perform relevant language tasks
 adapt their speech to circumstances
 make decisions under time pressure
 implement them fluently
 make any necessary adjustments as
unexpected problems arise
Test Taker
Context
Cognitive
Response
Scoring Validity
Score/Grade
Consequential
Validity
CriterionRelated Validity






Test-taker characteristics
Context focused
Cognitive focused (theory-based)
Scoring focused
Criterion focused (external)
Consequentially focused
 The abilities to be tested are demonstrated by
the mental processing of the candidate (the
cognitive dimension)
 The use of language in performing tasks is
viewed as a social rather than a purely
linguistic phenomenon (the socio- dimension)
 Test construct resides in interactions of
underlying cognitive ability and context of use,
i.e. socio-cognitive model

Executive processes
◦
◦
◦
◦
◦
◦
◦
◦
Conceptualizer
Pre verbal message
Linguistic formulator
Phonetic plan
Articulator
Overt speech
Audition
Speech comprehension
(Weir,2005)

Executive resources
◦ Language knowledge




Grammatical
Textual
Functional
Social
◦ Content knowledge
 Internal
 External


Routine Skills (Bygate, 1987)
Informational Routines
 expository routines:
involve factual information
 evaluative routines:
involve drawing of conclusions,
expression of reasoning; explanation,
predictions, justifications, preferences and
decisions


Routine Skills (Bygate, 1987)
Interactional Routines :
 telephone conversations
 interviews
 meetings
 discussions
 decision making tasks



Routine Skills (Bygate, 1987)
Interactional Routines :
They are assisted by…
 interaction management skills involving
content-focused agenda management and
interaction-focused turn-taking
Improvisational Skills
 brought into play when an interaction
falters




Informative Speech (individual) (10%)
Persuasive Speech (group) (20%)
Final Presentation (individual) (30 %)
No assessment of discourse and strategic
competences
Participants want more of the following:
 More focus on spontaneous speech
 A variety of speaking activities
 More presentations with smaller percentages
 Different types of presentations
 Focus on fluency
 Interactive activities
 Creativity and individuality
 Speech critiques







Debate
Role plays
Discussions
Impromptu speech
Pronunciation activities
Critical thinking activities
Asking & answering questions





Reporting the team’s observation of a store’s
merchandising strategies and store design
Organizing and sharing points in the group
Presenting findings
Giving evidence from the observations
Evaluating the effectiveness of the strategies

To inform and enlighten the audience about
a given topic on the general theme of
technology


Choosing a news item related to art,
analyzing it and presenting the analysis to
class
Includes:
 justification of the selection of the item
 reaction to points and style
 evaluation of the ideas


Giving a balanced view of the selected topic
(related to the five themes in the book)
Includes:
 extensive literature research
 reaction to the presented
material/discussion or evaluation of the
topic





Debates
Discussions
Panel discussion
Role plays
Conflict resolution tasks
Three different activities should be carried out
in class throughout the semester.

Setting: task
◦
◦
◦
◦
◦
◦

Purpose
Response format
Known criteria
Weighting
Order of items
Time constraints
Setting: administration
◦ Physical conditions
◦ Uniformity of administration
◦ Security
(Weir,2005)

Demands: task
◦ Linguistic
 Channel
 Discourse mode
 Text length
 Nature of information
 Topic familiarity
 Lexical range
 Structural range
 Functional range
1. Rubric

Is the rubric accurate and accessible?
2. Purpose

Is the purpose of the test made
unequivocally clear for the candidate?
3. Weighting

Are any weightings for different test
components adequately justified?
4. Known criteria

Are the criteria to be used in the marking of
the test explicit for the candidates and the
markers?
5. Time constraints: particularly important for
speech as normal speech takes place under
time pressure

Is the timing for each task (e.g. preparation
and completion) appropriate?
6. Discourse mode:
Reciprocity conditions are concerned with
the dimension of interpersonal interaction,
the relation between speaker and listener.

Are there reciprocal exchanges where
speakers have to adjust vocabulary and
message and take each other’s
contributions into account?
7. Type of information

Is the type of information appropriate for
the target situation requirements of the
students being tested?
8. Content knowledge

Is the content sufficiently familiar to
candidates so that they have sufficient
existing schemata to enable them to deploy
appropriate skills and strategies? (Urquhart
& Weir, 1998)
8. Content knowledge
“every attempt should be made to allow
background knowledge to facilitate
performance rather than allowing its absence
to inhibit performance” (Alderson, 2000)
30
9. Linguistic knowledge
 Lexical range
 Are the lexical items in the test both in
input text and required as output
appropriate for the level of the
candidates?

Syntax
Are the grammatical items in the test both
in input text and required as output
appropriate for the level of the
candidates?
9. Linguistic knowledge
 Function: is a term used to describe the
illocutionary force of what is said.
 Are the functions in the test both in
input text and required as output
appropriate for the level of the
candidates?
The variety of tasks with different functional
purposes and increasing number of tasks
improve the validity.

Rating
◦ Criteria / rating scale
◦ Rating procedures
 Rater selection
 Rater training
 Standardisation
 Moderation
 Rating conditions
 Statistical analysis
[email protected]