Transcript PowerPoint - University of Wisconsin

+
UW-Whitewater
Languages and Literatures
Ellen Titzkowski Boldt
[email protected]
August 6, 2014
Summer Assessment Workshop for High
School World Language Instructors
The Three Modes of Assessment:
Interpretive, Interpersonal, Presentational
+
Overview
UNDERSTANDING
ASSESSMENT
DESIGNING
ASSESSMENTS
IMPLEMENTING
ASSESSMENTS
EVALUATING
ASSESSMENTS
+
Overview
 Introduction




The Purpose of Assessment
Attitudes Towards Assessment
Intended Use of Assessments
Developing Effective Assessments
 Types
of Assessments
 Integrated


Performance Assessments (IPA)
The Three Modes of Communication
Assessing the Modes: Tasks & Strategies
 Designing
 Rubrics
Performance Assessments
Why is assessment
important?
+
Why do we keep doing it in
our classrooms, despite the
challenges it can present?
+
The Purpose of Assessment
 To
motivate students
 To
serve as more than a vehicle to assign a
grade
 To
drive the instruction (Sandrock 2010)
 To
show evidence that learning is occurring
 To
evaluate the effectiveness of instruction
 To
identify areas needed for improvement
+
The Purpose of Assessment
 Assessment
is used as a diagnostic tool to
describe what students have learned in the past
 shape future learning goals
 document progress towards student learning objectives
 identify areas needing improvement (in instruction and
student performance)
 measure language proficiency, communicative competency,
and cultural awareness
 evaluate teacher effectiveness*


WI Educator Effectiveness System  DPI

*Starting in 2014-2015, all WI educators will be evaluated on student
achievement & student learning objectives/outcomes (SLOs).

See http://ee.dpi.wi.gov/ for the latest information.
+
The Purpose of Assessment
 “Language
assessment is the process of using
language tests to accomplish particular jobs in
language classrooms and programs” (42).
 “To
keep our language assessment practice
purposeful, we therefore need to evaluate the
extent to which the language testing tools we
select and use are actually helping to accomplish
the jobs of language assessment in our classrooms
and programs” (44).
(Norris 2000/2012)
+
Attitudes towards Assessment
http://gratisography.com/
+
Attitudes towards Assessment
http://gratisography.com/
+
Assessment Stakeholders: How do
different groups view assessment and why?
 Stakeholders
 Teachers
 Students
 Administrators
 Students’ Families
 Curriculum
planners
 Future employers
 University admissions
counselors
(Norris 2000)
Step 1: Brainstorm individually
 Goals?
 Attitudes?
 Problems?
Step 2: Share and compare
 Discuss ideas with a
partner.
 Focus on differing
motivations.
+
What makes
a good
assessment?
Discuss in pairs.
+
Quality and Selection:
What makes a good assessment?
The selection and perceived quality of
an assessment depends on its intended
use.
+
Intended Use of Assessments
WHAT?
Test
Information
WHO?
Test
Users
INTENDED
TEST USE
IMPACT?
Test
Consequences
(Norris 2000)
WHY?
Test
Purposes
+
Developing Effective Assessments
 Key
questions to ask when creating/selecting the
right assessment:




Who uses the assessment?
What is being assessed?
Who/What is impacted?
Why? What is the purpose?
 Specify
the intended use of the assessment.
 Provide
a general description of the intended
assessment.
 Note
problems and possible solutions.
(Norris 2000)
+
Developing Effective Assessments
Purposeful
language assessment requires:
 Acknowledging
the context for assessment
 Focusing on assessment, not just on tests
 Specifying the intended use(s) of the
assessments(s)
 Evaluating the outcomes of the assessment(s)
(Byrnes 2001; Norris 2000)
+
Developing Effective Assessments
 What
does purposeful assessment look like?
 Identifying
the thematic and cultural contexts
 Setting attainable benchmarks or learning targets
 Creating classroom activities to support student
success on performance assessments
 Synthesizing language and content instruction
 Fostering task-based instruction
 Developing both formative and summative
assessments
 Involving all three modes of assessment
 Embedding assessments into curriculum
+
Assessment Models
 Common
 Level-
Assessments
and course-specific assessments
 Formative
 Exit
vs. Summative vs. Prototypical
interviews (OPI, SOPI, MOPI)
 Content-based, task-based, genre-specific
 Performance
 Three
Assessments
Modes of Communication
 5 C’s of the National Standards
+ Types of Assessments
Traditional
Assessment Tools

Focus on:
 grammatical accuracy
 focus on form
 vocabulary building
 discrete learning checks
Alternative Performance
Assessment Tools

Focus on:
 communication
 application of learning
 authentic language use
 performance of real world
tasks
 meaningful contexts
 proficiency development
 integration with standards
 3 modes of communication
 teaching to the “test”
(Sandrock 2010)
+
Types of Assessments
 Assessment
is a continuum.
 Teachers
need to provide students with a variety
of feedback on various types of assessment
across the spectrum, including:




specific and focused feedback
holistic and broad feedback
Formal (rubrics) and informal (learning checks)
feedback
A balanced assessment system = both
formative and summative assessments.
(Sandrock 2010)
+ Types of Assessments
Formative
Assessment
Summative
Assessment

Learning checks, guided
activities with teacher support

End-of-unit, end-of-course
assessment (no support)

Informs and modifies
instruction, classroom
activities and student learning

Demonstrates knowledge
gained without teacher
assistance

Builds students’ confidence

Motivates students

Scaffolds information to be
used in summative
performance assessments

Showcases application of
various skills learned via
formative assessments

May focus more on specific
learning targets (i.e. grammar
concepts, vocabulary)

Synthesizes a variety of
communication skills and
language concepts
+
Characteristics:
The 3 Modes of Communication

Interpretive Mode:
 listening, reading, viewing
 authentic, text-based (audio, written, video/film) materials
 monologic tasks (one-way communication)

Interpersonal Mode:
 spontaneous communication (oral or written)
 negotiation of meaning
 dialogic tasks (two-way communication)

Presentational Mode:
 speaking, writing
 monologic tasks (one-way communication)
 Rehearsed language usage
+ The 3 Modes of Communication
Interpersonal
Interpretive
Presentational
Active negotiation of meaning among
individuals
Interpretation of what the author,
speaker, or producer wants the
receiver of the message to
understand
Creation of messages
Participants observe and monitor one
another to see how their meanings
and intentions are being
communicated
One-way communication with no
recourse to the active negotiation of
meaning with the writer, speaker, or
producer
One-way communication intended to
facilitate interpretation by members
of the other culture where no direct
opportunity for the active negotiation
of meaning between members of the
two cultures exists
Adjustments and clarifications are
made accordingly
Interpretation differs from
comprehension and translation in that
interpretation implies the ability to
read (or listen or view) “between the
lines,” including understanding from
within the cultural mindset or
perspective
To ensure the intended audience is
successful in its interpretation, the
“presenter” needs knowledge of the
audience’s language and culture
Speaking and listening
(conversation); reading and writing
(text messages or via social media)
Reading (websites, stories, articles),
listening (speeches, messages,
songs), or viewing (video clips) of
authentic materials
Writing (messages, articles, reports),
speaking (telling a story, giving a
speech, describing a poster), or
visually representing (video or
PowerPoint)
(ACTFL Performance Descriptors for Language Learners 2012)
+ Standards-Based Performance Assessment
Interactive graphic: http://wimedialab.org/worldlanguageassessment/clover.htm
+ Assessment: Interpretive Mode
How do you typically assess students’
abilities to communicate in the interpretive
mode?
+ Assessment: Interpretive Mode

Interpretive Assessment Task =



Demonstrate literal comprehension (keys words, main ideas,
details) and interpretive comprehension (word and concept
inferences, cultural perspectives, author intent, text organization).
Use a comprehension guide (worksheets, Q&A, creating or
identifying images based on descriptions, etc.) to document both
levels of comprehension.
Strategies for Developing Interpretive Communication




Routinely incorporate authentic listening, viewing, and reading
texts/tasks into classroom instruction  skimming, scanning,
identifying language patterns.
Encourage focused listening/viewing/reading of smaller textual
chunks and teach comprehension strategies  context clues,
word families, textual organization (headings, captions, photos).
Design group activities that allow for collaborative interpretive
skills.
Assist students as they move from literal comprehension to
interpretive comprehension goals.
(Sandrock 2010: 83)
+
Considerations for text selection
(Intermediate Level)


Assessment:
Interpretive Mode


Assessment targets ability to detect
main ideas and supporting details.
Texts should focus on simple
narratives, routines, familiar
contexts, personal experience.
Include a range of sentence lengths
from simple to paragraph-length
text.
Topics should be of high interest to
students and include cultural
content from the target culture(s) to
allow comparison and contrast to
their own cultural practices.
(Sandrock 2010: 82)
+ Assessment: Interpretive Mode
Sources for authentic READING
texts(highly contextualized &
thematically appropriate)

product advertisements

public service campaign
announcements

interviews/surveys from youth
magazines
Sources for authentic LISTENING
& VIEWING texts (highly
contextualized & thematically
appropriate)

commercials (TV/radio)

public service campaign
announcements

simple TV or soap opera
segments

poems

simple stories

Interviews

genre-specific letters/email

talk show excerpts

advice columns

film excerpts

personal ads

songs

photo stories with captions

comic strips/cartoons

simple news articles

podcasts

resumes

descriptions (art/photos in
museum guides; directions)
(Sandrock 2010: 82)
+
Assessment: Interpersonal Mode
How do you typically assess students’ abilities to
communicate in the interpersonal mode?
+ Assessment: Interpersonal Mode

Interpersonal Assessment Task =




Two (or more) students exchange information spontaneously, as well
as express opinions, feelings, and emotions with each other
The student(s) may have information the other(s)do not have, thus
creating an information gap and purposeful exchange of real
information and negotiation of meaning.
Generally no notes or written support are allowed.
Strategies for Developing Interpersonal Communication

Incorporate regular classroom activities that require interpersonal
communication to
 lower students’ anxiety levels for later assessments
 practice spontaneous speech in non-evaluation situations
 use discourse markers and key phrases to negotiate meaning
 reduce reliance on written notes and support
 negotiate meaning via information gap tasks
 allow students to interact with a range of peers and their different
proficiencies (mutual benefit of modeling and support)
(Sandrock 2010: 83)
+
Assessment: Interpersonal Mode
 Example
Interpersonal Tasks
(with partners or in small groups)
Information gap tasks
 Interviews
 Cultural comparisons
 Discussions
 opinions
 reactions
 pros/cons
 personal perspectives

+
Assessment: Presentational Mode
How do you typically assess students’ abilities to
communicate in the presentational mode?
+
Assessment: Presentational Mode
 Presentational




Assessment Task =
Students create a message to communicate to an audience
by means of a written or spoken language product.
Monologic task  one-way communication
Often used as a summative task after building upon
previous interpretive and interpersonal tasks.
Presentational rubrics often evaluate “impact, which refers
to the degree to which the message maintains the attention
of the reader or listener. The teacher should explore with
students strategies for creating presentational products
that have impact (e.g. selection of topic, use of visuals,
choice of words, visual layout)” (Sandrock 2010: 84).
+
Assessment: Presentational Mode
Strategies for Developing Presentational Communication




Use a process-oriented approach (drafts, peer-editing, revisions,
rewrites, scripts, rehearsals, videotape) to allow for a variety of
feedback opportunities.
Incorporate peer- and self-evaluation into the feedback loop.
Address the issue of “impact” in your rubric.
 Discuss and practice evaluating the “impact” of various
presentational messages in a variety of authentic and studentproduced products.
Create a balanced rubric
 task appropriateness
 content
 language usage
(Sandrock 2010: 84)
+ Assessment: Presentational Mode
Example WRITING tasks
 Essay
 Poem
 Letter (genre-specific)
 Email (context-specific)
 Advertisement/Flyer
 Blog post
 Description
 Glogster
 Journal Entry
 Photo essay with
captions
Example SPEAKING Tasks
 Speech
 Monologue
 Voice mail
 Video
 Commercials
 Podcast
 Short play
 News broadcast
 Digital Story
 Genre-specific speech
(eulogy, campaign, etc.)
+
Designing Alternative Assessments
 Task
and assessment instruments must fit the
intended purpose for the student learning objective
and connect to the standards.
 Effective
assessments (formative, summative,
informal and formal) should be situated in a
meaningful thematic context with real world
application and authentic language use.
 Transparency: Assessment



≠ Mystery
What is the goal of the specific assessment?
Informing students about the expectations and
communicative goals fuels student motivation.
Discuss well-crafted rubrics at the beginning of the unit so
that students understand what the expected performance
involves.
(Sandrock 2010: 28)
+
Designing Performance Assessments
 Backward
 By
Design
first selecting the intended goal, performance
assessments can function as a filter for selection
the content needed to achieve the student
learning outcome(s) in a unit.
 What is necessary to complete the task(s)?
 Develop an essential question based on the
content/thematic unit in order to determine the
tasks.
 Create learning targets and formative
assessments to build towards summative
performance assessments in the three modes.
+
Templates for
Designing Performance Assessments
 Download
the Blank Curriculum Planning Template
 http://wimedialab.org/worldlanguageassessment/re
sources.htm
 Templates
for designing integrated performance
assessment tasks in future thematic units

Copies are in your folder for use in afternoon workshop
session (Sandrock 2010: 34) and with workshop
handouts/references online.
 Nature

Unit  detailed example
With workshop handouts/references online
+ Example Unit: Clothing (Intermediate Level)
Essential Question: What do clothes say about a person?
Mode
Interpretive
Read & answer
Performance questions from an
Assessment authentic article
in a fashion
magazine about
current trends.
Interpersonal
Presentational
Interview a partner
about their personal
style preferences in
various contexts and
what they project.
Fashion Show:Write and
present a text describing
and a peer’s outfit and
comment on what trends
& image(s) it projects in
which contexts.
In-Class
Practice
Read & analyze
culturally
authentic
clothing
advertisements
online.
Info-gap tasks
describing outfits/ style;
view/analyze YouTube
interviews  “Kleider
machen Leute”
outfit/personality
perceptions
Describe outfits in writing
(draft & revise); describe
orally what others are
wearing, how the clothes
they fit and what they
project about the wearer.
Necessary
Language,
Structures,
Functions,
Cultural Info
Clothing vocab
Adjectivesstyles/trends
Pricing in Euros
European sizing
Question-Answer
Dative verbs
Personal Pronouns
Comparative/Superlativ
e
Express opinions
Adjective endings
(synthesis and
application of everything
previously listed)
+
Sequencing Performance Assessments

ACTFL suggests a sequenced approach:
1. Interpretive assessment
2. Interpersonal assessment
3. Presentational assessment

This methods allows each assessment to build on the following
one, securing vocabulary and language functions from textual
modeling via authentic texts in the interpretive mode which are
needed to be successful in the interpersonal and presentational
modes when the language becomes productive.

There is some debate about the last two steps since spontaneous
dialogic communication with its inherent negotiation of meaning
may appear more complex than monologic presentational tasks,
such as writing and speaking (Tedick, D & Cammarata, L. 2014).
+
Rubrics
 Rubrics
are a necessary tool for teachers and
students to evaluate communication.
 Rubrics
must describe the expected performance
and provide useful, targeted feedback.



Does not meet expectations
Meets expectations
Exceeds expectations
 Rubrics
should demonstrate how increased
proficiency can be achieved so students know
what to aim for.
+
Rubrics

Share rubrics with students to allow familiarity and goals for
expected performance levels.

Model rubric evaluation with students via sample
performances.

Examples (included in your folders)
 ACTFL Performance Descriptors for Language Learners
(2012) can serve as a basis for creating rubrics.

UW-Whitewater Rubrics  based partially on the ACTFL
Performance Descriptors for Language Learners (1998
Version) & ACTFL Proficiency Guidelines
 Presentational-Writing Rubric
 Presentational-Speaking Rubric
 Interpretive Rubric (Assessing Textual Literacy)
+
K-12 Assessment Resources in WI
 Get
in the Mode: Assessment Videos (2008)
 http://www.youtube.com/watch?v=oAJ8ZqanZr4
&list=PL77DDFFFF389160BB&index=6
 http://wimedialab.org/worldlanguageassessmen
t/video.htm
Challenges
+ Time Constraints
Class size
Curriculum
Departmental Involvement
Student Attitudes  “test” or
“performance” anxiety
http://gratisography.com/
+
Final Thoughts
“Assessment is perhaps one of [the] most
difficult and important parts of our jobs as
teachers. Ideally, it should be seen as a means to
help us guide students on their road to learning.
No single procedure can meet the needs of all
learners and situations, so we need to
remember to incorporate a variety of tools to
help our students know how they are
progressing and to gauge the effectiveness of
our own methodology and materials.”
~ Jerrold Frank
(Frank 2012: 32)
+ Final Thoughts
Assessment…
…is more than a test.

…is all the things along the way that prepare students.

…builds on itself to help students progress.

…needs to be both formative and summative.

…can be informal and formal.

…provides opportunities for meaningful feedback.
…needs to be purposeful and meaningful for students.
…is motivational by setting a clear path for performance goals.
…drives instruction.
…is integrated in the classroom and allows for evaluation of all three
modes of communication  interpretive, interpersonal,
presentational.
+
Final Thoughts
“Language assessment…is much more than
simply giving a language test; it is the entire
process of test use. Indeed, the ultimate goal
of language assessment is to use tests to
better inform us on the decisions we make
and the actions we take in language
education.”
~John M. Norris
(Norris 2000/2012: 42)
+
References

American Council on the Teaching of Foreign Languages. (2012).
ACTFL performance descriptors for language learners, 2012
Edition. Alexandria, VA: ACTFL.


Byrnes, H. (2001). Faculty assessment and evaluation: additional
considerations. ADFL Bulletin, 32(3), 34-36.


http://www.actfl.org/sites/default/files/pdfs/PerformanceDescriptorsLa
nguageLearners.pdf
http://www.adfl.org/bulletin/V32N3/323034.htm
Frank, J. (2012). The roles of assessment in language teaching.
English Teaching Forum, 50(3), 32.

http://americanenglish.state.gov/files/ae/resource_files/50_3_7_frank.p
df
+
References

Norris, J. M. (2014, April). How do we assess task-based performance?
Invited LARC/CALPER testing and assessment webinar.


http://larc.sdsu.edu/testassesswebinar/jnorris/Norris_TaskBasedAssessment_PPT.pdf
Norris, J.M. (2012). Purposeful language assessment. English
Teaching Forum, 50(3), 41–45. (Reprinted from Norris, J.M.
(2000). Purposeful language assessment. English Teaching
Forum, 38(1), 18–23.)

http://americanenglish.state.gov/files/ae/resource_files/50_3_10_norris.pd
f

Sandrock, P. (2010). The keys to assessing language performance: A
teacher’s manual for measuring student progress. Alexandria,
VA: ACTFL.

Tedick, D & Cammarata, L. (2014). "Integrated Performance
Assessment: Adapting the Model for CBI." [online

http://carla.acad.umn.edu/cobaltt/modules/assessment/ipa/index.html
+
Questions? Feedback?
Contact me at [email protected]
http://gratisography.com/