Potentials and boundaries of formative assessment: Insights from New Zealand Yongqi Gu

Victoria University of Wellington New Zealand [email protected]

1

The big question

• What is assessment for?

2

Assessment purpose: norm- vs. criterion-referenced

• To rank or select among the students: Norm-referenced • To see who can achieve set goals or standards: Criterion referenced

3

Assessment purposes: Summative vs formative

Primary Purpose Proximity to Instruction Feedback Stakes

Formative

Adjusting instruction During Frequent, detailed Low

Summative

Assigning grades, accountability After Infrequent, general High 4

Purposes of classroom assessment

• To obtain info on how effective teaching has been • To provide feedback to learners: – Progress? – Weaknesses?

• To give learners an incentive to study • To clarify to both teacher and learners what learners need to achieve in terms of: – what to learn – amount of learning – level of performance • … 5

Language assessment in China

• Language assessment research – Review of 9 key journals – All assessment articles (total=91) published between 2001 and 2005 • Language assessment practice 6

Journals reviewed

1. Foreign Language Teaching and Research 2. Modern Foreign Languages 3. Journal of Foreign Languages 4. Foreign Language World 5. Foreign Languages and Their Teaching 6. Journal of PLA University of Foreign Languages 7. Foreign Language Research 8. Foreign Language Education 9. Foreign Languages Research 7

Journals reviewed (Chinese)

1.

外语教学与研究 2.

现代外语 3.

外国语 4.

外语界 5.

外语与外语教学 6.

解放军外国语学院学报 7.

外语学刊 8.

外语教学 9. 外语研究 8

Language assessment articles published in China: Categories Lable Topics

A Proficiency test B C D E F G H I Testing oral English Testing listening Testing writing Testing reading Integrative testing Computer assisted testing Reform in language testing Band 4 and Band 6 9

Language assessment articles published in China: Categories

J K L M N O P Q R S T U Test analysis Test taking strategies Test format SLA and testing Authenticity Curriculum and testing Social aspects of language testing Pragmatic aspects of testing Testing trends Washback Formative assessment Using tests for diagnostic purposes 10

Language testing articles published in China (2001-2005)

16 14 12 10 8 6 4 2 0 A B C D E F G H I J K L

Categories

M N O P Q R S T U 11

Research concentration (2001-2005)

G Computer assisted testing 7 S H Washback 7 Reform in language testing 9 I B Band 4 and Band 6 Testing oral English 10 14 12

Four articles on formative assessment

Article 在线同伴评价在写作能 力发展中的作用 形成性评估在中国大学 非英语专业英语写作教 学中的运用 自我评估同辈评估与培 养自主学习能力之间的 关系 学生英文写作能力的自 我评估 Author 蒋宇红 Journal 外语教学与研 究 外语教学 曹荣平 张文霞 周燕 楼荷英 外语教学 刘建达 现代外语 Issue 2005/03 2004/05 2005/04 2002/03 13

Using summative tests for diagnostic purposes

Article Author 大学英语校内测试 模式的调查与分析 郭丽 Journal 外语界 Issue 2003 /02 用重测法分析大学 新生英语水平 王丽琴 解放军外国 语学院学报 2001 /04 14

Summary

• Chinese researchers have concentrated on summative assessment (assessing OF learning), not on formative assessment (assessing FOR learning).

• No article touched on criterion-referenced assessment from 2001 to 2005.

• Chinese teachers have not extensively explored formative assessment as a teaching or learning tool for the classroom 15

Formative assessment

• Formative assessment refers to assessment that is specially intended to generate feedback on performance to improve and accelerate learning (Sadler, 1998) 16

Assessment that can be formative is assessment that

• Is embedded in a pedagogy of which it is an essential part (eg constructivist).

• Shares learning goals with students.

• Involves students in self-assessment.

• Provides feedback which leads to students recognising the gap and closing it.

• Is underpinned by confidence that every student can improve. • Involves reviewing and reflecting on assessment data.

(Harlen, 1998) 17

Three conditions for formative feedback

• Knowledge of standards (knowing what good performance is) • Comparing these standards (desired level of performance) to the student’s own current level of performance • Taking action to close the gap between current and desired performance Sadler (1989) 18

How does feebback work?

Feedback in a self-regulated learning framework Butler and Winne (1995) Teacher sets task Goals/criteria/standards Domain knowledge Strategy knowledge Motivational beliefs Student goals Processes internal to student Tactics & strategies Internal learning outcomes Paths of internal feedback Self-regulatory processes (Cognition, motivation & behavior) External feedback (Teacher/peers) Externally Observable outcomes 19

Does formative feedback work?

Empirical evidence Black and Wiliam (1998) summarized over 250 studies, and concluded that: 1. Improving formative assessment raises standards, dramatically. And FA helps low achievers more than other students.

2. Something should be done about it inside the classroom 3. Teachers can improve formative assessment 20

Factors that influence the effectiveness of FA

1. Providing effective feedback to students.

2. Student’s active involvement in their own learning.

3. Adjusting teaching to take account of the results of assessment.

4. Recognising the profound influence of assessment on students’ motivation and self-esteem.

5. Ensuring pupils reflect on their learning and understand how to improve.

21

Inhibiting factors

• A tendency for teachers to assess quantity and presentation of work rather than

quality of learning

.

• Greater attention given to marking and grading, much of it tending to lower self esteem of students, rather than

providing advice for improvement

.

• A strong emphasis on comparing students with each other, which demoralizes the less successful learners.

22

The New Zealand Experience

• Standards-based assessment (SBA) at the national level • Qualifications awarded by completing sets of unit standards (desired learning outcomes), not in terms of years of learning, and not by comparison with other students.

• Teachers design assessment tasks which relate to the specific learning needs of their learners and the context in which they are studying 23

The New Zealand Qualifications Framework

• Aim: a single, coherent framework for New Zealand • Unit standard: a set of specific learning outcomes and the criteria for their assessment • Assessment/performance level: achieved/not achieved • Particular set of unit standards are required for certificates and diplomas at each level.

24

The NZ qualifications system: 10 levels

25

Level 1: Description

PROCESS LEARNING DEMAND RESPONSIBILITY Carry out processes that: •are limited in range •are repetitive and familiar •are employed within closely defined contexts Employing: •recall •a narrow range of knowledge and cognitive skills •no generation of new ideas Applied: •in directed activity •under close supervision •with no responsibility for the work or learning of others 26

Level 3: Description

PROCESS LEARNING DEMAND RESPONSIBILITY Carry out processes that: •require a range of well developed skills •offer a significant choice of procedures •are employed within a range of familiar contexts Employing: •some relevant theoretical knowledge •interpretation of available information •discretion and judgement •a range of known responses to familiar problems Applied: •in directed activity with some autonomy •under general supervision and quality checking •with significant responsibility for the quantity and quality of output •with possible responsibility for the output of others 27

NCEA Assessment

• National Certificate of Educational Achievement ( NCEA): the National Qualifications Framework in schools • NCEA=Levels 1-3 in NQF: senior secondary school • 8 achievement standards for each school subject • Each stardard has a credit value and sits at a specified level on the NQF.

• Credits are accumulated towards a qulification 28

Stadands-based Assessment in NCEA

• Four levels of performance: – Excellence – Merit – Credit – Not achieved • Half of the standards for each subject are assessed by classroom teachers, the other half externally assessed nationally 29

An example: ESOL unit standard 17363

• • •

Read independently information texts, using ESOL Level: 3 Credit: 5

30

Summary of SBA in NZ

SBA in NZ is criterion-referenced assessment. It achieves a much better formative purpose than a summative one. This is because – SBA makes desired standards of learning explicit to both teachers and learners – It enables and encourages rich and frequent feedback from both the teacher and the students themselves – It does not emphasise the grade, and gives students multiple chances to achieve a standard.

31

What’s good about Standards based assessment? 1

• Clear and specific goals and standards for both learners and teachers • Students assessed in terms of what they can do, and not how they compare with peers • Slow learners are given multiple chances to achieve. 32

What’s good about Standards based assessment? 2

• Rich formative tasks during or at the end of particular units of work • peer, and self assessment based on observation, questioning, and feedback • give detailed information about students' learning needs • suggest appropriate teaching strategies to meet student needs 33

Problems with standards based assessment 1

• Assessment by unit standards greatly increased teacher workloads as they designed tasks and implemented them, kept records of student achievement, and participated in the necessary moderation activities.

• Unit standards are assessed on an achieved/not achieved basis. There is no provision for rewarding superior performance and academically able students may not be challenged to achieve to the best of their ability.

34

Problems with standards based assessment 2

• Despite moderaton procedures, ensuring consistent and fair assessment on a national basis is a tall order.

• SBA is good for low-stakes, formative purposes, but is used for high stakes purposes such as certification.

35

Insights for language assessment in China

1. At the system level, the main purpose of foreign language assessment in China is probably always going to remain norm referenced, summative, and selective.

2. At the pedagogical level, setting clear and realistic goals and standards for Chinese teachers and learners of English is a natural step we have to start. However, a NZ-style standards-based assessment is both impractical and unnecessary.

36

Insights for language assessment in China

3. At the classroom level, I see an urgent need for teachers to explore formative assessment tools such as self- and peer assessment, portfolios that provide rich feedback information.

4. At the learner level, formative assessment will not only improve learning results, but also encourage self-reflection, self monitoring, and self-regulation. 37

Insights for language assessment in China

5. At the research agenda level, it is not a question of whether, but a question of how formative assessment should be made use of systematically by Chinese teachers and learners to improve learning.

38

Purpose of assessment?

• If a major purpose of assessment is to inform teaching and learning, why aren’t we exploring formative assessment for its potentials in the foreign language classroom?

39

Thank you!

40