Transcript Applying an Evaluation and Assessment Framework: An Irish perspective Better Assessment and Evaluation to Improve Teaching and Learning EU Presidency Conference  Dublin  20

Applying an Evaluation and
Assessment Framework:
An Irish perspective
Better Assessment and Evaluation
to Improve Teaching and Learning
EU Presidency Conference  Dublin  20 March 2013
Dr Harold Hislop
Chief Inspector  Príomh-Chigire
Department of Education and Skills, Ireland
INSPECTORATE
FEABHAS NA FOGHLAMA A CHUR CHUN CINN
1
Purpose of the paper
• OECD: Education systems ought to view
arrangements for evaluation and assessment in
their countries as a structured and planned
system
• How could you apply the framework in Ireland?
• What questions and implications would this raise
for Ireland and others?
• What is the significance of the OECD framework?
2
Overarching themes
• Governance
• Design and procedures: the components that
one might consider using
• Capacity: the ability of the system and the
institutions and individuals to operate evaluation
and assessment arrangements
• Use of results
3
GOVERNANCE
4
GOVERNANCE
• Need for clear structures and defined
responsibilities for evaluation and
assessment OECD
• Ireland would appear to have at least the
main elements of governance in place,
though some challenges remain….
5
Significant legislation is in place
•
•
•
•
Education Act, 1998
Education Welfare Act, 2000
Teaching Council Act, 2001
Education for Persons with Special
Educational Needs Act, 2004
6
Minister responsible for…
• evaluation
To monitor and assess the quality, economy,
efficiency and effectiveness of the education
system provided in the State ……and publish, in
such manner as the Minister considers
appropriate, information relating to such
monitoring and assessment (Education Act,
section 7(2)(b))
7
Minister responsible for ….
• evaluation
• curricular policy
• making regulations concerning appointment and
qualifications of teachers
• determining policy framework in which the
Teaching Council operates
• making regulations concerning the inspection of
schools
8
Statutory bodies are in place
•
•
•
•
National Council for Curriculum & Assessment
State Examinations Commission
Inspectorate
Educational Research Centre
– (non-statutory)
• Teaching Council
9
Legislation contains checks and balances
• Powers are vested in Minister
BUT
• Minister can only exercise his powers following
consultation with the key stakeholders in the
education system
AND
• Statutory bodies must also consult with stakeholders
10
Consensus in policy making
Advantages
• Stakeholder “buy-in”
–
–
–
–
–
Teachers and teacher unions
School managers
School leaders and their national organisations
National organisations of parents and students
Business and community groups and interests
• Supports teacher professionalism
11
Consensus in policy making
Advantages
• Stakeholder “buy-in”
• Supports teacher professionalism
Disadvantages
• Responsiveness
• Sometimes radical change is needed
12
So, reflecting on governance…
• Structures of governance exist in Ireland
for an evaluation and assessment
framework,
BUT
• Have we articulated an overall policy
about evaluation and assessment in the
system?
13
AN OVERALL POLICY ABOUT
ASSESSMENT AND EVALUATION?
14
National Literacy and Numeracy Strategy, 2011
• the closest we have to a national statement
regarding how aspects of curriculum,
assessment and evaluation knit together
• but focussed on literacy
and numeracy
• linkages need to be more
explicit
15
A reflection on the OECD framework…
• We could find it beneficial to examine
deliberately how we want the essential
components of an evaluation and
assessment framework to develop in a
coherent way that will support the sort of
student learning to which we aspire in the
school system of the 21st century
16
DESIGN AND PROCEDURES
THE COMPONENTS OF A FRAMEWORK
17
DESIGN AND PROCEDURES
• Student assessment
• School evaluation
– Inspections and School Self-Evaluation
• Teacher appraisal &
Appraisal of school leaders
• Education system evaluation
18
Student assessment: primary
• Early adopter of standardised tests for national
sampling in 1970s– Educational Research Centre
• Standardised tests not used in consistent way in
schools until recently
• Strong reaction against testing and league tables
in 1990s
• Teacher education: not enough attention paid to
assessment skills and competences
19
Student assessment: secondary level
• Dominance of State examinations
• Externally set and marked examinations at
end of lower secondary level (15-16 years) and
end of upper secondary level (17-18 years)
• Less use of assessment for learning
• Teacher education: not enough attention paid to
assessment skills and competences
20
Student assessment: developments
• Curricula with learning outcomes
& examples of students’ work
• Assessment for Learning
• Standardised tests for literacy
and numeracy
• School based assessment &
certification at lower secondary
21
A number of policy directions highlighted in the
OECD Framework for Assessment and Evaluation
• Ensure that parents are engaged
• Support assessment for learning approaches
• Build assessment capacity in system
• A balance between formative/summative assessment
• A broader range of assessment methodologies
• Need to invest in and use ICT systems
• A balanced use of student assessment
– keep existing policies and arrangements under review
22
School evaluation: Inspection of schools
• Expanded and developed since 1998
• A well-embedded feature of primary and secondlevel schools
• Particular focus on teaching and learning
• Advice to teachers and school leaders
• Improvement agenda as well as accountability
• Inspection reports published since 2006
• Proportionate inspection in schools
23
School inspections: some features
• Range of inspection models, including
– Some notified inspections – examine leadership,
management, teaching and learning, capacity to improve
– Unannounced inspections – focussed on teaching and
learning; used to scan system and for risk planning
– Follow-up inspections – both generally and in schools
with severe weaknesses
– Confidential questionnaire data from parents and
students
24
Parents are very positive about many aspects
of the work of schools
• “The school is well run”
– Agree/strongly agree
90.2%
– Disagree/strongly disagree
4.5%
– Don’t know
5.4%
8,680 parental responses in
second-level schools,
provisional 2012 data, subject
to confirmation
Agree
Don't know
Disagree
25
Parents are very positive about teaching and
learning in the school
• “Teaching is good in the school”
– Agree/Strongly agree
– Disagree/Strongly disagree
– Don’t know
86.5%
7.5%
6.0%
• “My child is doing well in school”
– Agree/Strongly agree
93.0%
– Disagree/Strongly disagree
4.6%
– Don’t know
2.3%
8,680 responses; provisional data from 2012, subject to confirmation
26
Students are positive about many aspects of
their experience in school
• “There is a good
atmosphere in the
school”
– Agree/strongly agree
83.2%
– Disagree/strongly disagree
5.5%
– Don’t know
11.3%
12,893 responses, provisional
2012 data, subject to confirmation
Agree
Don't know
Disagree27
Students are generally positive about many
aspects of their experience in school
• “The school deals well
with bullying”
– Agree/strongly agree
69.6%
– Disagree/strongly disagree
10.7%
– Don’t know
19.8%
12,893 responses, provisional
2012 data, subject to confirmation
Agree
Don't know
Disagree28
School evaluation: School self-evaluation
• Less well-developed
• Guidelines/resources
29
A number of policy directions highlighted in the
OECD Framework for Assessment and Evaluation
• Clarity about role and purpose of school
evaluation
• Accountability or Improvement
• Public confidence and availability of data
• A framework of standards
• Capacity of school leaders and teachers
• Investment in data capture and usage
30
Teacher appraisal &
Appraisal of school leaders
• Formal systems: least well developed aspect of
system
• Teaching Council: “fitness to teach” enquiries,
power to de-register teacher
• Appraisal in probationary period and for underperforming teachers
• School leadership is considered in inspections but
• No regular school-based appraisal for teachers or
principal
31
A policy directions highlighted in the OECD
Framework for Assessment and Evaluation
• Given the experience of many countries
documented by the OECD, it’s not unreasonable
to ask
“Why has the Irish school system apparently
chosen not to use some form of teacher
appraisal as a component in its evaluation and
assessment arrangements?”
32
But what should teacher appraisal be about?
• Weakness in terminology and construct
• “Teacher appraisal” has implicit emphasis
on accountability
• We want reflective practitioners as well as
appraisal for evaluation
• Improvement as well as accountability
33
We should attempt to have…
• Reflective practice
• Collaborative, peer review
• Collaborative whole-staff conversations about
practice and how to improve
• Observation and constructive feedback from
peers and principal
• Collegial Professional Reflection & Appraisal
34
System evaluation
• National monitoring surveys of literacy and
numeracy (Educational Research Centre)
• Specialised evaluations and research (ERC
and others)
– initiatives and policies; disadvantaged groups
• Inspectorate reports
• International surveys
– TIMSS and PIRLS, PISA, PIAAC, etc.
35
IMPLICATIONS
36
IMPLICATIONS:
Concerns that may be expressed
• Over-use of student assessment
– “Weighing the pig will not make it any heavier”
• Effect of measurement, performance and
accountability
– Attempting to measure teaching and learning can alter
teacher and student behaviour
• Dynamic around international surveys
– Indiscriminate policy borrowing
37
IMPLICATIONS
• An honest, valuable and significant attempt to
avoid ill-informed policy making
• Getting the right mix of tools and approaches
• Move away from narrow focus on student
assessment – towards a balanced framework
• Takes into account country context and capacity
of each educational system
• Interesting challenges ahead!!
38
Go raibh maith agaibh!
Thank you!
Dr Harold Hislop
Chief Inspector
Department of Education and Skills, Dublin 1, Ireland
[email protected]
Full paper available at www.education.gov.ie
39