Transcript tools’ External evaluation

‘TOOLS’ KICK-OFF MEETING
BRUSSELS JANUARY 2012
EXTERNAL EVALUATION
ALB CONSEIL PARIS – EVALUATION SERVICES FOR EU PROJECTS
Monday, July 06, 2015
ALB Conseil - TOOLS Project
2
WHO WE ARE
• ALB CONSEIL , Paris, is a consortium of 3 professionals
from Belgium, France and Italy, with proved experience in
EU projects and LLP projects evaluation:
• LEONARDO, GRUNDTVIG, ERASMUS, COMENIUS
• KA2 LANGUAGES, KA3 ICT, E-LEARNING
• They teach, translate and work in total 6 languages:
English, Italian, French, German, Romanian, Spanish.
• Angelica Bucur and Lucia Leporati were in 2011
Quality experts for Languages (applications assessment)
and for Comenius (e-Twinning Pilot Assessment).
Monday, July 06, 2015
ALB Conseil - TOOLS Project
3
WHO IS WHO? TOOLS’ E.E. TEAM
• Angelica Lumi Bucur
Manager ALB CONSEIL
Economist, MBA, MD in EU
Affairs – Belgium
• 25 years of experience in
management, planning,
strategy, business, e-business,
ICT, education and training
• + 8 years assessment of EC
projects & LLP applications,
Progress and Final Reports.
• External (and co-) evaluator of 7
LLP projects LEO and GRU
• Lucia Leporati
Teacher ALB Consultant
MD in Modern Foreign Languages
Literatures, with specialization in
German, English, and Spanish
• 19 years of experience in
teaching in Italy, and from 2010
in Switzerland; CLIL expert
• Participated to Comenius 1, 2.1
and 2.2) and to European
Network Comenius3
• + 6 years assessment of EC
projects & LLP.
Monday, July 06, 2015
ALB Conseil - TOOLS Project
4
TOOLS PARTNERS: WHAT
ROLE FOR THE EXTERNAL
EVALUATORS (E.E.)?
WHAT EXPECTATIONS DO YOU HAVE?
HOW DO YOU WANT TO COLLABORATE WITH E.E.?
YOUR OPINIONS WILL BE INCLUDED IN THE E.E. STRATEGY
TO BE FINALIZED AFTER KICKK-OFF MEETING
Monday, July 06, 2015
ALB Conseil - TOOLS Project
5
EXTERNAL EVALUATORS of TOOLS:
WHAT WOULD WE LIKE TO DO?
• Go beyond the paradigm of evaluation as assessment:
• We aim to bring out the value of the project as a LLP
initiative and highlight the strengths of outcomes for
learning purposes and those ensuring outcomes ‘
sustainability.
• To act as critical friend of the TOOLS project and
partnership
• Find evidence of the quality achieved by the project,
according to the quality indicators defined in the
application and agreed by the whole TOOLS partnership
Monday, July 06, 2015
ALB Conseil - TOOLS Project
6
METHODOLOGY & EVALUATION AREAS
• We aim to provide a comprehensive model and
external evaluation program of the project
achievements through a range of activities and
assessments
• External evaluation is an integral part of quality evaluation
of TOOLS project work, achievements, contribution to
original aims and specific objectives
• Three areas: operational, summative, learning.
Monday, July 06, 2015
ALB Conseil - TOOLS Project
7
3 EVALUATION AREAS
• Operational formative evaluations will provide
information and recommendations /eventual
corrective measures on how the project is being
developed
• Summative evaluations for PR and FR time are based
on formative evaluations, and address the correlation
between expected results and achieved results
• Additional evaluation for learning purposes at end of
project will take place in month 30 :
• the updated assessment of all the project’s specific achievements
to LLP and KA2, to end-users,
• the added-value and EU-level contributions that can be drawn from
the project at the end of its EU-funded lifetime.
Monday, July 06, 2015
ALB Conseil - TOOLS Project
8
IMPLEMENTATION APPROACH AND ACTIVITIES
• Two-fold approach:
• ON-GOING MONITORING AND EVALUATIONS
• ACTIVITIES: Participation to project meetings, quantitative and
qualitative tools/questionnaires and interviews to analyse and
record progress against key indicators, taking in account at
what extent they match on the project own internal evaluation
and quality management strategy.
•
• REFLECTIVE REPORTS: Summative and/or formative
evaluations of the overall progress of the project, partners'
contributions in relation to end project achievements,
innovative aspects, range/means of communication and
dissemination activities, external impact, exploitation and
sustainability.
Monday, July 06, 2015
ALB Conseil - TOOLS Project
9
MONITORING AND EVALUATION MODEL -TOOLS
•Quality Plan, monitoring
•QA Activities, Reports,
Deliverables
• Quality indicators &
Success criteria
• Exploitation QA
•Quality Control Group
•All TOOLS members
•Users Participants to pilots,
Partners Peer reviews
•EACEA KA2 monitoring /
control
•Complement Internal QA,
Objective evaluation of
•Quality of Work-process and
results
•Added-Value of Content and
learning
INTERNAL
EVALUATION
EXTERNAL
EVALUATION
INTERNAL
STAKE
HOLDERS
EXTERNAL
STAKEHOLDE
R USERS
• CLIL in ERASMUS,
LEONARDO, GRUNDTVIG
•Teachers and Student testing
/ using TOOLS in 8 target
languages
•Teachers of other LWULs,
VOLL in ALL 4 EDU sectors
Monday, July 06, 2015
ALB Conseil - TOOLS Project
10
‘TOOLS’ EXTERNAL EVALUATION OVERVIEW
• EVALUATION SCOPE: Results (assessment in line with
•
•
•
•
Lifelong Programme framework) and the education learning
processes and relations between the actors involved (partners
/users /stakeholders and results).
KEY AIMS: Measure, forecast, control, support; highlight value
/changes.
JUDGEMENT: Factual and quality judgements (based on the
EACEA criteria and/or judgement), and value judgments based
on project success indicators and other criteria during
implementation.
METHODS: Quantitative AND Qualitative Evaluation methods
ACTION-PLAN: pre-established (in line with TOOLS plan
work-and LLP/EACEA assessment criteria) and participatory
during the whole duration of 2,5 years.
Monday, July 06, 2015
ALB Conseil - TOOLS Project
11
QUESTIONS , FEEDBACK
ABOUT THE PROPOSED
STRATEGY, ACTIVITIES AND
AREAS BY THE EXTERNAL
EVALUATION?
This discussion can be linked with the internal
evaluation and Quality Plan
Monday, July 06, 2015
ALB Conseil - TOOLS Project
12
INTERNAL EVALUATION:
TOOLS QUALITY INDICATORS
The application set-up for each WP clear quality
indicators and ambitious quantitative indicators
Need to review and agree on the key indicators
Monday, July 06, 2015
ALB Conseil - TOOLS Project
13
DISCUSSIONS and AGREEMENT?
Internal evaluation
Main aspects for Kick-off Meeting
External evaluation
• Complementarity
• Collaboration
• Who is in charge in each
• Data gathering
•
•
•
•
partner organisation
WPs internal indicators of
performance
Reports
Frequency
Etc.
• Frequency
• Means: e-mails, skype,
participation to meetings
• Reports
• Recommendations
Monday, July 06, 2015
ALB Conseil - TOOLS Project
14
WE THANK YOU !
Angelica L. Bucur
ALB CONSEIL Manager
[email protected]
Lucia Leporati
Consultant
[email protected]
ALB CONSEIL PARIS - EVALUATION SERVICES FOR EU PROJECTS