Transcript Slide 1

Erasmus Mundus
programme experience
Karsten Suhre, external expert to EACEA
Riga, 25.02.2011
1
Outline
• To write a successful proposal, it is essential to
do this from the point of view of the person who
eventually evaluates it.
• The following slides have been used by the
agency to brief its external experts for the
evaluation.
• Here I shall present, at hand of concrete
examples, how this briefing has been put into
practice.
2
An Erasmus Mundus Action 1 Joint
Programme (at masters or doctoral level)
is...
... an integrated study (/research) programme,
delivered (/managed) by a consortium of EU
and, if relevant, non EU HEIs (/research
organisation) that includes a mandatory
mobility component and leads to the award of a
fully recognised joint (/double, multiple)
degree.
3
Action 1 – Joint Programmes
Joint Programmes should have a double objective:
1. To offer high level training possibilities in a large
variety of thematic fields to excellent postgraduate students /
doctoral candidates from all over the word
 Proposals have to demonstrate their excellence, expertise and
innovation potential in the thematic field(s) concerned
2. To contribute to the shaping of a European Master
and European Doctorate model
 Proposals have to demonstrate strong programme integration
and institutional commitment, addressing the EM programme
requirements and recommendations but also those defined in the EU
education and research policies (Bologna and Ljubljana Processes)
4
An EM Joint Programme must be
• Excellent in all its dimensions
– Outstanding academic / research quality
– Professional and transparent admin. /fin. /H.R. management
• Integrated in all its dimensions
– Jointly designed
– Joint application, selection and assessment criteria
– Joint students registration and admission conditions
– Programme content fully recognised by all partners
• Sustainable beyond EU funding
– Strong Institutional Commitment
– Diversified sources of funding
• Closely linked to its socio-economic environment
– Added value for the academic / professional sector
– Role of “associated members” (programme’s sustainability and
evaluation; students /doctoral candidates employability)
5
Joint Programmes may vary for what
concerns:
•
•
•
•
•
•
•
•
their duration,
thematic area (broad or narrow)
role (/number, /origin) of individual partners,
duration (/number) of mobility tracks,
professional or research orientation,
placement/internships possibilities,
type of final degree(s) awarded,
etc.
There is no prescribed model, BUT the selected model
must solid, integrated, innovative, and take full
account of the learning (/research) objectives as well
6
as the participants’ needs and constraints.
Erasmus Mundus Action 1
Joint Doctoral Programmes
Specificities
7
WHAT IS AN EMJD (1) ?
o
A three- to four-year integrated postgraduate programme
at doctorate level,
o
designed and delivered by a consortium of universities
representing at least 3 different European countries,
o
including a mandatory training / research period in at
least 2 of the European consortium countries,
o
leading to the award of fully recognised double, multiple
or joint degree(s),
o
offering EM fellowships for European and non-European
doctorate candidates
8
WHAT IS AN EMJD (2) ?
o
EMJDs are based on the EMMCs model and on the
experience of Marie Curie fellowships
- Marie Curie focuses on the individual research of
its grantees in a given research area,
- EMJDs put more emphasis on a long-term
structured inter-university cooperation model and
the design of an integrated doctoral programme
o
EMJDS have a double objective
- To contribute to the shaping of doctoral studies in
the EAHE
- To support young researchers and offer them a
framework of excellence to implement their PhD
activities
9
AN EMJD MUST (1) :
o
Be fully developed at the time of the application and ready to
run for 5 consecutive editions (as from acad. year 2011/12)
o
Be an integrated programme in all its aspects
o
Define its own Participation Costs (/ programme fees); these
costs
-
must be transparent and independent from the candidate’s
mobility track / HEIs visited;
-
can be different for « laboratory » and « non laboratory »
based research;
-
should be co-funded by the consortium (/other sponsors) if
higher than the EM fixed contribution.
10
AN EMJD MUST (2) :
o
Be based on a consortium agreement signed by the governing
bodies of the partner HEIS
o
Recruit the doctoral candidates through employment contracts
(except in adequately documented cases or when national
regulation would prohibit this possibility)
o
Independently from the training/research language, provide
for the use of min. 2 European languages spoken in the
candidate’s host countries
o
Follow the principles described in the European Charter for
Researchers and the Code of Conduct for the recruitment of
Researchers
11
AN EMJD MUST (3) :
o
Sign detailed Doctorate Agreements with the
candidates defining the EMJD rules and both parties
rights and obligations
o
Where applicable, ensure that the relevant ethics
committees have approved the individual research
projects
o
Support third-country candidates in obtaining their
visas and residence permits
o
Take the necessary measures to promote the joint
programme and recruit top class doctoral candidates
12
EMJDs overall assessment
Will this proposal actively contribute to the training
of excellent young researchers and to the innovation
process in the thematic field(s) concerned ?
o
Will this proposal contribute to the shaping of a
strong and sustainable « European Doctorate » model
in line with the objectives of the Bologna Process and
which follows the prescriptions included in the « Code
of Conduct for the Recruitment of Researchers » and
the « European Charter for Researchers »?
o
Does this proposal indeed demonstrate the capacity
and the readiness to run if selected ?
o
13
EMJDs AWARD CRITERIA
Academic and Research Quality (25%)
o
o
How clearly described and justified are the:
o objectives and needs analysis (scientific and socioeconomic terms)
o Added value and distinctiveness (comparing to
other existing programmes) and
o Outcomes of the proposal
How does the proposal address:
o Originality and innovation
o Inter-sectorial and inter-organisational collabor°
14
EMJDs AWARD CRITERIA
Partnership experience and composition (25%)
o
How well does the proposal describe and justify:
o The
composition and the expertise of the
partnership and its relevance to the objectives of
the proposal
o Relevant international cooperation and networking
activities of all partners
o Diversity and complementarity of the partners in
terms of activities and types of organisations
o The
inclusion and appropriateness of the
professional sector
15
EMJDs AWARD CRITERIA
European integration and functioning of the EMJD (20%)
o
How clear is the description of integration aspects of the proposal
in terms of:
o
relevance and coherence of the mobility periods
o
Common mechanisms and standards for the
admission and review of candidates
o
the rationale and method of calculation of participation costs
and how these are going to be used within the consortium
o
Existence of joint supervision and monitoring mechanisms of
the candidates to ensure the highest quality of outcomes
o
Clarity about the degree to be awarded and its recognition and
possible measures for a possible joint degree
selection,
16
EMJDs AWARD CRITERIA
Provision for EMJD candidates and fellowship holders (15%)
o What does the proposal foresee for:
o The strategy to attact excellent candidates
o The services that will be offered to the selected candidates
o The linguistic support and policy in order to provide candidates
the possibility for the use of at least two different European
languages
o The overall management of the fellowship scheme, the
consortium mechanisms for the use of employment contracts
and a clear and documented justification in case of the use of
stipends
o The further career of the candidates and
o The clarity of rights and obligations of the candidate and the
consortium documented on a Doctoral Candidate Agreement
17
EMJDs AWARD CRITERIA
Programme management and quality assurance (15%)
o
How clear and sound are the proposed arrangements for cooperation
among the partners and how strong is the institutional commitment of all
of them
o
What is the planned sustainability strategy of the consortium to continue
after Community funding and how convincing is it (in terms of targets
and possible complementary funding possibilities explored and secured)
o
What will be the consortiums own internal evaluation methods and what
are provisions for using external bodies for review and feed-back?
18
Erasmus Mundus Action 1
Joint Masters Courses
Specificities
19
WHAT IS AN EMMC ?
o
A 1 to 2 years integrated programme (masters level)
o
delivered by a consortium of universities located in at
least 3 European countries
o
including a mandatory study period for students in at least
2 of the European countries of the consortium
o
leading to the award of a fully recognised double,
multiple or joint masters degree
o
offering EM scholarships for students and scholars (EU and
non EU).
20
AN EMMC MUST
o
Be fully developed at the time of the application - ready
to run 5 consecutive editions
o
Take the measures to recruit top class students and
invite high quality scholars
o
Support candidates in obtaining their visas & residence
permits
o
Independently from teaching language, provide for the
use of min. 2 European languages spoken in the
student’s host countries.
o
Provide insurance coverage to students
21
OVERALL ASSESSMENT
 What is the added value of the EMMC
proposals from the innovation, academic,
research, employability points of view, at EU
and worldwide level?
 Is the proposed joint course excellent, solid,
reliable and coherent enough, from the
content, organisational and structural points of
view, to guarantee that this expected added
value will be actually provided?
22
EMMC Award Criteria
A1 - Academic quality - Course content (30 %)
•
added value of the course in terms of academic
quality/innovation (EU and worldwide)? Contribution to
excellence of EU HEI
•
academic quality/coherence of the course? Contribution of
the partner HEIs in terms of resources and expertise
(added value of third country HEI if any)?
•
interaction with non educational actors and impact on
academic quality, long term sustainability, students
employment prospectives?
23
EMMC Award Criteria
A2 - Course integration (25%)
How well is the course integrated in all its aspects?
•
•
•
based on a truly integrated course recognised in
participating countries and leading to the award of a
double, multiple, joint (even better) degree?
joint student selection and examination procedures
defined by the consortium?
participation costs been jointly calculated taking into
account the needs and resources of the course as a
whole?
24
EMMC Award Criteria
A3 - Course management, visibility and sustainability
measures (20 %)
•
Has the consortium set up suitable management
mechanisms/tools/structures that will result in an
effective implementation of the course, and rely on the
partners’ high level of commitment and effective
cooperation?
•
How will this cooperation lead to sound management of
financial resources, ensure efficient course promotion
and attractiveness and lead to a strategy for long term
sustainability?
25
EMMC Award Criteria
A 4- Students’ services and facilities (15%)
•
Which services offered to students / invited scholars to
ensure that the best ones apply and fully devote their
time and attention to the course?
i.e information provided to students prior to their
enrolment , support for visa , hosting services,
insurance scheme, language facilities, networking
resources
26
EMMC Award Criteria
A 5 - Quality assurance and evaluation (10 %)
•
How will the consortium organise the evaluation of
the content but also of the management of the course
and involve students and scholars?
•
Will they develop joint evaluation mechanisms also
relying on the expertise of external experts?
27
Erasmus Mundus Action 1
The Assessment Procedure
and
The Role of Independent Experts
28
Experts’ Role and Obligations (1)
• Experts have been selected on the basis of their application to the
« EACEA Call for Experts »
• Their role is to
 Provide comments and recommendations to decision making
bodies AND to the applicants
• Their obligation is to
 Provide an independent, impartial and objective assessment
 Guarantee the total absence of conflict of interest with the
proposals assessed and/or discussed in the panels
 Keep all the information gathered during the assessment phase
confidential
 Respect the assessment timetable and practical instructions
 Provide coherent, meaningfull, usefull (and understandable!)
comments to the applicants
29
Experts’ Role and Obligations (3)
• Experts Assessment must
– be focused ONLY on the award criteria as defined in
the Call
EVERYTHING IS IN THE CALL AND THE EXPERTS MANUAL
– Cover both the academic excellence AND the quality of
the joint programme’s integration
• We have tried to map the proposal’s thematic field with the
experts profile but this has not always been possible
SECTORAL PANELS WILL BE USED TO CLARIFY PENDING
ISSUES (INCL. THEMATIC ONES)
30
Individual assessments
Experts have to:
• Provide comments for each Award Criteria covering each
individual issues / questions addressed
• Provide an individual score for each of issue / question
raised under each Award criteria
• Provide a selection recommendation
• Highly Recommended
• Recommended
• Not Recommended
• Provide Comments to EACEA (if applicable)
31
Marking Scale
Cannot be improved /High degree of
agreement among evaluators
Some very good points, Good overall with
respect to the criteria, improvement possible
Some good points and some weaknesses.
Fair overall with respect to criteria
Notable weaknesses in relation to the criterion
Poorly presented, confusing information or
poor technical content
Fails to address criteria or missing
information
5
4
3
2
1
0
Excellent
4.5
4.0
3.5
Very Good
Good
3.0
2.5
Fair
2.0
1.5
Poor
1.0
Marks must not go beyond half points
Mark of “0,5” should not be given
32
Consensus discussions
Purpose: to assure that both experts have a common
understanding of the proposal:
• If divergence of final scores is more than 20 points,
(globally OR per criterion) experts 1&2 discuss to see
reason for the differences. If necessary the lead expert can
help. If the divergence is confirmed a third expert will
assess the proposal
33
Consolidated Assessment
Expert 1, in close collaboration with Expert 2, will be
responsible for
• drafting the final text of the consolidated assessment
• providing the final scores (by default, the assessment tool
will suggest the mathematical average between the two
scores but experts can revise it in accordance with their
consensus discussion)
The consolidated Assessment will have to be endorsed
(/signed) by both experts
34
Role of Lead Experts
•
•
•
•
•
•
Experts experienced in assessing EM Action 1 Joint
Programmes
Will coordinate the work of the independent experts within
each of the 6 (2 x 3) thematic groups, from the content
(coherence, completeness, relevance of the comments and
scores) and procedural (/timing) points of view
Will stay in close contact with their experts and EACEA during
the assessment exercise
Will assist the individual pairs of experts during the consensus
discussions
Will act as 3rd assessor in case of divergence
Will chair the thematic panels and assist the Selection Board
for the production of the final ranking list of proposals
35
Questions?
36