Transcript Evaluation of SF Some German experience

Twinning Project RO/06/IB/SPP/01
Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds
National Evaluation Conference
Bucharest, 18.02.2009
Evaluation of SF
Some German experience
Benno Savioli
FEEDBACK-Bremen
[email protected]
Twinning Project RO/06/IB/SPP/01
Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds
Decentralised System of Implementation
 Federal State and 12 / 16 federal Länder
 Structural Funds since 1989 – now 5th funding period
 Mixing & switching implementation between centralisation &
decentralisation according to:



Funds, Funding Period,
Objective 1 (former GDR) and Objective 2/3 areas,
Single Programming Document (e.g. ESF 2000-2006) or multiple OPs
approach (current ESF) on side of COM
 Management of evaluation reflected this variety of
implementation settings
[email protected]
2
Twinning Project RO/06/IB/SPP/01
Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds
History of shifts in approach to Evaluation I
 Ideal function of evaluation

Instrument for critical reflection regarding to adequacy, efficacy and
efficiency of public policy interventions
 Introducing abstract public interest (general stakeholder) by objective
methodology  creating legitimacy
 Complement to NPM discourse
•
•
•
Active policy /conditional programmation
Commodification of public policy implementation
Functional equivalent to market prices  neutral assessment / judgment
 Actual function: Support to management



Monitoring substitute
Monitoring complement
Publicity & policy justification
 Administrative attitudes


Impingement and reluctance
Benchmarking /QMS
•

mainly focused to providers /administrative performance not that much
More active use as tool for publicity and as a rationale for action
[email protected]
3
Twinning Project RO/06/IB/SPP/01
Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds
History of shifts in approach to Evaluation II
 Selection of evaluators, types of evaluators, capacity building



Direct grants to public agencies supporting structural funds
implementation
Grants based on calls for proposals (NGOs, University institutes)
Public procurement (private research institutes, consultancies)
 Setting up networks & partnerships



Informal network of ESF evaluators for exchange of experience,
harmonisation of approaches, managed competition, active shaping
monitoring via evaluation sub-committees of monitoring committees
EES – European Evaluation Society – Professionalisation, Research and
exchange
German Evaluation Society (DeGEval) – Lobbying, standardisation and
recommendations, professionalisation by training and topic related
working groups
[email protected]
4
Twinning Project RO/06/IB/SPP/01
Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds
Effects of EU evaluation request
 Administration got used to it
Without Structural Funds this wouldn’t have happened
Number and scope of evaluation expanded (also outside SF)
Administration set up technical teams (capacity raised – partially by
hiring former evaluators)
 Administration took evaluations more under control (political risks)



 Methodology applied evolved


Net effects (control /comparison group designs)
Impacts and sustainability – macro-economic modelling
 Professionalisation in general



Standardisation
Post graduate courses
Business
 BUT: Moderate influence on policy design

Evaluation usually ‘comes to late’ for political process
[email protected]
5
Twinning Project RO/06/IB/SPP/01
Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds
Two highlights
 German ESF Stammblatt – Evaluation shapes Monitoring
 Common and overarching all ESF programmes approach for standardised data
gathering on
•
•
•

Projects (type of activity, target groups, benefits offered, region, financial data)
Project providers (type, region, size, economic classification)
Participants (target groups, gender, age, education, unemployment, participation in
active measures, individual results, labour market status change)
Common classifications and indicator definitions
 CIP EQUAL evaluation
 Systematic multi-level evaluation: EU-wide /national /Projects (DP)
 Intense exchange between national and DP-evaluations
•
•
•
•

Guidance function of national evaluation
Workshops for DP evaluators on SF-general, Innovation, Gender Mainstreaming, etc
Handouts on topics of common concern and interest
DP evaluators as source for national evaluation (regular questionnaires, interviews,
etc.)
Tremendous increase in evaluation experience by number of involved people
[email protected]
6
Twinning Project RO/06/IB/SPP/01
Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds
Positive influence of SF evaluation and promoting factors
 Professionalisation on both sides


Administration
Evaluators
 Good cooperation & coordination

Cooperation and professional exchange (even under potential
competitors)
 Active coordination of evaluation by MAs and monitoring subcommittees ( strong /well renowned topic owner)
 Independence and fairness
Procurement rules can’t guarantee independence
Independence is a matter of professional attitude and its evidence is
given in the process of examination and presentation of results
 Evaluation is not the judge himself but provides valid information for
rationale judgments


[email protected]
7
Twinning Project RO/06/IB/SPP/01
Support to Authority for Coordination of Structural Instruments (ACIS) to ensure a sound and efficient management of the EU Structural Funds
Thank you for your attention!
Now the floor
is open for you
!!!
[email protected]
8