OECD World Forum on Key Indicators

Statistics, Knowledge and Policy

Palermo, 10-13 November 2004

“Indicators and benchmarking as a support to the decision-making process: the Italian experience in active employment policies ”

Aviana Bulgarelli Director General for Vocational Training and Guidance Policies, Ministry of Labour and Social Policies, Italy.

Benchmarking in the public decision-making process

Benchmarking can be used for two types of tasks:

for a coordinated, consistent and systematic analysis

of performance, possibly tied to a system of

targets/objectives, with the definition of a set of significant and feasible indicators; for an identification of the most effective intervention

policies, models and instruments for achieving the

targets and pinpointing good/best practices to be transferred in various contexts.

Benchmarking in the public decision-making process

1.

2.

3.

4.

5.

Benchmarking process Identification of macro-objectives and corresponding performances to measure the success of implemented policies.

Identification of a system of elementary variables/indicators related to the macro-objectives.

Comparison between the national/regional situations examined and the reference benchmark. Understanding and explaining reasons for the differences observed in performance and identifying actions to be undertaken.

Monitoring policy implementation, analysing benchmarking results and, possibly, correcting the entire process.

Benchmarking in the public decision-making process

Key problems in public policies multiple objectives and, at the same time, specific intervention instruments and policies; alternative objectives and problems to identify the corresponding trade-offs; difficulty finding connections between specific policies/instruments available and the different objectives identified; plurality of stakeholders contributing to the identification of objectives and to the implementation and monitoring of policies.

European Employment Strategy (EES) and benchmarking

Benchmarking has often been used as a “surrogate” for evaluation, affecting therefore the decision-making process.

The EES is an example of multi-level strategic programming and represents the reference framework of a complex benchmarking exercise based on the open method of coordination.

Fixing objectives and guidelines for the EU with clear time-tables The open method of coordination Periodic monitoring, evaluation, peer review as a mutual learning process Establishing indicators and benchmarks to compare best practices Translating the guidelines into national and regional policies

European Employment Strategy (EES) and benchmarking Example from the guidelines for the employment policies of the Member States in the EES (Council Decision of 22 July 2003) Overarching objective: full employment

Overall employment rate: 67 % in 2005 and 70 % in 2010 Employment rate for women: 57 % in 2005 and 60 % in 2010 Employment rate for older workers (55 to 64): 50 % in 2010

Specific guideline: 4. Promote development of human capital and lifelong learning

By 2010: at least 85 % of 22-year olds in the EU should have completed upper secondary education the EU average level of participation in lifelong learning should be at least 12,5 % of the adult working-age population (25 to 64 age group).

European Employment Strategy (EES) and benchmarking: the Italian experience

EES process  Maastricht Stability Pact process but with a greater degree of freedom: Social and labour policies are more complex in terms of objectives, target population and effectiveness of intervention tools Complex decision-making mechanism involving a large number of stakeholders at different decision-making levels with high political, and managerial responsibility However, the EES has had an influence on the action of policy-makers: at a national level at a regional level at a local/provincial level in some cases

European Employment Strategy (EES) and benchmarking: the Italian experience

Reform process of labour market in Italy Implementation of labour monitoring system Influence of EES and EU benchmarking model stronger emphasis on specific targets (female and older population) even when the priority does not fully correspond to local labour market conditions multiplicity of stakeholders at many decision-making levels with strong vertical and horizontal subsidiarity (the statistical information system reflects this complexity) the objective of complying with the Lisbon benchmark has required the creation of many technical units involving a large number of stakeholders

European Employment Strategy (EES) and benchmarking: the Italian experience

Redirecting policies: the case of lifelong learning (I) Indicator for lifelong learning = participation rate of population aged 24 65 in permanent and continuing training Lisbon benchmark: 12.5% by 2010 Italy: 4.7% participation rate in 2003 EU average: 9.7% participation rate in 2003 Italian context: strong disparities among generations and among regions (especially in southern Italy)

+

time and budget constraints.

Problem:

Significant impact on policy-makers, but little room for manoeuvre  shifting resources towards lifelong learning policies to the detriment of alternative intervention policies would have political and organisational consequences.

European Employment Strategy (EES) and benchmarking: the Italian experience

Redirecting policies: the case of lifelong learning (II)

How to make the most of available resources?

According to Eurostat’s continuing vocational training survey (CVTS2), 24% of Italian enterprises carry out training activities compared to an EU average of 62%. Key factor: majority of small and medium-sized enterprises and traditional sectors, with little interest to offer training to employees: Investing in human capital has no tangible effects in the short term Greater incidence of training costs (both direct and indirect) Smaller organisational and logistical capacity Uncertainty regarding returns of training

Introduce new forms of interventions rather than financial incentives

European Employment Strategy (EES) and benchmarking: the Italian experience

Redirecting policies: the case of lifelong learning (III)

Creation of the Bilateral funds for continuing vocational training – Fondi paritetici interprofessionali per la formazione continua Direct involvement of social partners

(managing of financial resources, planning and directing interventions, organising and integrating local and sector needs, monitoring the outcomes of activities)

Aims:

encouraging a greater dissemination of the training culture and activities (especially with regards to smaller-sized enterprises) steering financing towards initiatives more in line with the actual requirements of companies

European Employment Strategy (EES) and benchmarking: the Italian experience

Influence of benchmarking at a local level The Italian Provinces are required to play a leading role in active labour policies and have also been involved in the benchmarking system proposed with the EES.

The experience of Local Action Plans for employment is a typical example, where many local authorities have set up integrated strategic programming tools for training, education and labour, within the framework of EES guidelines.

Indicators, benchmarking and evaluation

Main methodological and technical problems Coherence and comparability: better no comparison than a “wrong” one (e.g.: LLL indicator of Continuous Labour Force Survey  poor statistical visibility of short training courses) Relevance: in a multi-level programming system, important to understand “who has to decide what” and to provide suitable cognitive support (e.g.: Local Action Plans in Italy) Timeliness 1: indicators reflect an “out-of-date” vision of the economy  always ex-post and often available after a long delay (e.g.: monitoring process of EES implementation) Timeliness 2: acceleration of economic processes requires a switch from reactive to proactive approach  Can we construct a common system (methodologies and/or parameters) for estimating indicators?

Transferability: differences between contexts in which policies are implemented can have a great impact on their effectiveness

Indicators, benchmarking and evaluation

Relationship between benchmarking and policy evaluation How can we obtain information actually enabling to steer policy choices?

How can we overcome the ex-post nature of benchmarking?

How can we have a system consistent with the specific level of the decision-making process?

Debate: Can we raise benchmarking from a tool for comparing and analysing policy performance to an input for the evaluation process?