Transcript Methodology of evaluation of transnational cooperation

Transnational Cooperation and
Partnership in EQUAL
Lessons from to-date experience in the context
of 2007-2013 programming period
Cooperation and Partnership in Development of Education and Science in the Period of 2007-2013
11-12th of June, 2007, Vilnius
Krzysztof Jaszczołt, National Evaluator of EQUAL Initiative in Poland,
KDU-CASE Consortium, tel: +48 504071298; E-mail: [email protected]
Transnational Cooperation (TC) and
Partnership in EQUAL
Plan
• CI EQUAL in Poland
• Transnational Cooperation in EQUAL
- Framework
- Application
• Assessment of TC in EQUAL, Poland
• Conclusions and recommendations
• Key findings from other EQUAL evaluations
• TC in 2007-2013 programming period
• Example of the OP "Human Capital" in Poland
EQUAL in Poland
Key Facts
Community Initiative EQUAL
The objective of EQUAL is to test and promote – within
international co-operation – new ways of combating all forms
of discrimination and inequality on the labour market.
EQUAL in Poland
Program structure
3 Implementation Phases
Activity 1: Building Partnerships (1-6/2005)
Activity 2: Implementing Projects (7/2005 – 12/2007)
Activity 3: Disseminating Results (10/2007 – 3/2008)
5 Technical areas
A: “Access and return to the labour market”,
D: “Social Economy”,
F: “Adaptability of firms and employees”
G: “Reconciling family and professional life”
I: “Supporting the integration of asylum-seekers”
6 Principles
Thematic approach, Partnership, Empowerment,
Transnational Cooperation, Innovation, Mainstreaming
EQUAL in Poland
Intervention Logic
Developmental hypothesis of EQUAL
If we
- focus on selected fields (Thematic approach), and
- allow different sectors to join their efforts (Partnership), and
- invite beneficiaries to actively participate (Empowerment), and
- create conditions for international cooperation (Transnationality)
Then
- effective mechanism to develop innovative approaches will be created
- new solutions will be generated and tested (Innovation)
- successful ones will be widely adopted (Mainstreaming)
In result
- discrimination and inequalities in the labour market will be limited
EQUAL in Poland
Key Facts
- 99 Development Partnerships (DPs)
- 99 Leaders (NGO 59%, PA 16%, EDU 15%, Firms 10%)
- 690 Partners (average 7 / DP)
- Total DPs’ budget = EUR 150 mln (average EUR 1,5 mln / DP)
- Resource utilization by 3/2007: Time 67%, Budget 30%
Transnational Cooperation in EQUAL
Key Facts
Organization:
- TC is a Key Principle and integral part of every DP project
- Each DP is obliged to enter into at least one TCA
- TCA must be approved by Managing Authority in each country
- Common Database of EQUAL, ECDB – networking tool
Resources:
- Total TC budget = EUR 8,8 mln (6% of the program budget)
- Expenditures by 3/2007: EUR 1,7 mln (EUR 17 600 / DP)
Transnational Cooperation in EQUAL
Research basis:
3 Editions of on going evaluation research
-
Program monitoring data
-
Questionnaires for all DPs (leaders and partners)
-
Questionnaires for MA, NSS, MC
-
40 Case Studies
-
Interviews with the key actors in the program
Transnational Cooperation in EQUAL
TC is a meeting
"Transnational Cooperation is about meeting across borders and
cultures to take benefits of similarities and differences"
As any other form of meeting,
- it has its goal, participants, topic, context, etc
- its effectiveness will be higher, if it is well prepared and managed
Transnational Cooperation in EQUAL
TC is a meeting
Key questions:
Goal:
Why to meet?
Participants:
Who to meet with?
Topic:
What to meet about?
Context:
When and where to meet? Who pays the bill?
Management:
How to meet?
Transnational Cooperation in EQUAL
Why to meet?
Key advantages:
- to take benefit of multi-country experience
- to make a better use of results
Other advantages
- increased knowledge on discrimination and exclusion
- benchmarking across Member States
- transfer of managerial know-how
- credibility to the work carried out by international partnerships
Transnational Cooperation in EQUAL
Who to meet with?
Criteria used by Polish DPs w hen
selecting foreign partners
Facts:
96%
Field of activ ity
Willingness
40%
Creditworthiness
21%
Resources
21%
142 TC Agreements signed (1,4 / DP)
490 international partners (3,4 / TCA)
Country structure of Polish DPs:
Italy - 22%,
Similar culture
Spain - 12%,
17%
Germany 10%,
Language
6%
France 8%,
Other criteria
5%
Other countries 48% (Lithuania 1,6%)
0%
25%
50%
75%
100%
Transnational Cooperation in EQUAL
Who to meet with?
Sources of inform ation on
potential foreign partners
Observations:
ECDB data base
83%
Time pressure constraint
Partner initiativ e
82%
Effective process in quantitative terms
Recommendation
Earlier contacts
10%
Generally positive experience w/ ECDB
8%
Criteria were reasonable, but simple
Publications
6%
Personal contact
4%
Limited pool of partners and knowledge
Other sources
4%
Few “already known” partners
0%
25%
50%
75%
100%
Transnational Cooperation in EQUAL
What to meet about?
Transnational Coopperation - Expectations
% of respondents, w ho declared a priority
Models of TC
60%
Exchanging experience
Exchange of information / experience
42%
55%
Learning on others
Parallel development
46%
48%
45%
Access to tools
Import, export and adaptation
37%
Finding partners
29%
Common product
37%
32%
Joint project
0%
20%
DP Partners
40%
60%
80%
DP Liders
Exchange of people / services
Transnational Cooperation in EQUAL
What to meet about?
Observations:
Most DPs didn't expect "common
product” or “import/export” model.
Was the Scope of Work of your TNP
properly defined?
57%
60%
40%
34%
20%
5%
4%
neither "yes"
nor "no"
Rather no
0%
0%
Decisively yes
Rather yes
Decisively no
Preference to "info exchange" and
"learning" is natural, since DPs:
- have no experience in TC
- have limited knowledge on partners
- are building relations from scratch
- are delayed with their in own products
- are self-focused
Transnational Cooperation in EQUAL
When and where to meet?
Facts:
Frequency of direct m eetings w ith foreign
partners (annualized values)
6,0
6
5,0
4,4
3,5
4
3,9
4,6
3,5
2,6
1,9
1,6
2
1,1
1/05 - 6/05
In Poland
7/05 - 6/06
7/06 - 9/06
TCAs were concluded in the 2nd
quarter of 2005 - intense contacts
TC slowed down in the beginning of
Act.2
1,6
0
Abroad
Delay of Act.1 Agreements caused
problems in setting TCAs
Total
10/06 - 3/07
Since the 2nd quarter of 2006 TC is
systematically growing again.
Meetings abroad prevail in every
phase.
Transnational Cooperation in EQUAL
Who pays the bill?
DPs spendings on Transnational
Cooperation (by quarter)
DPs spendings on Transnational
Cooperation (% distribution by quarter)
'000
2500
2 115,5
1 867,6
2000
40%
39%
30%
1 498,4
26%
29%
20%
1500
20%
19%
892,1
1000
10%
6%
273,9
500
0
IV 05
IV 05
I 06
II 06
III 06
IV 06
5%
2%
0%
8,6
I 06
II 06
III 06
IV 06
I 07
Area A
Area G
I 07
Transnational Cooperation in EQUAL
TC framework
Observations:
Opinion on the size of TC budget Assessm ent by DP Leaders
40%
TC influence on DP project:
55,8%
60%
(1) project design phase – limited
(2) implementation – varies
(3) dissemination of results – ?
28,6%
20%
5,2%
10,4%
0,0%
0%
Decisively
sufficient
Rather
sufficient
Difficult to say
Rather
insufficient
Decisively
insufficient
Delay in financing slowed down the
process
DP Liders' opinion on Transnational
Cooperation rules in EQUAL
Relationship between national
projects and TNP is not clear
59,7%
60%
40%
Different timetables and approaches
expenditures among MS
32,5%
Budget is not a problem, liquidity is
20%
6,5%
1,3%
0,0%
Poor
Very poor
0%
Very good
Good
Average
Participants are positive on the
general framework for TC in EQUAL
Transnational Cooperation in EQUAL
How to meet?
Approach to coordination of TNP's
activities
70,8%
75%
50%
Facts:
Most TNPs are managed by a
coordinator (R1 partner or by rotation)
29,2%
25%
0%
Without coordinator
With coordinator
Method of cooperation in TNP
75%
50%
61,1%
38,9%
In most cases technical activities are
managed by a joint workplan
Preference to working-type relations &
coop. forms (workshops, study-tours)
25%
0%
Parallel workplans
Remaining TNPs establish a common
secretariat and a Steering Committee
Joint workplan
Transnational Cooperation in EQUAL
How to meet?
Means of TC - DP Leaders' opinion
3,8
E-mail exchange
97%
4,1
Workshop meeting
Conference / Seminar
3,7
78%
67%
4,0
Study tour - Beneficiaries
31%
Individual visits
31%
3,9
0
1
2
Utilization
3
4
5
Effectiveness
How m any partners are actively involved
in TNP operations?
60%
34%
18%
20%
5%
0%
0%
Majority
More sophisticated models can be a
challenge.
Issues in communication, coordination
and in-consistent expectations
Relevance of TC work in the context of
national projects
43%
Every one
Untli TC means "exchange of info"
current structures are sufficient.
4,4
Study tour - Experts
40%
Observations:
91%
Difficult to
say
Minority
No one
Transnational Cooperation in EQUAL
Assessment of TC
Two dimensions:
(1) Are there necessary conditions in place to
allow for effective TC?
(2) What are the results of TC?
Transnational Cooperation in EQUAL
Assessment of TC
TC Success Factors:
1. Willingness to cooperate and share experience
2. Ability to cooperate (knowing needs, having skills)
3. Possessing developed & documented products
4. Knowing rules and procedures of TC
5. Giving consideration to cultural context and legal limitations
Transnational Cooperation in EQUAL
TC Success Factors (1)
1. Willingness to cooperate and share experience
Analyzed elements
Conclusions:
- 97% of participants believe that TC is
-Attitude of participants towards TC
“useful” or “very useful” (improved).
- Readiness to share
- Cases of lack of willingness are marginal
- Way of integrating TC in national project
- DPs approach TC differently
2. Ability to cooperate (knowing needs, having skills)
Analyzed elements
Conclusions:
- Clear concept of DP's own product
- Product concepts were not ready for TC
- Clear role of TC in the develop. process
- "Info-Exchange" model prevails (bilateral)
- Experience in TC
- Lack of experience is an issue
3. Developed and documented own product to offer
Analyzed elements
- Being conscious of what we have
- Having it ready for exchange
Conclusions:
- Delayed DPs' own products
- Polish DPs rather “take” than “give”
- Experience could be interesting for NMS
Transnational Cooperation in EQUAL
TC Success Factors (2)
4. Knowing rules and procedures of TC
Analyzed elements
- Timetable
- Financing (budget, costs, documentation)
- TC mechanism (forms of coop, support)
Conclusions:
- Different TC timetables
- Shortened time for effective TC
- Periodic liquidity problems
- TC forms were found sufficient and useful
- 2 TNP seems optimal
5. Giving consideration to cultural context and legal limitations
Analyzed elements
Conclusions:
- Comparability of systems and conditions
- Not direct transfer but “creative adaptation"
- Cultural similarities and differences
- Positively on cultural differences
- Communication (language)
- Language is a problem (partial solutions)
Transnational Cooperation in EQUAL
Barriers
Problem s in TC
- Assessm ent by MA-NSS-MC
Problem s in TC
- % of DP liders w ho raised an issue
(groups of problems, 1-3 scale)
Communication problems
49%
Coordination problems
2. Ability to cooperate
2,06
3. Documented products to offer
2,05
46%
Different expectations
45%
Lack of common areas
24%
Financing problems
18%
Lack of willingness
5%
0%
1. Willingness to share
1,82
5. External conditions & limitations
1,81
4. TC rules and procedures
20%
40%
60%
1,67
80%
0
1
2
3
Transnational Cooperation in EQUAL
Barriers
Problem s in TC
Assessm ent by MA-NSS-MC
(individual problems, 1-3 scale)
DPs’ perspective:
2,3
lack of experience
2,2
products to offer
- Communication,
- Coordination
- Different expectations
2,1
confidence in TC
language problems
2,0
incoherent systems
2,0
different expectations
2,0
limited means of TC
2,0
agreement on financing
1,9
lack of common areas
1,9
knowledge of needs
1,9
open attitude
1,9
incoherent timetables
Other issues: personnel rotation,
approvals by MAs, lack of time for TC.
MA-NSS perspective:
1,8
coordination problems
1,7
- Ability to cooperate (experience, needs)
- Delay in product development
- Communication
1,6
different allowable costs
1,5
burden of DP's work
budget shortages
1,4
cultural differences
1,4
lack of time
1,3
0
1
2
3
Transnational Cooperation in EQUAL
Results of TC (1)
DP Liders' assessm ent of technical and
m anagerial benefits from TC (1-5 scale)
5
3,83
4
4,07
3,15
3,30
3,61
2,73
3
By managerial and technical dimensions:
2
1
Technical benefits
Managerial benefits
September '06
Average note
March '07
Exam ples of TC Benefits
Liders (87) and partners' (59) opinion
80%
72%
64%
36%
28%
Technical benefits
Managerial benefits
20%
0%
Liders
- TC results refer mostly to technical sphere
of projects
- Number of positive examples of TC
increased (9/06 – 65%; 3/07 – 73%),
60%
40%
- DP leaders’ opinion improved
Partners
Transnational Cooperation in EQUAL
Results of TC (2)
TC benefits by type of exchange
- information
- ideas & inspirations
- transfer of tools and methods
- development of common products
- exchange of people / services
Transnational Cooperation in EQUAL
Results of TC (3)
TC Benefits - Technical dim ension
(DP Liders' opinion, 63 cases, )
Info & experience
Observations:
20
Ideas
12
Applications
Benefits claimed by DPs are typical for an
early phase of TN relationship
16
Common product
9
Exchange of services
3
Others
3
0
10
20
30
- One of four DPs has actually applied a
solution generated through TC
TC Benefits - Managerial dim ension
(DP Liders' opinion, 24 cases)
- Participation in a joint TC project was
mentioned by only one of ten DPs
8
info & experience
Ideas
6
Applications
6
- Instances of exchanging services,
trainers or project staff are very rare (3%)
4
Others
0
- 53% of statements refer to "infoexchange" or "inspirations"
10
20
30
Transnational Cooperation in EQUAL
Results of TC (3)
Assessm ent of utility of TC in the context
of DP's goals
5
4
3,62
3,87
4,13
3,95
4,30 4,47
3,87
4,13
3,85 4,02
4,25
3
2
1
MA-NSS
June '06
Liders
Partners
September '06
Av erage
March '07
Observations:
In spite of the generic nature of to date
benefits, all EQUAL participants have
positive opinion on the utility of TC.
Assessment of this element of the
program is getting better in subsequent
editions of the evaluation research.
Transnational Cooperation in EQUAL
Conclusions / Recommendations:
1. Although the conditions created for TC were not optimal, to date results are
encouraging. Taking into account very positive opinions on utility of TC, we have
assessed this element of the program as “good” (4 pts in 1–5 scale).
2. Building TN relationship from the scratch is a process that requires significant time.
Linking it with a goal to develop complex technical project can bring an adverse effect
(constraining not supporting factor).
3. Depending on circumstances, TC can be considered as a value by itself, or as a
horizontal objective, subordinated to the primary goalsof the program. In the case of
demanding, innovative measures, as EQUAL is, we recommend the latter option.
(Simple exchange of information and experience is not sufficient to fully utilized
opportunities available through TC)
Transnational Cooperation in EQUAL
Conclusions / Recommendations:
4. In order to raise TC to more ambitious level (exchange of tools and services, joint
ventures, etc), it should be project- not partner-driven. To this end phase of setting
TNPs should be implemented when technical project is fully designed.
5. To realize benefits of international exchange at the project design phase separate
networking measures could be implemented before the main program is launched.
Encouraging joint proposals by international partners could additionally strengthen the
project itself, as well as increase potential of future TC.
6. Partner database proved to be very useful in facilitating “first contact” between
potential partners. Whatever approach to TC is selected, idea of releasing information
on local entities interested / involved in ESF measures is worth supporting.
Transnational Cooperation in EQUAL
Conclusions / Recommendations:
7. Facilitating TN cooperation is a demanding coordination task. Especially important
seems availability of time and financial resources. It is recommended that networking
efforts be carefully planned, taking into account process-related requirements.
8. A good communication skill of technical staff is a precondition of successful TC. An
approach to project staff selection should be reconsidered. It seems that solutions
applied in EQUAL were only partially effective in this regard.
9. Taking into account substantial costs of TC, as well as its potential, good practices in
utilizing TC results should be documented and promoted (integrating TC in project
proposals, arrangements for proper planning of study tours, methods of sharing
knowledge within project teams and among DP partners.
Transnational Cooperation in EQUAL
Other EQUAL evaluations:
Findings
Guidance provided at the European level was found useful
Geographic patterns of partner selection
Approaches to internal organization of technical works (groups, division of work)
Models of self-evaluation (external / internal evaluator; TC evaluation at the DP level)
Problems with TC feasibility after the end of the program
Recommendations:
EC should consider taking over a coordinating role for different types of TC
TN projects to be planned as own projects or simultaneously with national projects.
Organize close coordination between all the MS (a minimum of common issues)
Transnational Cooperation in EQUAL
2007-2013 Programming Period
Working Group on Transnational and Interregional Cooperation (1)
Programming TC - 3 options:
Option A: A dedicated transnational priority
Option B: A cross-cutting approach
Option C: A comprehensive transnational approach
Strategic building blocks for efficient TN activities
Define the issues for transnational activities
Ensure effective use of the results of TC
Specify the forms or types of eligible TN collaboration
Capacity building
Ensure monitoring and assessment of achievements
Transnational Cooperation in EQUAL
2007-2013 Programming Period
Working Group on Transnational and Interregional Cooperation (2)
Each OP should specify the type, scope and form of TC it wishes to support
Five common requirements:
Clear decisions on the eligible activities
Linking of individual project results to policies (thematic clusters ).
A platform for partner-search and matching.
Approval of TN activities rests with the MA who provides financing
MS is responsible for monitoring of TN (joint evaluation of partnerships and activities.)
Transnational Cooperation in EQUAL
2007-2013 Programming Period
TC in OP “Human Capital” (Poland)
Status:
Negotiations w/ the Commission to be closed in June
Initial version of TC guidance by June 15th
Detailed description of priorities by September
Draft guidance (1):
TC principal proved effective and added value to the EQUAL program in Poland
(innovation, institutional capacity, TN networks)
These benefits could be yet maximized if:
- TC stems directly from the needs of particular project
- Implementors receive support at design, implementation, mainstreaming phases
Transnational Cooperation in EQUAL
2007-2013 Programming Period
TC in OP “Human Capital” (Poland)
Draft guidance (2)
TC will be promoted in the HC OP through a cross-cutting, horizontal approach
- maximum flexibility in the TC implementation
- easy access to TN experience under all priorities
Three models of TN Cooperation:
- between projects that are implemented in different MS
- between networks of institutions that are active in similar technical areas
- between institutions that operate in similar fields and provide support to one another.
Thank you for your attention!