Innovation surveys and measurement of innovation activities
Download
Report
Transcript Innovation surveys and measurement of innovation activities
Innovation surveys: design, implementation,
lessons learnt
Micheline Goedhuys
Why do we need to measure innovation?
Scarcity of data in general, on innovation in
particular
Lack of policy tools for benchmarking
Insufficient monitoring and evaluation of policies
Nature of innovation calls for firm-level information
June 11 2008
DEIP, Amman June, 10-12 2008
2
Structure of session:
1.
Conceptual background
2.
Experiences with innovation surveys
3.
Methodological aspects
4.
Use of innovation survey data
5.
Opportunities and limitations to innovation data
collection: key issues
June 11 2008
DEIP, Amman June, 10-12 2008
3
1. Conceptual background
linear view that science, research and discovery
underlie innovation (science push)
innovation measured by science indicators:
R&D
engineers
patenting
bibliometrics, publications, citation indices
surveys (USA, 1960s) collecting R&D, patent data;
June 11 2008
DEIP, Amman June, 10-12 2008
4
1. Conceptual background
End 1980s, 1990s ‘activity approach’:
investigating the ‘black box’
innovation results from interaction firm-market,
learning, feedback (chain-link model of Kline and
Rosenberg 1986)
need for indicators capturing non-R&D activities
and incremental change
development of surveys asking firms about their
innovation process
June 11 2008
DEIP, Amman June, 10-12 2008
5
1. Conceptual background
Harmonisation of survey efforts in the
‘Oslo Manual’, 1992, 1997, 2005
basis for Community Innovation Surveys
innovation is measured as :
an activity (R&D, industrial design, acquisition of
machinery, external technology, training) and
an output (introduction of product or process
innovations)
Features: new-to-the-firm, significant improvements
June 11 2008
DEIP, Amman June, 10-12 2008
6
2. Experiences : CIS
CIS-1: 1990-92; first regional effort to collect
innovation data; 13 European countries,
CIS-2: 1994-1996; 17 countries
CIS-3: 1998-2000; more firms, more questions,
services, organisational change, 29 countries
CIS-light: 2000-2002, limited set of questions, 18
countries
CIS-4: 2002-2004: 29 countries, organisational
innovation and effect
CIS-2006: 2004-2006; 29 countries; no data
available yet
CIS-2008: 2006-2008
June 11 2008
DEIP, Amman June, 10-12 2008
7
2. Experiences : Latin America
Need of information to monitor the impacts of
economic reforms (trade liberalisation, privatisation,
deregulation, FDI,etc).
ARGENTINA
CHILE
COLOMBIA
MÉXICO
VENEZUE
LA
Survey
Number
I
I
I
I
I
Reference
period
1992-1996
1994-1995
1993-1996
1994-1996
1994-1996
Collection
period
1997
1995
1997
1997
1997
Agency
responsible
for survey
INDECSECYT
INE-SETPI
COLCIENCIA
-DNP
INEGICONACYT
OCEI
Source: Crespi, 2007
June 11 2008
DEIP, Amman June, 10-12 2008
8
2. Experiences : Latin America
Specific nature of innovation in Latin American
countries:
Importance of incremental innovation;
organisational and marketing innovation;
Importance of innovation embodied in machinery
and equipment (dissemination)
Less private and more informal R&D
Fragmented flows of information
June 11 2008
DEIP, Amman June, 10-12 2008
9
2. Experiences : Latin America
Need for changes to the survey instrument : Bogotá
Manual to complement OSLO Manual.
from innovations to firm-level innovative
activities and technology efforts
human resources, capabilities
enlarged data need on organisational, delivery
and design innovations
lack of centralised agency, different questionnaires
and sampling methodologies
June 11 2008
DEIP, Amman June, 10-12 2008
10
2. Experiences : Latin America
Second wave of Innovation surveys:
10 countries; 2000-2001; Argentina, Brazil, Chile,
Cuba, Ecuador, Mexico, Panama, Peru, Trinidad
& Tobago and Uruguay
more of uniformity but without common
questionnaire and sampling methodologies
revision of Bogotá Manual and Annex to Oslo
Manual (2005).
June 11 2008
DEIP, Amman June, 10-12 2008
11
2. Experiences : Latin America
Third wave of Innovation surveys:
5 countries; 2003-2005; Argentina, Brazil, Chile,
Colombia and Uruguay
A lot of exit and a core group of countries with
“consolidated” routines (but still with institutional
problems and financial issues).
ECLAC-RICYT-OAS network (2006) to create a
harmonized “core” questionnaire (plus access to
micro-data)
June 11 2008
DEIP, Amman June, 10-12 2008
12
2. Experiences : Asia, Africa
In Southeast Asia:
Malaysia (3), Taiwan (1), Singapore (1), Thailand
(2), China, India…
In Africa:
South Africa (2)
Planning to conduct an innovation survey in 20+
countries (NEPAD survey)
June 11 2008
DEIP, Amman June, 10-12 2008
13
2. Agenda:
ongoing debate to design innovation surveys to the
context of developing countries
concept of innovation : organisational, packaging,
delivery, design innovations, waste management
techniques, …
trade off between country/regional design and
benchmarking options
increasing policy relevance
inclusion of services and resource-based sectors
June 11 2008
DEIP, Amman June, 10-12 2008
14
3. Methodological aspects : questionnaire content
Basic information: name, location, industry,
ownership, year established…
Firm performance: sales, employment, …
Innovation activities: Investment, Training, intramural and external R&D, …and expenditures
Innovation outputs (product/process/organisational)
Sources of information for innovation
Cooperation for innovation
Government policy or incentives affecting
innovation
June 11 2008
DEIP, Amman June, 10-12 2008
15
3. Methodological aspects : questionnaire content
Objectives, goals or reasons for innovating
Impact of innovations on firm performance
Obstacles to innovation
June 11 2008
DEIP, Amman June, 10-12 2008
16
3. Methodological aspects
Organisation: national statistics agency, MOST,
universities, consultants
Reference period: 2 or 3 years (mostly 3)
Participation: voluntary, compulsary (in Latin
America)
June 11 2008
DEIP, Amman June, 10-12 2008
17
Venezuela
Uruguay
Thailand
Taiwan
South Africa
Slovenia
Singapore
Russia
Romania
M alaysia
Colombia
Chile
Brazil
Argentina
Turkey
Switzerland
South Korea
Slovak Republic
Poland
New Zealand
M exico
Hungary
European-17 (CIS)
Canada
Australia
0
NSA
June 11 2008
1
MOST/GOVT
2
UNIV/INST
DEIP, Amman June, 10-12 2008
3
CONSULT
18
0
st r
Ca alia
EU na
- 17 da
(C
Po IS)
l
S and
Ar .Kor
ge ea
nti
n
Br a
az
i
Co Ch i l
le
lo
Ma mbi
a
la
Th ysia
ai
S- Aland
f ri
Ru ca
s
H
Yu ung sia
go ar y
s
Ur lavia
ug
u
Me a y
xi c
o
Cz
e
Tu ch
r ke
Sw Slov y
iz
a
N. er la k
nd
Z
Ve ealan
ne
zu d
Ta ela
Slo iwa n
Ro veni
ma a
ni
Sin Pe a
ga r u
po
re
Au
Number of Year s
5
4
3
2
1
June 11 2008
DEIP, Amman June, 10-12 2008
19
3. Methodological aspects
Survey modalities: postal, PTEF follow up, personal
interview, telephone interview, online questionnaire,
CATI
Sector coverage: initially manufacturing,
increasingly services, resource based industries
Firm size: cutoff points: 5, 10, 20 or 50 workers
June 11 2008
DEIP, Amman June, 10-12 2008
20
4. Use of innovation surveys
by academics and researchers
Innovation and firm performance
Identify determinants/constraints to innovation
Innovation strategies
Regional and country studies
Industry studies
Innovation patterns over time
Developing innovation indicators: measurement
issues
June 11 2008
DEIP, Amman June, 10-12 2008
21
4. Use of innovation surveys
for policy making:
June 11 2008
Indicators for benchmarking
Mapping innovation ; innovation in new sectors
Assessing trends
Monitoring specific policy instruments
DEIP, Amman June, 10-12 2008
22
4. Example:
European innovation scoreboard
Uses 20 Indicators
Cross-country comparisons, industry comparisons
changes over time
consensus on policy action
uses CIS based indicators
% SMEs with in-house innovative activities
% SMEs that collaborate on innovation
total innovation expenditures as % sales
% new-to-market products/sales
% new-to-firm products/sales
June 11 2008
DEIP, Amman June, 10-12 2008
23
5. Opportunities and limitations
Innovation surveys have become key research
inputs of modern innovation studies (Crespi, 2007)
the use of CIS data in academic research
50
45
40
amount of publications
35
30
cis1
cis2
25
cis3
20
15
10
5
0
1
2
3
4
5
6
7
8
9
10
11
12
13
years since reference year
June 11 2008
DEIP, Amman June, 10-12 2008
24
5. Opportunities and limitations
Heterogeneity across questionnaires and
methodologies remains and is even on the rise due
to broadening concept of innovation, scope, …
Lots of country studies, little cross-country
comparisons in developing countries
This limits the use of survey data as benchmarking
tool (e.g. Crespi, 2007)
June 11 2008
DEIP, Amman June, 10-12 2008
25
5. Opportunities and limitations
On the questionnaire:
Need for the development of harmonized
guidelines with a core set of questions
Optional policy-relevant questions can be added
for policy monitoring
Methodology for country benchmarking:
Preferably common sampling methodology: size
cut-off point, industry coverage, …
Compulsory common (length of) reference
period, and participation mode
June 11 2008
DEIP, Amman June, 10-12 2008
26
5. Opportunities and limitations
Dissemination of non-aggregated micro-data is
crucial
Assessing trends: need for panel data
Need of involvement of stakeholders from the start
Need for strong coordination mechanism
June 11 2008
DEIP, Amman June, 10-12 2008
27
Useful links:
For a download of the CIS-4 questionnaire:
http://www.berr.gov.uk/files/file9688.pdf
Oslo Manual:
http://epp.eurostat.ec.europa.eu/cache/ITY_PUBLIC/OSLO/EN/OSL
O-EN.PDF
Bogotá manual:
http://www.ricyt.edu.ar/interior//difusion/pubs/bogota/bogota_eng.pdf
NEPAD study:
http://www.nepadst.org/doclibrary/pdfs/innopolicy_aug2004.pdf
June 11 2008
DEIP, Amman June, 10-12 2008
28