Transcript Measuring Research and Experimental Development

Innovation data collection:
Methodological procedures
& basic forms
Regional Workshop on Science, Technology and Innovation (STI) Indicators for Gulf countries
Doha, Qatar
15 to 17 October 2012
www.uis.unesco.org
Ch. 8 OM - Survey procedures
 Guidelines - collection and analysis of innovation
data;
 Comparable
results
over
time
and
across
countries;
 Particular
circumstances may
methodology  comparability.
require
other
www.uis.unesco.org
The survey approach
 The “subject” approach:

• Innovative behaviour and activities of the firm as a
whole;
 The “object” approach:
innovations (“significant innovation” of
some kind, firm’s main innovation).
• Specific
www.uis.unesco.org
Populations (1)
 The target population:
• Innovation
activities in the business enterprise
sector (goods-producing and services industries);
• Minimum:
all statistical units with at least ten
employees:
• Small: 10-49;
• Medium: 50-249;
• Large: 250 and above.
www.uis.unesco.org
Populations (2)
 The frame population:
• Units from which a survey sample or census is drawn;
• Based on the last year of the observation period for
surveys;
• Ideal frame = up-to-date official business register
 NSOs;
• If the register forms the basis for several surveys
(innovation, R&D, general business), the information
can be restricted to innovation.
www.uis.unesco.org
Survey methods (1)
 Mandatory surveys increase response rates;
 Census or sample surveys?
• Sample surveys - representative of target population
(industry, size, region)  stratified sample;
• Census - costly but unavoidable in some cases.
www.uis.unesco.org
Survey methods (2)
 Domains (sub-populations):
• Subsets of the sampling strata;
• Potential sub-populations: industry groupings, size
classes, regions, units that engage in R&D and
innovation-active;
• Guidelines:
» Same statistical units and classifications;
» Consistence of the methods for results calculation;
» Documentation of deviations in data treatment or differences in
the quality of the results from the domains.
www.uis.unesco.org
Survey methods (3)
 Sampling techniques:
• Stratified sample surveys: size and principal activity;
• Sampling fractions should not be the same for all
strata;
 Cross-sections: standard approach - new random
sample for each innovation survey;
 Panel data:
alternative/supplementary approach.
www.uis.unesco.org
Survey methods (4)
 Suitable respondents:
• Methods:
e.g.,
postal
surveys,
questionnaires, personal interviews;
web-based
• Unit’s most suitable respondent
- very specialised
questions that can be answered by only a few people;
• Try to identify respondents by name before data
collection starts.
www.uis.unesco.org
Survey methods (5)
 The questionnaire:
• Pre-test;
• Simple and short;
• Order of the questions;
• Questions on qualitative indicators - binary or ordinal
scale;
• International
innovation
translation and design;
• Short-form
surveys
-
questionnaires - units
innovation activity previously reported.
attention
with
to
little/no
www.uis.unesco.org
Survey methods (6)
 Combination of Innovation and R&D surveys:
 Reduction in the overall response burden;
 Scope for analysing the relations between R&D and
innovation activities;
 Increase in the frequency of innovation surveys;
 Country experiences - it is possible to obtain reliable results
for R&D expenditures;
 Longer questionnaire;
 Units not familiar with the concepts of R&D and innovation
may confuse them;
 Different frames for the two surveys.
www.uis.unesco.org
Survey methods (7)
 Guidelines for conducting combined surveys:
• Questionnaire: two distinct sections;
• Smaller individual sections;
• Comparison of results from combined and stand-alone
surveys should be done with care - surveying methods
should be reported;
• Samples extraction from a common business register.
www.uis.unesco.org
Estimation of results (1)
 Weighting methods:
• Weighting by the inverse of the sampling fractions
of the sampling units, corrected by the unit nonresponse;
• If
a stratified sampling technique with different
sampling fractions is used, weights should be
calculated individually for each;
• Based on the number of enterprises in a stratum;
• International
and
weighting method.
other
comparisons:
same
www.uis.unesco.org
Estimation of results (2)
 Non-response:
• Unit non-response: reporting unit does not reply at all;
• Item
non-response: response rate to a specific
question - % of blank or missing answers;
» Disregarding missing values + applying simple weighting
procedures based on the responses received assumes that
respondents and non-respondents are distributed in the
same way  biased results;
» Possibility: imputation methods.
www.uis.unesco.org
Presentation of results
 Descriptive analysis: no generalisation of results;
 Inferential
analysis: conclusions about target
population;
• Variance for the results: (average) values for innovation
indicators and their coefficients of variation and/or
confidence intervals;
 Results presentation: metadata (data collection
procedure, sampling methods, procedures
dealing with non-response, quality indicators).
for
www.uis.unesco.org
Frequency of data collection
 Every 2 years;
 If not economically feasible  frequency of 3 or
4 years;
 Specify an observation period;
• The length of the observation period for innovation
surveys should not exceed 3 years nor be less than
1 year.
www.uis.unesco.org
Annex A - 5. Methodological issues
for developing country contexts (1)
 Information system specificities:
• Relative weakness of statistical systems:
» Absence of linkages between surveys and data sets;
» Lack of official business registers;
• Involvement of NSOs;
• When
lacking, basic variables about firms’
performance can be included in the innovation
survey.
www.uis.unesco.org
Annex A - 5. Methodological issues
for developing country contexts (2)
 General methodological considerations:
• Survey application:
» In-person;
» Trained personnel;
• Questionnaire design:
» Sections can be separated to allow different persons in the
firm to reply them;
» Guidance/definitions;
» Language and translation of technical terms.
www.uis.unesco.org
Annex A - 5. Methodological issues
for developing country contexts (3)
 General methodological considerations:
• Frequency:
» Every 3 to 4 years (e.g., timed to CIS rounds);
» Update a minimum set of variables every year;
• Purpose of surveys;
• Clear questions;
• Adequate legislative base;
• The results should be published and distributed
widely.
www.uis.unesco.org
Example - product innovation/degree
of novelty
During the three years 200X to 200Y, did your
enterprise introduce:
(Product innovations)
New or significantly improved goods?
New or significantly improved services?
Yes
()
()
No
()
()
Were any of your product innovations during the three
years 200X to 200Y:
(Degree of novelty)
New to your market?
Only new to your firm?
Yes
()
()
No
()
()
www.uis.unesco.org
Example - innovation activities and
expenditures for product and process
innovations
During the three years 200X to 200Y, did your enterprise engage in the
following innovation activities:
(Innovation activities and expenditures for
process and product innovations)
In-house R&D
External R&D
Acquisition of machinery, equipment and software
Acquisition of external knowledge
Training for innovative activities
Market introduction of innovations
Other (preparations)
Yes
No
()
()
()
()
()
()
()
()
()
()
()
()
()
()
Amount of
expenditure
$
$
$
$
$
$
$
www.uis.unesco.org
Example - co-operation
During the three years 200X to 200Y, did your enterprise co-operate on any
of your innovation activities with other enterprises or institutions?
( ) Yes ( ) No
Type of innovation co-operation partner
A. Other enterprises within your enterprise group
B. Suppliers of equipment, materials, components, or software
C. Clients or customers
D. Competitors or other enterprises in your sector
E. Consultants, commercial labs, or private R&D institutes
F. Universities or other higher education institutions
G. Government or public research institutes
Location
Within
Abroad
country
()
()
()
()
()
()
()
()
()
()
()
()
()
()
www.uis.unesco.org
Example - hampering factors
If your enterprise experienced any hampering factors during the period 200X-200Y, please grade
the importance of the relevant factors.
Degree of importance
Hampering Factors
Excessive perceived economic risks
Economic
Innovation costs too high
factors
Lack of appropriate sources of finance
Organisational rigidities within the enterprise
Lack of qualified personnel
Internal
factors
Lack of information on technology
Lack of information on markets
Insufficient flexibility of regulations or standards
Other
factors
Lack of customer responsiveness to new goods or services
High
Medium
Low
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
()
Not
relevant
()
()
()
()
()
()
()
()
()
www.uis.unesco.org
Example - organisational innovation
During the three years 200X to 200Y, did your enterprise
introduce:
(Organisational innovations)
New business practices for organising
procedures
New
methods
of
organising
work
responsibilities and decision making
New methods of organising external relations
with other firms or public institutions
Yes
No
()
()
()
()
()
()
www.uis.unesco.org
Basic forms - CIS
 The Community Innovation Survey (2010) (CIS 2010):
1.
2.
3.
4.
General information about the enterprise
Product innovation
Process innovation
Ongoing or abandoned innovation activities
for pdt/pcs innovation
5.
Innovation activities and expenditures for pdt/pcs innovation
6.
Sources of information and co-operation for pdt/pcs innovation
7.
Objectives for pdt/pcs innovation
8.
Factors hampering pdt/pcs innovation activities
9.
Organisational innovation
10. Marketing innovation
11. Creativity and skills
12. Basic economic information on your enterprise
www.uis.unesco.org
Basic forms - AU/NEPAD
 AU/NEPAD Standard Innovation Questionnaire:
1.
2.
3.
4.
General information about the enterprise
Product innovation
Process innovation
Ongoing or abandoned innovation activities
for pdt/pcs innovation
5.
Innovation activities and expenditures for pdt/pcs innovation
6.
Sources of information and co-operation for pdt/pcs innovation
7.
Effects/Objectives of pdt/pcs innovation
Factors hampering (pdt/pcs) innovation activities
9. IPRs
10. Organisational and marketing innovation
11. Specific innovations by your enterprise
8.
www.uis.unesco.org
Thank you!
http://www.uis.unesco.org
[email protected]
www.uis.unesco.org