Innovation data collection: Advice from the Oslo Manual

South East Asian Regional Workshop on Science, Technology and Innovation Statistics Hanoi, Viet Nam 5-8 December 2011 Luciana Marins

www.uis.unesco.org

Ch 8 OM - Innovation Survey Procedures



Guidelines data

; - collection and analysis of

innovation



Comparable

countries;

results

over time and across  Particular methodology circumstances  may

comparability.

require other www.uis.unesco.org

The survey approach

“subject” approach:

 

The

• Innovative behaviour and activities of the whole;

firm

as a 

The “object” approach:

•

Specific innovations

some kind, (“significant innovation” of firm’s main innovation).

www.uis.unesco.org

Populations (1)



The target population:

• Innovation activities in the

sector business enterprise

(goods-producing and services industries); • Minimum: all statistical units with

employees

:

at least ten

• Small: 10-49; • Medium: 50-249; • Large: 250 and above.

www.uis.unesco.org

Populations (2)



The frame population:

• Units from which a survey sample or census is drawn; • Based on the surveys;

last year of the observation period

for •

Ideal frame

 NSOs; = up-to-date

official business register

• If the register forms the basis for several surveys (innovation, R&D, general business), the information can be restricted to innovation.

www.uis.unesco.org

Survey methods (1)



Mandatory

surveys increase response rates; 

Census or sample surveys?

•

Sample surveys

- representative of target population (industry, size, region) 

stratified sample

; •

Census

- costly but unavoidable in some cases.

www.uis.unesco.org

Survey methods (2)



Domains (sub-populations):

• Subsets of the sampling strata; •

Potential sub-populations:

industry groupings, size classes, regions, units that engage in R&D and innovation-active; •

Guidelines:

» Same statistical units and classifications; » Consistence of the methods for results calculation; » Documentation of deviations in data treatment or differences in the quality of the results from the domains.

www.uis.unesco.org

Survey methods (3)



Sampling techniques:

•

Stratified sample surveys:

size and principal activity; • Sampling fractions strata;

should not be the same

for all 

Cross-sections:

standard approach - new random sample for each innovation survey; 

Panel data:

alternative/supplementary approach.

www.uis.unesco.org

Survey methods (4)



Suitable respondents:

• Methods - ex:

postal surveys, personal interviews

; • Unit’s most suitable respondent - very questions that can be answered by only a

specialised few people

; •

Try to identify respondents by name before data collection starts.

www.uis.unesco.org

Survey methods (5)



The questionnaire:

•

Pre-test

; •

Simple and short

; •

Order

of the questions; • Questions on

qualitative indicators scale

; -

binary

or

ordinal

• International

translation

and innovation

design

; surveys attention to •

Short-form questionnaires

units innovation activity previously reported.

with little/no www.uis.unesco.org

Survey methods (6)



Combination of Innovation and R&D surveys:

 Reduction in the overall response burden;  Scope for analysing the relations between R&D and innovation activities;  Increase in the frequency of innovation surveys;  Country experiences - it is possible to obtain reliable results for R&D expenditures;  Longer questionnaire;  Units not familiar with the concepts of R&D and innovation may confuse them;  Different frames for the two surveys.

www.uis.unesco.org

Survey methods (7)



Guidelines for conducting combined surveys:

• Questionnaire -

two distinct sections

; •

Smaller individual sections

; •

Comparison of results

from combined and stand-alone surveys should be done with care - surveying methods should be reported; •

Samples extraction

from a common business register.

www.uis.unesco.org

Estimation of results (1)



Weighting methods:

•

Weighting by the inverse of the sampling fractions

of the sampling units, corrected by the unit non response; • If a stratified sampling

sampling fractions

technique is used, with

weights different

should be calculated

individually

for each; • Based on the

number of enterprises

in a stratum; •

International and weighting method.

other comparisons: same

www.uis.unesco.org

Estimation of results (2)



Non-response:

•

Unit non-response:

reporting unit does not reply at all; •

Item non-response:

response rate to a specific question - percentage of blank or missing answers; » Disregarding missing values

+

applying simple weighting procedures based on the responses received assumes that respondents and non-respondents are distributed in the same way 

biased results

; » Possibility:

imputation methods

.

www.uis.unesco.org

Presentation of results



Variance for the results:

innovation indicators and (average) values for their variation and/or confidence intervals; coefficients of 

Results presentation:

quality indicators.

metadata (including data collection procedure), sampling methods, procedures for dealing with non-response and www.uis.unesco.org

Frequency of data collection



Every two years;

 If

not economically feasible or four years;

 frequency of

three

 Specify an

observation period;

• The length of the observation period for innovation surveys

should not exceed three years nor be less than one year.

www.uis.unesco.org

Annex A - 5. Methodological issues for developing country contexts (1)



Information system specificities:

• Relative

weakness

of statistical systems: »

Absence of linkages

between surveys and data sets; »

Lack of official business registers

; • Involvement of

NSOs

; • When lacking,

performance

basic variables about

firms

can be included in the innovation

’

survey.

www.uis.unesco.org

Annex A - 5. Methodological issues for developing country contexts (2)



General methodological considerations:

•

Survey application:

» In-person; » Trained personnel; •

Questionnaire design:

» Sections can be separated to allow different persons in the firm to reply them; » Guidance/definitions; » Language and translation of technical terms.

www.uis.unesco.org

Annex A - 5. Methodological issues for developing country contexts (3)



General methodological considerations:

•

Frequency:

» Every three to four years (e.g., timed to

CIS

rounds); » Update a minimum set of variables every year; •

Purpose of surveys;

•

Clear questions;

•

Adequate legislative base;

•

The results should be published and distributed widely.

www.uis.unesco.org

Example - product innovation/degree of novelty

During the three years 200X to 200Y, did your enterprise introduce:

(Product innovations)

New or significantly improved goods?

New or significantly improved services?

Yes

( ) ( )

No

( ) ( )

Were any of your product innovations during the three years 200X to 200Y:

(Degree of novelty)

New to your market?

Only new to your firm?

Yes

( ) ( )

No

( ) ( ) www.uis.unesco.org

Example - innovation activities and expenditures for product and process innovations During the three years 200X to 200Y, did your enterprise engage in the following innovation activities:

(Innovation activities and expenditures process and product innovations)

In-house R&D External R&D Acquisition of machinery, equipment and software Acquisition of external knowledge Training for innovative activities Market introduction of innovations Other (preparations)

for Yes

( ) ( ) ( ) ( ) ( ) ( ) ( )

No

( ) ( ) ( ) ( ) ( ) ( ) ( ) $ $ $ $ $ $ $

Amount of expenditure

www.uis.unesco.org

Example - organisational innovation

During the three years 200X to 200Y, did your enterprise introduce:

(Organisational innovations)

New business procedures practices for organising New methods of organising responsibilities and decision making work

Yes

( ) ( )

No

( ) ( ) New methods of organising external relations with other firms or public institutions ( ) ( ) www.uis.unesco.org

Example - co-operation

During the three years 200X to 200Y, did your enterprise co-operate on any of your innovation activities with other enterprises or institutions?

( ) Yes ( ) No

Location Type of innovation co-operation partner

A. Other enterprises within your enterprise group B. Suppliers of equipment, materials, components, or software C. Clients or customers D. Competitors or other enterprises in your sector E. Consultants, commercial labs, or private R&D institutes F. Universities or other higher education institutions G. Government or public research institutes

Within country

( ) ( ) ( ) ( ) ( ) ( ) ( )

Abroad

( ) ( ) ( ) ( ) ( ) ( ) ( ) www.uis.unesco.org

Example - hampering factors

If your enterprise experienced any hampering factors during the period 200X-200Y, please grade the importance of the relevant factors.

Hampering Factors Economic

Excessive perceived economic risks Innovation costs too high

factors

Lack of appropriate sources of finance Organisational rigidities within the enterprise

Internal factors

Lack of qualified personnel Lack of information on technology

Other factors

Lack of information on markets Insufficient flexibility of regulations or standards Lack of customer responsiveness to new goods or services

High

( )

Degree of importance Medium

( )

Low

( )

Not relevant

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) www.uis.unesco.org

Thank you!

http://www.uis.unesco.org

[email protected]

www.uis.unesco.org