Transcript Measuring Research and Experimental Development

R&D Surveys:
Advice from the Frascati Manual
National training workshop
Amman, Jordan
18-20 October 2010
www.uis.unesco.org
Ch 7 FM - R&D Survey Methodology
 Statistics on R&D require regular, systematic and
harmonised special surveys
 Other sources provide info, but:
• concepts of R&D used often different from FM concepts
• concepts may change over time
• very difficult to obtain all data for the same period
• difficult to avoid double counting when tracking flows
from financial statements and other sources
 Estimates are a necessary supplement to surveys
• Especially in higher education sector
www.uis.unesco.org
Scope of R&D surveys
 R&D surveys should identify and measure all
financial and personnel resources devoted to all
R&D activities in all R&D units
 R&D surveys are mainly addressed to R&Dperforming units
 Chapter 7 of the FM only addresses performerbased surveys
 Statistical methodologies and other procedures
have to be established to capture all R&D,
especially for units in the business enterprise
sector with little R&D
www.uis.unesco.org
Identifying target population and survey
respondents - general
 Exhaustive survey not possible in most countries
 Constraints include:
• number of respondents may have to be restricted to keep
costs down
• R&D survey may have to be taken in conjunction with
another survey
• surveys of some groups may require the participation of
other agencies with different data needs and hence
different questions for respondents
 One size does not fit all: every country has different
constraints – advice is therefore of general nature
www.uis.unesco.org
Identifying target population and survey
respondents - business enterprise sector
 The enterprise is recommended as the main
statistical unit in the business enterprise sector
• Some enterprises perform R&D on a regular basis from
year to year, and may have one or several R&D units
• Other enterprises perform R&D only occasionally
 It is recommended that all enterprises performing
R&D, either continuously or occasionally, should
be included in R&D surveys.
www.uis.unesco.org
Survey population business enterprise
sector – first possible approach
1. A census-based survey of large enterprises and a
sample of smaller ones in order to identify R&D
performers and request the information from them
•
R&D performed in the past in the enterprise is not
considered
•
this is the approach followed in innovation surveys
•
very small enterprises and enterprises in certain less
R&D-intensive industries often excluded for cost reasons
•
when the sample size is very small, estimates may be
less reliable, owing to raising factors
•
Method not strictly followed in any country
www.uis.unesco.org
Survey population business enterprise
sector – second possible approach
2. Try to survey all enterprises known or assumed to
perform R&D, based on a register of R&Dperforming enterprises
•
•
•
•
•
•
lists of enterprises receiving government grants and
contracts for R&D
lists of enterprises reporting R&D activities in previous
R&D surveys, in innovation surveys or other enterprise
surveys
directories of R&D laboratories
members of industrial research associations
employers of very highly qualified personnel
lists of enterprises claiming tax deductions for R&D.
www.uis.unesco.org
Survey population business enterprise
sector – joint approach
Recommendation
 To include in R&D surveys of the business
enterprise sector all firms known or supposed to
perform R&D.
 To identify R&D performers not known or
supposed to perform R&D by a census/sample of
all other firms:
• In the industries on the next slide.
• In principle, enterprises in all size classes should be
included, but if a cut-off point is necessary, it should be
at ten employees.
www.uis.unesco.org
Industries to be included
Industry
ISIC Rev. 3/NACE Rev. 1
Mining
14
Manufacturing
15-37
Utilities, construction
40,41,45
Wholesale
50
Transport, storage and communication
60-64
Financial intermediation
65-67
Computer and related activities
72
R&D services
73
Architectural, engineering and other
technical activities
742
Plus any other industry relevant for the country
www.uis.unesco.org
Identifying target population and survey
respondents - government sector
 Units to include in surveys are:
• R&D institutes.
• R&D activities of general administrations of central or
state government, statistical, meteorological, geological
and other public services, museums, hospitals.
• R&D activities at the municipality level.
 Recommendation: the best way to survey is to
send questionnaires to all units known or assumed
to perform R&D.
www.uis.unesco.org
Identifying target population and survey
respondents - PNP sector
 The sources for identifying possible survey
respondents are mainly the same as for the
government sector.
 Register information may be less comprehensive
and could be completed by information from
researchers or research administrations.
 This sector may be more relevant for surveys on
R&D funding.
www.uis.unesco.org
Identifying target population and survey
respondents – higher education sector
 Recommendation: The surveys and estimation
procedures should cover all universities and
corresponding institutions, especially those
awarding degrees at the doctorate level. Other
institutions in the sector known or assumed to
perform R&D should also be included.
• Identification generally easy.
• preferable to use smaller units, such as departments or
institutes of the university, as statistical units.
www.uis.unesco.org
Working with respondents
 Questionnaire: simple and short, logical and with
clear definitions and instructions
• Optional: simpler survey for smaller units
 Test questionnaires on a sample of respondents
www.uis.unesco.org
Who is the right respondent?
R&D Manager
Accountant or personnel
manager
 Better understanding of
R&D and FM norms
 May not refer exactly to
R&D as defined in FM
 But may not be able to
supply exact figures
 But able to supply exact
figures
 Cooperation of all three may be needed
 Useful to identify in advance the person responsible for
providing information and for co-ordinating information
from smaller sub-units
www.uis.unesco.org
Encouraging co-operation
 Secure co-operation of respondent
• Make them appreciate the potential uses of the data
• Respect confidential data
• Minimise the response burden
• Share the results (option: customised information)
• Provide technical assistance and contact details
www.uis.unesco.org
Estimations
R&D measurement could be done in three stages:
 Identification of all specialised R&D units and
measurement of their total activity.
 Estimates of the non-R&D portions of their activity
and subtraction of these estimates from the total.
 Estimates of the inputs used for R&D in other units
and addition of these estimates to the total.
www.uis.unesco.org
Operational criteria
Tools for “translating” theoretical FM concepts into
practical questionnaire:
 Explanatory notes
 Hypothetical examples
Covered in FM
 Guidance to individual respondents
 Documentation on treatment of different cases
To be covered by data collection agency 
keeping good documentation is essential
www.uis.unesco.org
Estimation procedures
 Imputation methods for item non-response
• Use previous answer
• Hot decking (use info from same survey)
• Cold decking (use info from previous survey)
 Imputation methods for unit non-response
• Use past R&D data (adjusted for sales or employment
growth)
• Impute as a function of the relation to personnel or sales
(test with non-response analysis)
www.uis.unesco.org
Thank you!
http://www.uis.unesco.org
[email protected]
www.uis.unesco.org