Transcript Document 7195860

Three approaches to STI
measurement in developing
countries
Ernesto Fernandez Polcuch
UNESCO Institute for Statistics
International Conference on S&T Policy
Research and Statistical Indicators
08-10 November 2006 - Colombo, Sri Lanka
www.uis.unesco.org
Structure of the presentation
• UIS strategy in the field of S&T stats.
• Three approaches:
• Careers of Doctorate Holders (CDH)
• Innovation
• R&D
• Developing countries’ specificities and
way forward
www.uis.unesco.org
UIS S&T Strategy
www.uis.unesco.org
UNESCO International Review of
S&T Statistics and Indicators
Carried out jointly with UNESCO
Division of Science Analysis and Policy (SC/AP) and
Regional Office for S&T in Latin America and the Caribbean
(ROSTLAC) in 2002/2003.
Objectives:
• To review priority science policy information
needs.
• To examine existing S&T statistical and indicator
systems.
• To identify key areas for future development of
S&T statistics.
• To define the future role and strategy of the UIS.
www.uis.unesco.org
General S&T Policy Issues
ISSUES
Impact on Society
Dissemination of
Knowledge &
Technology transfer
S&T output
Institutions &
Mechanisms
Extremely
Important
Very
Important
%
%
%
76
53
17
29
7
18
53
33
14
45
41
38
41
Important
17
18
Extremely
Important
Impact on
Society
Very
Important
Dissemination of
knowledge & tech.
transfer
S&T
output
Institutions &
Mechanisms
0%
20%
40%
60%
80%
Not or Somewhat Important
Resources
Not/Somewhat Resources
100%
www.uis.unesco.org
Immediate term priorities
(Human & Financial Resources)
l
R&D personnel
l
Expenditure in R&D
l
Human resources devoted to S&T
l
Science education
l
Higher education
l
International mobility
l
Gender
www.uis.unesco.org
Medium term priorities
(Innovation)
l
l
l
l
l
Measurement of innovation in agriculture and
other non-manufacturing sectors
Promoting the use of indicators reflecting subnational (regional) innovation systems
Measuring minor or incremental innovation
Measuring innovative applications of existing
products or processes (surveys of use of
technologies)
Output
www.uis.unesco.org
Longer term priorities
(Output & Impact)
•
Output:
• Bibliometric tools adapted for the analysis of
scientific output in developing countries
• Technology output indicators
• Impact indicators:
• Measuring social impact of S&T,
• Impact of S&T on agriculture,
• Public perception of S&T
www.uis.unesco.org
Ongoing activities
• S&T Survey operation and data guardianship
• Training in S&T statistics: Workshops
• Standard setting/Methodological developments:
• Analysis / Publications
• Collaborations / Partnerships
www.uis.unesco.org
S&T Survey operation and data
guardianship
l
Conducting global Survey on Statistics
of Science & Technology: Biennially –
currently second round
l
Maintaining database on S&T statistics
l
Data publishing on the UIS website
l
Contributions to external agencies
www.uis.unesco.org
Survey on Statistics of Science
& Technology
• The 2004 S&T statistics survey was launched in
May, 2004. This was the first UNESCO statistical
questionnaire to be available for completion online via the internet. The latest resulting data were
released on the UIS website May 2006.
l
l
The 2006 S&T statistics survey was launched
in June/July 2006.
OECD and EUROSTAT provide data for their
Member States. RICYT provides data for Latin
America.
www.uis.unesco.org
Survey on Statistics of Science
& Technology
(continued)
l
Questionnaire on Statistics of Science and Technology; Instruction
Manual for completing questionnaire.
l
Data are collected from each country from either the institution
responsible for S&T policy or statistics (e.g. Ministry of Science and
Technology, Ministry of Research and Higher Education, National S&T
Council or similar organization) or the National Statistical Office;
through a single questionnaire which cover data on all institutions
carrying out R&D activities in the particular country.
l
Data collected:









R&D personnel by occupation & gender
R&D personnel by sector of employment & occupation
R&D personnel by sector of employment & gender
Researchers by formal qualification & sector of employment
Researchers by formal qualification & gender
Researchers by fields of science & sector of employment
Researchers by fields of science & gender
Total expenditure in R&D by sector of performance
Total expenditure in R&D by source of funds
www.uis.unesco.org
2004 Survey on Statistics of
Science & Technology
Results of the 2004 survey are now published on
the UIS website: http://www.uis.unesco.org
Data are available for 114 countries and territories.
Here is an overview of the sources:
UIS 2004 survey
UIS surveys 1998 or 2001
OECD (MSTI)
Eurostat (NewCronos)
RICYT
Total
49
7
35
5
18
114
www.uis.unesco.org
Evolution of 2006 survey
70
2004
2006
60
50
40
30
20
10
87 (end of S2004)
(…)
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10 (2006, 6Nov)
9
8
7
6
5
4
3
2
0
1
Number of received questionnaires (total)
2004 and 2006 Surveys
Received questionnaires comparison
Weeks since deadline
www.uis.unesco.org
http://www.uis.unesco.org
Data and Indicators published:
l
Number of Total R&D personnel (Headcount-HC and Full-time equivalent-FTE) by sex
l
Total R&D personnel (FTE) by sector of employment
l
Researchers (HC and FTE) by sex
l
Researchers (FTE) by sector of employment
l
Technicians and equivalent staff (HC and FTE) by sex
l
Other supporting staff (HC and FTE) by sex
l
Researchers per million inhabitants (HC and FTE)
l
Technicians per million inhabitants (HC and FTE)
l
Women as a percentage of R&D personnel (HC and FTE)
l
Women as a percentage of Researchers (HC and FTE)
l
Total Gross Domestic Expenditure on R&D (GERD) in local currency and international
Purchasing Power Parity (PPP) dollars
l
GERD by sector of performance (%)
l
GERD by source of funds (%)
l
GERD as a percentage of Gross Domestic Product (GDP)
l
GERD per capita
www.uis.unesco.org
UIS is UN lead agency for S&T
Statistics
Contributes data for:
l
l
UNESCO HQs: UNESCO Science Report
2005, UNESCO World Report, International
Report on Science Technology and Gender
2006.
UN Statistical Division: UN Statistical Year
Book
l
UNDP: Human Development Report
l
World Bank: World Development Indicators
l
Specific Requests
www.uis.unesco.org
S&T Statistics Workshops
Diagnosis:
l
Production of S&T statistics in many developing regions (particularly in Africa
and Asia) remains low. Capacity building is needed.
UIS Response: Regional workshops
l
l
l
l
l
l
To increase the number of countries regularly producing quality S&T indicators.
To create local capacities for the production of such indicators, with the final
aim of establishing sustainable local S&T statistics systems
To promote the use of S&T indicators, seeking comprehension for evidencebased S&T policy making.
To share experiences with other developing countries in the field of S&T
indicators, and to address the problems that countries may have encountered
in collecting S&T statistics.
To gain knowledge about the particular characteristics of S&T statistics data
collection and use in the context of countries in the same region.
To generate initiatives that could be used as a demonstration for good
practices in other countries of the region.
www.uis.unesco.org
S&T Statistics workshops: 2005/06
 Southern and Eastern Africa Regional Workshop – held in
Uganda, Sept. 2005
 South Asian Regional Workshop – held in India, Nov. 2005
 South-East Asian Workshop – held in Indonesia, March
2006
 Francophone Sub-Saharan Africa: held in Senegal, Oct.
2006
 Upcoming Workshops:
• Workshop in Central Asia: will be held in Kazakhstan, Nov. 2006
• Workshop in Arab States, will be held in Jan. 2006
• Workshop in South-East Europe: will be held in Croatia, Mar. 2006
www.uis.unesco.org
Standard setting/Methodological
developments:
l
Measuring Innovation in
Developing countries: Annex to
the Oslo Manual (2005):
Oslo Manual (Guidelines for collecting
and interpreting innovation data):
central reference document for the
statistical definition of innovation and
forms the basis for surveys of innovation
throughout the world.
www.uis.unesco.org
Careers of Doctoral Holders – CDH
Objectives:
l
to track the careers of doctoral holders, with the
aim of obtaining information on the loss of highly
qualified specialists, the so-called ‘brain drain’.
l
to design an international survey tool which would
help track the careers of doctorate holders which
would better inform policy makers worldwide, both
at global and local levels.
www.uis.unesco.org
Analysis / Publications
l
l
Data on the web site.
UIS Publications (S&T Bulletin):
1st – Investment in R&D;
2nd – Bibliometric Indicators;
3rd – Women in Science
(can be downloaded from the UIS website)
l
l
UNESCO Science Report 2005
International Report on Science, Technology and
Gender 2006
l
UNESCO World Report
l
History of Science Statistics at UNESCO
www.uis.unesco.org
Women in Science
International Report on Science,
Technology and Gender 2006
to be published by
UNESCO Natural Science Sector
 UIS Contribution: the chapter on ‘Statistics on
Science, Technology and Gender (STG)’ with an
Annex on statistical overview.
- Collaboration with European Commission (Research DirectorateGeneral) and a group of worldwide experts.
 UIS Bulletin on Women in Science: available online in Sept 2006.
www.uis.unesco.org
Collaborations / Partnerships
l
UNESCO HQs
l
OECD
l
Eurostat
l
RICYT (Latin America)
l
NEPAD / AU / ATPS (Africa)
l
ASEAN
l
IDRC
l
INRS (Quebec)
l
ISESCO
l
Inter-Academy Council
l
ISDB
www.uis.unesco.org
Quality of data
Efficient use of
resources
Consistency over
time and space
Accessibility and
affordability
Validity
and reliability
Relevance to
policy
Potential for
disaggregation
Comparability
through
standards
Clarity and
transparency
Currency and
punctuality
Coherence
across sources
www.uis.unesco.org
Careers of Doctorate Holders (CDH)
www.uis.unesco.org
UIS approach to Human Resources
statistics: CDH project
• Methodology developed “from the scratch” together with
OECD & Eurostat.
• Aimed both at developed and developing countries
• With participation from experts from both developed and
developing countries
• Now being piloted
• Promoting the methodology by encouraging developing
countries to conduct such surveys and produce crossnationally comparable statistics on careers of doctorate
holders
• What about the Sri Lankan Tracer Study of Graduates and
Postgraduates? Does it fit into this project?
www.uis.unesco.org
Relevance of the CDH project
• World economy is increasingly based on knowledge and
information.
• Knowledge is now recognised as the driver of productivity
and economic growth.
• As a result, there is a new focus on the crucial role of
highly qualified individuals who represent a key to the
production, application and transmission of knowledge.
• Statistics on the global trends in human resources for
Science and Technology (HRST) are very week.
• The quality and comparability of international data on
migration is particularly weak.
• Diversity of data collection methods hinders international
comparability, and does not provide information on career
paths and mobility patterns.
www.uis.unesco.org
Background and Process
Background:
 ‘Brain drain’ or loss of skilled personnel identified as priority by UIS and
UNESCO Paris
 OECD and EU are interested in
 skills loss from developing countries
 Moves from university to industry
 US NSF also interested in changing US patterns of academic
recruitment
Process:
 To meet the users demands UIS, OECD & Eurostat launched a project
on “Statistics on the Careers of Doctorate Holders”.
 Each organization took responsibility for issues appropriate to their
mandate. Expertise and resources are shared across the three
partners.
 Expert group: Argentina, Belgium, Canada, China, Denmark, France,
Germany, India, Italy, Japan, Malaysia, Portugal, Russian Federation,
Spain, Switzerland, Ukraine, Uganda, the United States.
www.uis.unesco.org
Objectives and Task forces
Objectives:
 To design an internationally comparable tool for tracking the
careers of doctorates holders and highly qualified people in
different countries.
 To pilot tests of this instrument in volunteer countries.
 to collect and exchange information on the career paths of
holders of doctorates from existing data sources and the new
survey tool.
Task forces:
 Production of output tabulations, and development of common
definitions (led by OECD and Canada)
 Drafting of methodological guidelines (led by Eurostat and
Portugal).
 Development of a model survey with a supporting methodology
(led by UIS).
www.uis.unesco.org
Output tabulation program
• Conceived to be used by countries when
delivering the requested data to the international
organizations;
• Consists of several tables organized in seven
broad groups: personal characteristics,
educational history, work history, scientific output,
perception of current work situation, international
mobility retrospective and future plans.
www.uis.unesco.org
Definitions
• Doctorate;
• Citizenship and residential status;
• Recent doctorate recipients;
• Employed persons;
• Unemployed and inactive persons;
• Researcher;
• Temporary/permanent employment, full-time and
part-time employment, etc.
• Postdoc
www.uis.unesco.org
Methodological guidelines
• Define and structure the target population
• Describe the sources for building sampling
frames
• Present some aspects of data collection,
data processing and results estimation.
• Should be seen as an orientation for
countries that plan to launch a CDH survey
as well as a tool to improve and align their
existing national surveys to an international
standard.
www.uis.unesco.org
Target population
Consists of individuals fulfilling the
following criteria:
• having an education at ISCED 6 level
(doctorates) obtained anywhere in the
world, and
• being resident (permanent or nonpermanent) within the national borders
of the surveying country.
www.uis.unesco.org
Target population structure
• Target population, residents in country X (the surveying country)
that are:
A. Citizens of country X with doctorate awarded within country X
B. Citizens of foreign countries with doctorate awarded in country X
C. Citizens of country X with doctorate awarded abroad
D. Citizens of foreign countries with doctorate awarded abroad
• Non-target population, residents outside country X that are:
E. Citizens of country X, with doctorate awarded within country X that have left
country X
F. Citizens of foreign countries, with doctorate awarded in country X that have
left country X
• What to do with doctorates abroad
- Groups E and F are usually taken into consideration in the reporting countries
where these doctorate holders are resident.
- If countries are able to construct a sample frame, and reach group E, the
survey could also be extended to this group, even if countries are not requested
to report information to the international organizations on them.
www.uis.unesco.org
Sources for constructing sampling
frames
• National education registers
• Universities
• International and national foundations
• National libraries
• Population censuses
• Research institutes
• Professional organizations
• Central registers of foreigners
• Alumni organizations
• Previously conducted surveys on doctorate holders
www.uis.unesco.org
Survey types
• Cohorts of recent graduates:
• administered on graduation and repeated every few years
• simple and straightforward, but
• limited coverage of the target population
• Snapshot surveys (cross-sectional retrospective
sample of all doctorates):
• one time, administered on an occasional basis
• more appropriate tool for collection of information related to the
entire target population
• covers all cohorts at the same time
• quickly provides information on the mobility and career
development of doctorate holders
• Should not be seen as mutually excluding but rather
complementary
www.uis.unesco.org
Instruments (developed by UIS)
• Core model questionnaire containing “core questions”
which would provide data to fill in the output tabulations.
The questionnaire will be accompanied with a manual
comprising the definitions and other guidelines.
• CDH questions data base comprises the set of
questions among which the suitable questions can be
chosen and applied by countries when designing CDH
surveys. CDH data base will be web-based and openly
accessible.
• The core model questionnaire is mainly foreseen for
countries which do not have CDH type surveys yet.
www.uis.unesco.org
Construction of Core model
Questionnaire
The following existing CDH questionnaires were used:
• USA (SED and SDR),
• Canada (SED),
• Switzerland (Survey of Tertiary Graduates plus Doctorates
surveyed the year after they received their degree),
• Portugal (Survey of former PhD Scholarships receivers
concerning their professional situation),
• India (Pilot study on the Career Profile and Professional
Achievements of Doctorates in science from the Selected
Central Universities/Institutions),
• Ukraine (Structure of database of Doctorate Holders in
Ukraine)
www.uis.unesco.org
Modules
• Doctoral Education (EDU)
• Recent graduates (REC)
• POSTDOCS (POS)
• Employment situation (EMP)
• Career-related experience and scientific
productivity (CAR)
• International mobility (MOB)
• Personal characteristics (PER)
www.uis.unesco.org
Doctoral Education
Information on educational history of
doctorates holders, such as:
• institution
• field of science
• duration
• sources of financial support
• country of previous degrees
www.uis.unesco.org
Recent graduates
• Data on the holders of doctorate who
received their degree in the last 2 years.
• Provides a complete educational history,
including names and locations of secondary
and each postsecondary institution, dates of
attendance, field of study, and date of
degrees.
• Enquires about the time elapsed before
obtaining first CAREER PATH JOB .
www.uis.unesco.org
POSTDOCS
• Would your principal job be considered a
POSTDOC position in your country of
residence? (PROBLEM WITH DEFINITION)
• Title, Field of S&T, total length of
POSTDOC
• Reasons for taking this POSTDOC
• Activities involved in POSTDOC (Research,
teaching, other)
• Main source of financial support
www.uis.unesco.org
Employment situation
• Detailed information on employment status,
working hours, principal employer, sector and type
of employment (postdoc, temporary or permanent
employment, part-time or full-time job ), annual
earnings, job-satisfaction.
• Relationship of job & doctorate degree.
• Retrospective career history within ten past years
(including occupation, dates, years of experience,
and reasons for leaving position).
• “multi-job” pattern is foreseen.
• Section for unemployed and inactive.
www.uis.unesco.org
International mobility
• Information on mobility patterns, including inflows
(group D) and outflows (group C).
• Enables to distinguish temporary mobility from
permanent mobility as well as to identify the
reasons for departure and return.
• Provides data on intentions to move out of the
country within the next year including the
destination planned.
• Enquires about linkages with country of origin.
www.uis.unesco.org
Career-related experience and
scientific productivity
• Data on scientific output and experience, as
well as teaching.
• Section for researchers, including reasons
for taking up research careers.
• Enquires about international cooperation
activities.
www.uis.unesco.org
Personal characteristics
• Marital status
• Number of dependents
• Place of birth, date of birth, citizenship
status (in order to better deal with dual
nationality as well) and residential status.
• Contact information
www.uis.unesco.org
Way forward
l Instruments are being tested.
l Joint publication with OECD & Eurostat
l Involve more countries to create an international
database
www.uis.unesco.org
Innovation
www.uis.unesco.org
UIS approach to innovation statistics
l Analyse experiences in innovation measurement
in developing countries.
l Propose amendments to Oslo Manual.
l Development of an Annex to the Oslo Manual
“Measuring Innovation in Developing Countries”,
with the participation of experts across the
developing world.
l Dissemination of Oslo Manual & Annex.
Discussion of its applicability in other developing
countries.
www.uis.unesco.org
Measuring Innovation in developing
countries
l After the publication of the 2nd Oslo Manual, also
developing countries started conducting innovation
surveys.
l The design of the surveys was intended to comply
with Oslo Manual standards, with adaptations for
capturing the particular characteristics of innovation
processes. Adaptations were prepared by each
country separately and with different approaches.
l Bogotá Manual published by RICYT (Ibero American
Network on S&T Indicators) first effort to compile
particularities and guide the design of crossnationally comparable innovation surveys.
www.uis.unesco.org
Annex to OM (continued)
l UIS circulated a base document prepared by
RICYT to a vast network of experts in the
developing world covering China, Thailand,
Singapore, Malaysia, Hungary, India, Lebanon,
South Africa, Tanzania, and Thailand.
l UIS drafted the final annex based on this input.
www.uis.unesco.org
Characteristics of innovation in
developing countries
l
Size and structure of markets and firms
l
Instability
l
Informality
l
Particular economic and innovation environments
l
Reduced innovation decision-making powers
l
Weak innovation systems
l
Characteristics of innovation
www.uis.unesco.org
Innovation measurement in
developing countries
l The definition of innovation needs to
remain unchanged, as well as those
concerning its subtypes.
l The concept of potentially innovative firm
is incorporated.
l Measurement priorities:
• Innovation capabilities (Human resources, Linkages,
Quality assurance systems, ICTs)
• Expenditure on innovation activities
• Organizational innovation
www.uis.unesco.org
Principal adaptations
l ICTs in innovation surveys
• strategic use of new technologies (“Front office” vs “Back office”)
l Linkages
• linkage agents and types of linkage
• geographical location of linkages
l Innovation Activities
•
•
•
•
•
“Hardware purchase”, and “Software purchase”
“Industrial design”, and “Engineering activities”
“Lease or rental of machinery, equipment and other capital goods”
“In-house software system development”
“Reverse engineering”
l Human resources + training
l Quality and environmental management
www.uis.unesco.org
Methodological issues for
developing country contexts
• Information systems specificities –
weakness of statistical systems
• Application of the survey
• Questionnaire design
• Frequency
• Publication
• Difficulties
www.uis.unesco.org
Thinking ahead
• The role of entrepreneurs and their attitudes towards
innovation.
• The intention to capture innovations driven by factors other than
market forces, and in particular innovations conducted by the
public sector.
• The adaptation of methodology to measure innovation in the
primary sector (particularly in agriculture).
• The need for better measuring minor or incremental
changes, including innovative applications of existing products
or processes, and the so-called 'backwards integration' of
technological capability.
• The development of indicators reflecting sub-national (regional)
innovation systems.
www.uis.unesco.org
Issues arising in the follow-up to the
Annex
• Innovation in informal sector?
• Innovation from traditional knowledge?
• Surveying innovation, rather than R&D, in
business (and informal) sector?
www.uis.unesco.org
Innovation data collection
• In 2007, innovation data and metadata will be
collected through a pilot project.
• Only countries with existing innovation surveys will
be approached.
www.uis.unesco.org
Research and Experimental
Development (R&D)
www.uis.unesco.org
UIS approach to R&D statistics
(currently)
• Applying Frascati Manual (FM) as much as
possible.
• Disseminate the FM methodology through
workshops
• Listen to countries (in workshops) and be
attentive to issues where the FM is not suitable for
developing countries and look for solutions
together with the country professionals.
www.uis.unesco.org
Breakdowns required by UIS
 R&D personnel by gender
 R&D personnel by occupation and gender
 R&D personnel by sector of employment and occupation
 R&D personnel by sector of employment and gender
 Researchers by formal qualification and sector of
employment
 Researchers by formal qualification and gender
 Researchers by fields of science and sector of employment
 Researchers by fields of science and gender
 Total expenditure in R&D by sector of performance
 Total expenditure in R&D by source of funds
www.uis.unesco.org
HOW do we collect data?
 R&D Surveys. Innovation surveys.
Combined R&D-innovation surveys.
-> Good quality questionnaires are needed!
 Administrative data (budget, personnel list)
 S&T management information systems
 Time-use surveys
 Estimations
Different strategies for different sectors:
one size does not fit all!
www.uis.unesco.org
Conclusions
www.uis.unesco.org
Conclusions: How do we ORGANIZE
a sustainable S&T statistics system?
 Convince policy-makers
 Involve multiple actors
 User/producer consultation
 Create national S&T statistics groups
 Shared ownership of data
 Quality is key – capacity building of producers
 Step-by-step approach
 Combine statistics with S&T information systems
 Network with colleagues from other (similar) countries
 Prioritize areas of work: R&D – innovation – CDH?
 Develop sector specific approaches and tools: GOV-HE-BUS-PNP-FOR
www.uis.unesco.org
Conclusions
• Three different approaches to development of
methodologies for developing countries:
• R&D, focus on comparability.
• Innovation, focus on adaptation.
• CDH, joint development from the scratch.
• The Annex to the Oslo Manual as an inspiration for
further work on other areas, particularly R&D.
• Developing countries are not homogeneous, UIS
is trying to get closer, but new developments will
need significant participation of national experts.
www.uis.unesco.org
Thank you!
http://www.uis.unesco.org
UNESCO Institute for Statistics
C.P. 6128 Succursale Centre-ville,
Montreal, Quebec, H3C 3J7,
Canada.
TP: (1 514) 343-6880
Fax: (1 514) 343-6872
www.uis.unesco.org