Transcript Econometric evaluation of public innovation subsidies: state of the art, limitations and future research Dirk Czarnitzki K.U.Leuven and ZEW Mannheim Paris, November 9th 2010

Econometric evaluation of public
innovation subsidies:
state of the art, limitations and
future research
Dirk Czarnitzki
K.U.Leuven and ZEW Mannheim
Paris, November 9th 2010
Introduction
• R&D is subject to market failure
– External effects
– Financial constraints
• Governments try to „correct“ market failure using
several policies
– Governments invest in public science
– Intellectual property rights systems
– R&D collaborations are exempt from anti-trust policy
– Public R&D grants or tax credits for companies
• Often preferred treatment for research consortia
• Recently especially industry science collaborations
Introduction: Direct subsidies
• Problem: crowding-out may occur!
– Once subsidies are available, companies have an incentive to
apply for any project (even the privately profitable ones) as
subsidy comes at marginal cost equal to zero.
– Subsidies may not only stimulate the projects with high social
return.
– In the worst case, private funding is simply replaced with public
funding.
– How can government select projects that were not carried out
otherwise?
• How to evaluate the „success“ of a policy?
An example of self-assessment by companies (259
subsidized German companies in 2001)
Project implementation became possible
Project start accelerated
Project duration reduced
Project scope extended
Increased technological level
Led to patent application
0% 10% 20% 30% 40% 50% 60% 70%
The Evaluation Problem
• Aim of quantitative methods of evaluation is the measurement of
effects generated by policy interventions on certain target variables;
• in innovation context, for example:
• impact of R&D subsidies on firms‘ private innovation/R&D
expenditure (input)
• or on other variables like patent applications etc. (output)
• David et al. (2000) review the literature on crowding-out effects and
Klette et al. (2000) survey microeconometric studies including output
analyzes (like firm growth, firm value, patents etc.)
• Critique of early surveys: results varied a lot, possibly because of
deficencies of methods that researchers used in the past
 selection bias (next slide)
• Recent survey, Cerulli (2010) surveys more recent literature that used
econometric methods for „treatment effect“ estimation.
The Evaluation Problem
In most cases, one is interested in the average „treatment effect on the
treated“ (TT), that is, the difference between the actual observed value
of the subsidized firms and the counterfactual situation:
„Which average value of R&D expenditure would the treated firms
have shown if they had not been treated“
Problem: The counterfactual situation is never observable and has to be
estimated!
TT  E Y | S  1  E Y | S  1
T
C
S:= Status of group, 1 = Treatment group; 0 = Non-treated firms
Outcome: YT = in case of treatment; YC = counterfactual situation
The Evaluation Problem
How can we estimate the counterfactual situation?
Problem: Those firms receiving a treatment may be different from those that
don‘t.
Thus: We cannot use a random sample of non-treated without any adjustment.
Example: Agencies that fund R&D follow a picking the winner strategy, as they
want to maximize the outcome of the funded projects.
 Firms that show high R&D in the past, professional R&D management, good
success with their other R&D projects will be preferably selected by policy
makers.
 Subsidy receipt becomes an endogenous variable (depending on the firms
characteristics).
 Solution: experimental setting, that is, a random assignment of treatments;
or the evaluation of policy via treatment effects estimators in non-experimental
settings.
Evidence on treatment effects
estimation
• „Buzzwords“:
– Econometric Matching
– Selection models (also parametric treatment effects models, or
control function approaches)
– Instrumental variable estimation
– (conditional) difference-in-difference estimation
– Regression discontinuity design
• Most recent studies find positive effects of R&D
subsidies on R&D investment
• However
– Researchers often only observed subsidized yes/no, instead of
amount of funding.
– Only some studies that use exact amount of funding/year.
Real effects or wage effects?
• Recent literature goes beyond the mere treatment effect
on the treated estimation.
• For instance, „Goolsbee (1998) critique“
– One may find increased investment because of subsidy
– However: risk that subsidy only ends up in higher wages of
researchers
– If wage increase does not coincide with productivity increase,
there would be no „real“ effect on knowledge creation.
• Recent evidence for Europe
– Kris Aerts, Ph.D. Thesis 2008 K.U.Leuven, for Flanders
– Pierre Mohnen and Boris Lokshin for NL
 „real“ R&D is also stimulated by subsidies.
Policy design: large versus small firms
• Researchers often find larger treatment effects for
smaller firms than for larger firms
• This may be a statistical artefact to some extent, though
– Often only small sample on really large firms (data limitation)
– Proportion of subsidized R&D is much smaller in large firms
compared to total R&D budget than in smaller firms
• Easier to find a „significant“ effect in small firms than in large firms.
• Measurement or specification problem
• See e.g. Aerts and Czarnitzki (2006), IWT study.
Policy design: size of subsidies
• Some evidence that small subsidies are „not useful“
– Gonzales et al. (2006)
• However, more research is needed here.
• Could also be a measurement problem as subsidy is
usually related to total R&D of the firm, and thus it is not
surprising that small subsidy has no large effects.
• If one applies an estimation technique that is very
flexible with regard to functional form assumptions
– (GPS method for estimating dose-response functions)
• …we can learn about about potential crowding out
effects in different areas of the subsidy distribution
-5
0
5
10
A dose response function
-6
-4
-2
ln(subsidy)
D ose--response function
yols
0
2
Nadaraya-Watson estimates
lnfue
0
1
2
3
4
5
A dose response function
-6
-4
-2
ln(subsidy)
Dose--response function
yols2
0
2
yols
0
.5
1
Estimated elasticities
-6
-4
-2
ln(subsidy)
hy
E OLS SQ
0
E OLS LIN
2
Policy design: e.g. collaborative
research
• Often preferential treatment of R&D consortia,
especially industry science collaborations
• External effects can be internalized within consortia by
collaboration
• Duplicate research avoided
• Participants can benefit from bundling knowledge and
realizing economies of scope (knowledge from a
collaborative project can be used for other projects)
• Thus, there may be a „money effect“ and a „spillover
effect“.
Evidence from German policy
Division of collaborative research grants by type of research consortia
Granted amount in million EUR (nominal) .
450
400
only firms
350
only science
300
firms+science
250
200
150
100
50
0
1985
1987
1989
1991
1993
1995
1997
Source: PROFI database from Germany’s Federal Ministry of Education and Research;
own calculations.
1999
2001
2003
2005
Policy design: e.g. collaborative
research
• Evidence that spill-over effects are present and that
treatment effect of collaborative research is larger than
for „individual subsidies“
• Also: spill-over effects larger for collaboration with
science
– Projects more basic, i.e. more generic in terms of knowledge
creation?
– Leading to higher economies of scope?
• Branstetter and Sakakibara (2002), Czarnitzki and Fier
(2003), Czarnitzki, Ebersberger, Fier (2007), Czarnitzki
(2009).
Policy design: type of R&D
• What type of R&D is actually funded by governments in
the business sector?
• Is it mainly basic research?
• Or rather applied research and technological
development?
• Market failure may be larger for basic research than for
other types
– Basic research further away from market
– Much higher uncertainty about oucomes and industrial
applications
• Does the agency behave similar to a bank? That is,
also prefer less risky projects when making a grant
decision?
Policy design: type of R&D
• Czarnitzki, Hottenrott, Thorwarth (2010) find indeed that firms
suffer more from financial constraints with regard to „Research“
than for „Development“
• Also: firms that receive subsidies for basic research show no
sensitivity to financial constraints, but non-subsidized do.
Total
Number of
submitted projects
Grant rate
(Strategic) Basic
Research
Experimental
Mixed projects
Development and
Prototyping
3506
1389
829
1288
81%
75%
91%
82%
Grant rate of submitted project proposal by type in Flanders
Note: The data were kindly provided by IWT Flanders (own calculations).
Policy Mix:
Subsidies versus R&D tax credits
• How should the government decide on which
instrument to use?
• Not so much evidence!
– Berube and Mohnen (2009) for Canada: among R&D tax credit
recipients, firms that receive direct subsidies invest additional
funds.
– Takalo, Tanayama, Toivanen (2009), structual model on
application decision of the firm, grant decision of agency
(yes/no and subsidy rate), investment decision of firm
– Model allows simulating absence of subsidies, or „tax credits
only“ versus „direct grants only“
– Similar effects of subsidies and tax credits
Distributional effects
• Czarnitzki and Ebersberger (2010) apply a standard
treatment effects estimation,
• but then use the results to derive a Lorenz curve of
R&D concentration for Germany and Finland
• (Why Germany and Finland? Only direct grants, but no
R&D tax credits available)
• R&D concentration is lower in actual situation (policy
regime with direct subsidies) than in counterfactual
situation (absence of any policy)!
Distributional effects
Lorenz-curve for the distribution of
R&D personnel (Finland)
Lorenz-curve for the distribution of
R&D personnel (Germany)
Going beyond „treatment on the
treated“ on innovation INPUT
• It could be of interest whether firms that are currently not
benefitting from subsidies would invest more into R&D if they would
receive subsidies
– „Treatment on the Untreated“
• Some recent evidence: cross-country comparison Spain, Belgium,
Germany, Luxembourg, South Africa (Czarnitzki and Lopes Bento,
2010)
• However, also evidence that currently funded companies do not
invest more than actually non-funded firms would invest if they
received subsidies.
• More research is needed here! Our study suffers from severe data
limitations.
• One could also estimate which firms out of non-recipients would
invest most if they would get subsidies
– Ongoing research…..
Going beyond „treatment on the
treated“  innovation OUTPUT
• Does subsidized R&D leads to more innovation output eventually
• Subsidized projects may fail more frequently than others. So, more
patents? Higher sales with new products?
• After treatment effects estimation, total R&D can be decomposed
into 2 components
– The R&D that the firm would have conducted anyway
– and R&D that was induced by subsidy
(subsidy + additionally triggered R&D)
• Some evidence that both components of R&D have a positive
impact on patents and new product sales
(Czarnitzki and Hussinger, 2004, Czarnitzki and Licht, 2006,
Hussinger, 2008).
• However: subsidized component‘s productivity is slightly lower then
pure privately financed R&D. (consistent with neoclassical theory)
• Limitations: Timing of output relative to input?
Conclusions and discussion
• Many research questions have been addressed.
• Yet, the evidence for actual policy making is still limited
• Questions:
– Young Innovative Companies (YICs), see Veugelers (2009),
Schneider and Veugelers (2010)
– What is the „optimal“ policy in terms of size of tax credit or mix
with direct R&D grants?
– What is a superior design of a funding instrument, e.g.:
• Application and grant decision versus:
• 2-stage process as in U.S. SBIR program: 1st stage conceptual feasibility
study (small amount of subsidy), 2nd stage large subsidy (only 30% of 1st
stage participants survive, on average).
– Sound cost-benefit analysis: treatment effects vs. cost for
society.