Transcript Folie 1 - CertCost

Improving the Organic Certification System
Workshop in Brussels, October 14, 2011
Risk Based Inspections
in organic farming
Raffaele Zanoli
Università Politecnica delle Marche, IT
Seventh Framework
Programme
Grant Agreement No.
207727
A working definition of a RBI
The goal of Risk Based Inspections (RBIs) is to develop a
cost-effective inspection and maintenance program that
provides assurance of acceptable integrity and reliability of
a control system
A risk based approach to inspection planning is used to:
Ensure risk is reduced as low as reasonably practicable
Optimize the inspection schedule
Focus inspection effort onto the most critical areas
Identify and use the most appropriate methods of
inspection
Modelling RBI systems: Objectives
1. Assessment and description of the current
inspection practices in terms of risk and
efficiency
2. Define a probabilistic model to increase the
efficiency of the system based on probability
theory
3. Optimisation of enforcement measures to
reduce the occurrence of objectionable
organic production
Modelling RBI systems: Data required
In order to predict the risk of non-compliance
At farmer/operator level
Depending on crop type, farm type, geographic
location, operators characteristics, etc.
We need data on:
detected non-compliances;
structural, financial and managerial information
at operator level
Modelling RBI systems: Data available
Collected during CERTCOST EU project
Data from 6 different European CBs (from CH,
CZ, DE, DK, IT, UK)
Three years covered (2007-2009)
We used standard data that is routinely recorded
by inspection bodies
Available data do not match the
requirements
Databases mainly contain structural data
CBs collect NC data with non-homogenised textual
descriptions: hard to rank NC severity
Sanction data are more standardised, but:
they are only a proxy of NC
no common definition of sanctions across CBs /
countries
no clear relationship between NC and sanctions (with
some exceptions);
no information available about why an operator
receives a sanction (e.g. use of pesticides in wheat
production, use of unauthorised feed for livestock, etc.)
Homogenisation of sanctions across CBs
and countries
IT, CZ (and UK) CBs use a similar 4 sanction category (UK:
NC)classification
Further aggregation in terms of slight and severe sanction categories
IT, CZ, UK straightforward interpretation; DE, DK, CH: input from CBs
to correctly classify sanctions
Distribution of farms, by sanction
category, country, and year
Modelling RBI systems: Analytical tools
Econometric
models
Parametric approach:
a-priori distributional
assumptions
Bayesian
Networks
Semiparametric approach:
more flexible, no individual
factor testing allowed
Statistical testing of
each risk factor, count
models
Single and multiple risk
factors impact evaluation
Good for standardised
analysis across
countries/CBs
Good for farm type
simulations
Potential risk factors
46 hypothesis concerning factors affecting the probability for
an operator to get a sanction has been generated with
collaboration from partners
The hypothesis refers to the following aspects:
general risk,
structural / managerial for farms,
structural/managerial for processors,
specific crop, livestock and product variables,
control related issues
Some of the hypothesis cannot be tested for all
countries/years due to missing data (eg processor turnover,
risk class)
Factors increasing/decreasing risk
Factors increasing/decreasing risk
Factors increasing/decreasing risk
Few risk factors found relevant for all countries: Past behaviour, Farm
Size, Bovine livestock
History dependence: operators who are not compliant tend to continue
to be so
if one operator has been non compliant the previous year is more
likely to be non compliant in the next year
If one operator has committed minor irregularities is more likely to be
found to have committed major infringements
No overall risk pattern for crop types, though country specific risks
For livestock, bovines and pigs entail higher risk
In countries where (slight) non compliances are more numerous (DK,
UK, partly CH) there might be a higher farms homogeneity, hence lower
discrimination effects of explanatory variables
Personal, farmer-specific variables are probably crucial in explaining risk
but we have VERY limited data on these
General conclusions
We can say with some confidence which factors contribute to
risk, but we cannot rule out those who don’t
As a consequence, we cannot define low risk operator types
To implement more efficient Risk Based Inspection procedures
CBs would need better or different datasets
RBI based on past experience can limit predictable risk, but
cannot avoid potential ‘catastrophic’ events
uncertainty is an essential factor that should inform inspection
procedures (black swans): think what can impact (the sector,
the consumer, the CB, etc.) most, even if the risk (probability) of
occurrence is low (but maybe the cost of detection is also low)
Some statements to open discussion
Harmonised RBI is fundamental to guarantee integrity, improve
efficiency and reduce the cost of inspection: a growing body of small
“organic” farmers and growers are refusing certification and inspection
schemes and selling on alternative short supply-chains – this creates
further confusion among consumers
Without clear and uniform criteria for classifying non-compliances as
irregularities or infringement AND without better data and better
information systems, no RBI system can work on a global scale
Without global trust on certification and inspection procedures no
global organic trade can survive
Grazie!
Thank you!
[email protected]
Seventh Framework
Programme
Grant Agreement No.
207727
Limitations of the study
Data issues:
Data suffer from censoring (i.e. missing data): we only have
information on NCs that were detected by the CBs, but we have no
idea how many and what kind of NCs have NOT been detected
Inspection data contain varying quality/quantity of management &
structural data, but little/no personal information on operators
All operators should be inspected at least once per year (legal
requirement), but the share of subsequent inspections (either
unannounced or follow ups) varies across countries and CBs
Data are little/no harmonised both within a country and across various
countries
Limitations of the study (2)
Epistemological/methodological issues:
What is the data generating process (DGP)? Since CBs are actually using
some form of internal RBI protocol to inform timing of compulsory
announced inspections as well as follow-ups and unannounced
inspections, the risk factors that we have observed may simply depend on
their inspection planning and NOT actual risk (confirmation bias)
Due to limited amount of severe NCs and related sanctions in the
database, the reliability of the analysis of factors influencing severe risks is
limited by statistical reasons