Transcript A Systems Approach to Establishing Scientific Integrity in

A Systems Approach to
Establishing Scientific Integrity in
Evidence Based Policy Making
Prof. Dr. Wijnand J. Swart
Centre for Plant Health Management (CePHMa)
Faculty of Natural and Agricultural Sciences
University of the Free State
Bloemfontein
South Africa
[email protected]
EVIDENCE in Policy Context
• Sound, credible and robust evidence, whether :
–
–
–
–
–
–
–
–
–
quantitative
qualitative
statistical
economic
attitudinal / behavioural
anecdotal
social
opinion based
or review based .....
......is an essential and necessary part of the
enabling environment for formulating policies
that are coherent and effective in terms of their
outcomes.
The Policy Making Process
• Techniques, analyses and judgements used to
evaluate and formulate data and information into
knowledge / evidence for making effective policies
are critical.
There are policies that:
Use GOOD data &
information….
…and use it
WELL…
Use POOR data &
information….
…and use it
POORLY…
Source: DEFRA, UK
What is ‘Sound Evidence?’
• Concept of “sound and credible evidence”
is very complex.
• Dependant on inter alia:
–
–
–
–
–
types
sources
generating techniques
context
‘understanding’
...... of data / information / knowledge
‘Sound Evidence’: Context & Understanding
TRUTH?
• Knowledge comprises
strategy, practice, method, or
approach (how).
• Wisdom embodies principle,
insight, morals, or archetypes
(why).
• Absolute Truth ?
CONTEXT
• Information relates to
description, definition, or
perspective (what, who, when,
where).
INDEPENDENCE
• Data are facts (e.g. numbers,
names, symbols) and have little
value in themselves.
WISDOM
understanding
principles
KNOWLEDGE
understanding
patterns
INFORMATION
understanding
relations
DATA
UNDERSTANDING
Knowledge vs Science
• Science is organized knowledge. Herbert Spencer
• Science: a knowledge of principles and causes.
(Webster's Revised Unabridged Dictionary)
• Science is a way of thinking much more than it is a body
of facts. Carl Sagan
• “Good Science” could be defined as those practices
which contribute most to advances in understanding.
Sindermann: The Joy of Science 1985.
• Science is to see what everyone else has seen but think
what no one else has thought. Albert Szent-Gyorgyi
Science and Policy Making
• Creative and innovative contributions
of scientists and policy makers and the
trust engendered in them by the public,
to whom they are accountable is of
paramount importance.
• Follows therefore that fostering an
environment that promotes research or
scientific integrity is an integral part of
that accountability and the pursuit of
new knowledge.
‘Scientific Integrity’
•
“Integrity” by definition:
– “honesty”
– “a state of being entire or whole”
•
Perspectives of ‘scientific integrity’ :
1.
Ethical issues relating to misconduct (fraud) or
manipulation, suppression, or distortion of facts.
2.
Striving towards “wholeness” or excellence in the
search for knowledge.
Research Integrity
•
Essential for maintaining scientific excellence and for
keeping the public’s trust.
•
Research integrity characterizes :
a.
Institutional integrity:
 creating an environment that promotes responsible conduct and
high levels of integrity
 embracing standards of excellence, trustworthiness, and
lawfulness
b.
Individual integrity: Scientist’s commitment to intellectual honesty
and personal responsibility and is an aspect of moral character and
experience. A good scientist must:
 communicate well
 obtain research grants
 excel in teaching and mentoring
 engage in ethical decision making
 use knowledge ‘wisely’ to plan and execute research.
Institutional Integrity:
Politics vs Science
• Politicization of science as old as science
itself e.g Galileo's theory that the Earth
revolves around the sun perceived as a
challenge to the authority of the Catholic
church.
• Political interference threatens the
integrity of government science and policy
making all over the world.
• Manipulation, suppression, and distortion
of government science misinforms public
and leads to poor policy decisions.
• Especially rife in developing countries,
e.g. the assertion of the South African
president Thabo Mbeki that AIDS is not
caused by HIV flew in the face of decades
of research and threatened to undermine
proper treatment of the disease.
Bush Administration's Misuse of Science
• On February 18, 2004, 62 pre-eminent
scientists AND researchers released a
statement titled Restoring Scientific
Integrity in Policy Making in the USA.
• Scientists charged the Bush administration
with widespread and unprecedented
“manipulation of the process through which
science enters into its decisions.”
• Scientists accused Bush administration of :
1. Epidemic altering and concealing of
scientific information by senior officials in
various federal agencies.
2. Active censorship of scientific information
that the administration considered
threatening to its own philosophies
3. Restriction of the ability of governmentsupported scientists to freely communicate
scientific ideas related to "sensitive" issues
.
Integrity of Research Institutions
• The organizational structure and processes that typify the
mission and activities of a research institution can either
promote or detract from the responsible conduct of research.
• These process are in part determined by the external
environment and are influenced by the dynamics between and
among organizational members.
• Any element or part of an organization can be viewed as a
system in and of itself.
• External conditions influence the inputs into an organization,
affect the reception of outputs from an organization’s
activities, and directly affect an organization’s internal
operations.
Open systems model of internal environmental elements of a research
organization showing:
 Inputs / resources for organizational functions
 Structures and processes that define an organization’s operation
 Outputs / outcomes of activities carried out by individual scientists, research
groups or teams, and other research-related programs.
Source: National Academy of Sciences - http://www.nap.edu
• Interrelatedness between research organizations and the various
external influences that have an impact on integrity in research.
• Systems and subsystems of the external-task environment are
embedded within the general sociocultural, political, and economic
environment.
• Relationships also exist between and among the elements within
the external environment.
Source: National Academy of Sciences - http://www.nap.edu
Holistic Knowledge / Evidence
• Rather than focus on the ethical or moral aspects of
scientific integrity, focus here is on the process of
generating data and information and integrating it into
sound knowledge (sound evidence) for decision-making.
• “Integrity” by definition:
– “honesty”
– “a state of being entire or whole”
• “Integrate” by definition:
– “to combine parts into a whole”
Jan Christian Smuts,
Holism and Evolution
1926
“A whole.. is more than the sum of its parts.”
‘Sound Evidence’: A Holistic View
• A collection of data is not information.
• A collection of information is not knowledge.
• A collection of knowledge is not wisdom.
• Information, knowledge, and wisdom are more than
simply collections
• Each concept represents more than the sum of its parts
and has a synergy of its own.
A “WHOLE” AS A SYSTEM
• “ A system is defined as a set of interacting units with
relationships among them. The properties (or behaviour)
of the system as a whole emerge out of the interaction of
the components comprising the system.”
• The interactions of the parts become more relevant to
understanding the system than understanding the parts.
• This definition of a system implies something beyond
cause and effect.
The Ultimate System
• In truth only one system, the “Universe"
• All systems are sub-systems of a larger system.
COMPLEXITY
?
universe
galaxy
solar system
world
nation
state
community
person
organ
cell
molecule
atom
particle
NUMBER OF SUB-SYSTEMS
Systems Thinking and Policy
• Science is a way of thinking much more than it is
a body of facts. Carl Sagan
• “Systems thinking” offers a conceptual
framework or model for ‘thinking differently’.
• Systems thinking has permeated many scientific
fields including: education, business
management, human development, sociology,
psychology, agriculture, ecology and biology,
earth sciences.
Hard vs Soft Systems
• Adopting a systemic perspective on solving policy
problems therefore appear to offer a useful way of
correcting these deficiencies.
• In 1960s, a ‘hard’ (quantitative) systems approach was
touted as the policy science.
• However, hopes not realized for variety of reasons; its
comprehensive modelling too information-intensive and
mathematical.
• The ‘soft’ (qualitative) systems approach of systems
thinking has increasingly been used since the 1990’s as a
paradigm in policy planning and implementation.
• Soft systems methods stress the self-organizing and
adaptive capacities of appropriately designed systems.
Hard vs Soft Systems
Hard systems methods
•
•
•
•
•
•
•
•
•
Objective philosophy
Computer science + systems theory
Rigid method
Data, process, database technical
focus
Scientifically analytical
Analyst is expert
Analyst dominated
Computer design outcomes
One ‘correct’ solution
Soft systems methods
•
•
•
•
•
•
•
•
•
Subjective (interpretive) philosophy
Systems + sociological theory base
Flexible methodology
Organizational problem-solving
focus
Creative / intuitive
Analyst is facilitator
Participative
Organizational learning outcomes
Several ambiguous outcomes
Agro-ecosystem
A hard systems view of a
farming system; a
biological network suitable
for mathematical solution.
A soft systems view of a
farming system; an arena for
gaining experience and
increased understanding.
Source: Robinson, B. 2003. 11th Australian Agronomy Conference)
Soft Systems Methodology (SSM)
• Can be used both for general problem solving and in
the management of change.
• Used in the analysis of complex situations where there
are divergent views about the definition of the problem
— "soft problems" or policy options (e.g. How to
improve health services delivery; How to manage
disaster planning).
• At the heart of SSM is a comparison between the world
as it is, and some models of the world as it might be.
• Out of this comparison comes a better understanding
of the world ("research"), and some ideas for
improvement ("action").
SSM for Problem Solving
• ‘Classic form’ of SSM consists of seven steps:
– Problem unstructured by researchers
– Problem situation expressed to capture “rich picture”
– Create root definitions of relevant systems (i.e. social, political &
environmental)
– Making and testing conceptual models based upon world views
– Comparing conceptual models with reality
– Identifying feasible and desirable changes
– Acting to improve the problem situation
reality
understanding
and
improvement
conceptual
models
• Differences between models and reality become the basis for
planning and policy making process.
Multi-agent systems (MAS)
• Policy increasingly has to address topics that have to do with
disequilibrium, dynamics, and locality.
• The overwhelming complexity of biophysical and socioeconomic constraints that increasingly characterize rural areas
in developing countries necessitates the development of more
sophisticated tools to support policy making in these areas.
• Multi-agent systems (MAS) are a relatively new field in
computer science that have been proposed as a modelling
approach for establishing even higher levels of scientific
integrity in the generation and evaluation of evidence for
making policies.
• Analogous to artificial intelligence.
MAS for Policy Making
• Multi-agent models might be the preferred choice when
heterogeneity and interactions of agents and environments are
significant and policy responses cannot be aggregated linearly.
• MAS can thus complement bio-economic simulation models which
cannot fully capture the heterogeneity in biophysical and socioeconomic constraints and the interactions between them.
• There are several policy questions in the context of agricultural
development of rural areas where MAS simulations may generate
useful information for decision making on public investments in
R&D and the targeting of policy interventions.
• Examples of such policy questions:
– Should funds be spent on crop breeding for stress resistance
or in research for improved crop management?
– Should micro-finance be promoted or should agricultural inputs
be subsidized?
Agents
• “An agent is anything that can be viewed as
perceiving its environment through sensors and
acting upon that environment through effectors”
EFFECTOR
AGENT
SENSOR
OUTPUT
INPUT
SYSTEM
ENVIRONMENT
• Agents may be persons, farms, markets, computer programmes
or anything that is reactive, autonomous, and goal-oriented.
• Agents may have the ability to communicate with other agents,
learning, mobility, and flexibility. May even have personality and
show emotions!
Agent Flexibility
• An intelligent agent is capable of flexible autonomous
action.
• FLEXIBLE:
– Reactive: A reactive system is one that maintains an
ongoing interaction with its environment, and
responds to changes that occur in it (in time for the
response to be useful)
– Pro-active: Generating and attempting to achieve
goals; not driven solely by events; taking the initiative
i.e. goal directed behavior; recognizing opportunities
– Social: Ability to interact with other agents via some
kind of agent-communication language, and perhaps
even cooperate with others.
PROBLEM SOLVING
REACTIVE
Identifying a
problem
PROACTIVE
Definition of the
system
ANALYSIS
ANALYSIS
Understanding the
problem
Understanding
the system
POLICY
POLICY
MANAGEMENT
MANIPULATION
“Improving”
problem
Maintaining
the system
BIBLIOGRAPHY
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Balmann, A. 2000. Modeling land use with multi-agent systems: perspectives
for the analysis of agricultural policies.
Berger, T. and Ringler, C. 2002. Trade-offs, efficiency gains and technical
change – Modeling water management and land use within a multiple-agent
framework, Quarterly Journal of International Agriculture 41:119–144.
Berger, T., Schreinemachers, p., and Woelcke, J., 2006. Multi-agent
simulation for the targeting of development policies in less-favored areas.
Agricultural Syatems 88:28-43.
Checkland, P. 1981 Systems thinking, systems practice. Chichester: Wiley.
Checkland, P., and Holwell, S. 1998 Information, systems, and information
systems: making sense of the field. Chichester, UK: Wiley.
Checkland, P. and Scholes, J. 1991 Soft systems methodology in
action. Chichester: Wiley.
Harrison MI. 1994. Diagnosing Organizations: Methods, Models, and
Processes, 2nd ed. Thousand Oaks, CA: Sage.
Schreinemachers, P. Berger, T. and Aune, J.B., 2007. Simulating soil fertility
and poverty dynamics in Uganda: A bio-economic multi-agent systems
approach. Ecological Economics 64:387-401
Union of Concerned Scientists. 2004. Scientific integrity in policy making: An
investigation into the Bush administration's misuse of science. Cambridge
(Massachusetts): Union of Concerned Scientists; 49 pp.
Woolridge, M. 2002. An Introduction to Multiagent Systems by John Wiley &
Sons (Chichester, England).
THANK YOU!