Transcript Document

http://heande.pyrkilo.fi
National Public Health Institute, Finland
Open risk assessment
Lecture 1: Introduction
www.ktl.fi
Jouni Tuomisto
KTL, Finland
1
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Outline
www.ktl.fi
• What are the challenges of the current risk
assessment?
• Why risk assessment is needed in the future?
• What is needed from the new risk assessment?
• Can it work?
• Practical examples and links
2
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
Disclaimer
• Open (risk) assessment is not officially accepted
method or thinking anywhere
• It has been developed in KTL within a risk
research group lead by J.T.
• The idea is actively promoted within several EUfunded research projects (Intarese, Heimtsa,
Beneris, Hiwate, Hitea) without official success so
far
• The idea is promoted in practice by running a
website for open assessments
3
http://heande.pyrkilo.fi
National Public Health Institute, Finland
What are the challenges of the current
risk assessment?
www.ktl.fi
•
•
•
•
•
•
4
Limited area of application
Lack of flexibility and breadth
Inefficiency and slowliness of the process
Deliberate biases towards "safety"
Communication problems
Lack of acceptability among stakeholders
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Limited area of application
• Only a few chemical groups require pre-market RA:
– Pesticides, drugs, food additives
– This will improve with Reach but not disappear
• RA not triggered for many important "natural" exposures:
www.ktl.fi
– Traditional foods and food items vs. GMO
– Environmental exposures: moldy buildings vs. PM
• Often limited to situations where the release links to
someone's economic interest
• Who can and should trigger a RA?
5
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Lack of flexibility and breadth
• Each discipline has developed an own framework
–
–
–
–
–
–
Scientific opinions on food issues by EFSA
Chemical risk assessment for pesticides
Safety assessment for drugs
Life cycle assessment for consumer products
Environmental impact assessments for major construction sites
"Not tested with animals" for cosmetics
www.ktl.fi
• Is this just cultural diversity or a problem of
administration and a health hazard?
6
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Importance of boundaries
• Risk-benefit analysis of farmed salmon (Tuomisto et al, Science 2004)
If risk managers assume responsibility of
total health effect of salmon consumption
-206
30900
-154
23400
www.ktl.fi
0
BAU
Restrict fish use
25000
30000
If risk managers care only for cancer due to
pollutants
7
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Inefficiency and slowliness of the process
• Inefficiency: it takes a lot of person-months to complete
– A lot of expensive expert work
– The risk assessments done are not available for others in a
useful format
• A description of work in a form of a report may be available, but the
actual work, i.e., data, calculations, reasoning, and detailed results are
not.
• Slowliness: it takes a lot of calendar months to complete
www.ktl.fi
– The process has data collection, systematic literature
searches, public hearings, reviews, scientific advisory
panels…
– The dioxin RA by the U.S.EPA:
• a draft was published 1996
• a second draft was published 2000
• …we are still waiting for the final version
• With the same money, there could be more better RAs
8
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Major chemical reviews in IRIS
Chemicals in IRIS website
The most recent major update
600
Number of chemicals
500
400
300
200
www.ktl.fi
100
0
1985
1987
1989
1991
1993
1995
1997
Date of update
9
1999
2001
2003
2005
2007
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Deliberate biases towards "safety"
• Approaches to minimize the false negative error
– Reference dose=NOAEL/UFa/UFi
– BMDL: lower CI of the benchmark dose
– LMS (q1*): linearized multistage
www.ktl.fi
• Poorly known chemicals are perceived worse than
well known major hazards
• The problems tend to fall out of YOUR mandate
(to others to solve (or ignore))
10
http://heande.pyrkilo.fi
National Public Health Institute, Finland
On whose side does the problem fall?
• Risk-benefit analysis of farmed salmon (Tuomisto et al, Science 2004)
If risk managers assume responsibility of
total health effect of salmon consumption
-206
30900
-154
23400
www.ktl.fi
0
BAU
Restrict fish use
25000
30000
If risk managers care only for cancer due to
pollutants
11
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Why is risk assessment needed in the future?
www.ktl.fi
• The real problems in the future are NOT those that the
current risk assessment was developed for:
– Drugs
– Pesticides
– Food additives
• It is needed because it would be nice to do something
useful for the real risks of the future. The risks that are so
complex that no single expert is an expert in all parts of
the issue. These risks are such as...
14
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Climate change
www.ktl.fi
• Uppsala glacier, Patagonia, Argentina: above, in
1928 and below, in 2004.
15
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
Fine particle air pollution
16
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
Energy efficiency
17
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
Urban living environment
18
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
Drinking water amount and quality
19
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
Biodiversity
20
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
Population growth
21
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
Global environmental taxes
22
http://heande.pyrkilo.fi
National Public Health Institute, Finland
What is needed from the new assessment?
Limited area of application
Adoptable by any area of administration or policymaking
Lack of flexibility and breadth
Fully scalable to very simple and very complex
questions
Inefficiency and slowliness of
the process
Info structured & directly reusable
Delegation, non-experts included
Routines automated
Best estimates (incl uncertainty) used
www.ktl.fi
Deliberate biases towards
"safety"
Communication problems
Everything available for clarification questions
Lack of acceptability among
stakeholders
Stakeholders must have a say on everything in
advance
Value judgements included in the assessment
23
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Paradigm shift
www.ktl.fi
• Thomas Kuhn (1922-1996)
• Science progresses in a regular
way until too many faults are
identified in the current
paradigm. Then, there is a
period of extraordinary science,
which leads into a shift of
paradigm
24
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
Pyrkilo
• Jouni Tuomisto (1997): It is possible to develop
such a system, pyrkilo, that transforms ideas,
information, and people's opinions into a
description that tends to converge towards
scientific validity.
• The assessments that are produced using the
pyrkilo method we call open assessments.
• After several years of work, the Heande website
was opened Sept 2006.
25
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Open assessment
• The objectives:
– Find solutions to ALL the challenges at the same time
– Systematize and "industrialize" the risk assessment
– Maintain high scientific quality
• The current situation with open assessment: there are
suggested methods to all challenges listed previously
www.ktl.fi
– Many of the suggestions have not been tested in practice
– Not everything will probably work
• However, there is already a critical mass of solutions
available so that full-scale testing can be started
• Further problems should be solved as they appear
26
http://heande.pyrkilo.fi
National Public Health Institute, Finland
What is the acceptability of the idea of
open assessment?
• Poll (informal, based on observations of several
audiences):
–
–
–
–
30 % think it is a stupid idea
50 % think it cannot work
15 % find it interesting, but…
5 % are fond of the idea
www.ktl.fi
• In which category do you fall in?
27
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
Assessment: Gasbus
28
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Assessments in practice
• They look like Wikipedia articles
• They are written in much the same way
• Substance is on the main page
– Research question
– Definition: how to find an answer
– Result
www.ktl.fi
• Discussion about the substance is on a separate
discussion page
– Discussion can be free or structured
29
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Open assessment
• The research question for the (pyrkilo) method:
– "How can scientific information and value judgements be
organised for societal decision-making in such a way that open
participation is possible?"
• Full range of development
www.ktl.fi
–
–
–
–
30
a new ontological foundation
strictly object-oriented approach
a new structure for information objects
traditional RA methods for processing information, but
organised in a more systematic way
– tools that enable open collaboration
– data sources that are directly available and applicable
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Highlights of the theory behind
• The basic principles are
– Open participation at all phases of the assessment
– Requirement of scientific method (openness to scientific
critique) at all phases
– Reusability of information from one assessment to another
www.ktl.fi
• A uniform information structure is used:
– Assessment: specific information need for a policy decision
– Variable: a truthful description of a particular part of reality; it is
independent of assessments given its scope.
– Both objects have 4 attributes: name, scope (research
question), definition (how to answer), and result
31
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Assessments are performed openly in the
Internet
• Parts needed to run open assessments
– Web pages with method descriptions for the
assessors (Guidebook)
– Web pages about useful data (Resource Centre)
– Web pages about actual assessments
www.ktl.fi
• Descriptions of assessments and models used (Warehouse)
• Actual models (Toolbox)
• Results of assessments in a uniform structure (Result
database)
32
– Web pages for collaborative work on assessments
(Collaborative workspace)
Red parts are officially parts of the Intarese project
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Quality control
• Is quality control possible if pages are available for
all to contribute?
• Our first thoughts (no experience yet, because
work is practically within research projects)
– Pages can be freely added and edited
– Advanced pages are protected from direct edits
www.ktl.fi
• Then, the edits are made through an open discussion on a
separate page; results are transferred to the actual page
– Very important pages go through a peer review and
get a quality label
33
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Can open assessment work?
www.ktl.fi
• I am convinced it can work
• I am convinced the remaining problems can be
solved
• However, this does not mean that it WILL
succeed, at least in our time…
34
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Leonardo's parachute ca. 1500
www.ktl.fi
– first applications in 20th century
35
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Bayes' theorem ca. 1750
• Reverend Thomas Bayes published the Bayes'
theorem in ca. 1750
www.ktl.fi
– first real applications in 1960's
36
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
What is the potential for mass collaboration in
environmental health risks?
• 6,000,000,000 people in the world
• 1,000,000,000 of them have access to Internet
• 10,000,000 of them are seriously thinking about
environmental and/or health problems: ”What
could I do?”
• 1,000,000 of them can speak English
• 100,000 of them have a good background for the
work (e.g. university degree)
• 10,000 of them are willing to spend 1 h/week on
this
• 250 person-weeks/week work force available
37
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
How would the world look like with fullscale open assessments?
• The turnover of scientific information speeds up
• Scientific information is easily available in a readily
useful form always, from anywhere, and in your
own language.
• Time is spent on assessing solutions to problems,
not on talking about assessing them.
• People start to respond to the politicians
suggesting something stupid: ”Did you not even
check Heande?”
38
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
The ORA report
39
http://heande.pyrkilo.fi
National Public Health Institute, Finland
www.ktl.fi
Falsification
• Karl Popper (1902-1994)
• Science consists of statements
(theories) that can be falsified
• Science is an evolutionary
process where poor theories are
falsified
• The current knowledge consists of
those theories that have not (yet)
been falsified
40
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Bayes' theorem
www.ktl.fi
• Thomas Bayes (1702-1761)
• A posterior probability given new data can be
calculated from a prior and the likelihood of the data
41
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Decision theory
www.ktl.fi
• Howard Raiffa
• Decision analysis is a rational
method for making decisions. In
addition, the use of subjective
(Bayesian) probabilities in
decision analysis should be
promoted.
42
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Quality of an estimate
www.ktl.fi
• Roger Cooke
• The quality of a quantitative
estimate (probability distribution)
can be evaluated against a golden
standard using informativeness
and calibration
43
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Vines in Bayesian belief network
www.ktl.fi
• Roger Cooke
• BBNs describe the reality by using
conditional probabilities
• These probability distributions can
have any form and they can still
be solved analytically, if vines are
used
44
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Argumentation
www.ktl.fi
• Frans van Eemeren
• Disputes can be solved by
using formal argumentation that
consists of attacks and defends
of specified statements
45
http://heande.pyrkilo.fi
National Public Health Institute, Finland
PSSP
www.ktl.fi
• Veikko Pohjola
• A system can effectively be
described using two kinds of
objects: processes and
products that are produced by
these processes. Each object
has attributes purpose,
structure, state, and
performance.
46
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Wisdom of crowds
www.ktl.fi
• James Surowiecki
• A group of people is likely to
outperform an individual expert, if
they can use individual knowledge,
act independently and in a
decentralized way, and their
opinions are effectively aggregated
47
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Web encyclopedias
www.ktl.fi
• Jimbo Wales
• Encyclopedia that anyone can
edit: It is possible to motivate a
very large group in collecting
information and write articles
about important issues.
48
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Mass collaboration
www.ktl.fi
• Don Tapscott, Anthony Williams
• A large group of unorganised
people are able to produce
complex artefacts, if the product is
information or culture, the work
can be chopped into bite-size
pieces, and the pieces can be
effectively synthesised.
49
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Summary
• The purpose of open assessment is to improve
and make more efficient the use of scientific
information
• The basic principles are
www.ktl.fi
– Open participation at all phases of the assessment
– Requirement of scientific method (openness to
scientific critique) at all phases
– Reusability of information from one assessment to
another
• Researchers are needed to test the ideas of open
assessment
• If successful, scientific practices may change
50
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Practical examples and links
• The open website for open assessments:
http://heande.pyrkilo.fi
– Case studies (see Category:Risk assessments):
• impacts of bus engine type on health
• Farmed salmon: pollutants vs. fish oils
• Municipal solid waste incinerator
– Workshop material (February 2008)
• Heande: Kuopio Risk Assessment Workshop 2008
• This lecture and much more
www.ktl.fi
– To access non-published material:
51
• Create a user account
• Fill in the form on “Information about page protection”
• Your username will be added to the group of insiders
http://heande.pyrkilo.fi
National Public Health Institute, Finland
Links
• Risk assessment workshop
• http://heande.pyrkilo.fi/heande/index.php/Kuopio_Risk_Asses
sment_Workshop_2008
• Gasbus case on PM and bus engines
• http://heande.pyrkilo.fi/heande/index.php/Gasbus__health_impacts_of_Helsinki_bus_traffic
• Farmed salmon csae
www.ktl.fi
• http://heande.pyrkilo.fi/heande/index.php/Benefitrisk_assessment_on_farmed_salmon
• Guidebook (not openly available)
• http://heande.pyrkilo.fi/heande/index.php/Guidebook
52