Transcript Analysis of the results of the past IRRS missions

Presentation held at the Workshop on Lessons Learned from IRRS Missions
Moscow, Russian Federation
9-11 December 2014
IAEA
International Atomic Energy Agency
Outline
• Basis of presentation
• Analysis of references
• Lessons Learned on the
Regulatory Framework
IAEA
USNRC IRRS Training - Lessons Learned
2
BASIS OF PRESENTATION
IAEA
USNRC IRRS Training - Lessons Learned
3
Missions in 2006-2014
Korea (f)
Niger
UAE
Zimbabwe
Canada (f)
France
Netherlands
Mexico
Germany
Mauritius
Ukraine
Russia
Switzerland
Cameroon
Sierra Leone
UK (part 2)
Australia (f)
Belgium
Cameroon
Kenya
Namibia
Vietnam
Slovenia
Czech Rep.
Vietnam (f)
Uganda
Madagascar
Lebanon
Ukraine (f)
Germany (f)
Finland
Russia (f)
Slovenia (f)
France
Gabon
Botswana
Canada
USA
Korea
Greece
UK (f)
Jordan
UK
Australia
Spain
Peru
China
Spain (f)
Slovakia
Poland
Pakistan
Romania
Japan
Cote d'Ivoire
France (f)
Iran
Romania
Sweden
Bulgaria
USA (f)
2006
2007
2008
2009
2010
2011
2012
2013
2014
3
9
8
7
4
9
4
6
10
IAEA
USNRC IRRS Training - Lessons Learned
4
IAEA Nuclear Safety Action Plan
IAEA is to “Strengthen IAEA peer reviews in order to
maximize the benefits to Member States”.
In specific:
2.4.1 IAEA Secretariat to review the effectiveness of
their peer reviews
To set the basis for this action an analysis of past
IRRS missions has been performed
IAEA
USNRC IRRS Training - Lessons Learned
5
Missions analysed
France
UK
Romania
Niger
Mexico
Mauritius
Cameroon
Kenya
Uganda
Gabon
Australia
Japan
Germany
Ukraine
Sierra Leone
Namibia
Madagascar
Botswana
Spain
Cote d'Ivoire
Russia
UK (part 2)
Vietnam
Lebanon
Canada
Peru
France (f)
2006
2007
2008
2009
Ukraine (f)
USA
China
Iran
UAE
Canada (f)
Switzerland
Australia (f)
Slovenia
Germany (f)
Korea
Spain (f)
Romania
2010
2011
Finland
Greece
Slovakia
Sweden
Belgium
Czech Rep.
Russia (f)
UK (f)
Poland
Bulgaria
USA (f)
2012
2013
2014
• Initial (22) an follow-up (9) missions in 20062014 to countries with nuclear power plants
IAEA
USNRC IRRS Training - Lessons Learned
6
Details of analysis reported
http://gnssn.iaea.org/
regnet/irrs/Pages/IR
RS_pub_docs.aspx
IAEA
USNRC IRRS Training - Lessons Learned
7
IRRS Modules analysed
GENERAL
CORE
REGULATORY
FUNCTIONS
SPECIFIC
IAEA
USNRC IRRS Training - Lessons Learned
8
IAEA Safety Standard Requirements
IAEA
USNRC IRRS Training - Lessons Learned
9
Observations and their types
observations
Observation: focused results of the review process,
based on facts and related to possible ways of
improvement or achievements to recognise, with
reference to requirements in IAEA Safety Standards
IAEA
• Observations lead to:
• Recommendation
• Suggestion
• Good Practice
Findings
USNRC IRRS Training - Lessons Learned
10
Format of an Observation
A description of what the reviewer
observed, that has lead to the present
finding or good practice
Quotation from the relevant IAEA
Safety Standard that is to be / has
been followed.
Recommendation / suggestion / good
practice offered by the reviewer
This BASIS is a Reference to GSR Part 1 Requirement 4
IAEA
USNRC IRRS Training - Lessons Learned
11
ANALYSIS OF REFERENCES
IAEA
USNRC IRRS Training - Lessons Learned
12
References to IAEA safety standards
GP References to GSR Part 1 and elsewhere from Modules
R&S references to GSR Part 1 and elsewhere from Modules
to GSR Part 1
to other standards
100%
100%
90%
90%
80%
80%
70%
70%
Number of references
Number of references
to GSR Part 1
60%
50%
40%
30%
60%
50%
40%
30%
20%
20%
10%
10%
0%
to other standards
0%
1
2
3
IAEA
4
5
6
Module number
7
8
9
10
1
2
3
4
5
6
Module number
7
8
9
10
GSR Part 1 is indeed the
IAEA safety standard that
plays a central role in the
IRRS process, further
basic references are GSR3 and
USNRC IRRS Training
- Lessons
Learned GS-R-2.
13
Frequently referred Requirements
Most frequent references to GSR Part 1
EU missions
• R24: Demonstration of safety
all missions
4.50
5
3
2.41
2.36
3
9
2.09
2.09
1
7
6
1.68
1.64
1.59
7
Number of references per missions
4.00
for the authorization of
facilities and activities in
Module 5;
• R18: Staffing and competence
of the regulatory body in
Module 3;
• R20: Liaison with advisory
bodies and support
organizations in Module 3;
• R32: Regulations and guides in
Module 9.
IAEA
non-EU missions
3.50
3.00
2.50
2.00
1.50
1.45
1.00
0.50
0.00
24
18
20
32
2
27
GSR Part 1 Requirement number
25
29
Findings, referencing GSR
Part 1 Requirements relating
to core regulatory functions
from initial missions comprise
a dominant part of all
references
USNRC IRRS Training - Lessons Learned
14
Frequency of references
Cumulative reference rate of findings to GSR Part 1
from largest
from smallest
1.00
References per missions to GSR Part 1
0.90
EU missions
Average number of references per missions
0.80
0.70
Cumulative rate
non-EU missions
all missions
4.00
0.60
0.50
0.40
0.30
0.20
3.50
3.00
2.50
2.00
1.50
1.00
0.50
0.00
18 24 32 20 2 27 25 29 4 31 33 22 15 26 16 19 36 23 11 1 8 3 30 5 10 28 7 14 17 35 21 34 6 13 9 12
GSR Part 1 Requirement number
0.10
0.00
24 18 20 32 2 27 25 29 4 31 33 15 22 26 16 19 36 23 11 1 8 3 30 5 14 28 7 17 35 10 21 34 6 13 9 12
GSR Part 1 Requirement number
• 10 Requirements (28%) obtain about 60% of the references
• 17 Requirements (~ 47%) have 80%
• The 9 (25%) least referenced Requirements have ~ 5%
IAEA
USNRC IRRS Training - Lessons Learned
15
LESSONS LEARNED ON THE
REGULATORY FRAMEWORK
IAEA
USNRC IRRS Training - Lessons Learned
16
Statistics of observations analysed
Number of observations in all missions
Recommendations
Suggestions
Observations in initial Missions
Good Practices
R
Number of observations
160
Number of observations
140
120
100
80
60
40
20
S
GP
R+S
Ʃ
70
60
50
40
30
20
10
0
0
1
2
3
4
5
6
Modules
7
by Modules
IAEA
8
9
10
A B C D E
F G H
I
J K L M N O P Q R S
Missions
T U V
by missions
USNRC IRRS Training - Lessons Learned
17
Correlation of observations
Correlation of Recommendations and Good Practices
R
30
25
20
15
10
5
0
• compensation by reviewers
• some topics are more frequently
considered than others
O E
S
R
J
G T
F
H U P I N Q
Mission
L
V A M C D K
B
Correlation of Recommendations and Suggestions
R
S
35
3) The reasons for R – S are:
Number of findings
30
• R -> S conversion
• S -> R conversion
or the specificities of the given
mission
IAEA
GP
35
Number of observations
Expectations:
a) R and G are negatively correlated
b) R and S are positively correlated
1) The expected correlations are not
present in all missions
2) The reasons for R – G are:
25
20
15
10
5
0
O
E
S
R
J
G
T
F
USNRC IRRS Training - Lessons Learned
H
U
P I N
Mission
Q
L
V
A M C
D
K
B
18
The Subject Group approach
• In order to analyse the technical contents of the
findings they are classified into Subject Groups (SGs).
• The purpose of this grouping is to collect the
observations having similar characters into statistically
meaningful groups
• E.g.:
SG 1d): Providing/using legal framework for regulatory
activities includes findings related to
• revision of regulations to provide authority;
• reflecting independence of the regulatory body;
• issuance of decrees and orders;
• relieving time constraints on decision making.
IAEA
USNRC IRRS Training - Lessons Learned
19
The most populated SGs
Subject Groups with the most findings
50
Number of findings per missions
45
40
35
30
25
20
15
10
5
0
1.d 7.a 9.b 7.c 3.a 4.a 10.e 3.f 4.b 9.a 3.c 3.g 1.a
Subject Group
Subject Groups with the most GPs
18
Number of Good Practices
16
14
12
10
8
6
4
2
0
3.f
IAEA
3.g
7.d
USNRC IRRS Training - Lessons Learned
1.e
10.b 9.a 6.a
Subject Group
7.a
2.a
3.a
10.d
20
Subject Groups with the most findings
50
Characteristic examples of issues (1)
Number of findings per missions
45
40
35
30
25
20
15
10
5
0
1.d 7.a 9.b 7.c
3.a 4.a 10.e 3.f
4.b 9.a
3.c
Subject Group
The most populated Subject Group, Providing/using
legal framework for regulatory authorities, (Module 1)
includes the following typical recommendations
The government should
• establish the legal framework for an effectively independent,
unambiguously authorized regulatory body with clear
division of responsibilities;
• provide the regulatory body with the authority to issue or the
involvement in issuance of regulatory requirements;
• provide full provisions for appeals
The addressee of these findings is the government and
the issue raised is lack of specific legal provisions
necessary for discharging regulatory responsibilities.
IAEA
USNRC IRRS Training - Lessons Learned
21
3.g 1.a
Subject Groups with the most findings
50
Characteristic examples of issues (2)
Number of findings per missions
45
40
35
30
25
20
15
10
5
0
1.d 7.a 9.b 7.c
3.a 4.a 10.e 3.f
4.b 9.a
3.c
Subject Group
Developing inspection programme (Module 7) is the
second most frequent SG. Typical findings therein are:
The regulatory body should (or should consider to)
• expand the inspection programme (in scope, in types, in
frequency);
• develop further its inspection system (initiation,
methodology, monitoring, evaluation);
• improve inspection planning.
This is a fairly homogeneous group of findings, the
addressee of which is the regulatory body and the main
issue is shortcomings in the inspection methodology
applied.
IAEA
USNRC IRRS Training - Lessons Learned
22
3.g 1.a
Subject Groups with the most findings
50
Characteristic examples of issues (3)
Number of findings per missions
45
40
35
30
25
20
15
10
5
0
1.d 7.a 9.b 7.c
3.a 4.a 10.e 3.f
4.b 9.a
3.c
Subject Group
SG Reviewing regulations and guides (Module 9) is the
third among the most frequent ones with typical findings:
The regulatory body should (or should consider to)
• systematically and periodically review and revise as
necessary the regulations, regulatory requirements and
guidance;
• introduce formalized gap analysis between IAEA
requirements and national regulations.
This group also addresses the regulatory body and points
to weaknesses in review and revision of regulations.
IAEA
USNRC IRRS Training - Lessons Learned
23
3.g 1.a
Subject Groups with the most GPs
18
Characteristic examples of issues (4)
Number of Good Practices
16
14
12
10
8
6
4
2
0
3.f
3.g
7.d
1.e
10.b
9.a
6.a
Subject Group
7.a
2.a
3.a
The most frequent Subject Group of Good Practices is
External involvement in the regulatory process (Module 3)
Typical observations are
• taking advice and assistance from advisory committees,
Technical Support Organizations;
• regular contact with the licensees and manufacturers.
The most frequently commended activity is accepting
technical assistance.
(Note that this activity is fairly common among the
regulatory bodies and as such it would not qualify for a
Good Practice – pointing to a weakness in the process).
IAEA
USNRC IRRS Training - Lessons Learned
24
10.d
Dual issues
• Certain subjects appear both as
findings and as Good Practices:
• Developing the inspection program
(2nd in findings, 8th in GPs)
• Staffing of RB (5th in findings, 10th
in GPs)
• External involvement (1st in GPs, 8th in
findings)
IAEA
1)
2)
3)
4)
5)
6)
7)
8)
9)
10)
USNRC IRRS Training - Lessons Learned
R+S
1.d
7.a
9.b
7.c
3.a
4.a
10.e
3.f
4.b
9.a
GP
3.f
3.g
7.d
1.e
10.b
9.a
6.a
7.a
2.a
3.a
25
Balance values
Balance value of a Subject Group = difference of relative
number of Good Practices and findings in the Subject Group
Balance-values of Subject Groups
7.d
3.f
1.e
3.g
10.b
2.a
6.a
6.h
9.a
2.b
1.g
10.d
6.g
3.h
6.f
5.d
7.f
4.e
5.a
6.b
8.d
7.e
3.a
7.g
8.a
4.d
6.e
3.d
5.c
1.f
6.i
7.a
3.e
5.b
5.e
4.f
7.b
7.h
10.a
10.c
5.f
6.c
6.d
8.c
9.c
4.c
1.b
1.c
3.b
9.e
8.b
9.d
4.a
10.e
9.b
1.a
4.b
3.c
7.c
1.d
6.00
5.00
4.00
3.00
2.00
1.00
0.00
-1.00
-2.00
-3.00
-4.00
-5.00
IAEA
USNRC IRRS Training - Lessons Learned
26
Follow-up missions
• Limited amount of data from 9 missions
• Progress can be characterized by the number of
issues remained open and by the number of new
issues
Progress in follow-ups by Modules
Initial findings
open issues
new findings
60
number of findings
50
40
30
20
10
0
1
IAEA
2
3
4
5
6
7
8
9
10
USNRC IRRS Training - Lessons Learned
27
Issues remained open
• 69 open issues
• The most populated Subject Groups are:
• Details of the MS, developing MS manual (Module 4) – 6 pcs
• Resources of the RB (Module 3) – 5 pcs
Ratio of open and new issues
Ratio of open issues in follow-ups
Recommendations
Suggestions
1.00
All
0.90
number ratio of findings
0.80
open issues/intial issues
0.70
0.60
0.50
0.40
0.30
0.80
0.70
0.60
0.50
0.40
0.30
0.20
0.20
0.10
0.10
0.00
1
0.00
Minimum
IAEA
Average
Maximum
2
3
4
5
Modules
USNRC IRRS Training - Lessons Learned
6
7
8
9
28
THANK YOU FOR YOUR ATTENTION
This activity is conducted by the IAEA, with funding by the
European Union. The views expressed in this presentation do not
necessarily reflect the views of the European Commission
IAEA
USNRC IRRS Training - Lessons Learned
Photo: V. Friedrich
29