Transcript 独立行政法人の 独立行政法人による

A Novel Evaluation System of
Public Research Institute
in Japan
International Workshop on the Evaluation of
Publicly Funded Research
Berlin, 26-27 September 2005
Naoto Kobayashi
National Institute of Advanced Industrial Science and Technology (AIST)
CONTENTS
1.Missions and Principles of AIST
2.Evaluation Policy in AIST
3.Novel Evaluation System from the
Viewpoint of Outcome
4.Evaluation in the Future for the
Publicly Funded Research Institutes
Missions of AIST
in association with the Ministry of Economy,
Trade and Industry (METI)
Missions of AIST: for all mankind and global consciousness to our future
The mission of AIST is to generate innovative technologies for the new
industries and to contribute to the economic development of our country and to
the lives of people and , thus, finally to the sustainable global world!!
Interdisciplinary Researches
to enable to plan long-range
Environment
governmental policies
attributing to the current
Energy
and future needs of society
Life Science
Resources
Type-II Basic Research
& Full Research
Information
Communication
Electronics
Manufacturing
Standard & Measurements
Machine
Nanotechnology
Oceanography
Material
Geology
Advanced Researches
to explore broad spectra of
research fields and to integrate
multidisciplinary subjects
Basic Researches
to develop and maintain high standards of scientific and engineering research
3
President
Researchers: ~2,500
Administrative staff: ~720
Budget: ~900M$
Senior Vice-President
Vice-President
Auditor
Advisory Board
Auditors’ Office
Planning
Headquarter
General
Administration
Headquarter
Evaluation Dept.
Safety &
Environmental
Protection Dept.
Fellow
Research
Coordinator
Emeritus
Advisor
Emeritus
Councilor
Public Relations Dept.
Research support
departments
Research Institute (21)
Technology Information Dept.
Research Initiative (5)
Tsukuba Advanced Computing Center
International Patent Organism Depository
AIST Innovation Center for Start-ups
Geo-information Center
Metrology Management Center
Collaboration Dept.
Intellectual Property Dept.
AIST Innovations
(TLO)
International Affairs Dept.
Research administration
departments
〔Sites〕
AIST Hokkaido
AIST Tohoku
AIST Tsukuba
AIST Tokyo
AIST Chubu
AIST Kansai
AIST Chugoku
AIST Shikoku
AIST Kyushu
Research Center (28)
General Administration Dept.
Human Resource Dept.
Financial Affairs Dept.
Research Facilities Dept.
June, 2005
4
産総研
Research Bases of AIST
old AIST
AIST Hokkaido
15 Research Institutes
8 Institutes in Tsukuba
7 Institutes in other regions
In 2001
new AIST
AIST Kansai
Merged into 1 Institute
9 Research Basis
AIST Chugoku
AIST Tohoku
AIST Tsukuba
AIST Tokyo Waterfront
Tokyo Headquarters
AIST Kyushu AIST Shikoku
AIST Chubu
5
Concept of Full Research in AIST
AIST defines Type-II Basic Research, which integrates multiple disciplines and creates
methods for the use of integrated knowledge; traditional basic research is defined as
Type-I Basic Research, which is the pursuit and discovery of novel laws, principles
and theories that govern natural phenomena. AIST places its highest priority on the
pursuit of complete research, "Full Research", ranging from Type-I Basic Research to
the development of “products” by conducting intensive Type-II Basic Research.
Contribution to the Society
Application
Development
Finding of
New Knowledge
Integration of
Knowledge
Theory
Principle
Unknown
Phenomena
Economic
& Social
Needs
Theorem
President
Dr. Yoshikawa
Type-I Basic Research
Law
Type-II Basic Research
Coherent and concurrent research based on scenario
Full Research
6
Sustainable Development by Full Research
Outcomes
AIST
Society
Public resources
Innovation!!
People’s
Interest
(applause)
dream
Type-1 basic
research
Universities
nightmare
Type-2 basic
research
reality
Industrial
development
Industries
AIST
Full Research
7
Council for Science and Technology Policy
R&D Evaluation Guideline (1997, 2001, and 2005）：
・appropriate and fair evaluation→ flexible, competitive, and open research environment
・encouraging evaluation → original and promising R&D, researchers with great ambitions and
abilities , and significant policy and program
・announcement of results of evaluation→ accountability to the nation
・strict evaluation→ concentrated and effective allocation of resources
⇒ internationally high level of R&D which contributes to society and economy, and
to development of new discipline
Independent Administrative Institution Act
2001
Introduction of evaluation system in AIST
・setting Evaluation Department
・annual evaluation
2001～2004
AIST Evaluation System Advisory Committee
How to evaluate R&D in AIST
as an Independent Administrative Institution
promoting R&D
・auto-evaluation
・evaluation interval
・outcome
2003～5
Evaluation from the viewpoint
of Outcome
8
Evaluation System of AIST
Medium-Term Plan
Evaluation
Committee for IAI
in METI
Medium-Term Target
Annual Report
President
Report
Evaluation
Department
Evaluation of
Activity
Evaluation of Each Unit’s
Research Resources
(Peers, Specialists or Intellectuals,
Internal Reviewers)
Comment
Evaluation Committees
Reflection on
Future
Management
Comment for
Management
Advisory
Board
Research Units (Center, Institute and Initiative)
9
Significance of Evaluation

To evaluate research activities is to encourage
researchers with great ambitions and abilities and to assist
research organizations which contribute to the development
of economy and the quality of our lives.
 The top management of AIST requires the recognition of
check and balance function in the evaluation of research
activities, to consider the allocation of resources for research
and to be equal to the occasion such as the reorganization.
 AIST is also responsible for the accountability for public
resources by showing performances and achievements to
the society.
10
Novel Evaluation System of Research Units
Purpose: 
Activation and Increasing in Efficiency
 Making sure the research strategy
 Reference to the Managements
The 1st Term (FY2001-2004)
The 2nd Term (FY2005-2009)
Based on Output
Toward Outcome
Every year
Change
Every a few years
Evaluation Intervals
Startup
Research Units
Interim
Summarized
RI and RC
Following
RC
Roadmap toward ・Reconsideration of roadmap
・Output Indexes to outcomes
outcomes
・Management procedure
11
Pattern of Outcome Creation
(Product Type)
Solar Panel
Startups
Companies、New Startups
Production・Marketing
Collaboration/ Technology Transfer
(Prototype・Improvement）
New Industry
New Market
Research in AIST
Liquid Crystal Display
Novel Idea
Output
Sample
Transfer
Market
Market Share
Outcome
Acceptance in Market & Society
12
Pattern of Outcome Creation
(Database Type)
α Version，β Version
Supply
Organic Compounds
Spectrum DB
Extension of
External Access
DB Open to the
Public
System Adjust and Implementation in
Startups and/or Companies
New Industry
New Market
Startups
Collaborative Work
DB
Construction
System Adjust and Implementation in AIST
Acceptance in
Domestic and
International
Society
Research in AIST
System Development
Novel Idea
Output
System Improvement・Data Accumulation
Transfer Sample
Market
Market Share
Outcome
Acceptance in Market & Society
13
Evaluation from the Viewpoint of the Outcome
①： Evaluation of Realized Outcomes
②： Evaluation of Scenario/Roadmap
③： Evaluation of Outputs to contribute to Outcomes
④： Evaluation of Management to contribute to Outcomes
Present Research
Past
Outputs in the Past
Future
① ： Realized Outcomes
② ：Roadmaps
③ ：Outputs
④ ：Management
Expected Outcomes
in the Future
Evaluation is performed through Items ②～ ④
14
Evaluation
from the viewpoint of outcome
to reflect the future
Element of Evaluation to evaluate the process of R&D
Roadmap
Output
Management
15
Procedure of Evaluation
Strategy of AIST
external reviewers:
Specialists or Intellectuals
internal reviewers:
＊
Roadmap Evaluation
Evaluation
Roadmap
Principal Reviewers
Evaluation
＊Management Evaluation
＊Output
＊Roadmap for outcomes
＊Output indexes
＊Management
Research Units
Evaluation: Rating with comments
Evaluation Department
Nation
President of AIST
Evaluation Committee
16
Evaluation of Roadmap
to make sure the research strategy
Evaluation Index
To know
Outcome
Social and economical
effects
Milestone
Progress plan
Technical element
Core technique
Benchmark
Comparison with others
17
Example of Roadmap for Outcomes:
Ultra-Low-Loss power devices
（Power Electronics Research Center）
2002
Subjects
(Targets)
2004
2006
2010
3 inch Wafer
Epitaxy
> Few 10μm/h
4 inch Wafer
6 inch Wafer
Bipolar-Tech.
High-reliable SiO2
T j >250℃
Self-Alignment
R&D in AIST
Low-Temp Process
Wire-less
（ＦＥＴ）
（ＩＧＢＴ）
600V10A
600V100A
Low-EMI
Heatproof
Lamp Ballasts
Applications
2020
Multi-Wafer-Epi.
low-Ron
Packaging
FET Devices
Modules
2012
The 2nd Term
The 1st Term
Substrate / Wafer
Processing
2008
Air-Con.
2.5kV100A
Bipolar Devices
Technology transfer
Capacitor-less
Dispersed
Power supply
General
Inverter
Factory
Motor drive
Power Transmission
System
Instruments
Electric Power Distribution System
18
Procedure of Evaluation
external reviewers:
internal reviewers:
Specialists or Intellectuals
Principal Reviewers
＊Roadmap
Evaluation
＊
OutputEvaluation
Evaluation
Output
＊Management Evaluation
Strategy of AIST
＊Roadmap for outcomes
＊Output indexes
＊Management
Research Units
Evaluation: Rating with comments
Evaluation Department
Nation
President of AIST
Evaluation Committee
19
Evaluation of Output Indexes
to know the level of research unit
Papers
Press releases
Patents
Prizes
Standards
Prototypal products
Activities at academic party
etc.
20
Procedure of Evaluation
external reviewers:
internal reviewers:
Specialists or Intellectuals
Principal Reviewers
＊Roadmap
Evaluation
＊Output Evaluation
Management
＊
ManagementEvaluation
Evaluation
Strategy of AIST
＊Roadmap for outcomes
＊Output indexes
＊Management
Research Units
Evaluation: Rating with comments
Evaluation Department
Nation
President of AIST
Evaluation Committee
21
Evaluation of Management
Concept of Full Research : Policy statement
Leadership
Project formation
Budget
Intellectual property
Human resource
Technology transfer
Collaboration
Start-ups etc.
22
Procedure of Evaluation
external reviewers:
internal reviewers:
Specialists or Intellectuals
Principal Reviewers
＊Roadmap
Evaluation
＊Output Evaluation
＊Management Evaluation
Strategy of AIST
＊Roadmap for outcomes
＊Output indexes
＊Management
Research Units
Evaluation: Rating with comments
Evaluation Department
Nation
Nation
President of AIST
Evaluation
Evaluation Committee
Committee
23
Novel Evaluation System of Research Units
Purpose: 
Activation and Increasing in Efficiency
 Making sure the research strategy
 Reference to the Managements
The 1st Term (FY2001-2004)
The 2nd Term (FY2005-2009)
Based on Output
Toward Outcome
Every year
Change
Every a few Years
Evaluation Intervals
Startup
Research Units
Interim
Summarized
RI and RC
Following
RC
Roadmap toward ・Reconsideration of roadmap
・Output Indexes to outcomes
outcomes
・Management procedure
24
Mid-term Objectives
by the minister (e.g. METI)
Mid-term Plans
by each Institute
3-5 years
Check of Performances & Achievements
Evaluation Committee in each ministry
Evaluation Committee in Ministry of
Internal Affairs and Communications
25
President
Researchers: ~2,500
Administrative staff: ~720
Budget: ~900M$
Senior Vice-President
Vice-President
Auditor
Advisory Board
Auditors’ Office
Planning
Headquarter
General
Administration
Headquarter
Evaluation Dept.
Safety &
Environmental
Protection Dept.
Fellow
Research
Coordinator
Emeritus
Advisor
Emeritus
Councilor
Public Relations Dept.
Research support
departments
Research Institute (21)
Technology Information Dept.
Research Initiative (5)
Tsukuba Advanced Computing Center
International Patent Organism Depository
AIST Innovation Center for Start-ups
Geo-information Center
Metrology Management Center
Collaboration Dept.
Intellectual Property Dept.
AIST Innovations
(TLO)
International Affairs Dept.
Research administration
departments
〔Sites〕
AIST Hokkaido
AIST Tohoku
AIST Tsukuba
AIST Tokyo
AIST Chubu
AIST Kansai
AIST Chugoku
AIST Shikoku
AIST Kyushu
Research Center (28)
General Administration Dept.
Human Resource Dept.
Financial Affairs Dept.
Research Facilities Dept.
June, 2005
26
Evaluation of Research Administration Departments
and Research Support Departments
Aim: Support for research activities of AIST
More effective execution of administrative and supportive work
Improvement in services
・Setting of “Proposal Box” for accepting opinions to improve administrative work
Increase in efficiency
・List-up and sorting of the importance of work
・Re-arrangement of the staffs so as to maximize the total efficiency of the work
Activation of operation
・Quantitative target-setup
・Preparation of “Work Flow Manual”
Strategy of Evaluation
Evaluation of R&D Policy
Strategy
Do
Implementation
Plan
Evaluation
See
Evaluation of R&D in the Research Institute
Action
Plan
Planning
Input
Management
Report
Proposal
Advice
Research
Check
Evaluation
Output
Do
28
What is Innovation?
◆Adequate
Evolution of the Economy, Society
and Environment based on Novel Science &
Technology
The trend to intensify the interface of Science and
Industrial Innovation is prevailing at universities
and public research industries in the world.
Especially it is necessary to make breakthroughs by
technological fusion in the interdisciplinary fields
(ICT, Nanotechnology, Biotechnology etc.).
Conclusion: Evaluation for Innovation
R&D
Industrial Society
Innovation!!
Industrial Transformation is necessary toward the
Sustainable Global Society.
Innovation on the Basis of Novel R&D is necessary
for the Industrial Transformation.
Adequate Evaluation System of R&D which
contributing to the Innovation is important,
especially for the Publicly Funded Research Institutes.
Evaluation from the Viewpoint of the Outcome is one
of the attractive tools for the novel evaluation.
Notes:
•
•
•
•
•
•
•
Evaluation is for Encouraging with Love.
Evaluation is for Communication.
Evaluation is for Strategy.
Evaluation is for Creation.
Evaluation is to be fed back.
Evaluation reflects the Future.
Evaluation is like a Compass for Voyage.
31
Thank you !
32
Appendix
33
Definition
of independent administrative institution
【Performance】
- to be highly public
- need not to be performed by government directly
- hard to be achieved by private organization
independently
① effective and with high quality
② independent
③ open
① To perform effectively
and
qualitatively
- to set mid-term objectives
and to receive periodical evaluation by third
party
- to entrust some service to private organization
or to abolish institutions
- to manage as a private corporation
- to organize minimum size
35
② To perform independently
- to concentrate authority to president
- to exclude the excess involvement of minister
- to have an appropriate personnel
including executives from private corporation
③ To perform openly
- to open all performances and achievements
to the public
36
Evaluation Committee
【outside of AIST】
Specialists
e.g. Professor, or Head of other IAI
Intellectuals
e.g. Head of enterprise, or Journalist
【inside of AIST】
Principal Reviewers
etc.
37
Evaluation Interval
in the 2nd Research Phase
研究センター
Research Center
・産総研の看板
・top down
・時限的
・time limit
Middle Evaluation
Start up
Middle Evaluation
Terminal Evaluation
Advise for
Roadmap
研究部門
Research Institution
・技術ポテンシャルの発展
・bottom up
・no time limit
・継続的
Middle Evaluation
Terminal Evaluation
Advise for
Roadmap
研究ラボ
Research
Initiative
・研究センターの創出
・to be Research Center
・時限的
・time limit
Middle Evaluation
Advise for
Roadmap
A new Research Center
38
History of AIST
1948
1960
1970
1990
1980
2000
Establishment of Former AIST
1903 Weighing & Testing
National Research Laboratory of Metrology
Mechanical Engineering Laboratory
Machinery
Chemical
1900 Tokyo National Research Laboratory Technology
Alcohol
Microorganic Industry
Fermentatio
nTextile
Fiber & Polymer
1882 Geological Survey
1891 Electrical
Mining Resources
2 Regional Institutes
Production
Science
Nat’l Inst.for Advanced
Interdisciplinary Research
Geological Survey of Japan
Electronics
Electrotechnical Laboratory
Resources
& Pollution National Institute for Resources and Environment
7 Regional
Institutes
7 Regional National Industrial Research Institutes
Weights and Measures Training Institute
Concept for Reorganization
Industry Type
Research Areas
Reorganization by Unification
Industrial Arts
National Institute of Materials
and Chemical Research
National Institute of Bioscience
and Human-Technology
Merging research fields
and Administrative needs
National Institute of
Advanced Industrial Science and Technology
～ Organizational shift in accordance with society needs at the times～
Recovery
Aiming at
Aiming at
Catching up
from World
COE
Front Runner
advanced nations
War II
39