Transcript Grids - ENEA

Oxford University
e-Science Centre
e-Science, the Grid
and…
will they change research?!
Prof. Paul Jeffreys
Director Oxford e-Science Centre
http://e-science.ox.ac.uk/
Professorial Fellow, Keble College
[email protected]
1
Building the Information Society in Europe
Introduction
Oxford University
e-Science Centre
• There is an activity which:–
–
–
–
–
Tony Blair (and many other leaders!) has (have) enthused about
The UK Office of Science and Technology has invested £0.25b
The investment of public funds is estimated to be at least € 2b
Has resulted in world-leading new research
Addresses issues in the Lambert Review of Business-University Collaboration
• (http://www.hm-treasury.gov.uk/consultations_and_legislation
/lambert/consult_lambert_index.cfm)
– and .. if you believe the previous Director General of the Research Councils..
• “will change the dynamic of the way science is undertaken"
2
Building the Information Society in Europe
Talk Outline
Oxford University
e-Science Centre
• Preliminaries and definitions
– Example
• UK e-Science and the Grid
– International developments
• Overview of e-Science in Oxford
• Future vision for e-Research
– Change way research is done
– Component of Information Society
3
Building the Information Society in Europe
Oxford University
e-Science Centre
Preliminaries
4
Building the Information Society in Europe
Front page FT, 7th Mar 2000
Oxford University
e-Science Centre
“‘The Grid’, as it is provisionally
known, will work far more quickly and
reliably than today’s internet. It should
eventually enable computer users to
receive exactly the information they
want from anywhere in the world within
seconds – and without having to go
through a tortuous search process.”
5
Building the Information Society in Europe
Blair’s speech on British Science
Oxford University
e-Science Centre
•
http://politics.guardian.co.uk/speeches/story/0,11126,721029,00.html
“It's significant that the UK is the first country to develop a national e-Science
grid, which intends to make access to computing power, scientific data
repositories and experimental facilities as easy as the web makes access to
information. One of the pilot e-science projects is to develop a digital
mammographic archive, together with an intelligent medical decision
support system for breast cancer diagnosis and treatment. An individual
hospital will not have supercomputing facilities, but through the grid it could
buy the time it needs.”
PM Tony Blair, July 2002
6
Building the Information Society in Europe
What is the Grid?
Oxford University
e-Science Centre
• “… a software infrastructure that enables
flexible, secure, coordinated resource sharing
among dynamic collections of individuals,
institutions and resources”
[The Grid, eds. Foster & Kesselman]
• “an emergent infrastructure capable of
delivering dependable, pervasive and uniform
access to a set of globally distributed,
dynamic and heterogeneous resources. It
brings challenges of scalability,
interoperability, fault tolerance, resource
management and security”
[Tony Hey]
7
Building the Information Society in Europe
e-Science
Oxford University
e-Science Centre
• John Taylor, previous Director General of the Research Councils, OST
– is about research increasingly done through distributed global
collaborations enabled by the Internet (e.g. human genome
program, LHC/CERN)
– uses very large data collections, terascale computing resources,
high performance visualisation
– and col-laboratories – support for trusting teams
e-Science will change the dynamics of how research is done
8
Building the Information Society in Europe
A Definition of e-Research
Oxford University
e-Science Centre
‘e-Research is about global collaboration in key research
areas, and the next generation of infrastructure that will
enable it.’
9
Building the Information Society in Europe
Oxford University
e-Science Centre
“Behind the Wall”, “In Front of the Wall”,
“Through the Wall”
“Behind the wall”
“In front of the wall”
Information
Utility
- resources
Users
- people
- devices
- compute
- data
- comms
“Through the Wall” Col-laboration & interaction between people
10
Building the Information Society in Europe
Oxford University
e-Science Centre
“Behind The Wall”: today
- many “bits of walls”, ad hoc Client-Server
HPC
Experiment
Analysis
Storage
HPC
Scientist
Experiment
Computing
Storage
Analysis
HPC
11
Building the Information Society in Europe
Oxford University
e-Science Centre
“Behind The Wall”: next generation
-Information Utilities and col-laboratories
Scientist
Scientist
Scientist
G
R
I
D
M
I
D
L
E
W
A
R
E
Experiment
Analysis
Computing
Storage
Storage
Analysis
Experiment
Computing
Storage
Computing
12
Building the Information Society in Europe
Oxford University
e-Science Centre
An example to catch the imagination
13
Building the Information Society in Europe
14
Breast cancer
facts
•
•
•
•
•
•
•
•
•
•
10% of Western women develop breast cancer
19% cancer deaths, 24% cancer cases
500,000 cases annually in EC and USA
early diagnosis massively improves prognosis
– screening programs, eg in UK 3 million mammograms per year
– 55 million mammograms per year world wide
20% cancers are missed by radiologists at screening
70-80% biopsies turn out to be benign
30% inter- and intra-radiologist variability
22% of films are “lost” between visits
5% of images need to be re-taken
a 1cm tumor has typically been in the body for 6-8 years
15
UK Breast
Screening –
Today
Paper
Began in 1988
Film
Women 50-64
Screened
Every 3 Years
1 View/Breast
1.5M - Screened in 2001-02
65,000 - Recalled for Assessment
8,545 – Cancers detected
300 - Lives per year Saved
Scotland,
Wales,
Northern Ireland
England
(8 Regions)
92 Breast
Screening
Centres
230 – Radiologists “Double Reading”
Statistics from NHS Cancer Screening web site
Each centre 16
sees
5K-20K images/yr
UK Breast
Screening –
Challenges
Digital
Digital
2,000,000 - Screened every Year
120,000 - Recalled for Assessment
10,000 - Cancers
1,250 - Lives Saved
Women 50-70
Screened
Every 3 Years
2 Views/Breast
+ Demographic
Increase
Scotland,
Wales,
Northern Ireland
England
(8 Regions)
92 Breast
Screening
Programmes
230 - Radiologists “double Reading”
50% - Workload Increase
Up to 50K/yr
per
17
centre
eDiamond aims
• construct a federated database of mammograms
• contribute to Grid middleware development
• contribute to HealthGrid development in UK, Europe
• aims to support the UK Breast Screening Program
Novel image analysis, federation of large data sets
owned by hospitals, and levels of access to that data
18
end-user project goals
• Teaching tool for radiologists, radiographers
 St George’s Hospital
• Tele-diagnosis
 Edinburgh Breast Screening Unit, W. of Scotland
• Algorithm development: data mining
 Oxford Radcliffe Breast Care Unit
• Epidemiology
 Guy’s Hospital, London
• Quality control
 Oxford Medical Vision Laboratory
Clinicians want to use the Grid & they profoundly wish
19
to remain ignorant about how it works
For several years, I had wanted to find a way to gain the
statistical power I needed for
medical
image
Why
is the
Gridanalysis –
the Grid offers the potential to provide it!
needed?
And, not just for medical image analysis …
• mammograms are typical of medical images
many parameters (potentially) of interest
relatively few images gathered at each individual centre
• insufficient statistical power in the database garnered
from a small number of centres
• The Grid provides the statistical power at acceptable
bandwidth and with guarantees on secure image/data
transmission
20
The Grid is for all of
scholarship
• Specialised image corpora & knowledge are widely
dispersed through the world
• The humanities have much to teach science about
curation of large datasets, ontology development, and
development of metadata
21
Oxford University
e-Science Centre
UK e-Science Programme &
International Developments
22
Building the Information Society in Europe
SR2000 e-Science Allocation
Oxford University
e-Science Centre
DG Research Councils
E-Science
Steering Committee
Director’s
Awareness and Co-ordination Role
Grid TAG
Director
Director’s
Management Role
Generic Challenges
Academic Application Support
EPSRC (£15m), DTI (£15m)
Programme
Research Councils (£74m), DTI (£5m)
PPARC (£26m)
BBSRC (£8m)
MRC (£8m)
NERC (£7m)
£80m Collaborative projects
ESRC (£3m)
EPSRC (£17m)
CLRC (£5m)
Industrial Collaboration (£40m)
23
Building the Information Society in Europe
SR2000+SR2002 e-Science Funding
Oxford University
e-Science Centre
Total for e-Science from Spending Reviews
£M
MRC
BBSRC
NERC
EPSRC
Of which:HPC
Core Prog
PPARC
ESRC
CCLRC
TOTAL
2001/2
1.0
1.0
1.0
6.0
2002/3
2.0
2.0
2.0
13.0
2003/4
5.0
5.0
4.0
22.0
2004/5
6.9
5.0
4.0
17.2
2005/6
6.2
5.0
4.0
19.5
TOTAL
21.1
18.0
15.0
77.7
0.0
3.0
3.0
0.0
1.0
13.0
3.0
6.0
8.0
1.0
1.5
29.5
6.0
6.0
15.0
2.0
2.5
55.5
0.0
8.2
16.4
5.5
2.5
57.5
2.5
8.0
15.2
5.1
2.5
57.5
11.5
31.2
57.6
13.6
10.0
213.0
24
Building the Information Society in Europe
UK e-Science Grid
Oxford University
e-Science Centre
Edinburgh
Glasgow
DL
Belfast
Newcastle
Manchester
Cambridge
Oxford
Cardiff
RAL
London
Hinxton
Southampton
25
Building the Information Society in Europe
e-Science Centres of Excellence
Oxford University
e-Science Centre
•
•
•
•
•
•
•
Birmingham/Warwick – Modelling
Bristol – Media
UCL – Networking
White Rose Grid – Leeds, York, Sheffield
Lancaster – Social Science
Leicester – Astronomy
Reading - Environment
26
Building the Information Society in Europe
UK e-Science Grid – phase 2
Oxford University
e-Science Centre
Edinburgh
Glasgow
DL
Belfast
Newcastle
Manchester
Cambridge
Oxford
Cardiff
RL
London
Hinxton
Soton
27
Building the Information Society in Europe
UK e-Science Timeframes
Oxford University
e-Science Centre
SR2000
2001 2002 2003 2004 2005 2006 2007
*
*
*
SR2002
*
*
SR2004
*
*
SJ5/AAA Service
LHC/LCG
*
*
*
*
*
28
Building the Information Society in Europe
Particle Physics Grid
Oxford University
e-Science Centre
• Growing links with Particle Physics Grid
– Crucially important for next generation of experiments at CERN
– Huge investment (UK £100+m in capital equipment alone)
– Oxford - integral part of ‘Southern Tier 2’ in UK particle physics Grid
• Important factor in development towards national ‘persistent Grid’
– EGEE instrumental
• International compatibility
– PP Grid has to be connected internationally!
29
Building the Information Society in Europe
Recent International Developments
Oxford University
e-Science Centre
• Enterprise Grid Alliance:
– “Leading technology companies today launched the Enterprise Grid Alliance
(EGA), a consortium formed to develop enterprise grid solutions and accelerate
the deployment of grid computing in enterprises.
– http://xml.coverpages.org/ni2004-04-20-a.html
– The EGA consortium has been formed to "encourage and accelerate movement
to an open grid environment through interoperability solutions."
– Companies having representatives on the EGA Board of Directors include EMC,
Fujitsu-Siemens, HP, Intel, NEC, Network Appliance, Oracle, and Sun.”
• “Microsoft, IBM, and BEA Systems have released a trio of proposed Web
Services standards to address several unmet requirements to realise the
promises of the services-oriented application model.”
– Will underpin “Grid Services”
30
Building the Information Society in Europe
Information-based society..
Oxford University
e-Science Centre
• e-Research and the Grid are contributors to the Information-based society
– e-Research/Grid part of the ‘forces driving change’:• UK Grid
• e-Science projects are stretching the underpinning IT infrastructure
– e-Research/Grid also part of process of migration to Information Society:• On-Demand resources
• Integration (especially of databases)
• Inter-connection or col-laboratories (eg Oxford and Auckland)
– Irving Wladawsky-Berger:
• “We see a world where more integration is needed, better management of
information, and greater flexibility”
– Vision applies directly to e-Research;
• Information Society operating within academic framework…
– NB e-Research is closely coupled to industry as will be demonstrated!!
31
Building the Information Society in Europe
Oxford University
e-Science Centre
Oxford e-Science
32
Building the Information Society in Europe
Oxford e-Science Centre
Oxford University
e-Science Centre
• Summary:
– In 2.5 years - grown to significant activity
– Supports and expands knowledge in, and use of, e-Science/Grid
• e-Science activities in at least 15 departments
– Portfolio of exciting research projects
– Strong contribution to UK e-Science Core Programme
• Creating persistent, robust and reliable national infrastructure
• Part of UK national Grid (one of 4 nodes)
– £20m flowed into and through University
– Offers e-Science support for region
– Close relationships with IBM and CCLRC (UK national laboratory)
33
Building the Information Society in Europe
OeSC ‘Objectives’
Oxford University
e-Science Centre
• Establish Oxford as regional centre on UK national Grid
– Thereby establish Grid connections for our researchers
– Make our resources available on the Grid
• Support groups throughout University undertaking national and international
e-Science projects (and other Grid activities), and link with companies
– Provide support infrastructure:- registration, certificate authorisation, training,
documentation, security, services
– Share development, coordinate and optimise across projects
– Disseminate
• Commission ‘intranet Grid’
– Share resources across university
– 3000 cpus !
34
Building the Information Society in Europe
Collaborating OU Departments
Oxford University
e-Science Centre
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Biochemistry
Atmospheric, Oceanic and Planetary Physics
Engineering
Materials
Wellcome Trust Centre for Human Genetics
Zoology
Physics
Oxford Internet Institute
Said Business School
Begbroke Business Park
University Library Services
Clinical Trials Unit
Pharmacology and NTRAC
Departments in Humanities
35
Building the Information Society in Europe
2 Crucial e-Science Components
Oxford University
e-Science Centre
• Software Engineering Programme Team
http://www.softeng.ox.ac.uk/
– Essential contribution to OeSC
– Contribute five academic staff, plus a number of dedicated researchers, to the eScience team
– Expertise in design, requirements, and security
• Doctoral Training Centre
http://www.stats.ox.ac.uk/~mcvean/DTC.htm
– Providing training in general research and communication skills is crucial to the
future development of interdisciplinary research
– Opportunity to share much of the training needed for e-Science
36
Building the Information Society in Europe
IBM
Southampton
Birkbeck, LRC,
Birmingham, Nottingham,
York
Cambridge
NTRAC
UCL
Univ. Wales
Manchester
(Singapore)
Others..
CLRC
BioSimGrid
All Universities
in UK PP
Integrative Biology
Grid PP
Tier 2
e-DiaMoND +
Resource Man.
2 Globus Gatekeepers
- Linux Cluster (Condor)
- Supercomputer
CLRC
EDG
Access Grid nodes
JISC testbed cluster
Network Monitoring
DCOCE
Remote Microscopy
Climate Prediction
L2G/ETF/STF/TAG/GOC/ATF
JISC
Collaborative Visualisation
MIAS-Grid
National Cosmos Grid + Rem. Vis.
Geodise
Reality Grid
Oxford e-Science Centre
Dynamic brain Atlas
Dame
St. Georges,
Guy's,
Churchill, St.
Thomas' NHS
Trust Hospitals;
Breast
Screening
Centres in
Edinburgh,
Glasgow,
Aberdeen Univ.
IBM, Virage, Boxer
System Ltd, Square
Box Systems, Int DO
CLRC
Oxford Brookes
http://e-science.ox.ac.uk
MIMAS,
Eduserve
IBM, Mirada
Video Works
OeSC
Security + Data Man. EDG
CERN
IBM
NCRI
Tissue Bank
High Throughput
Structural Biology
CERN
CLRC
Nottingham
Leeds
UCL
Birmingham
Auckland
JEOL
CLRC
Open University
Strategic Partnership with IBM
Oxford University
e-Science Centre
• Relationship built over many years – now reached new levels
• Strategic alliance in e-Science with emphasis on Life Sciences
• Partnership framework signed on 21 January by VC and Director of Hursley
– “A partnership between IBM and the University of Oxford will create a
framework for recognising, consolidating, and sustaining the collaboration that
already exists. It will take advantage of emerging opportunities for
collaboration across the disciplines, and promote the exchange of ideas,
resources, and talent between the two organisations.”
• University:
– academic research and scientific vision
• IBM:
– expertise in industrial research and development
– options for deployment and exploitation
• Built on excellent collaboration forged in e-DiaMoND
38
Building the Information Society in Europe
Oxford University
e-Science Centre
e-Science/e-Research Vision
39
Building the Information Society in Europe
e-Research – a new paradigm
Oxford University
e-Science Centre
• The invention and exploitation of advanced IT
– to generate, curate and analyse research data
• From experiments, observations and simulations
• Quality management, preservation and reliable evidence
– to develop and explore models and simulations
• Computation and data at extreme scales
• Trustworthy, economic, timely and relevant results
– to enable dynamic distributed virtual organisations
• Facilitating collaboration with information and resource sharing
• Security, reliability, accountability, manageability and agility
• Training and teaching crucially important
40
Building the Information Society in Europe
e-Research
Oxford University
e-Science Centre
• Developing e-Research
– Represents a new academic paradigm
– Requires a combination of expertise and resources
– Facilitates world leading research, new opportunities for deployment, exciting
partnerships
• In Oxford – driven as application-led e-Research
– Embracing computer science and computer services
– Includes Humanities
• Blended within University: OeSC, OSC, SEP and DTC
• Set of skills completed - through partnership with IBM
– Expertise and resources for the realisation and deployment of designs, on a
national, industrial scale
• e-Research -- an approach which goes beyond existing University structures
and discipline boundaries
41
Building the Information Society in Europe
Research – has it changed?
Oxford University
e-Science Centre
• e-Science has already changed research in Universities
– e-DiaMoND, Integrative Biology, …
– New capabilities to form col-laboratories (eg Oxford-Auckland)
• But .. e-Science is a path to development of interdisciplinary research
– Very exciting opportunities
• Vision for future:– e-Science/e-Research acting as a catalyst for interdisciplinary advancement underpinned by a new IT infrastructure - facilitating new kinds of research
• e-Research recognised as an academic pursuit (not just infrastructure)
– Component of the new Information Society
42
Building the Information Society in Europe
Conclusions
Oxford University
e-Science Centre
• e-Science activity has grown rapidly
• e-Research and Grid will continue to grow in importance
• Flagship projects..
– e-DiaMoND and many others
• … demonstrate that research has already changed
• e-Research is a new paradigm which enriches academia and changes
research
– catalyst for interdisciplinary activities offering new possibilities
• Relationship with IBM strategic for Oxford University
• e-Research component of the new Information Society
43
Building the Information Society in Europe