Transcript Controls Systems Upgrade Strategy

SLAC National Accelerator Laboratory
Linac Controls Upgrade Status
Tom Himel
SLAC
November 11, 2008
Nov 11 2008
Linac Controls Upgrade Status
1
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Outline
Introduction
Phasing
New architecture
Long lead items status
Planning
Switchover
Summary
Nov 11 2008
Linac Controls Upgrade Status
2
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
What a difference one letter makes
I started managing the linac controls
upgrade a couple of months ago.
Last Friday morning, John Galayda told me I
would no longer be a reviewer on the FAC.
Instead I would be a reviewee.
What a difference a single letter makes
Nov 11 2008
Linac Controls Upgrade Status
3
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Introduction
Linac (and DR and PEP and ESA) use
control system built in early 80’s for the SLC.
PEP-II used EPICS for new functionality with
some connections between SLC and EPICS
LCLS used EPICS for all new devices.
Provided connections so SLC high level
apps could be used for new EPICS devices
Nov 11 2008
Linac Controls Upgrade Status
4
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Introduction (2 of 2)
Through retirements and layoffs, have lost most
people who know how to maintain the SLC control
system.
Have decided to upgrade the SLC control system
to EPICS with a series of AIP projects.
This will reduce chances of long downtimes due to some
problem where expertise is lacking causing a long
MTTR.
Hardware is old. Upgrade is overdue.
SLC control system has many years of applications
being added and fine tuned. Main difficulty of the
upgrade is to not lose too much this functionality.
Nov 11 2008
Linac Controls Upgrade Status
5
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
SLC control system architecture
Nov 11 2008
Linac Controls Upgrade Status
6
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Phases
Phase 1 makes LCLS independent of the Alpha and
multibus micros
Replaces about 15 micros with VME crates, CPUs, and CAMAC
interface.
1 micro in each of last 10 sectors, few in BSY, few system micros (e.g.
MPG)
Main work is software
Phase II replaces the CAMAC with more modern hardware
Main expense is hardware
Phase III-N propagates above upgrades to other regions of
the accelerator complex
Little planning here yet
There are different types of functionality and hardware modules in
other regions so not just simple replication
Nov 11 2008
Linac Controls Upgrade Status
7
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
SLC control system architecture
Phase I
Phase II
Nov 11 2008
Linac Controls Upgrade Status
8
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Phase I Hardware
Replace 15 micros with VME crates/CPUs/EVRs
Develop VME to SLAC serial CAMAC interface
Run Ethernet to new VME crates
Replace few small systems with new hardware
instead of developing new software.
Expect that BPMs will be upgraded
All quite straight forward
Nov 11 2008
Linac Controls Upgrade Status
9
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Phase I software
Replace >1,000,000 lines of software on Alpha
Replace >100,000 lines of software on micros
Not as bad is it appears
Some functionality no longer used at all (e.g. lattice
diagnostics)
Some functionality not used by LCLS (PEP-II
applications)
Much already exists in EPICS (e. g. data archiving and
configurations and distributed real-time database)
Some are high level applications being developed for
LCLS by the HLA group
It is still the major headache of phase I.
Nov 11 2008
Linac Controls Upgrade Status
10
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Long lead items started
VME CAMAC interface prototype working
Can read the position of a switch with a CAMAC module
and display it on an EPICS display. Used:
New VME to CAMAC hardware interface designed by
Jeff Olsen.
New CAMAC software drivers written by Bob Sass.
New test software written by Sheng Peng.
Prelimary design for simple CAMAC module
support (SAM, IDIM, LDIM) done
Design for RF support which has most comple
module (PIOP) started a month ago.
Have purchased micro TCA hardware and about to
start phase II RF hardware design effort
Nov 11 2008
Linac Controls Upgrade Status
11
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Planning
A great deal of effort has gone into documenting
existing functionality and determining which parts
need to be implemented in EPICS.
Biggest schedule uncertainty is amount of software
that must be implemented before we can switch
over.
Making list and will get agreement (signatures)
with users.
There are serious, tough decisions to make.
Need to agree on some functionality that will NEVER be
needed and others that can be delayed until after switchover.
Nov 11 2008
Linac Controls Upgrade Status
12
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
High Level Applications
Most HLAs are being written by Patrick
Krejcik’s group and are not officially part of
the upgrade.
They are essential for a successful upgrade
and also determine the architecture for HLAs
and mid level applications specific to the
upgrade.
I’m working closely with Patrick’s group.
Nov 11 2008
Linac Controls Upgrade Status
13
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Schedule
There was a very aggressive schedule proposed early in this
project.
At an earlier review, reviewers were concerned that 9 FTE-years was
a significant underestimate of the software work.
Also concerned that that project start will be slow because manpower
will continue to be used to make LCLS controls work well.
Plan to take considerably more time. Maybe 2 years to
complete phase 1 and finish the R&D for phase 2.
Still working on more detailed schedule. Concentrating on
documenting scope of the project.
Controls is training people to keep the SLC control system
running
Will be risk of significant unscheduled downtimes due to obscure
problems where expertise is lacking.
Nov 11 2008
Linac Controls Upgrade Status
14
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Switchover
Thorough testing will be done in sectors 17
and 18 (not used by LCLS).
Via a PV gateway and aliases HLAs will
support devices both before and after
switch-over.
Can switch a sector at a time instead of
everything at once.
Will need some scheduled LCLS downtime
to test and implement the switchover.
Nov 11 2008
Linac Controls Upgrade Status
15
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Planning – task list
Short description
SAM driver
SAM fast read or read on request
PDU driver
regional beam codes
timing control
BPM readout
BPM private calibrations
replace BPM electronics
BPM display + orbit fit
Correlation plots
Crate verifier test and display
Network display
CAMAC crate location
CamDmp
Nov 11 2008
Linac Controls Upgrade Status
?
16
Project
Design Himel
ideas priority
systemtype
stat Low
1.1
stat Low
3.0
time Low
1.1
time Low+Mid
3.0
time Low+Mid
1.2
bpm Low+Mid
2.5
bpm Low+Mid
3.0
bpm Elec+Low
1.2
HLA HLA
1.3
HLA HLA
1.3
sys Low+Mid
1.5
sys Low
1.2
sys Low
1.4
sys Low
1.2
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Worries
1.
2.
These are mostly not under my control so lab
management needs to be aware of them.
As implementation of phase I will take longer than
originally planned and as expertise in the SLC control
system is needed until the rest of the linac, damping
rings, NLCTA, the gun lab, and the test lab are
converted, people experienced with the SLC control
system must be kept or new ones trained.
If FACET proceeds on its schedule of completion in Oct
2009, it clearly will not use the new control system and
that much more expertise in the SLC control system will
need to be maintained. If FACET proceeds on a slower
schedule, management will need to decide soon
whether a controls upgrade is done as part of the
project, or with operating or AIP funds before the
project, or if it will use the SLC control system.
Nov 11 2008
Linac Controls Upgrade Status
17
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Worries
3.
4.
This is a large, important project. Even so, many LCLS
needs discovered during the commissioning process will
be higher priority. It is better to do this project right and, if
necessary, slowly, than to rush it to meet a schedule
deadline and lose too much functionality when it is turned
on.
There are 2 steering feedback in the region being
converted. If they need to run at 120 Hz, then the EPICS
version of 120 Hz feedback must be done before the
switchover. (Planned for Jan 2010.)
Nov 11 2008
Linac Controls Upgrade Status
18
Tom Himel
[email protected]
SLAC National Accelerator Laboratory
Summary
Linac upgrade is needed
Can survive while we work on the upgrade
Intense work on the upgrade is awaiting
personnel being freed from LCLS project
completion and commissioning
Plans are progressing but still need detailed
schedule which is in progress.
Nov 11 2008
Linac Controls Upgrade Status
19
Tom Himel
[email protected]