Transcript Slide 1

The LHC
Control
System
B. Frammery
For the CERN - AB/CO Group
CERN – AB Department
b. frammery -10.10.2005
1
Content
•
•
•
•
•
•
•
•
•
•
A brief introduction
The hardware infrastructure
Machine timing & sequencing
Data management
Communications
Software frameworks
General services
Critical systems for LHC
The CERN Control Center
A brief status and a few issues
CERN – AB Department
b. frammery -10.10.2005
Presentation or Poster
to this conference
[THx.y-zz]
2
A brief INTRODUCTION
CERN – AB Department
b. frammery -10.10.2005
3
CERN machines
(LEP)
LHC
Until 2003
Since 2003
AB Department
SL Division
PS Division
CERN – AB Department
b. frammery -10.10.2005
4
In 2003 & 2004
for physics
(LEP)
LHC
for R&D
CERN – AB Department
b. frammery -10.10.2005
5
In 2005
(LEP)
LHC
for physics
for R&D
Commissioning for LHC
CERN – AB Department
b. frammery -10.10.2005
6
Strategy
• Develop new software and hardware infrastructures
• For LHC
• To be used & tested on all the new developments
• To be spread over all the CERN accelerators at a later stage
• Integrate industrial solutions as much as possible
Meaning that, meanwhile, the “legacy” controls for LINAC2,
the PSB, the PS and the SPS are to be maintained
CERN – AB Department
b. frammery -10.10.2005
7
Hardware
infrastructure
CERN – AB Department
b. frammery -10.10.2005
8
LHC control hardware infrastructure
• The network : the Technical Network
o
o
o
o
o
Dedicated to accelerators & technical services
No direct connectivity to the outside world
Linked to the office network (the Public Network)
Security strategy to be deployed from 2006 [Oral TU3.4-30]
Gigabit backbone
• A 3-tier structural layout
o
o
o
Resource tier (Front Ends for equipment)
Business tier (servers for general services)
Presentation tier (consoles for GUIs)
CERN – AB Department
b. frammery -10.10.2005
9
The CERN Technical Network
CERN
control
rooms
LHC
Local
Points
Computer
Center
CC
CCC
CERN
Control Center
CERN – AB Department
b. frammery -10.10.2005
10
LHC controls architecture diagram
CERN GIGABIT ETHERNET TECHNICAL NETWORK
PUBLIC
ETHERNET
NETWORK
TCP/IP communication services
TCP/IP communication services
TCP/IP communication services
BEAM POSITION MONITORS,
BEAM LOSS MONITORS,
BEAM INTERLOCKS,
RF SYSTEMS, ETC…
CERN – AB Department
QUENCH PROTECTION AGENTS,
POWER CONVERTERS FUNCTIONS
GENERATORS, …
LHC MACHINE
b. frammery -10.10.2005
ANALOGUE
SIGNAL
SYSTEM
ACTUATORS AND SENSORS
CRYOGENICS, VACUUM, ETC…
LHC MACHINE
11
LHC controls architecture diagram
TCP/IP communication services
TCP/IP communication services
RT/LynxOS
VME Front Ends
350
200
Linux/LynxOS PC
Front Ends
30 FrontcPCIEnds
160
PLCs
TCP/IP communication services
WorldFIP SEGMENT
(1, 2.5 MBits/sec)
CERN GIGABIT ETHERNET TECHNICAL NETWORK
PUBLIC
ETHERNET
NETWORK
BEAM POSITION MONITORS,
BEAM LOSS MONITORS,
BEAM INTERLOCKS,
RF SYSTEMS, ETC…
CERN – AB Department
QUENCH PROTECTION AGENTS,
POWER CONVERTERS FUNCTIONS
GENERATORS, …
LHC MACHINE
b. frammery -10.10.2005
ANALOGUE
SIGNAL
SYSTEM
FIP/IO
PROFIBUS
OPTICAL
FIBERS
All the front-end equipment is located in surface buildings in
non-radioactive areas (ease of maintenance)
ACTUATORS AND SENSORS
CRYOGENICS, VACUUM, ETC…
LHC MACHINE
12
LHC controls architecture diagram
CERN GIGABIT ETHERNET TECHNICAL NETWORK
PUBLIC
ETHERNET
NETWORK
TCP/IP communication services
Linux/HP ProLiant
APPLICATION SERVERS
FILE SERVERS
PVSS/Linux
SCADA SERVERS
TCP/IP communication services
TIMING GENERATION
RT/LynxOS
VME Front Ends
Linux/LynxOS PC
Front Ends
cPCI
Front Ends
T T T T
PLCs
T
BEAM POSITION MONITORS,
BEAM LOSS MONITORS,
BEAM INTERLOCKS,
RF SYSTEMS, ETC…
CERN – AB Department
T
QUENCH PROTECTION AGENTS,
POWER CONVERTERS FUNCTIONS
GENERATORS, …
LHC MACHINE
b. frammery -10.10.2005
T
ANALOGUE
SIGNAL
SYSTEM
FIP/IO
T
PROFIBUS
T
OPTICAL
FIBERS
T
WorldFIP SEGMENT
(1, 2.5 MBits/sec)
TCP/IP communication services
ACTUATORS AND SENSORS
CRYOGENICS, VACUUM, ETC…
LHC MACHINE
13
LHC controls architecture diagram
FIXED
DISPLAYS
CENTRAL OPERATOR
CONSOLES
CERN GIGABIT ETHERNET TECHNICAL NETWORK
PUBLIC
ETHERNET
NETWORK
LOCAL OPERATOR
CONSOLES
TCP/IP communication services
Linux/HP ProLiant
APPLICATION SERVERS
FILE SERVERS
PVSS /Linux PC
SCADA SERVERS
TCP/IP communication services
TIMING GENERATION
RT/LynxOS
VME Front Ends
Linux/LynxOS PC
Front Ends
cPCI
Front Ends
T
T
T
T
PLCs
T
BEAM POSITION MONITORS,
BEAM LOSS MONITORS,
BEAM INTERLOCKS,
RF SYSTEMS, ETC…
CERN – AB Department
T
QUENCH PROTECTION AGENTS,
POWER CONVERTERS FUNCTIONS
GENERATORS, …
LHC MACHINE
b. frammery -10.10.2005
T
ANALOGUE
SIGNAL
SYSTEM
FIP/IO
T
PROFIBUS
T
OPTICAL
FIBERS
T
WorldFIP SEGMENT
(1, 2.5 MBits/sec)
TCP/IP communication services
ACTUATORS AND SENSORS
CRYOGENICS, VACUUM, ETC…
LHC MACHINE
14
Machine Timing
& sequencing
CERN – AB Department
b. frammery -10.10.2005
15
CERN machines
CERN – AB Department
b. frammery -10.10.2005
16
Timing & Sequencing (2)
21.6 s
CERN – AB Department
b. frammery -10.10.2005
17
Timing & Sequencing (3)
External
events
Basic Period
1200/900/600 ms
40MHz
PLL
GPS
One pulse
per Second
(1 PPS)
Advanced
(100us)
Edited
events
Central
Timing
Generator
Module
1PPS
1 PPS
Smart
clock
PLL
10 MHz
RS485
Timing
Distribution
CERN UTC Time
Timing events
Telegrams
Synchronized
1KHz
Phase looked
40 MHz
Event
encoding clock
Delay
Phase locked
10MHz
Control System
Central Timing
Generation
(25ns
steps)
Timing
Receivers
PSB
PS
SPS
LHC
UTC time
(NTP or GPS)
CERN – AB Department
b. frammery -10.10.2005
18
Data Management
CERN – AB Department
b. frammery -10.10.2005
20
Databases : the 4 domains of data
Serial Number
•Physical equipment
•Use of the general CERN MTF
database for asset management
Physical Equipment
CERN – AB Department
b. frammery -10.10.2005
21
Databases : the 4 domains of data
Installed
Equipt
Type
Machine Layout
Optics
Powering
LHC machine description
•LHC layout (mechanical, optics,
Equipment
Catalogue
Serial Number
1’612 electrical
circuits
80’000 connections
CERN – AB Department
electrical)
•DC magnet powering
Physical Equipment
b. frammery -10.10.2005
22
Databases : the 4 domains of data
Installed
Equipt
Type
Controls Configuration
Machine Layout
Optics
Powering
Computer
Address
Equipment
[MO4A.1-70]
Catalogue
Serial Number
Controls Configuration
Physical Equipment
CERN – AB Department
b. frammery -10.10.2005
•PS Model extended to LHC
23
Databases : the 4 domains of data
Installed
Equipt
Type
Controls Configuration
Machine Layout
Optics
Powering
Computer
Address
Equipment
Catalogue
Serial Number
Operational Data
Physical Equipment
CERN – AB Department
b. frammery -10.10.2005
>200’000
signals
Settings
Measurements
Alarms
Logging
Post-Mortem
24
Databases : the 4 domains of data
Installed
Equipt
Type
Controls Configuration
Machine Layout
Optics
Powering
Computer
Address
Equipment
Catalogue
Serial Number
Operational Data
Physical Equipment
CERN – AB Department
b. frammery -10.10.2005
Settings
Measurements
Alarms
Logging
Post-Mortem
25
Communications
CERN – AB Department
b. frammery -10.10.2005
26
The Controls MiddleWare (CMW)
• Ensemble of protocols, Application Programming Interfaces (API)
and software frameworks for communications.
• Two conceptual models are supported:
o the device access model (using CORBA). Typical use is
between Java applications running in the middle tier and
equipment servers running in the resource tier. Unique API for
both Java and C++.
o the messaging model (using the Java Message Service). Typical
use is within the business tier or between the business tier and
applications running in the presentation tier.
CERN – AB Department
b. frammery -10.10.2005
27
Software
frameworks
CERN – AB Department
b. frammery -10.10.2005
28
The software frameworks (1)
• Front-End Software Architecture (FESA)
[FR1.2-5O]
Complete environment for Real-Time Model-driven control software
implemented in C++ for the LynxOS and Linux platforms
• Java framework for accelerator controls
Uses J2EE application servers with lightweight containers
o Plain Java objects (no EJB beans)
o Applications can run (for test) in a 2-tier setup [TU1.3-5O]
o Unified Java API for Parameter Control (JAPC) to access any kind of
parameter. [TH1.5-8O]
o
CERN – AB Department
b. frammery -10.10.2005
29
The software frameworks (2)
[WE2.2-6I]
• UNified Industrial Control System (UNICOS)
o
o
[WE3A.2-60]
Complete environment for designing, build and programming
industrial based control systems for the LHC.
Supervision layer: PVSS II (SCADA from ETM)
UNICOS & the Java framework for accelerator controls use
the same graphical symbols and color codes
CERN – AB Department
b. frammery -10.10.2005
30
GENERAL SERVICES
CERN – AB Department
b. frammery -10.10.2005
31
The Alarm System
• LHC Alarm SERvice (LASER)
LASER
Service
PS alarm
Gateway
Legacy PS
alarms
LEIR
(FESA)
Broke
r
LHC
(FESA)
[TH2.2-70]
•« Standard » 3-tier architecture
•Java message service (JMS)
•Subscription mechanism
CAS alarm
Gateway
New SPS
alarms
(FESA)
Legacy CAS
alarms
(SPS, TCR,
CSAM)
Current
New
Gateway
CERN – AB Department
b. frammery -10.10.2005
32
Logging
• Several 105 parameters will be logged
• Every data or setting is timestamped (UTC)
• Parameters are logged
o
o
o
on regular intervals (down to 100 ms)
on request
on-change
CERN – AB Department
b. frammery -10.10.2005
33
Analogue signals
The ancestor
Open Analogue Signals Information System (OASIS)
o
o
o
o
o
[TH3A.1-50]
To visualize and correlate in Real-Time time critical signals in
the control room
~500 signals for LHC – 50 MHz bandwidth (+ ~1000 in PS/SPS)
Distributed cPCI system using analogue MPX and oscilloscope
modules (Acqiris or other types) close to the equipment
Triggers through the timing network for precise time
correlations
Standard 3-tier architecture.
CERN – AB Department
b. frammery -10.10.2005
34
Core control application
software (LSA)
• Normalized data model valid for
o
[TU1.3-5O]
Settings, measurements, optics parameters
• Set of software modules for
o
o
o
Optics definition
Setting generation & management
“Trims” (coherent global modifications of settings)
• Set of generic applications
Developed together with OP, based on experience with LEP
and tested already for 2 new extractions from SPS (CNGS, TI8)
CERN – AB Department
b. frammery -10.10.2005
35
Post Mortem
To take a snapshot of the LHC vital systems.
• Automatic (typ. when an interlock appears) or manual trigger
• No beam allowed if PM not ready
• Capture of
o
o
o
o
Logged data
Alarms (LASER)
Transient recorder signals (OASIS)
Fixed displays
• Analysis
o
o
o
o
A few Gigabytes per Post Mortem capture
Structured sorting of causes & effects
Needed from October 2005 for Hardware commissioning
Continuous development effort for the years to come
CERN – AB Department
b. frammery -10.10.2005
36
Critical systems
for LHC
CERN – AB Department
b. frammery -10.10.2005
37
Powering Interlock System (1)
IP5
IP6
•For POWERING, LHC
is equal to 8 sectors
IP4
IP7
IP3
IP8
IP2
IP1
IP1
6 large cryostats
Atlas
CERN – AB Department
IP8
LHC-b
b. frammery -10.10.2005
38
Powering Interlock System (1)
13
SIEMENS
14
SIEMENS
26
SIEMENS
43
PLCs
SIEMENS
15
SIEMENS
14
SIEMENS
Control
Network
to Beam Interlock System
To protect 1612 electrical
circuits with 10’000
supraconducting magnets
CERN – AB Department
b. frammery -10.10.2005
39
Powering Interlock System (2)
[PO2.036-3]
Technical Network
PVSS Console and Server
(monitoring & configuration)
Siemens PLC
(process control &
configuration)
Profibus
Magnet/
QPS
QPS
Quench
QPS
ProtectS
ystem
UPS
PC_PERMIT
CIRCUIT_QUENCH
/ MAGNET
OVERTEMP
PC_FAST_ABORT
Patch
Panels and
Electronics
POWERING_FAILURE
Beam Permit
AUG
CERN – AB Department
Powe
Powe
Powe
rr
Power
r
Conv
Conver
Conv
Conv
erter
ters
erter
erter
HW Current loops for
connections of clients
Beam Interlock system
Hardware system
b. frammery -10.10.2005
40
Beam Interlock System (1)
optical fiber loop
10 MHz signal
(as RHIC / BNL)
BIC
BIC
BIC
vacuum
Pt.4
Beam Dump
Momentum
cleaning
Pt.3
BIC
BIC
BEAM 1
clockwise
BEAM 2
counterclockwise
BIC
BIC
Connects the Beam Loss
Monitors and many others
systems to the Beam Dump
request.
CERN – AB Department
Pt.8
BIC
Pt.1
BIC
b. frammery -10.10.2005
BIC
BIC
Betatron
cleaning
ATLAS
BIC
Injection
BEAM I
from SPS
BIC
BIC
Pt.7
LHC-B
ALICE
Pt.2
•
Pt.6
RF
…others..
Two independent hardware
loops as « beam permit »
signal transmission.
BIC
CMS
access
•
BIC
Pt.5
powering and
protection
beam loss monitors
BEAM DUMP
CONTROLLERS
BIC
BIC
BIC
BIC
Injection
BEAM II
from SPS
41
Beam Interlock System (2)
[PO2.031-3]
Java
Application
16 VME CRATES
Technical Network
User Interfaces
(installed in User’s rack)
Beam
Permit
Beam
Permit
#1
copper cable
#2
Patching
Beam
Permit
+
#3
Test &
Monitoring
Module
F.O.
interface
Beam Interlock Controller
CERN – AB Department
b. frammery -10.10.2005
Safe Beam
Par.
(via Timing)
Core
module
up to 1200 meters
Safe
Beam
Parameter
Receiver
Beam
Permit
Loops
42
Real-Time Feedback systems
• LHC orbit feedback
2000 Beam position parameters
o 1000 steering dipoles
o 10 Hz frequency
o
• LHC tune feedback
•Modest system – 4 parameters and some 30 PCs (up to 50 Hz ?).
• LHC Chromaticity feedback
•Considered but difficulty to have reliable measurements
CERN – AB Department
b. frammery -10.10.2005
43
Orbit Feedback system
FB
•Centralized architecture
•> 100 VME crates involved
•Through the Technical network
•Tests on SPS in 2004 successful
•Simulations show 25Hz capability
CERN – AB Department
b. frammery -10.10.2005
44
Quench Protection System
PVSS Expert GUI
LHC Logging
Retrieve and
present data
Post-mortem
Send data
Alarms (LASER)
PC Gateway
Power Interlocks
PVSS Data Server
Supervision/Monitoring
LHC
CERN – AB Department
Supraconducting
b. frammery -10.10.2005
DQRMB
DQSMB
DQHDS
DQHDS
DQQDI
DQQDC
DQQDG
DQQDC
DQQDG
DQQDC
DQQDG
DQQDC
DQQDG
DQQDC
DQHDS
DQHDS
DQQDL
WorldFIP
magnets
45
Controls for cryogenics
PVSS
Data
Servers
Central Cryogenic
control room
Local Cryogenic
control room
Quantum <–> S7-400.
Gateway <–> Quantum.
WFIP
Gateways
(LINUX)
Schneider
PLCs
Siemens
PLCs
•130 PLCs ( Schneider & Siemens)
•Application built on UNICOS framework
PROFIBUS DP
networks
WFIP
Networks (4)
PROFIBUS PA networks
[WE3A.2-60]
CERN – AB Department
b. frammery -10.10.2005
46
Collimation System (1)
Compulsory to gain 3 orders of magnitude in
performance beyond other hadron colliders.
• 162 collimators when fully deployed
• 5 degrees of freedom & 10 measurements
of absolute and relative positions and
gaps per collimator
• Synchronous move with 10 mm precision
within a few 10 ms in relation with
o
o
Local orbit
Beam loss measurements
[PO2.016-2]
CERN – AB Department
b. frammery -10.10.2005
48
The CERN
CONTROL CENTER
(CCC)
CERN – AB Department
b. frammery -10.10.2005
50
The CERN Control Center
• A single control room for CERN to control
o
o
All accelerators
All technical services
• Grown from the SPS (LEP) control room on the French
CERN site (Prévessin)
• Work started in November 2004, to be delivered in
October 2005 & to be operational in February 2006
• All CERN machines operated from the CCC in 2006
CERN – AB Department
b. frammery -10.10.2005
51
The CERN Control Center
The current aspect
The Architect drawing
CERN – AB Department
b. frammery -10.10.2005
52
The CERN Control Center
SPS
Tech. services
+ cryoplants
PS Complex
LHC
40 console modules
16 large LCD displays
The current aspect
The architect’s view
CERN – AB Department
b. frammery -10.10.2005
53
The CERN Control Center
Erich Keller
One of the 20 workplaces of the CCC (for 2 operators )
CERN – AB Department
b. frammery -10.10.2005
54
A brief Status
OF the LHC
Control System
CERN – AB Department
b. frammery -10.10.2005
55
Status : the basic infrastructure
Basic infrastructure
conception
implementation
comments
Network
done
done
CERN security strategy to be applied
VME FECs
purchased
done
LEIR: 100% installed,
LHC Hardware Commissioning :50% installed
PC gateways
purchased
done
LHC Hardware Commissioning : 50% installed
PLC FECs
purchased
done
Cryogenics : 60% installed
Powering Interlock system : 30% installed
WorldFIP
done
done
tunnel & Surface buildings: deployed 100%,
qualified : 35%
Remote reboot
done
done
Installed sectors 7-8, 8-1
Servers
purchased
provisional installation
to be installed in CCC < Feb 2006
Consoles
equipment defined and to be delivered
purchased
in oct.05
Central Timing
Timing distribution
& receivers
done
done
to be installed in CCC before March 2006
done
done for all modules
installed in LHC Points 1, 7 & 8
CERN – AB Department
b. frammery -10.10.2005
to be installed Nov 2005 - March 2006 for CCC
Installed in Field CR - UA83
56
Status : the software components
[TH4.2-10]
Post
Logging
Mortem
Subsystems
Control .
Timing
Alarms
(LASER)
.
Powering Automated Analogue
Interlocks
Test
Signals
Procedures (OASIS)
CMW
FESA
PVSS/
UNICOS
Application
software/
LSA core
BOTH
Both OK
Test
opportunities
TT40/TI8
NO
YES
Partial
NO
YES
NO
YES
BOTH
NO
YES
YES
YES
YES
NO
YES
BOTH
NO
YES
NO
YES
NO
NO
NO
NO
YES
YES
QPS surface tests
YES
NO
NO
NO
NO
NO
NO
FESA
NO
NO
LSS8L tests
YES
YES
YES
YES
YES
YES
NO
BOTH
YES
Partial/OK
electrical
circuit
commissioning
YES
YES
YES
YES
YES
YES
NO
BOTH
YES
Partial/OK
SPS/TI2/CNGS
YES
YES
YES
YES
YES
NO
YES
BOTH
YES
Partial/OK
extraction test
LEIR beam
Commissioning
1st
QRL tests
BOTH
(vacuum)
Generic
applics
Large
Tests in progress
CERN – AB Department
Tests already done
b. frammery -10.10.2005
57
Issues
(1)
• Basic Infrastructure
o
o
Security policy to be implemented on the Technical Network
without jeopardizing the deployment of the Consoles & servers.
Deployment of the new timing system on the pre-injectors.
• Software
o
o
While generic application and general services are in line,
specific application programs for LHC cannot yet be specified.
Software modules not tested at full scale.
CERN – AB Department
b. frammery -10.10.2005
58
Issues (2)
• Hardware commissioning
o
o
Time to commission the LHC becomes thinner and thinner.
Manpower very limited to face both LHC installation, hardware
commissioning and support to operational machines:
• Beam commissioning
o
o
Some critical systems are pretty late (ex:collimation)
Strategy to be found to inject some beam despite of all the security
systems!!
• The “legacy software”
To get the manpower for LHC, the existing controls infrastructures
have been somewhat neglected.
o The restart of the machines in 2006 will be difficult.
o
CERN – AB Department
b. frammery -10.10.2005
59
Conclusion
• The basic LHC control system exists today.
• There is a strong commitment by everyone
to be ready to start LHC with beam in
Summer 2007.
• More news in October 2007 …
CERN – AB Department
b. frammery -10.10.2005
60
Thank you
for your attention
CERN – AB Department
b. frammery -10.10.2005
61
CERN – AB Department
b. frammery -10.10.2005
62