Transcript LCLS LLRF Distributed Control System

LCLS LLRF Distributed
Control System
Dayle Kotturi
Controls Department
SLAC National Accelerator Lab
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
1
1
Dayle Kotturi
[email protected]
LCLS LLRF Distributed Control System
Outline
Scope
Global Overview
General stability requirements
Principal motivator
Solutions
Throughput measurement
Conclusions
Additional resources
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
2
2
Dayle Kotturi
[email protected]
Scope
The low level RF controls system consists of RF
phase and amplitude controls at these locations:
Laser
Gun
L0-A (a.k.a. L0-1)
L0-B (a.k.a. L0-2)
L0 Transverse cavity
L1-S
L1-X
L2 – using 2 klystrons to control avg phase/ampl of L2
L3 Transverse cavity
L3 - here is a bit different (lots of klystrons!)
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
3
3
Dayle Kotturi
[email protected]
LLRF Global Overview
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
4
4
Dayle Kotturi
[email protected]
General stability requirements
For LCLS, the general RF stability requirements
are: 0.1 deg phase and 0.1% amplitude in L0
and L1 for S band.
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
5
5
Dayle Kotturi
[email protected]
Principal motivator
Placing the digitizers next to the low noise RF
components eliminates transmission of low
noise analog signals outside the chassis.
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
6
6
Dayle Kotturi
[email protected]
476 MHz RF Reference clock distributed to all 30 sectors in the Linac and beyond
RF Phase and Amplitude correction at 120 Hz for:
laser, gun, L0-A, L0-B, L1-S, L1-X, T cav, L2 and S25 Tcav
Solution 2: Multiple VME crates with in-house modules
Sector 25 T Cav (new 4/2005)
L2: in sector 24, there are 3 stations to adjust in order to accurately control phase and amplitude for long, beam-based fast feedback
T Cav
L1-X
L1-S
L0-B
Slow adjustments to
L0-A
allow rotation of the
gun
reference phase
laser
RF Reference/4 = 119 MHz
stabilized to 50 fs jitter
PAD
I and Q
Demodulator
A
D
C
F
I
F
O
s
D
A
C
Controller
with
ethernet
s
l
o
w
Local trigger
RF Reference*6 = 2856 MHz
stabilized to 50 fs jitter
PAC
Controller
with
ethernet
I and Q
Modulator
Private ethernet
8 kBytes at 120 Hz
VME Crate
1 trigger
for 4
channels
of 1k
samples
C
P
U
F
I
F
O
s
D
A
C
E
V
R
Eth
recvr
A
D
C
f
a
s
t
Other
waveforms
Fast, but
not 119
MHz. 59.5
MHz ok
Private ethernet
4 kBytes at 120 Hz
A
D
C
C
P
U
Beam-based
longitudinal
fast feedback
gigabit
ethernet
s
l
o
w
Global
longitudinal
beam-based
feedback VME
crate
Thermocouple system
100 mW
Controls gigabit ethernet
(interface to MCC)
Possibly combined into one module
IQ Modulator: a
phase shifter
and an
attenuator
1 trigger to travel
up to ½ sector
away
Lin
100 mW
ac
/A
cc
All except laser RF
ele
La
o
rat
rR
A
c/
1 kW
photodiode
Amps
el
Klystron
or
at
er
Q) F
(I& R
cc
ut
FO
rR
119 MHz
120 Hz
60 MW
photodiode
UV
SLED
cavity
In
Q)
na
(I& )
Li
Hz (I&Q
z
20
F 1 MH
r R 119
F
se
se
La
119 MHz
Laser
Oscillator
Solid State Sub Booster
Gun
&
(I
NB: For the gun, SLED
cavity is shorted out
)
Q
HPRF
240 MW
1 kW
1 kW
60 MW
10' accelerator
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
7
7
Dayle Kotturi
[email protected]
476 MHz RF Reference clock distributed to all 30 sectors in the Linac and beyond
RF Phase and Amplitude correction at 120 Hz for: L3
RF Reference/4 = 119 MHz
stabilized to 50 fs jitter
C
P
U
Beam-based
longitudinal fast
feedback gigabit
ethernet
DAC (slow)
Updates
@120 Hz;
stable 20
µs before
beam arrives
and able to
distinguish
beamcodes
C
P
U
RF Reference*6 = 2856 MHz
stabilized to 50 fs jitter
IQ Modulator
(driving subbooster klystron
in sectors 29)
Global
longitudinal
beam-based
feedback VME
crate
IQ Modulator
(driving subbooster klystron
in sectors 30)
Controls gigabit ethernet (interface to MCC)
100 mW
100 mW
Sub Booster Klystron
Sub Booster Klystron
500 W
500 W
8 copies
klystron
8 copies
klystron
60 MW
60 MW
SLED
cavity
SLED
cavity
HPRF
240 MW
HPRF
240 MW
1 kW
1 kW
60 MW
1 kW
10' accelerator
1 kW
60 MW
10' accelerator
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
8
8
Dayle Kotturi
[email protected]
RF phase and amplitude correction and global feedback at 120 Hz for LCLS LINAC
RF Dist’n
Laser
Gun
L0-A
PAC
PAD
PAD
L0-B
Key:
Indicates located in RF Hut
Otherwise at Klystron
SPAC
SPAC
SPAC
SPAC
SPAC
PAD
PAC
SPAC
PAD
PAC
PAD
PAD
PAD
PAD
Indicates may be needed
The maybe is included in
counts below
PAC
PAD
PAD
L0-Tcav
Eth
recvr
PAC
PAD
PAD
PAC
PAD
PAD
PAD
L1-S
PAC
PAD
Beam Phase
Monitor
C
P
U
E
V
R
VME Crate at S20
running
longitudinal,
beam-based
feedback.
L1-X
PAC
PAD
PAD
PAD
S20
L24-1
PAC
Fast PACs:
Slow PACs (SPACs):
PADs:
VME crates:
8
6
19
1
S24
4
2
2
1
Total
12
8
21
2
L24-2
PAC
L24-3
PAC
Tcav L24-8
PAC
PAD
PAD
Eth
recvr
C
P
U
E
V
R
VME Crate at S24
running
longitudinal,
beam-based
feedback.
S29
SPAC
S30
SPAC
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
9
9
Dayle Kotturi
[email protected]
Example of generic LLRF control system instance
PAD
Temperature monitors
D
Coldfire A
F
I and Q
CPU C
A I
Demorunning
D F
dulator
RTEMS s
C O
and l
s
EPICS o
w
4 channels
of 1k
samples
PAC
VME
C
P
U
Coldfire
CPU
running
RTEMS
and
EPICS
E
V
R
1 trigger
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
1 trigger
10 10
Temperature monitors
D
A
C
FPGA
IQ Modulator
gives phase
and amplitude
control
D
A
s
C
l
o
w
2 channels
of 1k
samples
Dayle Kotturi
[email protected]
Phase/Amplitude Detector -> VME
LLRF VME
(Handles phase and amplitude
conversions, feedback calculations and
setting of PAC)
LLRF PAD
(Phase and amplitude detector)
ADC
Channel
0
EVR timing trigger
Private gigabit switch
ADC
Channel
1
ADC
Channel
2
Coldfire
Use secondary ethernet interface
to send 4 channels of PAD data
via UDP broadcast via private
switch. Time required: 100 nsec
mvme6100
- Receive I and Q averages
- Convert to 4 channels of Phase
and amplitude
- Compute weighted averages
- Trigger BSA
- Do feedback calculation
- Send I and Q adjustment to PAC
ADC
Channel
3
LCLSCA network
Change PAD channels’
size and offsets
Set PDES, ADES and
control RF feedback
Update GUIs
with PAD
waveforms,
temperature,
I and Q
actuals every
2 seconds
Update GUIs with
actual phases
amplitudes,
feedback
corrections, etc,
every 2 seconds
MCC workstation running
lclshome GUI
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
11
11
Dayle Kotturi
[email protected]
OS, BSP and EPICS versions
PAD:
rtems4.9.1
m68k uC5282
epics-R3.14.10
VME:
rtems4.9.1
powerpc beatnik (mvme5500/mvme6100)
epics-R3.14.8.2
PAC:
rtems4.9.1
m68k uC5282
epics-R3.14.10
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
12 12
Dayle Kotturi
[email protected]
PAD waveform readout
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
13 13
Dayle Kotturi
[email protected]
PAD IOC Stats
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
14 14
Dayle Kotturi
[email protected]
VME Feedback Calculation
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
15 15
Dayle Kotturi
[email protected]
VME IOC Stats
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
16 16
Dayle Kotturi
[email protected]
PAC waveform control
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
17 17
Dayle Kotturi
[email protected]
Throughput Time: PAD->VME->PAC
1
2
3
4
5
6
7
8
9
10
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
18 18
Dayle Kotturi
[email protected]
Throughput steps: PAD->VME->PAC
Event (#
matches
diagram)
Absolute
time after
trigger
(μsec)
Description
-
0.0 Trigger delivered to phase/amplitude detector digitizer (PAD)
1
7.2 ISR: signals “data ready” to wake up DAQ task
2
102.0 Channel0 readout and data processing begins (40 Is and 40 Qs)
3
583.2 Channel1 readout and data processing begins (40 Is and 40 Qs)
4
1041.6 Channel2 readout and data processing begins (40 Is and 40 Qs)
5
1501.6 Channel3 readout and data processing begins (40 Is and 40 Qs)
6
1963.2 PAD starts stream of processed values to VME over private net
7
2349.6 PAD streaming completed. VME parses, does feedback, sends.
8
2508.0 Phase/amplitude controller (PAC) receives new setpts from VME
9
2524.0 PAC writes to FPGA which calcs new WF to send next trigger
10
2529.2 Data is ready
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
19 19
Dayle Kotturi
[email protected]
Conclusions
At 120 Hz operation, time budget=8.333
ms
LLRF PAD->VME->PAC throughput
measured=2.529 ms for 4 channels of 40
points each, with no offsets,
adjust: subtract 2.5 μsec per pair of IRQ
raise/lower calls (8 pairs = 20 μsec)
adjust: one socket sends to multiple PACs;
add switching time
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
20 20
Dayle Kotturi
[email protected]
Acknowledgements
Thanks to Ron Akre and Klystron Department for
setting up hardware, scopes and signal
generators
Thanks to SLAC NAL Controls Group
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
21 21
Dayle Kotturi
[email protected]
Additional Information
Details of the PAD->VME transfer
Details of the VME->PAC transfer
RF stability measurement
PAD
PAD Block diagram
LCLS LLRF website:
http://www.slac.stanford.edu/grp/lcls/controls/global/subsystems/llrf
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
22 22
Dayle Kotturi
[email protected]
Details of the PAD->VME transfer
http://www.slac.stanford.edu/grp/lcls/controls/glo
bal/sw/epics/epics%20team%20meetings/prese
ntations/lanIpBasic.pdf
Raw ethernet packets with IP and UDP headers.
Similar to BSD sockets.
Solution is for low end CPU on small LAN.
Requirement: ship 1 KB of data in ~200 μsec
VME initializes, starts and stops PAD streaming
When PAD is streaming, device support for
waveform on VME parses out the values and
uses them in the feedback calculations of new
setpoints.
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
23 23
Dayle Kotturi
[email protected]
Details of the VME->PAC transfer
On VME, a subroutine record that has
calculated new setpoints calls a driver
routine that sends the values to the PAC
via udp socket
PAC is has thread waiting to receive
packet
When packet arrives, it parses out the
setpoints and puts them into mem mapped
FPGA
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
24 24
Dayle Kotturi
[email protected]
LCLS Jitter Specification for 2 Seconds is 0.14% Amplitude and 0.14 degree Phase
Feedback ON 20 Second Plot shows
Feedback ON 20 Second Plot shows
Phase Jitter 0.043 degrees
Phase Jitter 0.043 degrees
Amplitude Jitter 0.022%
Amplitude Jitter 0.024%
Short Term RF Jitter Specification for L0B are well Exceeded.
This is as good as it gets – Don’t tell Physicists or they will expect it. Ron Akre 2007
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
25 25
Dayle Kotturi
[email protected]
About the PAD
CPU is MCF5282 (64MHz)
The digitizer used is the Linear
Technologies LTC2208. It was the first 16
bit digitizer chip on the market capable of
running at 119MHz, it is specified to run up
to 130MHz.
At SLAC NAL, PAD digitizer used for RF,
beam position monitors, beam charge
monitors and bunch length monitors.
Pohang Light Source is also using PAD for
new RF system.
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
26 26
Dayle Kotturi
[email protected]
Phase/amplitude detector (PAD) Block
25.5MHz IF
16bit DATA
ZX60-4016E
Chan. 1
IF
WCLK
RF LO
CHAN 2
RF INPUT
RP N
16 bit
DATA
FIFO
64k words
FILTER 25.5MHz BP
16bit DATA
MIXER
Chan. 2
IF
WCLK
CS/
CLK
FIFO
64k words
CONTROL /
Arcturus uC5282
Microcontroller Module
with 10/100 Ethernet
RF LO
16bit DATA
MIXER
Chan. 3
WCLK
FIFO
64k words
RAW
ETHERNET
CHAN 3
INPUT
RP BNC
+18dB
16bit DATA
CHAN 4
RF INPUT
RP N
-8dBm
10dBm
Chan. 4
WCLK
10dBm
IF
-2dBm
ETHERNET COM
CHAN 1
RF INPUT
RP N
Control Board
4 X 16 bit ADC
102MHz Clock
LTC2208
Transformer Coupled Inputs
RF Board
FIFO
64k words
RF LO
Control
MIXER
CPLD
ZN4PD1-50
LO INPUT
2830.5MHz
RP N
5VDC
0.8A x 2 Analog
10dBm
LO OUTPUT
2830.5MHz
FP N
QSPI
5VDC
0.5A Digital
CLOCK INCLOCK Mon
102MHz 102MHz
RP N
FP N
TRIG In
120Hz
RP BNC
TRIG Mon
FP BNC
20 pin ribbon
QSPI
3x 75mA
TE62054-ND 2x18V 0.417A
YEL
BLK
GRN
RED
RED
VIO
BRN
BLU
24Bit
Analog
Input
Board
12VDC
LM340Reg
5 ohm
12VDC
LM340Reg
10 ohm
ANALOG IN ANALOG IN
110VAC
5VDC
LM340Reg
110VAC
Power Module
Corcom
6EHL1SC
1A FB
1 ohm
YEL
BLK
GRN
RED
RED
VIO
BRN
BLU
TE62071-ND 2x9V 1.94A
5VDC
LM340Reg
1 ohm
5VDC
LM340Reg
2 ohm
Ron Akre
FRONT PANEL LEDs
5VD 5VA2 5VA1 12V2 12V1
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
27 27
Dayle Kotturi
[email protected]
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
28 28
Dayle Kotturi
[email protected]
LCLS LLRF Distributed Control System
Outline
Scope
General stability requirements
Principal motivator
Solutions
Throughput measurement
Conclusions
Additional resources
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
29 29
Dayle Kotturi
[email protected]
Scope
The low level RF controls system consists of RF
phase and amplitude controls at these locations:
Laser
Gun
L0-A (a.k.a. L0-1)
L0-B (a.k.a. L0-2)
L0 Transverse cavity
L1-S
L1-X
L2 – using 2 klystrons to control avg phase/ampl of L2
L3 Transverse cavity
L3 - here is a bit different (lots of klystrons!)
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
30 30
Dayle Kotturi
[email protected]
General stability requirements
For LCLS, the general RF stability requirements
are: 0.1 deg phase and 0.1% amplitude in L0
and L1 for S band.
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
31 31
Dayle Kotturi
[email protected]
Principal motivator
Placing the digitizers next to the low noise RF
components eliminates transmission of low
noise analog signals outside the chassis.
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
32 32
Dayle Kotturi
[email protected]
476 MHz RF Reference clock distributed to all 30 sectors in the Linac and beyond
RF Phase and Amplitude correction at 120 Hz for:
laser, gun, L0-A, L0-B, L1-S, L1-X, T cav, L2 and S25 Tcav
Solution 2: Multiple VME crates with in-house modules
Sector 25 T Cav (new 4/2005)
L2: in sector 24, there are 3 stations to adjust in order to accurately control phase and amplitude for long, beam-based fast feedback
T Cav
L1-X
L1-S
L0-B
Slow adjustments to
L0-A
allow rotation of the
gun
reference phase
laser
RF Reference/4 = 119 MHz
stabilized to 50 fs jitter
PAD
I and Q
Demodulator
A
D
C
F
I
F
O
s
D
A
C
Controller
with
ethernet
s
l
o
w
Local trigger
RF Reference*6 = 2856 MHz
stabilized to 50 fs jitter
PAC
Controller
with
ethernet
I and Q
Modulator
Private ethernet
8 kBytes at 120 Hz
VME Crate
1 trigger
for 4
channels
of 1k
samples
C
P
U
F
I
F
O
s
D
A
C
E
V
R
Eth
recvr
A
D
C
f
a
s
t
Other
waveforms
Fast, but
not 119
MHz. 59.5
MHz ok
Private ethernet
4 kBytes at 120 Hz
A
D
C
C
P
U
Beam-based
longitudinal
fast feedback
gigabit
ethernet
s
l
o
w
Global
longitudinal
beam-based
feedback VME
crate
Thermocouple system
100 mW
Controls gigabit ethernet
(interface to MCC)
Possibly combined into one module
IQ Modulator: a
phase shifter
and an
attenuator
1 trigger to travel
up to ½ sector
away
Lin
100 mW
ac
/A
cc
All except laser RF
ele
La
o
rat
rR
A
c/
1 kW
photodiode
Amps
el
Klystron
or
at
er
Q) F
(I& R
cc
ut
FO
rR
119 MHz
120 Hz
60 MW
photodiode
UV
SLED
cavity
In
Q)
na
(I& )
Li
Hz (I&Q
z
20
F 1 MH
r R 119
F
se
se
La
119 MHz
Laser
Oscillator
Solid State Sub Booster
Gun
&
(I
NB: For the gun, SLED
cavity is shorted out
)
Q
HPRF
240 MW
1 kW
1 kW
60 MW
10' accelerator
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
33 33
Dayle Kotturi
[email protected]
476 MHz RF Reference clock distributed to all 30 sectors in the Linac and beyond
RF Phase and Amplitude correction at 120 Hz for: L3
RF Reference/4 = 119 MHz
stabilized to 50 fs jitter
C
P
U
Beam-based
longitudinal fast
feedback gigabit
ethernet
DAC (slow)
Updates
@120 Hz;
stable 20
µs before
beam arrives
and able to
distinguish
beamcodes
C
P
U
RF Reference*6 = 2856 MHz
stabilized to 50 fs jitter
IQ Modulator
(driving subbooster klystron
in sectors 29)
Global
longitudinal
beam-based
feedback VME
crate
IQ Modulator
(driving subbooster klystron
in sectors 30)
Controls gigabit ethernet (interface to MCC)
100 mW
100 mW
Sub Booster Klystron
Sub Booster Klystron
500 W
500 W
8 copies
klystron
8 copies
klystron
60 MW
60 MW
SLED
cavity
SLED
cavity
HPRF
240 MW
HPRF
240 MW
1 kW
1 kW
60 MW
1 kW
10' accelerator
1 kW
60 MW
10' accelerator
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
34 34
Dayle Kotturi
[email protected]
RF phase and amplitude correction and global feedback at 120 Hz for LCLS LINAC
RF Dist’n
Laser
Gun
L0-A
PAC
PAD
PAD
L0-B
Key:
Indicates located in RF Hut
Otherwise at Klystron
SPAC
SPAC
SPAC
SPAC
SPAC
PAD
PAC
SPAC
PAD
PAC
PAD
PAD
PAD
PAD
Indicates may be needed
The maybe is included in
counts below
PAC
PAD
PAD
L0-Tcav
Eth
recvr
PAC
PAD
PAD
PAC
PAD
PAD
PAD
L1-S
PAC
PAD
Beam Phase
Monitor
C
P
U
E
V
R
VME Crate at S20
running
longitudinal,
beam-based
feedback.
L1-X
PAC
PAD
PAD
PAD
S20
L24-1
PAC
Fast PACs:
Slow PACs (SPACs):
PADs:
VME crates:
8
6
19
1
S24
4
2
2
1
Total
12
8
21
2
L24-2
PAC
L24-3
PAC
Tcav L24-8
PAC
PAD
PAD
Eth
recvr
C
P
U
E
V
R
VME Crate at S24
running
longitudinal,
beam-based
feedback.
S29
SPAC
S30
SPAC
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
35 35
Dayle Kotturi
[email protected]
Example of generic LLRF control system instance
PAD
Temperature monitors
D
Coldfire A
F
I and Q
CPU C
A I
Demorunning
D F
dulator
RTEMS s
C O
and l
s
EPICS o
w
4 channels
of 1k
samples
PAC
VME
C
P
U
Coldfire
CPU
running
RTEMS
and
EPICS
E
V
R
1 trigger
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
1 trigger
36 36
Temperature monitors
D
A
C
FPGA
IQ Modulator
gives phase
and amplitude
control
D
A
s
C
l
o
w
2 channels
of 1k
samples
Dayle Kotturi
[email protected]
Phase/Amplitude Detector -> VME
LLRF VME
(Handles phase and amplitude
conversions, feedback calculations and
setting of PAC)
LLRF PAD
(Phase and amplitude detector)
ADC
Channel
0
EVR timing trigger
Private gigabit switch
ADC
Channel
1
ADC
Channel
2
Coldfire
Use secondary ethernet interface
to send 4 channels of PAD data
via UDP broadcast via private
switch. Time required: 100 nsec
mvme6100
- Receive I and Q averages
- Convert to 4 channels of Phase
and amplitude
- Compute weighted averages
- Trigger BSA
- Do feedback calculation
- Send I and Q adjustment to PAC
ADC
Channel
3
LCLSCA network
Change PAD channels’
size and offsets
Set PDES, ADES and
control RF feedback
Update GUIs
with PAD
waveforms,
temperature,
I and Q
actuals every
2 seconds
Update GUIs with
actual phases
amplitudes,
feedback
corrections, etc,
every 2 seconds
MCC workstation running
lclshome GUI
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
37 37
Dayle Kotturi
[email protected]
OS, BSP and EPICS versions
PAD:
rtems4.9.1
m68k uC5282
epics-R3.14.10
VME:
rtems4.9.1
powerpc beatnik (mvme5500/mvme6100)
epics-R3.14.8.2
PAC:
rtems4.9.1
m68k uC5282
epics-R3.14.10
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
38 38
Dayle Kotturi
[email protected]
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
39 39
Dayle Kotturi
[email protected]
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
40 40
Dayle Kotturi
[email protected]
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
41 41
Dayle Kotturi
[email protected]
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
42 42
Dayle Kotturi
[email protected]
Measuring Throughput: PAD->VME>PAC
1
2
3
4
5
6
7
8
9
10
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
43 43
Dayle Kotturi
[email protected]
Throughput steps: PAD->VME->PAC
Event (#
matches
diagram)
Absolute
time after
trigger
(μsec)
Description
-
0.0 Trigger delivered to phase/amplitude detector digitizer (PAD)
1
7.2 ISR: signals “data ready” to wake up DAQ task
2
102.0 Channel0 readout and data processing begins (40 Is and 40 Qs)
3
583.2 Channel1 readout and data processing begins (40 Is and 40 Qs)
4
1041.6 Channel2 readout and data processing begins (40 Is and 40 Qs)
5
1501.6 Channel3 readout and data processing begins (40 Is and 40 Qs)
6
1963.2 PAD starts stream of processed values to VME over private net
7
2349.6 PAD streaming completed. VME parses, does feedback, sends.
8
2508.0 Phase/amplitude controller (PAC) receives new setpts from VME
9
2524.0 PAC writes to FPGA which calcs new WF to send next trigger
10
2529.2 Data is ready
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
44 44
Dayle Kotturi
[email protected]
Conclusions
At 120 Hz operation, time budget=8.333
ms
LLRF PAD->VME->PAC throughput
measured=2.529 ms for 4 channels of 40
points each, with no offsets,
adjust: subtract 2.5 μsec per pair of IRQ
raise/lower calls (8 pairs = 20 μsec)
adjust: one socket sends to multiple PACs;
add switching time
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
45 45
Dayle Kotturi
[email protected]
Acknowledgements
Thanks as always to Ron Akre and Klystron
Department for setting up hardware, scopes and
signal generators
Thanks to SLAC NAL Controls Group
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
46 46
Dayle Kotturi
[email protected]
Additional Information
Details of the PAD->VME transfer
Details of the VME->PAC transfer
RF stability measurement
PAD
PAD Block diagram
LCLS LLRF website:
http://www.slac.stanford.edu/grp/lcls/contr
ols/global/subsystems/llrf
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
47 47
Dayle Kotturi
[email protected]
Details of the PAD->VME transfer
http://www.slac.stanford.edu/grp/lcls/controls/glo
bal/sw/epics/epics%20team%20meetings/prese
ntations/lanIpBasic.pdf
Raw ethernet packets with IP and UDP headers.
Similar to BSD sockets.
Solution is for low end CPU on small LAN.
VME initializes, starts and stops PAD streaming
When PAD is streaming, device support for
waveform on VME parses out the values and
uses them in the feedback calculations of new
setpoints.
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
48 48
Dayle Kotturi
[email protected]
Details of the VME->PAC transfer
On VME, a subroutine record that has
calculated new setpoints calls a driver
routine that sends the values to the PAC
via udp socket
PAC is has thread waiting to receive
packet
When packet arrives, it parses out the
setpoints and puts them into mem mapped
FPGA
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
49 49
Dayle Kotturi
[email protected]
LCLS Jitter Specification for 2 Seconds is 0.14% Amplitude and 0.14 degree Phase
Feedback ON 20 Second Plot shows
Feedback ON 20 Second Plot shows
Phase Jitter 0.043 degrees
Phase Jitter 0.043 degrees
Amplitude Jitter 0.022%
Amplitude Jitter 0.024%
Short Term RF Jitter Specification for L0B are well Exceeded.
This is as good as it gets – Don’t tell Physicists or they will expect it. Ron Akre 2007
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
50 50
Dayle Kotturi
[email protected]
About the PAD
The digitizer used is the Linear
Technologies LTC2208. It was the first 16
bit digitizer chip on the market capable of
running at 119MHz, it is specified to run up
to 130MHz.
At SLAC NAL, PAD digitizer used for RF,
beam position monitors, beam charge
monitors and bunch length monitors.
Pohang Light Source is also using PAD for
new RF system.
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
51 51
Dayle Kotturi
[email protected]
Phase/amplitude detector (PAD) Block
25.5MHz IF
16bit DATA
ZX60-4016E
Chan. 1
IF
WCLK
RF LO
CHAN 2
RF INPUT
RP N
16 bit
DATA
FIFO
64k words
FILTER 25.5MHz BP
16bit DATA
MIXER
Chan. 2
IF
WCLK
CS/
CLK
FIFO
64k words
CONTROL /
Arcturus uC5282
Microcontroller Module
with 10/100 Ethernet
RF LO
16bit DATA
MIXER
Chan. 3
WCLK
FIFO
64k words
RAW
ETHERNET
CHAN 3
INPUT
RP BNC
+18dB
16bit DATA
CHAN 4
RF INPUT
RP N
-8dBm
10dBm
Chan. 4
WCLK
10dBm
IF
-2dBm
ETHERNET COM
CHAN 1
RF INPUT
RP N
Control Board
4 X 16 bit ADC
102MHz Clock
LTC2208
Transformer Coupled Inputs
RF Board
FIFO
64k words
RF LO
Control
MIXER
CPLD
ZN4PD1-50
LO INPUT
2830.5MHz
RP N
5VDC
0.8A x 2 Analog
10dBm
LO OUTPUT
2830.5MHz
FP N
QSPI
5VDC
0.5A Digital
CLOCK INCLOCK Mon
102MHz 102MHz
RP N
FP N
TRIG In
120Hz
RP BNC
TRIG Mon
FP BNC
20 pin ribbon
QSPI
3x 75mA
TE62054-ND 2x18V 0.417A
YEL
BLK
GRN
RED
RED
VIO
BRN
BLU
24Bit
Analog
Input
Board
12VDC
LM340Reg
5 ohm
12VDC
LM340Reg
10 ohm
ANALOG IN ANALOG IN
110VAC
5VDC
LM340Reg
110VAC
Power Module
Corcom
6EHL1SC
1A FB
1 ohm
YEL
BLK
GRN
RED
RED
VIO
BRN
BLU
TE62071-ND 2x9V 1.94A
5VDC
LM340Reg
1 ohm
5VDC
LM340Reg
2 ohm
Ron Akre
FRONT PANEL LEDs
5VD 5VA2 5VA1 12V2 12V1
LCLS LLRF Distributed Control System LCLS LLRF Distributed Control System
EPICS Collaboration Meeting, Vancouver 1 May 2009
52 52
Dayle Kotturi
[email protected]