Status of MicroTCA LLRF Development

Zheqiao Geng

On behalf of the LLRF AIP team

6/4/2012

Outline

       History of LLRF at SLAC MicroTCA Based LLRF System Overview FPGA Firmware Design EPICS Software Design Applications for Operation System Tests Summary 2020/4/30 Zheqiao Geng, MicroTCA LLRF 2

History of LLRF at SLAC

2020/4/30 Zheqiao Geng, MicroTCA LLRF 3

SLAC Linac

2020/4/30 Zheqiao Geng, MicroTCA LLRF 4

Existing RF System of Linac

   30 Sectors (LCLS uses the last 10 of them) Each sector contains of 8 klystrons and 1 sub-booster Every RF station has 2 racks for controls 2020/4/30 Zheqiao Geng, MicroTCA LLRF 5

WG & Cable Penetration to Tunnel

PIOP CAMAC Module Controls IPA, PAD, and MKSU. Interface to control system Existing Controls Racks

Existing Linac Klystron Station RF Control, Monitoring, and Interlocking System

IPA Chassis Controls RF Phase and Amplitude PAD Chassis Measures RF Phase and Amplitude MKSU Chassis Interlock and Control for Klystron SLED Support Systems 7

PIOPs (4) PDU Timing Controls Upgrade Implementation 12-01-10 8

Controls Upgrade Implementation 12-01-10 9

 

LLRF for LCLS

Several critical RF stations were upgraded for LCLS with newly designed Phase and Amplitude Detector (PAD), Phase and Amplitude Controller (PAC) and VME running EPICS The old IPA, MKSU and CAMAC system is kept (

not shown in the diagram below

) 2020/4/30 Zheqiao Geng, MicroTCA LLRF 10

LLRF for LCLS (cont.)

  PAD : Custom chassis with 4 channels of down mixers and ADCs PAC : Custom chassis with a DAC board and an I/Q modulator 2020/4/30 Zheqiao Geng, MicroTCA LLRF 11

   

Some Limitations of PAD/PAC

A feedback control loop has to follow the chain of PAD-VME-PAC connected with Ethernet, the real-time performance is limited. It is not possible to do intra pulse control (pulse width ~ 3 µs) Computation power of the Coldfire MCU used in PAD/PAC chassis is quite limited. One more Channel Access client connected to the EPICS software in the Coldfire MCU can significantly degrade its real-time performance One PAD chassis (2U or 3U) only contains 4 ADC channels. The density is too low to efficiently use the rack space Custom designed chassis is difficult to maintain

The MicroTCA based LLRF system presented in this talk tends to upgrade the PAD/PAC system to be more compact, flexible, maintainable and reliable…

2020/4/30 Zheqiao Geng, MicroTCA LLRF 12

Overview of the MicroTCA Based LLRF System

2020/4/30 Zheqiao Geng, MicroTCA LLRF 13

PAD/PAC LLRF VS MicroTCA LLRF

2020/4/30 Zheqiao Geng, MicroTCA LLRF 14

RF Support Chassis

2020/4/30 Zheqiao Geng, MicroTCA LLRF 15

MicroTCA Crate

2020/4/30 Zheqiao Geng, MicroTCA LLRF 16

AMC Carrier + PMC EVR

2020/4/30 Zheqiao Geng, MicroTCA LLRF 17

AMC ADC Board – SIS8300



Struck SIS8300 Board

   4 lane PCI Express 10 Channels 125 MS/s 16-bit ADC Two 16-bit DACs for Fast Feedback Implementation   Twin SFP Card Cage for High Speed System Interconnects Virtex 5 FPGA

The board is equivalent to the digital parts of 2.5 PADs + 1 PAC!

2020/4/30 Zheqiao Geng, MicroTCA LLRF 18

Summary of Hardware Architecture

 Compared to the PAD/PAC chassis, the MicroTCA based LLRF system uses commercial digital boards to reduce the R&D time    MicroTCA system has much higher density of ADC/DAC channels. The system is more compact Compared to the PAD/PAC chassis, the digital hardware of the new design is installed in the MicroTCA crate. The boards are hot-swappable and easy to maintain 

To be improved (nice to have): An AMC EVR board will be introduced to route trigger signals via the backplane. ADC boards will take triggers from backplane to remove the trigger cables

 

The RTM with S-band down mixers from DESY will be evaluated. The system can be more compact by moving the down mixers and up converter to the RTM board from the RF support chassis The klystron beam voltage conditioner board will be improved to directly measure the flattop of the voltage pulse by adding offset to the klystron beam voltage signal

2020/4/30 Zheqiao Geng, MicroTCA LLRF 19

FPGA Firmware (for SIS8300) Design

2020/4/30 Zheqiao Geng, MicroTCA LLRF 20

Major Requirements to the Firmware

    Intra-pulse phase feedback control Provide 64K sampling buffer for each ADC channel (10 channels) DAC can be used as an arbitrary waveform generator Exception detection and handling 2020/4/30 Zheqiao Geng, MicroTCA LLRF 21

Overview of the Firmware

2020/4/30 Zheqiao Geng, MicroTCA LLRF 22

Algorithm for Intra-pulse Phase Control

Q Demodulation Vector Rotation Q Correction I I Q I   2020/4/30 Align the vector along the I axis so that Q components will be proportional to the phase jitter Phase error is estimated at the first part of the RF pulse and the correction is applied at the second part of the RF pulse (latency < 1 µs) Zheqiao Geng, MicroTCA LLRF 23

ADC Data Acquisition



64 K samples from ADC2 Spectrum by FFT (SNR = 77 dBFS)

Allow up to 64K point data acquisition for ADC SNR calculation 2020/4/30 Zheqiao Geng, MicroTCA LLRF 24

DAC Waveform Generation

  

Waveforms Generated by 2048 point Buffers (triggered output) Waveforms Generated by 2048 point Buffers (CW output)

Specify arbitrary I/Q waveforms in two 2048-point buffers for two DACs Allow CW output regardless of trigger

Example: In-phase and Quadrature waveforms for single side band modulation

Zheqiao Geng, MicroTCA LLRF 25

RF System Simulator

Simulated Klystron Output (25.5 MHz IF) Input to the RF System Simulator

  Simulate the klystron output and SLED output The simulator can be used to test most of the functions in the lab 2020/4/30

Simulated SLED Output (25.5 MHz IF)

Zheqiao Geng, MicroTCA LLRF 26

EDM Panel for Firmware Control

2020/4/30 Zheqiao Geng, MicroTCA LLRF 27

Summary of Firmware Design

     MicroTCA based LLRF system connects ADCs and DACs to the same FPGA to enable the intra-pulse control The powerful FPGA is used to implement most of the complex real-time functions to relax the CPU load. 360 Hz operation or multi-bunches operation can be well supported PCI Express links the FPGA and CPU to enable faster data transfer which improves the data acquisition capability 

To be improved (basic): The firmware will be extended to be general and configurable for all RF stations (CW control, Laser control, RF Gun control, X-band and S-band klystron control)



Intra-pulse phase feedback will be upgraded for both amplitude and phase control. I/Q control scheme will be used

 

To be Improved (nice to have): Improve the intra-pulse control refer to the klystron high voltage jitters if they have stronger correlations Improve the up conversion algorithm to reduce the non-linearity of the phase and amplitude actuation

2020/4/30 Zheqiao Geng, MicroTCA LLRF 28

EPICS Software Design

2020/4/30 Zheqiao Geng, MicroTCA LLRF 29

Software Architecture

2020/4/30 Zheqiao Geng, MicroTCA LLRF 30

EDM Panels – RF Station Top

2020/4/30 Zheqiao Geng, MicroTCA LLRF 31

EDM Panels – Pulse-pulse Phase Control

2020/4/30 Zheqiao Geng, MicroTCA LLRF 32

EDM Panels – LLRF Timing Settings

2020/4/30 Zheqiao Geng, MicroTCA LLRF 33

EDM Panels – RF Synchronous DAQ

2020/4/30   Save all phase and amplitude values of the RF signals for the same RF pulse synchronously up to 65536 pulses Save all waveforms for the same RF pulse synchronously up to 2048 pulses Zheqiao Geng, MicroTCA LLRF 34

EDM Panels – RF Waveforms

2020/4/30 Zheqiao Geng, MicroTCA LLRF 35

Summary of Software Design

      PAD/PAC based LLRF system has software pieces in PAD CPU, PAC CPU and VME CPU, they communicate with each other via UDP. The architecture is complex and difficult to maintain. The computation power of these CPUs are quite limited MicroTCA based LLRF system has one much more powerful CPU. Real-time Linux OS will be used and it is much more flexible to be adapted to the newest multi-core CPUs MicroTCA software is compiled to a single IOC process so the number of maintenance points is reduced Data and waveforms can be saved at 120 Hz for diagnostics. Later this function can be synchronized by Timing System for all RF stations so that the behavior of the entire machine can be analyzed within one RF pulse Software architecture is more modular and understandable  

To be improved (basic): Software should be improved to fit the extended firmware MicroTCA infrastructure (such as software development tools, EPICS base, boot up tools) needs to be improved. This topic will be covered by Charlie Xu’s talk

2020/4/30 Zheqiao Geng, MicroTCA LLRF 36

LLRF Applications

2020/4/30 Zheqiao Geng, MicroTCA LLRF 37

LLRF Applications

 

Examples of Applications for A RF Station

Measure the klystron energy no-load   Measure klystron saturation curve Measure the beam phase with beam induced signal      Calibration of the imbalance of the I/Q modulator and DAC offset Loop phase correction for intra-pulse feedback control Intra-pulse feedback gain optimization DAC waveform generation for single side band up-conversion Set klystron mode to ACC or standby

Most of the applications already exist for the old RF stations. The new LLRF software will inherit the existing algorithms and codes

2020/4/30 Zheqiao Geng, MicroTCA LLRF 38

System Test

2020/4/30 Zheqiao Geng, MicroTCA LLRF 39

ADC Noise Measurement

510-04516D 100MHz 1/4 Freq Divider 25MHz CH0 CH1 CH2 CH3 CH4 CH5 CH6 CH7 CH8 CH9 100MHz CLK2 CLK1 CLK0

At Lab At 28-2 ADC0

77.4

76.1

ADC0 ADC1

78.6

76.7

ADC1

Signal to Noise Ratio (dBFS)

ADC2 ADC3 ADC4 ADC5 ADC6

78.2

76.7

79.4

77.2

78.2

76.5

78.8

77.4

77.0

76.7

ADC2

Crosstalk Matrix (dB)

ADC3 ADC4 ADC5 ADC6 ADC7

78.8

77.6

ADC7 ADC8

79.1

77.4

ADC8 ADC9

79.2

77.4

ADC9 ADC0 ADC1 ADC2 ADC3 ADC4 ADC5 ADC6 ADC7 ADC8 ADC9

2020/4/30 99.3

0 82.7

82.3

95.8

91.5

99.2

100.2

98.9

100.5

88.9

0 84.1

94.7

93.4

101.1

101.4

102.4

102.1

102.3

87 87.1

0 81.2

83.8

95.1

94.3

103.5

105.4

104.8

97.3

98.5

86.1

0 97.7

94.9

93.6

102.4

104.6

104.8

95.2

97 87 97.1

0 81.3

81 94.9

96 100.9

111.6

105 98.6

98.5

85.5

0 88.5

97.8

106.2

108.5

96.8

98.9

85.1

92.5

0 81.7

112.2

112.9

113 109.8

98.6

101.1

87.2

0 103.5

104.6

104.4

103.9

98.6

100.4

88.9

84.7

94.7

84.3

81.1

0 106.4

106.5

101.5

92.4

Zheqiao Geng, MicroTCA LLRF 95.2

0 112.3

108.9

112.2

111.5

109.9

111.8

107.3

109.7

6570 6560 6550 6540 6530 0 2 2.019

x 10 4 2.018

2.017

2.016

2.015

0 2 I 4 Amp 4 -1.906

x 10 4 -1.907

-1.908

-1.909

6 x 10 4 8 -1.91

0 -70.9

-70.95

2 Q 4 Pha/deg 6 x 10 4 8 -71 -71.05

6 x 10 4 8 -71.1

0 2 4 6 x 10 4 8 -60 -80 -100 -120 0 -20 -40 Data Noise Floor Signal Level -140 0 5 10 15 20 25 30 Frequency / MHz 35 40 45 40 50

Phase and Amplitude Measurement Noise

119 MHz clock jitter: 257 fs integrated from 10 Hz to 40 MHz     With a -1dBFS 25.5 MHz IF input and 257 fs clock jitter, the measurement noises (in full bandwidth of ADC) are expected to be: Phase : 0.01 deg RMS Amplitude: 0.02 % RMS

Considering the bandwidth of the RF system in the view of the beam (~1.2 MHz), the measurement noises will meet the LCLS-II requirements (0.07 deg RMS phase jitter and 0.06 % RMS amplitude jitter for the most critical RF station of L1S)

2020/4/30 Zheqiao Geng, MicroTCA LLRF 41

Installation at LI28-2

SSSB RF Support Chassis Zheqiao Geng, MicroTCA LLRF MicroTCA Crate MKSUII 42 2020/4/30

System Test at LCLS Linac (LI28-2)

- RF Signal Measurement

2020/4/30 Zheqiao Geng, MicroTCA LLRF 43

RF Reference Signal

2020/4/30 Zheqiao Geng, MicroTCA LLRF 44

I/Q Modulator Output Signal

2020/4/30 Zheqiao Geng, MicroTCA LLRF 45

Klystron Drive Signal

2020/4/30 Zheqiao Geng, MicroTCA LLRF 46

Klystron Output Signal

2020/4/30 Zheqiao Geng, MicroTCA LLRF 47

SLED Output Signal

2020/4/30 Zheqiao Geng, MicroTCA LLRF 48

System Test at LCLS Linac (LI28-2)

- Reference Tracking

2020/4/30 Zheqiao Geng, MicroTCA LLRF 49

Reference Tracking

   2020/4/30 Subtract the phase of the reference signal from other RF signal phases Remove the common mode error caused by the RF detectors experiencing the same temperature fluctuations Remove the phase jump caused by the clock re-synchronization Reference signal

Reference Phase

59.5

59.4

59.3

59.2

59.1

59 0 5000 10000 15000 Klystron output signal

Klystron Output Phase

57 56 55 54 53 0 5000 10000 15000 SLED output signal

SLED Output Phase

104.5

104 103.5

103 102.5

102 0 5000 10000 15000 Time / s Zheqiao Geng, MicroTCA LLRF 50

System Test at LCLS Linac (LI28-2)

- Pulse-pulse Feedback

2020/4/30 Zheqiao Geng, MicroTCA LLRF 51

Long Term Stability with Feedback

31 30.5

30 29.5

29 0 1000 2000 3000 4000 Time / s 5000 SLED phase with feedback 6000 7000 8000 31 30.5

30 29.5

29 0 1000 2000 SLED phase without feedback 3000 4000 Time / s 5000 6000 7000 8000

2-hour phase measurement

2020/4/30 Zheqiao Geng, MicroTCA LLRF 52

System Test at LCLS Linac (LI28-2)

- Intra-pulse Feedback

2020/4/30 Zheqiao Geng, MicroTCA LLRF 53

Intra-pulse Feedback

  Phase error is estimated at the first part of the RF pulse and the correction is applied at the second part of the RF pulse Loop delay should < 1 µs  Phase jitter of different parts of the RF pulse should be correlated The entire loop delay is ~ 600 ns, quite promising for intra pulse feedback. 2020/4/30 Zheqiao Geng, MicroTCA LLRF 54

Phase Jitter Correlation

Correlation coefficient of the phase jitter between different parts of the RF pulse ~ 0.3-0.4

2020/4/30 Zheqiao Geng, MicroTCA LLRF 55

Intra-pulse Feedback Gain Sweeping

   Feedback Gain Effects of the intra-pulse feedback on the beam can only be tested at the sensitive RF stations (like L1S) Correlation of the phase jitters of different parts of the RF pulse is not so strong

How about the correlation between phase jitter and the klystron high voltage?

2020/4/30 Zheqiao Geng, MicroTCA LLRF 56

System Test at LCLS Linac (LI28-2)

- Intra-pulse phase slope compensation

2020/4/30 Zheqiao Geng, MicroTCA LLRF 57

Motivations

   At rising edge of the klystron signal, the phase changes > 100 deg, which will lower the efficiency to fill the SLED cavities Klystron signal after PSK has a phase change > 60 degree, which will lower the integrated E field seen by the beam Idea: Remove the phase slope with feed forward to possibly increase the energy gain from the klystron 2020/4/30 Zheqiao Geng, MicroTCA LLRF 58

Phase Slope Compensation with Feed Forward

2020/4/30 Zheqiao Geng, MicroTCA LLRF 59

SLED Amplitude with Feed Forward Iterations

  Amplitude of SLED output can be increased by ~ 3 % Note: There is still 10 degree phase change at the SLED output pulse during the part that interacts with the beam!

The MicroTCA based LLRF system provides a very powerful platform to implement the new ideas from physicists

2020/4/30 Zheqiao Geng, MicroTCA LLRF 60

System Test at LCLS Linac (LI28-2)

- Phasing the klystron

2020/4/30 Zheqiao Geng, MicroTCA LLRF 61

Phasing the Klystron with Matlab



Without phase feed forward control With 50 iterations of phase feed forward control

Need further measurement to clarify if we can really increase the energy gain or not! 2020/4/30 Zheqiao Geng, MicroTCA LLRF 62

More Tests Need to be Done

    Intra-pulse I/Q control need to be tested at critical RF stations like L1S before the first bunch compressor. The effects of the intra-pulse control can be examined by monitoring the beam stability Measurement of beam induced signal need to be done at LI28-2. To do this test, the klystron need to be in standby state Correlation between RF amplitude/phase jitters and klystron high voltage jitter need to be tested. This will tell us if we can use the klystron high voltage jitter for intra-pulse feedback. As mentioned before, a new klystron beam voltage conditioner board need to be designed More tests need to be done to check if the intra-pulse phase slope compensation can increase the energy gain or not 2020/4/30 Zheqiao Geng, MicroTCA LLRF 63

Summary

2020/4/30 Zheqiao Geng, MicroTCA LLRF 64

Summary of MicroTCA System

     MicroTCA introduces intra-pulse control to reduce fast jitters MicroTCA system uses PCI express for faster data acquisition which enables to save the RF waveforms at 120 Hz or even at 360 Hz MicroTCA system uses more powerful FPGA and CPU which enables to upgrade the system for 360 Hz or multi-bunch operation without changing the hardware MicroTCA system contains more ADC channels in a single board which enables to implement the reference tracking to remove the phase jump problem in digital I/Q demodulation algorithm The PAD/PAC system uses Coldfire MCU which is a bottleneck for real-time performance. The PAD-VME-PAC chain is connected with Ethernet and it is not possible to perform intra-pulse control. 120 Hz waveform saving is poorly supported as well 2020/4/30 Zheqiao Geng, MicroTCA LLRF 65

Summary of MicroTCA System (cont’d)

    MicroTCA has better upgradability: overhead in data transfer speed (PCI Express) and computation power (FPGA + multi-core CPU) MicroTCA has better maintainability: hot-swappable – reducing repair time – more availability MicroTCA has better reliability – simpler and more compact system structure; redundant MCH and power supply MicroTCA has better platform management – IPMI   MicroTCA is a new platform which needs more development efforts Cost is relatively high due to small market 2020/4/30 Zheqiao Geng, MicroTCA LLRF 66

Thank you!

2020/4/30 Zheqiao Geng, MicroTCA LLRF 67

Backup Slides

2020/4/30 Zheqiao Geng, MicroTCA LLRF 68

Phase and Amplitude Actuation Noise

   I/Q modulator Input Jitters: Phase 0.083 deg RMS, Amplitude 0.01% I/Q modulator Output Jitters: Phase 0.074 deg RMS, Amplitude 0.03% The phase and amplitude actuation does not introduce extra noise (or at least neglectable) 2020/4/30 Zheqiao Geng, MicroTCA LLRF 69