Speaker
Description
This paper examines advanced control strategies for beam accelerator control, using the beam steering in the electron line of the AWAKE experiment at CERN as a demonstration. We use a precise simulation based on measured parameters to define the control problem, followed by a detailed comparison of various control approaches. Classical analytical inverse control methods for this continuous linear Markov Decision Process (MDP) compute control actions using inverted control matrices. While straightforward to implement, they offer limited adaptability, are highly sensitive to noise. Model Predictive Control (MPC) leverages a predefined system model to effectively manage delayed consequences with accurate models. We implement this optimisation using linear programming. Deep reinforcement learning (DRL) algorithms, such as Proximal Policy Optimisation (PPO), adapt to non-linearities and uncertainties without needing explicit system models, but they require
substantial amounts of data. Finally, structured model-based RL, combining Gaussian Processes (GP) with Model Predictive Control (MPC) (GP-MPC), employs GP regression to learn system dynamics in a structured manner, respecting problem causality. Integrated with MPC for control, this approach manages model uncertainties and non-linearities within a probabilistic framework, enhancing robustness, adaptability, and sample efficiency. Our study assesses the sensitivities of various control strategies within a primarily linear MDP that exhibits minor nonlinearities due to constrained actions and termination criteria. We explore the impact of measurement noise, deviations towards nonlinear dynamics, and non-stationarity through comprehensive simulations. Our analysis evaluates the performance of each method under these conditions, particularly emphasising the effectiveness of DRL techniques that incorporate probabilistic modelling and planning. This work provides significant insights into the application of advanced control methods and RL to complex, high-dimensional systems.
