DeepMind has announced the use of reinforcement learning to control the plasma state during a thermonuclear reaction. The DeepMind algorithm made it possible to increase the stability of the process of controlled thermonuclear fusion on a tokamak.
Nuclear fusion is a promising source of virtually unlimited energy. In terrestrial conditions, the temperatures required for the implementation of thermonuclear fusion should exceed 100 million degrees Celsius. Such temperatures are reached in a tokamak, which can contain plasma hotter than the core of the Sun. However, plasma in tokamaks is unstable, which makes it difficult to maintain the process necessary for nuclear fusion. To ensure that the plasma never touches the walls of the tokamak, which will lead to heat loss (and possibly damage), the control system must coordinate the coils and adjust the voltage on them thousands of times per second.
To solve this problem, DeepMind collaborated with the Swiss Plasma Center EPFL. The DeepMind reactor control system is trained using simulations performed by EPFL researchers. In thermonuclear fusion research, the need for simulation is due to the fact that currently operating reactors can maintain plasma in one experiment only for a few seconds, after which they need time to restart.
The DeepMind system learned how to operate a tokamak — in particular, a TCV tokamak at the Swiss Plamsa center – in the process of modeling a tokamak on a timeline of about several hours, and then transferred these skills to a real TCV. While traditional controllers use algorithms to evaluate plasma properties in real time and adjust the magnet voltage accordingly, the DeepMind system uses a single algorithm to control all 19 tokamak coils simultaneously, automatically determining which voltages are best suited to achieve the “plasma configuration” directly from the sensors.
As a demonstration, DeepMind showed for the first time that the system can control aspects of plasma with a single controller. The laboratory then used the system to create a number of plasma structures, which are being studied by plasma physicists for their usefulness for energy generation.
