Sony AI company made a breakthrough in the field of artificial intelligence and created a so-called intelligent agent, or program, which surpassed the best drivers in the simulator Gran Turismo Sport. The authors called their development Gran Turismo Sophy (GT Sophy). It is the world’s first racing AI agent with a deep understanding of the dynamics and forces acting on the car. Sophy is able to drive far beyond human capabilities, dynamically selects the right tactics, but is also trained in the basics of sports etiquette.
GT Sophy is a joint development of Sony AI, Polyphony Digital, and Sony Interactive Entertainment. It uses a new Deep RL method based on a complex system of rewards and QR-SAC algorithm, which encodes the rules of racing and a special training mode needed to master all the nuances of driving. Usually with the help of Deep RL “pump up” the game AI in real-time network strategies, as RL agents calculate their actions far ahead, but it seems that Sony first applied this methodology in a racing simulator.
To train GT Sophy, Sony AI experts created a web-based DART (Distributed, Asynchronous Rollouts and Training) platform that had access to a thousand PlayStation 4 consoles and Sony’s cloud-based gaming platform. Each console collected the data needed for training or evaluated an already-trained version of GT Sophy. DART enabled hundreds of simultaneous simulations, each time improving the AI agent’s interaction with the environment and enhancing its driving skills.
What can Sophy do? First, she understands the dynamics of the car and instantly calculates the ideal driving trajectory. Secondly, she adopts the right strategy in fractions of a second. For example, she can make use of the under-pressure zone behind the car ahead to overtake or block her rival without letting him overtake her. Finally, Sophie is trained in the basics of sports etiquette. According to the authors, this means no intentional crashes or attempts to push the other driver off the track.
Last summer Sophie won two races of the Race Together 2021 Challenge, and in October she won the FIA Gran Turismo format race. But development doesn’t stop there. It has already been announced that Sony AI will continue to improve the GT Sophy, and will also explore ways to integrate the AI agent in future Gran Turismo series simulators.
Source: Sony AI