Self-driving cars are considered the technology of the future and have attracted the attention of many major companies over the past 5 years, including Apple. While we do have some automated cars, this technology still has several limitations that need to be addressed in the future. Most importantly, ensuring user safety must take precedence. Additionally, the issue of cost is a significant barrier preventing self-driving cars from becoming truly widespread globally.
However, all these hurdles can be easily resolved if you bring this technology into… video games. Recently, 15-year-old programmer Leon Hillman utilized the open-source code from Comma.ai, a company specializing in self-driving car research, and applied it to the game Grand Theft Auto 5 (GTA 5).
If you didn’t know, controlling vehicles, especially cars, is an essential part of any version of the GTA series. However, the problem is that most players will never adhere to traffic laws in these games. As soon as they get in a car, they will immediately speed straight to their mission location without caring about the roads or traffic conditions, sometimes causing many catastrophic virtual accidents.
Not to mention some gamers are so lazy that they don’t even want to drive at all and are willing to pay a little money to take a taxi instead. But the downside of this feature is that you won’t get to enjoy the stunning scenery that GTA creates; only those who drive themselves can appreciate it. Therefore, Leon’s initiative can be seen as a perfect solution, allowing you to keep your hands free from controlling the car while comfortably enjoying the virtual street scenery in GTA 5.
Car crashes and accidents are an “essential specialty” in the GTA series.
On the other hand, Comma.ai has recently allowed users to equip additional software called Openpilot for Honda and Toyota models. This software can automatically detect lanes and control speed, making it suitable for use in everyday traffic as well as on highways. Moreover, Openpilot is open source, allowing anyone to integrate support for various other vehicles.
Inside the vehicle, Comma.ai equips a camera system that both displays the driver’s image and provides a view of the road; a tablet to relay notifications to the driver; and a cable system that connects the vehicle to the Openpilot software. The official Comma.ai website explains the basic operating mechanism of this software: “Openpilot works by collecting data from radar integrated within the vehicle and combining it with the camera integrated into the hardware to determine tasks such as acceleration, braking, or steering.”
Comma.ai is a company specializing in self-driving car research.
Returning to our teen “hacker”, Leon “littlemountainman” Hillmann is a programmer living in Munich. He shared that the biggest challenge in integrating Openpilot with GTA 5 was enabling the two programs to recognize and interact with each other. Openpilot collects data from both the game’s camera (the player’s perspective) and the cameras on the virtual cars within the game.
Leon stated: “Cars in GTA do not have CAN (control area network) – a very efficient serial communication protocol for real-time control applications with a high level of safety; or the IMU (inertial measurement unit) module, which includes accelerometers and gyroscope sensors. Therefore, the first step I had to take was to create an entirely new vehicle interface for GTA, based on a 2018 Honda Civic, before I could integrate Openpilot into the game.”
Leon spent about 2 weeks writing code for tasks such as braking, refueling, and steering control. After that, he continued to process everything he programmed to work with the Xbox controller. Now, he has his own version of Openpilot for GTA 5.
Leon had to use 2 different computers along with 1 webcam to complete his project.
Sharing on GitHub, he stated: “You will need 2 PCs, 1 running Windows with all the Xbox drivers, and 1 running Ubuntu 16.04, connected to a webcam to run Openpilot. I think this webcam should have a minimum resolution of 1080p; personally, I use the Logitech C920, but the quality is still not very outstanding.”
The idea of bringing self-driving technology into video games has actually been around for a few years. Some companies have even applied this method to train their AI. This is a safe approach to minimize accidents involving people during the testing of this technology.
According to VICE