Latest news with #Zenseact
Forbes
3 days ago
- Automotive
- Forbes
Volvo: Gaussian Splatting Is Our Secret Ingredient For Safer Cars
For decades, the Volvo brand has been synonymous with safety. But keeping passengers secure is no longer just about a strong cabin or cleverly designed crumple zones. Increasingly, safety is about semi-autonomous driving technology that can mitigate collisions or even avoid them entirely. Volvo intends to be ahead of the game in this era too. Its secret weapon? Something called 'Gaussian Splatting'. I asked Volvo's Head of Software Engineering Alwin Bakkenes and subsidiary Zenseact's VP Product Erik Coelingh exactly what this is and why it's so important. 'We have a long history of innovations based on data,' says Bakkenes. 'The accident research team from the 70s started with measuring tapes. Now in the digital world we're collecting millions of real-life events. That data has helped us over the years to develop a three-point safety belt and the whiplash protection system. Now, we can see from the data we collect from fleets that a very large portion of serious accidents happen in the dark on country roads where vulnerable road users are involved. That's why, with the ES90 that we just launched, we are also introducing a function called lighter AAS where we have enabled the car to steer away from pedestrians walking on the side of the road or cyclists, which in the dark you can't see even if you have your high beam on. This technology picks that up earlier than a human driver.' The Volvo EX90 SUV will also benefit from this technology. 'If you want to lead in collision avoidance and self-driving, you need to have the best possible data from the real world,' adds Bakkenes. 'But everyone is looking also at augmenting that with simulated data. The next step is fast automation, so we're using state-of-the-art end-to-end models to achieve speed in iterations. But sometimes these models hallucinate. To avoid that, we use our 98 years of safety experience and these millions of data points as guardrails to make sure that the car behaves well because we believe that when you start to automate it needs to be trusted. For us every kilometer driven with Pilot Assist or Pilot Assist Plus needs to be safer than when you've driven it yourself. In the world of AI data is king. We use Gaussian Splatting to enhance our data set.' 'Cars are driven all around the world in different weather and traffic conditions by different people,' says Coelingh. 'The variation is huge. We collect millions of data points, but it's still a limited amount compared to reality. Gaussian Splatting is a new technology that some of our PhD students have been developing the last few years into a system where you can take a single data point from the real world where you have all the sensor, camera, radar and LIDAR sequences and then blow it up into thousands or tens of thousands of different scenarios. In that way, you can get a much better representation of the real world because we can test our software against this huge variation. If you do it in software, you can test much faster, so then you can iterate your software much more quickly and improve our product.' 'Gaussian Splatting is used in different areas of AI,' continues Coelingh. 'It comes from the neural radiance fields from nerves.' The original version worked with static images. 'The first academic paper was about a drum kit where somebody took still pictures from different angles and then the neural net was trained on those pictures to create a 3D model. It looked perfect from any angle even though there was only a limited set of pictures available. Later that technology was expanded from 3D to 4D space-time, so you could also do it on the video set. We now do this not just with video data, but also with LiDAR and radar data.' A real-world event can be recreated from every angle. 'We can start to manipulate other road users in this scenario. We can manipulate real world scenarios and do different simulations around this to make sure that our system is robust to variations.' Volvo uses this system particularly to explore how small adjustments could prevent accidents. 'Most of the work that we do is not about the crash itself,' says Coelingh. 'It's much more about what's happening 4-5 seconds before the crash or potential crash. The data we probe is from crashes, but it's also from events where our systems already did an intervention and in many cases those interventions come in time to prevent an accident and in some cases they come late and we only mitigated it. But all these scenarios are relevant because they happen in the real world, and they are types of edge case. These are rare, but through this technology of Gaussian Splatting, we can go from a few edge cases to suddenly many different edge cases and thereby test our system against those in a way that we previously could not.' This is increasingly important for addressing the huge variation in global driving habits and conditions a safety system will be expected to encounter. 'Neural Nets are good at learning these types of patterns,' says Coelingh. 'Humans can see that because of the behavior of a car the driver is talking into their phone, either slowing down or wiggling in the lane. If you have an end-to-end neural network using representations from camera images, LiDAR and radar, it will anticipate those kinds of things. We are probing data from cars all around the world where Volvo Cars are being driven.' The system acts preemptively, so it can perform a safety maneuver for example when a pedestrian appears suddenly in the path of the vehicle. 'You have no time to react,' says Coelingh. Volvo's safety system will be ready, however. 'Even before that, the car already detects free space. It can do an auto steer and it's a very small correction. It doesn't steer you out of lane. It doesn't jerk you around. It slows down a little bit and it does the correction. It's undramatic, but the impact is massive. Oncoming collisions are incredibly severe. Small adjustments can have big benefits.' Volvo has developed one software platform to cover both safety and autonomy. 'The software stack that we develop is being used in different ways,' says Coelingh. 'We want the driver to drive manually undisturbed unless there's a critical situation. Then we try to assist in the best possible way to avoid collision, either by warning, steering, auto braking or a combination of those. Then we also do cruising or L2 automation.' Volvo demonstrated how it has been using Gaussian Splatting at NVIDIA's GTC in April. 'We went deeply into the safe automation concept,' says Bakkenes. 'Neural nets are good at picking up things that you can't do in a rule-based system. We're developing one stack based on good fleet data which has end-to-end algorithms to achieve massive performance, and it has guard rails to make sure we manage hallucinations. It's not like we have a collision avoidance stack and then we have self-driving stack.' 'There was a conscious decision that if we improve performance, then we want the benefits of that to be both for collision avoidance in manual driving and for self-driving,' says Coelingh. 'We build everything from the same stack, but the stack itself is scalable. It's one big neural network that we can train. But then there are parts that we can deploy separately to go from our core premium ADAS system all the way to a system that can do unsupervised automation. Volvo's purpose is to get to zero collisions, saving lives. We use AI and all our energy to get there.'
Yahoo
25-03-2025
- Automotive
- Yahoo
Will This Tech Replace Real-World Safety Tests?
Volvo is using AI to create life-like virtual environments to test its ADAS technology that used to require on-road testing in controlled environments. The automaker is using Gaussian splatting to create realistic 3D environments and subjects sourced from real-world visual data to train its systems. The technology, which was unavailable a decade ago, now promises to train ADAS and autonomous software quicker and more thoroughly by training it on "edge cases." A decade ago on-road testing was seen as an irreplaceable part of the development of advanced driver assistance systems (ADAS), with automakers racking up tens of thousands of miles to fine-tune various features. But slowly, the virtual world has been taking over this arena, especially when it comes to the development of higher levels of autonomous driving. Volvo is now using AI to create life-like, virtual worlds to aid in the development of its safety software including ADAS features—something that wasn't available just a few years ago. Specifically, the automaker is using a method called Gaussian splatting to create from scratch realistic 3D environments and subjects sourced from real-world visual data collected by cars. What's more, these environments can be easily tweaked as needed, adding or changing the behavior and number of various road users and traffic patterns to test out how a safety system would react in various scenarios. "We already have millions of data points of moments that never happened that we use to develop our software," says Alwin Bakkenes, Head of Global Software Engineering at Volvo Cars. The footage of the environments themselves wouldn't fool a human, but it's realistic enough for camera-based ADAS to train themselves, and for developers to examine the various systems' reactions. One of the advantages of generating realistic but artificial environments with minute variations is to discover dangerous and improbable edge cases—a term that has received plenty of attention in the autonomous vehicle sphere in recent years. Encountering such rare scenarios in traffic would ordinarily take tens of thousands of miles of driving data, but reproducing them in virtual worlds while adding small variations in each iteration helps Volvo train its safety systems. The environments were created with help from Zenseact, an AI and software startup owned by Volvo. "Thanks to Gaussian splatting we can select one of the rare corner cases and explode it into thousands of new variations of the scenario to train and validate our models against," Bakkenes adds. "This has the potential to unlock a scale that we've never had before and even to catch edge cases before they happen in the real world." Still, there are limits to virtual environments, even very photorealistic ones, as more advanced ADAS now rely on multiple camera, lidar, and radar inputs simultaneously. So the amount of information going into ADAS decision-making is expanding as well. If there is another limit to AI-created environments, it's that they are not speeding up the creation of a single regulatory jurisdiction for higher-level autonomous systems even in the US, even as they are speeding up and improving testing. Of course, such techniques are not a 100% replacement for real-world, on-road testing. So Volvo and other automakers aren't relying entirely on 3D, AI-generated environments, even with a number of companies already specializing in creating such worlds for testing autonomous systems, including SAE Level 3 and Level 4 systems. But the auto industry could be approaching a stage where the majority of time and miles are accumulated in completely artificial environments, with various systems being trained and validated on hundreds of thousands of miles of simulated highways and city streets. Will we see a single, national standard emerge in the US for autonomous systems in the coming years, including SAE Level 3 that Volvo and Polestar want to offer, or will such systems remain governed on a state-by-state basis? Let us know what you think in the comments below.
