The World Economic Forum, Deepen AI, and WMG, an academic department at the University of Warwick, have announced the launch of the Safety Pool Scenario Database. The database houses different driving scenarios and situations that industry and academia can utilize when testing and validating new ADAS innovations and autonomous driving platforms. The Safety Pool Scenario Database is also expected to help inform lawmakers as they institute new policies and regulatory guidelines around higher levels of autonomy.
Safety, Efficiency & Inclusion
The Safety Pool Scenario Database is a living library of relevant, meaningful, and diverse use cases tagged according to their operational design domain. “Which makes them immediately actionable for OEMs, Tier1s, and system developers for testing and validation purposes,” said Nicola Croce, Technical Program Manager at Deepen AI. “Operational design domain tags are also mapped to specific test track routes or real-world routes, contributing to a seamless integration of the full spectrum of system validation: from simulation to test track to public roads.”
Individual features like the operational design domain tags hint at the larger vision behind the Safety Pool Scenario Database: one of efficiency, accessibility, and safety. With this in mind, it’s important to note how engineers and researchers often work with a great degree of vigor to benchmark their ADAS innovations. The general rule of thumb is that any next-generation offering should be lighter and more cost-effective, yet without sacrificing performance. By this logic, we could reasonably say that the better benchmarked a particular item is, the more efficient, accessible, and safer it becomes in the long term.
Though the Safety Pool Scenario Database is not a traditional product offering (like we might think of a camera or LiDAR or an entire automobile), it follows a similar train of thought. The World Economic Forum, Deepen AI, and WMG want something that helps mitigate cost without compromising efficiency or safety, which they have done with the Safety Pool Scenario Database.
“Most AV developers create their own libraries which they use to test and validate their software in simulation, but Safety Pool is unique in that it is an open, shared library accessible to any developers, which enables those with less resources to access a massive dataset without the cost of developing it themselves,” explained Tim Dawkins, Lead, Automotive & Autonomous Mobility at the World Economic Forum. “This should level the playing field in enabling all developers to access a robust dataset to ensure their vehicles can handle a range of challenges before hitting the road.”
“It is of paramount importance to have a common and shared repository worldwide where industrial actors can contribute corner cases or even accident scenarios,” Croce said. “In this way, every organization can make sure that their system is tested against those risky situations and that similar accidents never happen again. Safety Pool’s goal is to be this global reference platform, with safety, transparency, and inclusion as its foundational pillars.”
Diversification of The Dataset
As Croce explains, the Safety Pool Scenario Database starts from a significant collection – currently near 100,000 and counting – of curated scenarios related to a diverse set of use cases, such as lane-keeping assist. Each scenario is carefully curated and vetted for quality and diversity. Industrial organizations can access the public scenario archive and contribute scenarios to the Safety Pool Scenario Database. They can even access their own private section to collect and organize their test scenarios.
“The goal is for it to become the reference initiative worldwide for safe testing and deployment of AVs, and the database welcomes contributions from all of its members,” Croce continued. “Industrial players are incentivized to share new, relevant, and diverse scenarios through a tokenized mechanism that rewards them with a credit score they can use to access other scenarios.”
“Safety Pool is set up as an incentive-based brokerage to encourage industry developers to put data in and receive data in exchange,” Dawkins added. “So the cost of acquiring a broad and diverse dataset is vastly reduced, and the incentive system is intended to encourage and prioritize diversification of the dataset.”
The Value of Simulation
The launch of the Safety Pool Scenario Database coincides with the ongoing discussion about the benefits of simulation relative to achieving higher levels of autonomy. This discussion is already familiar territory for WMG. Near the end of 2020, WMG announced a project to help correlate simulated testing of autonomous applications on public roads in the UK (the Midlands Future Mobility testbed is a key part of this project).
“Safety of automated driving systems is a hard research challenge and can only be solved by national and international collaboration and knowledge sharing,” said Dr. Siddartha Khastgir, Head of Verification & Validation of Connected and Autonomous Vehicles at WMG, University of Warwick. Dr. Khastgir’s UKRI Future Leaders Fellowship is also an integral part of the aforementioned project, along with the Midlands Future Mobility testbed. “Testing and validating automated driving systems transparently in an integrated simulation-based framework and in real-world scenarios will not only provide insights into the readiness of autonomous driving systems but also speed up the adoption globally.”
“In order to overcome the impractical endeavor of driving 11 billion test miles to prove statistical safety, the AV community have converged towards scenario-based approaches to ensure the safety of the ADS systems,” Croce added. “There’s a ton of variability in what can happen on the road, with lots of edge cases being observed and collected every day.”
The Role of Law & Policy
Along with industry and academia, government officials can also access the Safety Pool Scenario Database. Like robust simulation testing, sound laws and policies are a fundamental and indispensable part of the future mobility equation (and greater future mobility ecosystem if we consider things like infrastructure, yearly traffic fatalities, and climate change).
As part of the World Economic Forum’s Shaping the Future of Mobility platform, the Safe Drive Initiative leverages a proposed framework through the Forum’s Automotive and Autonomous Mobility community that ultimately serves as a scenario-based safety assurance program for policymakers. Among the Safe Drive Initiative’s many goals is providing government officials with accurate and uniform testing data so they can make informed choices regarding the implementation of autonomous vehicles.
“Safety Pool uses a machine-readable scenario description language to annotate the parameters of each scenario to allow the AV developers to input them to their suite of development tools,” Dawkins said on the importance of the database to the Safe Drive Initiative. “Other scenario datasets also do this, but Safety Pool catalogs the scenario dataset so that non-technical people can browse the library, such as regulators.”
Safety Pool Scenario Database: Learn More
More information about the Safety Pool Scenario Database, including its Steering Committee members, can be found on its official site. Industry, academia, and government can request access to the database here.
“Across the board, our mission here at the Forum is to improve the state of the world, and that means making sure that the future of mobility is safe, clean, and inclusive,” Dawkins said. “We view technologies like autonomous vehicles and ADAS as essential enablers to that mission, and hence we work with industry and governments to collaborate and share knowledge, so the benefits of these technologies are realized as soon as possible.”
“With the launch of Safety Pool Scenario Database, we are inching closer to seeing automated driving systems on the roads,” Dr. Khastgir added. “We are excited to be at the forefront of this revolution.”
