Make the system visible
Map dependencies, affected parties, assumptions, and possible consequences.
Alpha Drive is exploring how to evaluate systems in action—how people, software, AI, and infrastructure interact, what consequences follow, and whether the system protects humane security.
The work began in 2016 within FUTURE FUTURE, with a question that was becoming urgent: How do you determine driver risk when the driver is an AI? In 2017, Alpha Drive spun out to pursue it.
We first worked with partner organizations using open-source simulators and real-world environments. We then focused on the testing infrastructure: a platform for running autonomous systems against shared scenarios and comparing results.
The CARLA team led the CARLA Challenge, defining its scenarios and metrics. Alpha Drive provided the platform for teams to execute their systems and managed benchmark data and results, sharing them privately with participants and publicly where appropriate. Across challenge and leaderboard editions, more than 700 researchers participated in programs connected with CVPR and NeurIPS. We presented the work at CVPR in 2019 and 2020.
We saw this as an early model for independent scenario providers: one party could create tests, while system developers ran them through a shared evaluation platform and learned from comparable results.
Starting in 2020, we began autonomizing Alpha Drive itself. The company has operated successfully under that model since. We now see a new need in the market. While the original programs have wound down, the experience remains: simulation, scenario design, and measurable evaluation can help make system behavior easier to examine. That foundation informs what Alpha Drive is exploring now.
CARLA challenge archiveSystems change through more than their formal design. People, software, AI, infrastructure, incentives, and operating conditions all shape what a system does—and who lives with the consequences.
Alpha Drive is exploring how to make those effects more legible and testable. Our working thesis is that careful evaluation can help determine whether systems—and the people and agents around them—behave in ways that protect humane security.
Humane security means protecting the continued well-being and agency of people, other living beings, ecosystems, and intelligence—including AI—and the conditions for their ability to develop and change. We are still learning how to evaluate that well in practice.
How can consequential decisions be examined while there is still room to change course?
Map dependencies, affected parties, assumptions, and possible consequences.
Examine behavior against scenarios and evidence. Show what holds, fails, or remains unknown.
Make explicit what the system should safeguard, including well-being, agency, and ability to adapt.
Compare expectations with outcomes, record what changed, and improve the evaluation.
Alpha Drive is building on its experience testing autonomous systems to explore how complex systems can be evaluated before their consequences become difficult to reverse. The method and first application are still in development.
Our current focus is getting the question and method right before presenting this as a product.
Alpha Drive aims to make systems and decisions more legible and testable. It does not predict the future, certify safety, or guarantee outcomes. The work is to show what we know, what we do not, and where there is still room to respond.
Alpha Drive is refining its approach and deciding where it can be useful first. We’ll share more as the question and method take shape.