This presentation, “Using Apache Airflow to Automate Autonomous Driving Tests” by Bosch, details the significant challenges of testing software for autonomous vehicles and how Apache Airflow provides a robust solution.
The core problem lies in the sheer scale of testing required: to statistically prove that autonomous vehicles are safer than human drivers (e.g., 20% lower fatality rate with 95% confidence), an astronomical 14 billion kilometers of testing is needed. Physical testing alone would take 400 years with a fleet of 100 vehicles operating non-stop, making it impractical and statistically impossible for ensuring safety. Moreover, the “chaos of reality” (as illustrated by a chaotic street scene) demands testing across an immense number of complex scenarios. Standard CI/CD tools fall short here, as they are designed for short-lived code builds, not the massive test volumes, dynamic workflows, complex dependencies, and specialized hardware environments inherent to autonomous driving development.
Bosch, in collaboration with Cariad through the “Automated Driving Alliance,” adopted Apache Airflow as their orchestrator to manage thousands of parallel test executions. Airflow was chosen for its large community, Python-based workflow-as-code approach, enterprise-readiness, scalability, vendor/tech neutrality (Kubernetes, Spark/Hadoop, Docker), and Apache license, avoiding vendor lock-in. They even leverage Airflow for “Edge Worker” deployments to manage testing on remote sites with specialized hardware.
Key lessons learned include the importance of building on mature products rather than developing in-house solutions, leveraging the community for support, and prioritizing upstream contributions to minimize custom code. Bosch actively contributes to Airflow, helping shape its development in a direction that meets their critical safety needs.
Indeed, Bosch has been an extremely active contributor to the Apache Airflow project, making over 900 contributions, including new features, bug fixes, and improvements. They are deeply involved in the Airflow community through conferences, podcasts, and discussions, demonstrating a strong commitment to open-source collaboration and development. This extensive contribution highlights how they not only use Airflow but also actively help evolve it to meet the demanding requirements of autonomous driving test automation.
