A novel test for autonomy

27/06/2023 · Technology

Learning to drive

Driving lessons in the European Union (EU) follow a structured plan that covers a diverse range of tasks, such as high-speed lane changes on the Autobahn and parking. By design, these lessons capture the breadth of driving skills required to be mastered before the learner is deemed safe for public roads.

At the writing of the blog, the current generation of AVs (Autonomous Vehicles) fall well below the safety standards set for human drivers, but they both have a common goal. Executing driving tasks safely while generalizing to new locations.

AV development is accelerated by close loop testing in simulation as a proxy for generalizing to new environments. The scenarios in simulation do not test AVs for the local intricacies (inter/intra city) and variance in the real traffic behavior (time of day).

To bridge the gap between simulation and real world, Yaak is working with driving schools and building Open Scenario Map. An open-source scenario catalogue that enables AV developers to test in unseen, unbiased expert-curated scenarios.

To source diverse and critical scenarios, we partnered driving instructors who use Yaak's data collection platform to log the driving lessons they run. The driving lessons are enriched with incidents logged by driving instructors and driving task labels added from map data, e.g. unprotected right turns with a bike lane.

Sample drive data: Scenario captured with eight cameras, GNSS, and a 5G modem.

dual policies

  1. Instructor policies: Driving instructors operate vehicles on the routes where they normally schedule their driving lessons multiple times to cover seasonal and intra-day variations.

  2. Student policies: During driving lessons, students operate the vehicle and learn to execute driving tasks. The instructor present in the vehicle uses an app to annotate driving incidents.

Incident: incorrect positioning within lane, visible in the top-row, middle camera.

To log severe incidents, driving instructors use the dual pedals to break or accelerate in the event the student's driving behavior is unsafe. 38,1% of the reported incidents are severe.

Severe Incident: Improper observance of yield sign causes the instructor to slam the brakes.

Scaling scenarios

Across 30 German cities, driving instructors add to our drive dataset (expert and student policies) every day. With a backlog of pre-orders we aim to to cover new cities and countries to further capture unseen and unbiased scenarios. Our goal is to build the world's largest open-source repository of scenarios for developing a novel test of generalization for autonomy through Open Scenario Map.

2021
2021

YEAR

1
1

Cities

1
1

Experts

893
893

Incidents

342
342

Severe incidents

4839
4839

KM