Join the AI Driving Olympics, 6th edition, starting now!

Posted on November 1, 2021 | by Ivano Marocchi

The 2021 AI Driving Olympics

Compete in the 2021 edition of the Artificial Intelligence Driving Olympics (AI-DO 6)!

The AI-DO serves to benchmark the state of the art of artificial intelligence in autonomous driving by providing standardized simulation and hardware environments for tasks related to multi-sensory perception and embodied AI.

Duckietown traditionally hosts AI-DO competitions biannually, with finals events held at machine learning and robotics conferences such as the International Conference on Robotics and Automation (ICRA) and the Neural Information Processing Systems (NeurIPS).

AI-DO 6 will be in conjunction with NeurIPS 2021 and have three leagues: urban driving, advanced perception, and racing. The winter champions will be announced during NeurIPS 2021, on December 10, 2021!

Urban driving league

The urban driving league uses the Duckietown platform and presents several challenges, each of increasing complexity.

The goal in each challenge is to develop a robotic agent for driving Duckiebots “well”. Baseline implementations are provided to test different approaches. There are no constraints on how your agents are designed.

Each challenge adds a layer of complexity: intersections, other vehicles, pedestrians, etc. You can check out the existing challenges on the Duckietown challenges server.

AI-DO 2021 features four challenges: lane following (LF), lane following with intersections (LFI), lane following with vehicles (LFV) and lane following with vehicles and intersections, multi-body, with full information (LFVI-multi-full).

All challenges have a simulation and hardware component (,), except for LFVI-multi-full, which is simulation () only.

The first phase (until Nov. 7) is a practice one. Results do not count towards leaderboards.

The second phase (Nov. 8-30) is the live competition and results count towards official leaderboards.

Selected submissions (that perform well enough in simulation) will be evaluated on hardware in Autolabs. The submissions scoring best in Autolabs will access the finals.

During the finals (Dec. 1-8) one additional submission is possible for each finalist, per challenge.

Winners (top 3) of the resulting leaderboard will be declared AI-DO 2021 winter champions and celebrated live during NeurIPS 2021. We require champions to submit a short video (2 mins) introducing themselves and describing their submission.

Winners are invited to join (not mandatory) the NeurIPS event, on December 10th, 2021, starting at 11.25 GMT (Zoom link will follow).

Overview

🎯	Goal: develop robotic agents for challenges of increasing complexity
🚙	Robot: Duckiebot (`DB21M/J`)
👀	Sensors: camera, wheel encoders

Schedule

	Practice: Nov. 1-7
	Competition: Nov. 8-30
	Finals: Dec. 1 – 8
	Winners: Dec. 10

Rules

	Practice: unlimited non-competing submissions
	Competition: best in sim are evaluated on hardware in Autolabs
	Finals: one additional submission for Autolabs
	Winners: 2 mins video submission description for NeurIPS 2021 event.

The challenges

Lane following

LF – The most traditional of AI-DO challenges: have a Duckiebot navigate a road loop without intersection, pedestrians (duckies) nor other vehicles. The objective is to travel the longest path in a given time while staying in the lane, i.e., not committing driving infractions.

Current AI-DO leaderboards: LF-sim-validation, LF-sim-testing.

Previous AI-DO leaderboards: sim-validation, sim-testing, real-validation.

Lane following with intersections

LFI – This challenge builds upon LF by increasing the complexity of the road network, now featuring 3 and/or 4-way intersections, defined according to the Duckietown appearance specifications. Traffic lights will not be present on the map. The objective is to drive the longest distance while not breaking the rules of the road, now more complex due to the presence of traffic signs.

Current AI-DO leaderboards: LFI-sim-validation, LFI-sim-testing.

Previous AI-DO leaderboards: sim-validation, sim-testing.

Lane following with vehicles

LFV – In this traditional AI-DO challenge, contestants seek to travel the longest path in a city without intersections nor pedestrians, but with other vehicles on the road. Non-playing vehicles (i.e., not running the user’s submitted agent) can be in the same and/or opposite lanes and have variable speed.

Current AI-DO leaderboards: LFV-sim-validation, LFV-sim-testing.

Previous AI-DO leaderboards: (LFV-multi variant): sim-validation, sim-testing, real-validation.

Lane following with vehicles and intersections (stateful)

LFVI-multi-full – this debuting challenge brings together roads with intersections and other vehicles. The submitted agent is deployed on all Duckiebots on the map (-multi), and is provided with full information, i.e., the state of the other vehicles on the map (-full). This challenge is in simulation only.

Leaderboards: LFVI_multi_full-sim-validation

Getting started

All you need to get started and participate in the AI-DO is a computer, a good internet connection, and the ambition to challenge your skills against the international community!

We provide webinars, operation manuals, and baselines to get started.

May the duck be with you!

Thank you to our generous sponsors!

AI Driving Olympics 5th edition: results

Posted on December 19, 2020 | by Ivano Marocchi

AI-DO 5: Urban league winners

This year’s challenges were lane following (LF), lane following with pedestrians (LFP) and lane following with other vehicles, multibody (LFV_multi).

Let’s find out the results in each category:

LF

Andras Beres 🇭🇺
Zoltan Lorincz 🇭🇺
András Kalapos 🇭🇺

LFP

Bea Baselines 🐤
Melisande Teng 🇨🇦
Raphael Jean 🇨🇦

LFV_multi

Robert Moni 🇭🇺
Márton Tim 🇭🇺
Anastasiya Nikolskay 🇷🇺

Congratulations to the Hungarian Team from the Budapest University of Technology and Economics for collecting the highest rankings in the urban league!

Here’s how the winners in each category performed both in the qualification (simulation) and in the finals running on real hardware:

Andras Beres - Lane following (LF) winner

Melisande Teng - Lane following with pedestrians (LFP) winner

Robert Moni - Lane following with other vehicles, multibody (LFV_multi) winner

AI-DO 5: Advanced Perception league winners

Great participation and results in the Advanced Perception league! Check out this year’s winners in the video below:

AI-DO 5 sponsors

Many thanks to our amazing sponsors, without which none of this would have been possible!

Stay tuned for next year AI Driving Olympics. Visit the AI-DO page for more information on the competition and to browse this year’s introductory webinars, or check out the Duckietown massive open online course (MOOC) and prepare for next year’s competition!

AI-DO 5 leaderboard update

Posted on December 9, 2020 | by Jacopo Tani

AI-DO 5 pre-finals update

With the fifth edition of the AI Driving Olympics finals day approaching, 1326 solutions submitted from 94 competitors in three challenges, it is time to glance over at the leaderboards!

Leaderboards updates

This year’s challenges are lane following (LF), lane following with pedestrians (LFP) and lane following with other vehicles, multibody (LFV_multi). Learn more about the challenges here. Each submission can be sent to multiple challenges. Let’s look at some of the most promising or interesting submissions.

The Montréal menace

Raphael Jean at Mila / University of Montréal is a new entrant for this year.

An interesting submission: submission #12962

All of raph’s submissions.

The submissions from the cold

Team JetBrains from Saint Petersburg was a winner of previous editions of AI-DO. They have been dominating the leaderboards also this year.

Interesting submissions: submission #12905

All of JetBrains submissions: JBRRussia1.

BME Conti

PhD student Robert Moni (BME-Conti) from Hungary.

Interesting submissions: submission #12999

All submissions: timur-BMEconti

Deadline for submissions

The deadline for submitting to the AI-DO 5 is 12am EST on Thursday, December 10th, 2020. The top three entries (more if time allows) in each simulation challenge will be evaluated on real robots and presented at the finals event at NeurIPS 2020, which happens at 5pm EST on Saturday, December 12.

Join the AI Driving Olympics now!

Test your skills in the urban league challenges and qualify for the finals at NeurIPS!

AI-DO 5

AI-DO 5 Update

Posted on December 5, 2020 | by Andrea Censi

AI-DO 5 Update

AI-DO 5 is in full swing and we want to bring you some updates: better graphics, more maps, faster and more reliable backend and an improved GUI to submit to challenges!

Challenges visualization

We updated the visualization. Now the evaluation produces videos with your name and evaluation number (as below).

Challenges updates

We fixed some of the bugs in the simulator regarding the visualization (“phantom robots” popping in and out).

We updated the maps in the challenges to have more variety in the road network; we put more grass and trees to make the maps more joyful!

Faster and more reliable backend

The server was getting slow given the number of submissions, and sometime the service was unavailable. We have revamped the server code and added some backend capacity to be more fault-tolerant. It is now much faster!

Thanks so much to the participants that helped us debug this problem!

More evaluators

We brought online many more CPU and GPU evaluators. We now encourage you to submit more often as we have a lot more capacity.

Submit to testing challenges

We also remind you that the challenges on the front page are the validation challenges, in which everybody can see the output. However what counts for winning are the testing challenges!

To do that you can use dts challenges submit with the –challenges option

Or, you can use a new way using the website that we just implemented, described below.

Submitting to other challenges

Step 1: Go to your user page, by clicking “login” and then going to “My Submissions”.

Step 2: In this page you will find your submissions grouped by “component”.

Click the component icon as in the figure.

Step 3: The page will contain some buttons that allow you to submit to other challenges that you didn’t submit to yet.

Join the AI Driving Olympics, 5th edition, starting now!

Posted on November 5, 2020 | by Ivano Marocchi

Compete in the 5th AI Driving Olympics (AI-DO)

The 5th edition of the Artificial Intelligence Driving Olympics (AI-DO 5) has officially started!

Duckietown hosts AI-DO competitions biannually, with finals events held at machine learning and robotics conferences such as the International Conference on Robotics and Automation (ICRA) and the Neural Information Processing Systems (NeurIPS).

The AI-DO 5 will be in conjunction with NeurIPS 2020 and have two leagues: Urban Driving and Advanced Perception.

Urban driving league challenges

This year’s Urban League includes a traditional AI-DO challenge (LF) and introduces two new ones (LFP, LFVM).

Lane Following (LF)

The most traditional of AI-DO challenges: have a Duckiebot navigate a road loop without intersection, pedestrians (duckies) or other vehicles. The objective is traveling the longest path in a given time while staying in the lane.

Lane following with Pedestrian (LFP)

The LFP challenge is new to AI-DO. It builds upon LF by introducing static obstacles (duckies) on the road. The objectives are the same as for lane following, but do not hit the duckies!

Lane Following with Vehicles, multi-body (LFVM)

In this traditional AI-DO challenge, contestants seek to travel the longest path in a city without intersections nor pedestrians, but with other vehicles on the road. Except this year there’s a twist. In this year’s novel multi-body variant, all vehicles on the road are controlled by the submission.

Getting started: the webinars

We offer a short webinar series to guide contestants through the steps for participating: from running our baselines in simulation as well as deploying them on hardware. All webinars are 9 am EST and free!

Introduction

Learn about the Duckietown project and the Artificial Intelligence Driving Olympics.

ROS baseline

How to run and build upon the “traditional” Robotic Operation System (ROS) baseline.

Local development

On the workflow for developing and deploying to Duckiebots, for hardware-based testing.

RL baseline

Learn how to use the Pytorch template for reinforcement learning approaches.

IL baseline

Introduction to the Tensorflow template, use of logs and simulator for imitation learning.

Advanced sensing league challenges

Previous AI-DO editions featured: detection, tracking and prediction challenges around the nuScenes dataset.

For the 5th iteration of AI-DO we have a brand new lidar segmentation challenge.

The challenge is based on the recently released lidar segmentation annotations for nuScenes and features an astonishing 1,400,000,000 lidar points annotated with one of 32 labels.

We hope that this new benchmark will help to push the boundaries in lidar segmentation. Please see https://www.nuscenes.org/lidar-segmentation for more details.

Furthermore, due to popular demand, we will organize the 3rd iteration of the nuScenes 3d detection challenge. Please see https://www.nuscenes.org/object-detection for more details.

AI-DO 5 Finals event

The AI-DO finals will be streamed LIVE during 2020 edition of the Neural Information Processing Systems (NeurIPS 2020) conference in December.

Learn more about the AI-DO here.

Thank you to our generous sponsors!

The Duckietown Foundation is grateful to its sponsors for supporting this fifth edition of the AI Driving Olympics!

AI-DO 3 – Urban Event Winners

Posted on January 14, 2020 | by Kirsten Bowser

In case you missed it AI-DO 3 has come and gone. Interested in reliving the competition? Here’s the video.

We had a great time at NeurIPS hosting the Third Edition of the AI Driving Olympics. As usual the sound of Duckies attracted an engaging and supportive crowd.

Racing Event

The competition began with the Racing Event, hosted by AWS DeepRacer. They ran their top 10 submissions and selected the winner by who could complete the fastest lap.

Racing Event Winner
Ayrat Baykov at 8:08 seconds

Advanced Perception Event

The winners of the Advanced Perception Event hosted by APTIV and the nuScenes dataset were announced. Luckily a member of the winning team was present to accept the award.

Rank 3
CenterTrack – Open and Vision

Rank 2
VV_Team

Rank 1
StanfordlPRL-TRI

Urban Event

The competition culminated with Duckietown’s own Urban Driving Event, where we ran the top submissions for each of the three challenges on our competition tracks.

Winners

Lane Following

JBRRussia1: Konstantin Chaika, Nikita Sazanovich, Kirill Krinkin, Max Kuzmin

Lane Following with Vehicles

phmarm

Lane Following with Vehicles and Intersections

frank_qcd_qk

Final Scoreboard

A few pictures from the event

: AWS DeepRacer competition

: The crowd at AI-DO3

: Command central

: It is all action in Duckietown

: Trying to get a glimpse of the action

: Urban Event – live finals!

: Winner LFVI frank_qcd_qk

: Winner LFV phmarm

: Duckietown Team

: The Duckietown Team

Congratulations to all the winners and thanks for participating in the competition. We look forward to seeing you for AI-DO 4!

Round 3 of the the AI Driving Olympics is underway!

Posted on October 29, 2019 | by Liam Paull

The AI Driving Olympics (AI-DO) is back!

We are excited to announce the launch of the AI-DO 3, which will culminate in a live competition event to be held at NeurIPS this Dec. 13-14.

The AI-DO is a global robotics competition that comprises a series of events based on autonomous driving. This year there are three events, urban (Duckietown), advanced perception (nuScenes), and racing (AWS Deepracer). The objective of the AI-DO is to engage people from around the world in friendly competition, while simultaneously benchmarking and advancing the field of robotics and AI.

Check out our official press release.

Learn more about the AI-DO competition here.

If you've already joined the competition we want to hear from you!

Share your pictures on facebook and twitter.

Congratulations to the winners of the second edition of the AI Driving Olympics!

Posted on June 3, 2019 | by Kirsten Bowser

Team JetBrains came out on top on all 3 challenges

It was a busy (and squeaky) few days at the International Conference on Robotics and Automation in Montreal for the organizers and competitors of the AI Driving Olympics.

The finals were kicked off by a semifinals round, where we the top 5 submissions from the Lane Following in Simulation leaderboard. The finalists (JBRRussia and MYF) moved forward to the more complicated challenges of Lane Following with Vehicles and Lane Following with Vehicles and Intersections.

If you couldn’t make it to the event and missed the live stream on Facebook, here’s a short video of the first run of the JetBrains Lane Following submission.

Thanks to everyone that competed, dropped in to say hello, and cheered on the finalists by sending the song of the Duckie down the corridors of the Palais des Congrès.

A few pictures from the event

: the validation tracks

: Team meeting

: Competitors at work

: DuckiePond

: Duckietown Junior

: A young Forbes roboticist

: Street name matching game!

: Street name matching game!

: Street name matching game!

: Street name matching game!

: Street name matching game!

: Street name matching game!

: Countdown to the finals

: Last minute touches on the slides

: Platform gurus

: Intro to the AI-DO, it’s as easy as a few lines of code

: The audience of AIDO2

: JetBrains saying a word or two

: Group shot with our winners!

Don't know much about the AI Driving Olympics?

It is an accessible and reproducible autonomous car competition designed with straightforward standardized hardware, software and interfaces.

Get Started

Step 1: Build and test your agent with our available templates and baselines

Step 2: Submit to a challenge

https://challenges.duckietown.org

Check out the leaderboard

View your submission in simulation

Step 3: Run your submission on a robot

https://get.duckietown.org/

in a Robotarium

AI-DO Robotarium Evaluations Underway

Posted on May 16, 2019 | by Liam Paull

We have started evaluating the submissions in our “robotarium”:

Duckiebot onboard camera feed	Robotarium watchtower camera feed

To queue your submissions for robotarium evaluation, please follow these instructions:

You need to use the –challenge option to specify 3 challenges: the two simulated ones (testing and validation) and the hardware one:

dts challenges submit –challenge aido2-LF-sim-validation,aido2-LF-sim-testing,aido2-LF-real-validation
dts challenges submit –challenge aido2-LFV-sim-validation,aido2-LFV-sim-testing,aido2-LFV-real-validation
dts challenges submit –challenge aido2-LFV-sim-validation,aido2-LFVI-sim-testing,aido2-LFVI-real-validation

We will evaluate submissions by participants that are in the top part of the leaderboard in the simulated testing challenge.

The robotarium evaluations are limited, and we will do them in a round robin strategy for each user. We aim to evaluate all in the top 10 of the simulated challenge; and then more if there is the possibility.

Participants can have multiple submissions in the “real” challenges. We will evaluate first according to “user priority” or by most recent. The priority is settable through the web interface by using the top right button.

Deadlines

The challenges will close May 21 at 8pm Montreal (EDT) time. Please check the server timestamp for the precise time in your time zone.

Update to Dynamics Model in Duckietown Simulator

Posted on May 13, 2019 | by Liam Paull

We have implemented an improved dynamics model in the simulator. If you are using the simulator to:

Train your agent with reinforcement learning
Generate data for imitation learning
Test and debug your submission

then you may want to retrain/retest with the new dynamics model. This model is much closer to the true Duckiebot and should permit much easier transfer from simulation to the real robot hardware.