Imitation Learning Approach for AI Driving Olympics Trained on Real-world and Simulation Data Simultaneously

Imitation Learning Approach for AI Driving Olympics Trained on Real-world and Simulation Data Simultaneously

The AIDO challenge is divided into two global stages: simulation and real-world. A single algorithm needs to perform well in both. It was quickly identified that one of the major problems is the simulation to real-world transfer. 

Many algorithms trained in the simulated environment performed very poorly in the real world, and many classic control algorithms that are known to perform well in a real-world environment, once tuned to that environment, do not perform well in the simulation. Some approaches suggest randomizing the domain for the simulation to real-world transfer.

We propose a novel method of training a neural network model that can perform well in diverse environments, such as simulations and real-world environment.

Dataset Generation

To that end, we have trained our model through imitation learning on a dataset compiled from four different sources:

  1. Real-world Duckietown dataset from logs.duckietown.org (REAL-DT).
  2. Simulation dataset on a simple loop map (SIM-LP).
  3. Simulation dataset on an intersection map (SIM-IS).
  4. Real-world dataset collected by us in our environment with car driven by PD controller (REAL-IH).

We aimed to collect data with as many possible situations such as twists in the road, driving in circles clockwise/counterclockwise, and so on. We have also tried to diversify external factors such as scene lighting, items in the room that can get into the camera’s field of view, roadside objects, etc. If we keep these conditions constant, our model may overfit to them and perform poorly in a different environment. For this reason, we changed the lighting and environment after each duckiebot run. The lane detection was calibrated for every lighting condition since different lighting changes the color scheme of the image input.

We made the following change to the standard PD algorithm: since most Duckietown turns and intersections are standard-shaped, we hard-coded the robot’s motion in these situations, but we did not exclude imperfect trajectories. For example, the ones that would go slightly out of bounds of the lane. Imperfections in the robot’s actions increase the robustness of the model. 

Neural network architecture and training

Original images are 640×480 RGB. As a preprocessing step, we remove the top third of the image, since it mostly contains the sky, resize the image to 64×32 pixels and convert it into the YUV colorspace.

We have used 5 convolutional layers with a small number of filters, followed by 2 fully-connected layers. The small size of the network is not only due to it being less prone to overfitting, but we also need a model that can run on a single CPU on RaspberryPi.

We have also incorporated Independent-Component (IC) layers. These layers aim to make the activations of each layer more independent by combining two popular techniques, BatchNorm and Dropout. For convolutional layers, we substitute Dropout with Spatial Dropout which has been shown to work better with them. The model outputs two values for voltages of the left and the right wheel drives. We use the mean square error (MSE) as our training loss.

Results

For the training evaluation, we compute the mean square error (MSE) of the left and the right wheels outputs on the validation set of each data source. 

The first table shows the results for the models trained on all data sources (HYBRID), on real-world data sources only (REAL) and on simulation data sources only (SIM). As we can see, while training on a single dataset sometimes achieves lower error on the same dataset than our hybrid approach. We can also see that our method performs on par with the best single methods. In terms of the average error it outperforms the closest one tenfold. This demonstrates definitively the high dependence of MSE on the training method, and highlights the differences between the data sources.

The next table shows simulation closed-loop performance for all our approaches using the Duckietown simulator. All methods drove for 15 seconds without major infractions, and the SIM model that was trained specifically on the simulation data only drove just 1.8 tiles more than our hybrid approach.

The third table shows the closed-loop performance in the real-world environment. Comparing the number of tiles, we see that our hybrid approach drove about 3.5 tiles more than the following in the rankings model trained on real-world data only.

Conclusion

Our method follows the imitation learning approach and consists of a convolutional neural network which is trained on a dataset compiled from data from different sources, such as simulation model and real-world Duckietown vehicle driven by a PD controller, tuned to various conditions, such as different map configuration and lighting. 

We believe that our approach of emphasizing neurons independence and monitoring generalization performance can offer more robustness to control models that have to perform in diverse environments. We also believe that the described approach of imitation learning on data obtained from several algorithms that are fitted to specific environments may yield a single algorithm that will perform well in general.

 —
 JBRRussia1 team

Learn at Home

These are difficult times for us all. 

With physical distancing directives issued across the globe and many people restricted to their homes, we want to reach out (virtually) and offer our support.

To help you beat the isolation blues, the Duckietown Foundation is 

towards the next 100 orders of Duckiebots, Starter Kits, and Navigation Packs.

Remember that you can still learn about robotics without a robot!

Almost all of our resources remain open and available for your use.

Join us on 
Slack, peruse our library, or start training for the Urban League of the AI Driving Olympics.

Due to the closure of academic institutions the Duckietown Autolabs are temporarily closed. 

Coming soon: online demonstrations and tutorials to help you get started!

Have fun learning and stay safe!

AI-DO 3 – Urban Event Winners

In case you missed it AI-DO 3 has come and gone. Interested in reliving the competition? Here’s the video.

We had a great time at NeurIPS hosting the Third Edition of the AI Driving Olympics. As usual the sound of Duckies attracted an engaging and supportive crowd.

 

Racing Event

The competition began with the Racing Event, hosted by AWS DeepRacer. They ran their top 10 submissions and selected the winner by who could complete the fastest lap.

Racing Event Winner 
Ayrat Baykov at 8:08 seconds

 

Advanced Perception Event

The winners of the Advanced Perception Event hosted by APTIV and the nuScenes dataset were announced. Luckily a member of the winning team was present to accept the award.

Rank 3
CenterTrack – Open and Vision

Rank 2
VV_Team

Rank 1
StanfordlPRL-TRI

 

Urban Event

The competition culminated with Duckietown’s own Urban Driving Event, where we ran the top submissions for each of the three challenges on our competition tracks.

Winners

 

Lane Following 

JBRRussia1: Konstantin Chaika, Nikita Sazanovich, Kirill Krinkin, Max Kuzmin

Lane Following with Vehicles

phmarm

Lane Following with Vehicles and Intersections

frank_qcd_qk

 

Final Scoreboard

A few pictures from the event

Congratulations to all the winners and thanks for participating in the competition. We look forward to seeing you for AI-DO 4!

Community Spotlight: Kirill Krinkin – STEM Intensive Learning Approach


In the world of engineering education, there are many excellent courses, but often the curriculum has one serious drawback – the lack of good connectivity between different topics. Over in Saint Petersburg, Russia, 
Kirill Krinkin from SPbETU and JetBrains Research has been using Duckietown to address this problem through an intensive STEM winter course.

STEM Intensive Learning Approach

by Kirill Krinkin

The first part of the school program was a week of classes in the base topic areas which were chosen to complement each other and help students see the connection between seemingly different things – mathematics, electronics and programming.

Of course, the main goal of the program was to give students the opportunity to put their new found knowledge into practice themselves.

Duckietown was the perfect fit for our course because it offered a hands-on learning experience for all of our main topics areas, and once we covered those subject in the first lessons, we challenged the students with much more complex tasks – in the form of projects – in the second half of the course. It made for an exciting and engaging curriculum because students could address a problem, write a program to solve it, and then immediately launch it on a real robot. 

The main advantage of Duckietown compared to many other platforms is that there is a very small learning curve: people who knew nothing about programming and robotics started working on projects after only a few days!

Overview of the course

Part 1 – Main Topic Areas

Subject 1: Linear Algebra

Students spent one day studying vectors and matrices, systems of linear equations, etc. Practical tasks were built in an interactive mode: the proposed tasks were solved individually, and the teacher and other students gave comments and tips.

 

Subject 2: Electricity and Simple Circuits

Students studied the basics of electrodynamics: voltage, current, resistance, Ohm’s law and Kirchhoff’s laws. Practical tasks were partially done in the electric circuits simulator or performed on the board, but more time was devoted to building real circuits, such as logic circuits, oscillatory circuits, etc.

 

Subject 3: Computer Architecture

In a sense, a bridge connecting physics and programming. Students studied the fundamental basis, the significance of which is more theoretical than practical. As a practice, students independently designed arithmetic-logic circuits in the simulator.

 

Subject 4: Programming

Python 2 was chosen as the programming language, as it is used in programming under ROS. After we taught the material and gave examples of solving problems, students were challenged with their own problems to solve, which we then evaluated. 

 

Subject 5: ROS

Here the students started programming robots. Throughout the school day, students sat at computers, running the program code that the teacher talked about. They were able to independently launch the basic units of ROS, and also get acquainted with the Duckietown project. At the end of this day, students were ready to begin the design part of the course – solving practical problems.

Part 2 – Projects

1. Calibration of colors

Duckiebots needs to calibrate the camera when lighting conditions change, so this project focussed on the task of automatic calibration. The problem is that color ranges are very sensitive to light. Participants implemented a utility that would highlight the desired colors on the frame (red, white and yellow) and build ranges for each of the colors in HSV format.

2. Duck Taxi

The idea of this project was that Duckiebot could stop near some object, pick it up and then continue along, following a certain route. Of course, a bright yellow Duckie was the chosen passenger. The participants divided this task into two: detection and movement along the graph.

drive while Duckie is not detected

Duckie identified as a yellow spot with an orange triangle 🙂

Building a route according to the road graph and destination point

3. Building a road map

The goal of this project was to build a road map without providing a priori environmental data for the Duckiebot, relying solely on camera data. Here’s the working scheme of the algorithm developed by the participants:

4. The patrol car

This project was invented by the students themselves. They offered to teach one Duckiebot, the “patrol”, to find, follow, and stop an “intruding” Duckiebot. The students used ArUco markers to identify the Intruder on the road as they are easy to work with and they allow you to determine the orientation and distance of the marker. Next, the team changed the state machine of the Patrol Duckiebot so that when approaching the stop-line the bot would continue through the intersection without stopping. Finally, the team was able to get the Patrol Duckiebot to stop the Intruder bot by connecting via SSH and turning it off. The algorithm of the patrol robot can be represented as the following scheme:

Summary

Students walked away from our STEM intensive learning program with the foundations of autonomous driving, from the theoretical math and physics behind the programming and circuitry to the complex challenges of navigating through a city. We were successful in remaining accessible to beginners in a particular area, but also providing materials for repetition and consolidation to experienced students. Duckietown is an excellent resource for bringing education to life.

After our course ended students were asked about their experience. 100% of them said that the program exceed their expectations. We can certainly say that the Duckietown platform played a pivotal role in our success.

Congratulations to the winners of the second edition of the AI Driving Olympics!

Team JetBrains came out on top on all 3 challenges

It was a busy (and squeaky) few days at the International Conference on Robotics and Automation in Montreal for the organizers and competitors of the AI Driving Olympics. 

The finals were kicked off by a semifinals round, where we the top 5 submissions from the Lane Following in Simulation leaderboard. The finalists (JBRRussia and MYF) moved forward to the more complicated challenges of Lane Following with Vehicles and Lane Following with Vehicles and Intersections. 

Results from the AI-DO2 Finals event on May 22, 2019 at ICRA

If you couldn’t make it to the event and missed the live stream on Facebook, here’s a short video of the first run of the JetBrains Lane Following submission.

Thanks to everyone that competed, dropped in to say hello, and cheered on the finalists by sending the song of the Duckie down the corridors of the Palais des Congrès. 

A few pictures from the event

Don't know much about the AI Driving Olympics?

It is an accessible and reproducible autonomous car competition designed with straightforward standardized hardware, software and interfaces.

Get Started

Step 1: Build and test your agent with our available templates and baselines

Step 2: Submit to a challenge

Check out the leaderboard

View your submission in simulation

Step 3: Run your submission on a robot

in a Robotarium

AI-DO Robotarium Evaluations Underway

We have started evaluating the submissions in our “robotarium”:

Duckiebot onboard camera feed Robotarium watchtower camera feed

To queue your submissions for robotarium evaluation, please follow these instructions:

You need to use the –challenge option to specify 3 challenges: the two simulated ones (testing and validation) and the hardware one:

  • dts challenges submit –challenge aido2-LF-sim-validation,aido2-LF-sim-testing,aido2-LF-real-validation
  • dts challenges submit –challenge aido2-LFV-sim-validation,aido2-LFV-sim-testing,aido2-LFV-real-validation
  • dts challenges submit –challenge aido2-LFV-sim-validation,aido2-LFVI-sim-testing,aido2-LFVI-real-validation

We will evaluate submissions by participants that are in the top part of the leaderboard in the simulated testing challenge.

The robotarium evaluations are limited, and we will do them in a round robin strategy for each user. We aim to evaluate all in the top 10 of the simulated challenge; and then more if there is the possibility.

Participants can have multiple submissions in the “real” challenges. We will evaluate first according to “user priority” or by most recent. The priority is settable through the web interface by using the top right button.

Deadlines

The challenges will close May 21 at 8pm Montreal (EDT) time. Please check the server timestamp for the precise time in your time zone.

Update to Dynamics Model in Duckietown Simulator

We have implemented an improved dynamics model in the simulator. If you are using the simulator to:

  • Train your agent with reinforcement learning
  • Generate data for imitation learning
  • Test and debug your submission

then you may want to retrain/retest with the new dynamics model. This model is much closer to the true Duckiebot and should permit much easier transfer from simulation to the real robot hardware.

AI-DO 2 Validation and Testing Registration

We are in the final countdown to AI-DO 2 at ICRA!

Now is the time to let us know if you will be using the validation and testing facilities at the Duckietown competition ground. Please register below!