Leak in leaderboard (it's bad).

Front page Forums 🤖 🚗 AI Driving Olympics 🏆 Rules discussion Leak in leaderboard (it's bad).

This topic contains 1 reply, has 2 voices, and was last updated by https://www.duckietown.org/wp-content/uploads/ap_avatars/c203d8a151612acf12457e4d67635a95.jpg Kirsten Bowser 3 years, 11 months ago.

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
  • #29183



    I’m really excited to participate in this challenge. But there are a few things that make this competition not objective at all for now:
    1) “Leaderboard” is showing top 16 submits from all users. Instead it should show one best submit for each person.
    2) The evaluation if scores is super leaky. Because we do see the evaluation map. Someone may create this map and train his agent only on this map, while the agent may perform really good on this map — it may not be good on another maps. Please follow the practice, like kaggle does — 2 leaderboards. And on private leaderboard there is private map (that no one from participants have ever seen) — or even better multiple private maps.

    Thank you!

    • This topic was modified 3 years, 11 months ago by  heyt0ny.
    Kirsten Bowser

    Thanks for the note @heytony – we’re on it!

Viewing 2 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.