Front page Forums 🤖 🚗 AI Driving Olympics 🏆 Rules discussion Leak in leaderboard (it's bad).

This topic contains 1 reply, has 2 voices, and was last updated by https://www.duckietown.org/wp-content/uploads/ap_avatars/c203d8a151612acf12457e4d67635a95.jpg Kirsten Bowser 10 months, 3 weeks ago.

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • #29183

    heyt0ny
    Participant

    Hello!

    I’m really excited to participate in this challenge. But there are a few things that make this competition not objective at all for now:
    1) “Leaderboard” is showing top 16 submits from all users. Instead it should show one best submit for each person.
    2) The evaluation if scores is super leaky. Because we do see the evaluation map. Someone may create this map and train his agent only on this map, while the agent may perform really good on this map — it may not be good on another maps. Please follow the practice, like kaggle does — 2 leaderboards. And on private leaderboard there is private map (that no one from participants have ever seen) — or even better multiple private maps.

    Thank you!

    • This topic was modified 10 months, 3 weeks ago by  heyt0ny.
    #29186
    https://www.duckietown.org/wp-content/uploads/ap_avatars/c203d8a151612acf12457e4d67635a95.jpg
    Kirsten Bowser
    Keymaster

    Thanks for the note @heytony – we’re on it!

Viewing 2 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.

Close Menu