Front page › Forums › 🤖 🚗 AI Driving Olympics 🏆 › Embodied tasks › Local evaluation
This topic contains 15 replies, has 6 voices, and was last updated by Andrea Censi 2 years, 5 months ago.
-
AuthorPosts
-
October 20, 2018 at 1:09 pm #28544
Hello,
I am facing a problem with local evaluation.
The command ‘dts challenges evaluate’ fails.
It says it can’t build a connect to docker daemon.I attach the logs
https://drive.google.com/file/d/1Nlx-CoukWoFkW8Vfg5QX6Mk2a40zFOT1/view?usp=sharingBest regards,
Vadim VolodinOctober 20, 2018 at 2:12 pm #28547This might be a problem with your docker configuration. Could you send us the output of
docker info
, please?October 20, 2018 at 4:49 pm #28548This is a known problem – we are trying to understand the reason.
At the bottom here you can find a workaround:
October 20, 2018 at 6:28 pm #28563Oh, I didn’t try the troubleshooting, thanks.
Here isdocker info
output https://drive.google.com/file/d/1XpVHTUgaOK8z1YsFfmrJlpPBxrMbKgkq/view?usp=sharingOctober 23, 2018 at 4:01 pm #28916Hello!
I’m facing error when running
dts challenges evaluate
. I tried it on all repos with “template” on github and it raises:`
ERROR:runner-local:Traceback (most recent call last):
File “/project/src/duckietown_challenges_runner/runner_local.py”, line 35, in runner_local_main
runner_local_main_()
File “/project/src/duckietown_challenges_runner/runner_local.py”, line 84, in runner_local_main_
client = check_docker_environment()
File “/project/src/duckietown-shell/lib/dt_shell/env_checks.py”, line 36, in check_docker_environment
check_user_in_group(‘docker’)
File “/project/src/duckietown-shell/lib/dt_shell/env_checks.py”, line 79, in check_user_in_group
active_groups = get_active_groups(username=None)
File “/project/src/duckietown-shell/lib/dt_shell/env_checks.py”, line 104, in get_active_groups
raise Exception(str(e))
Exception: Command ‘[‘groups’]’ returned non-zero exit status 1INFO:dts:status: exited
ERROR:dts:The container exited.
INFO:dts:Logs saved at evaluator.log
`
evaluator.log is empty.
October 23, 2018 at 7:51 pm #28969Hello, my logs now are as following, but my username is in group docker
DEBUG:dts:checking docker environment
INFO:dts:The server URL is: https://challenges.duckietown.org/v3
INFO:dts:Updating container duckietown/dt-challenges-evaluator:v3
INFO:dts:Starting container local-evaluator with duckietown/dt-challenges-evaluator:v3
INFO:dts:Container command: dt-challenges-evaluate-local –output output
INFO:dts:status: running
INFO:dt-challenges-runner:dt-challenges-runner 3.1.41
INFO:duckietown-challenges:duckietown-challenges 3.2.3
INFO:dts:duckietown-shell 3.0.24
DEBUG:runner-local:Running in directory /home/polyprogrammist/Documents/duckietown/agent/lf1first
DEBUG:dts:Checking docker environment
ERROR:runner-local:Traceback (most recent call last):
File “/project/src/duckietown_challenges_runner/runner_local.py”, line 35, in runner_local_main
runner_local_main_()
File “/project/src/duckietown_challenges_runner/runner_local.py”, line 84, in runner_local_main_
client = check_docker_environment()
File “/project/src/duckietown-shell/lib/dt_shell/env_checks.py”, line 36, in check_docker_environment
check_user_in_group(‘docker’)
File “/project/src/duckietown-shell/lib/dt_shell/env_checks.py”, line 86, in check_user_in_group
raise InvalidEnvironment(msg)
InvalidEnvironment: The user is not in group “docker”.It belongs to groups: root.
Note that when you add a user to a group, you need to login in and out.
October 23, 2018 at 10:37 pm #28970I did the troubleshooting. It helped. But for the second attempt it broke.
Step 10/10 : CMD python solution.py
—> Using cache
—> dd5a79cf7b39
Successfully built dd5a79cf7b39
Successfully tagged myimage:latest
INFO:runner-local:steps: [u’step1-simulation’, u’step2-scoring’]
INFO:runner-local:Now considering step “step1-simulation”
ERROR:runner-local:Traceback (most recent call last):
File “/home/polyprogrammist/.local/lib/python2.7/site-packages/duckietown_challenges_runner/runner_local.py”, line 35, in runner_local_main
runner_local_main_()
File “/home/polyprogrammist/.local/lib/python2.7/site-packages/duckietown_challenges_runner/runner_local.py”, line 129, in runner_local_main_
shutil.rmtree(wd)
File “/usr/lib/python2.7/shutil.py”, line 261, in rmtree
rmtree(fullname, ignore_errors, onerror)
File “/usr/lib/python2.7/shutil.py”, line 261, in rmtree
rmtree(fullname, ignore_errors, onerror)
File “/usr/lib/python2.7/shutil.py”, line 261, in rmtree
rmtree(fullname, ignore_errors, onerror)
File “/usr/lib/python2.7/shutil.py”, line 266, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File “/usr/lib/python2.7/shutil.py”, line 264, in rmtree
os.remove(fullname)
OSError: [Errno 13] Permission denied: ‘output-local-evaluation/step1-simulation.tmp/challenge-evaluation-output/episodes/ep000/log.bag’October 27, 2018 at 6:02 pm #29097test
October 27, 2018 at 6:09 pm #29098From Anna Tsalapova. She couldn’t create a post here because of some issue of the forum
>Hello
I’m facing a similar problem. When I first rundts challenges evaluate
command (on the random template) it works fine. Then I changesolution.py
a bit and rundts challenges evaluate
again and I failes with the problemERROR:runner-local:Traceback (most recent call last):
File “/project/src/duckietown_challenges_runner/runner_local.py”, line 35, in runner_local_main
runner_local_main_()
File “/project/src/duckietown_challenges_runner/runner_local.py”, line 124, in runner_local_main_
shutil.rmtree(wd_final)
File “/usr/lib/python2.7/shutil.py”, line 261, in rmtree
rmtree(fullname, ignore_errors, onerror)
File “/usr/lib/python2.7/shutil.py”, line 261, in rmtree
rmtree(fullname, ignore_errors, onerror)
File “/usr/lib/python2.7/shutil.py”, line 261, in rmtree
rmtree(fullname, ignore_errors, onerror)
File “/usr/lib/python2.7/shutil.py”, line 266, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File “/usr/lib/python2.7/shutil.py”, line 264, in rmtree
os.remove(fullname)
OSError: [Errno 13] Permission denied: ‘output/step1-simulation/challenge-evaluation-output/episodes/ep002/log.bag’Full logs you can find here: https://drive.google.com/file/d/1iMuoBbhU9BeJugatU9AdE9Hqkm17cBjS/view?usp=sharing
October 29, 2018 at 12:34 pm #29113It seems like you may have an older version of
duckietown-shell
If you are running locally, can you do
sudo rm -rf output && dts update
If you’re running in Docker, just run the second command.
Try this and see if it works?
October 30, 2018 at 8:06 am #29127I’m running locally and it did not help, I got the same error.
October 31, 2018 at 2:37 am #29132Could you explain what was changed in
solution.py
? Some lines inside are critical to correct scoring.October 31, 2018 at 7:15 am #29181I just changed threshold value in our simple algorithm, just one number.
November 1, 2018 at 2:10 pm #29295@polyprogrammist – can you remove the directory “output” and try again?
I think it’s a left over from before when we wrote the wrong permissions.
Alternatively use
dts challenges evaluate --output DIR
where DIR does not exist yet.November 3, 2018 at 4:58 pm #29364Hello, It works with removing an output. As soon as I understood, we recalculate a lot of data doing this way? If so, it’s easier to send submission to server than evaluate locally.
We changed the rights of the following directory and that worked: output/step1-simulation/challenge-evaluation-output/episodes
Could you make the permissions for it, please? -
AuthorPosts
You must be logged in to reply to this topic.