Local evaluation – Duckietown

This topic contains 15 replies, has 6 voices, and was last updated by Andrea Censi 2 years, 5 months ago.

Viewing 15 posts - 1 through 15 (of 16 total)

1 2 →

Author

Posts
October 20, 2018 at 1:09 pm #28544

PolyProgrammist
Participant

Hello,
I am facing a problem with local evaluation.
The command ‘dts challenges evaluate’ fails.
It says it can’t build a connect to docker daemon.

I attach the logs
https://drive.google.com/file/d/1Nlx-CoukWoFkW8Vfg5QX6Mk2a40zFOT1/view?usp=sharing

Best regards,
Vadim Volodin

October 20, 2018 at 2:12 pm #28547

Florian Golemo
Participant

This might be a problem with your docker configuration. Could you send us the output of docker info, please?

October 20, 2018 at 4:49 pm #28548

Andrea Censi
Keymaster

This is a known problem – we are trying to understand the reason.

At the bottom here you can find a workaround:

http://docs.duckietown.org/DT18/AIDO/out/cm_first.html

October 20, 2018 at 6:28 pm #28563

PolyProgrammist
Participant

Oh, I didn’t try the troubleshooting, thanks.
Here is docker info output https://drive.google.com/file/d/1XpVHTUgaOK8z1YsFfmrJlpPBxrMbKgkq/view?usp=sharing

October 23, 2018 at 4:01 pm #28916

heyt0ny
Participant

Hello!

I’m facing error when running dts challenges evaluate. I tried it on all repos with “template” on github and it raises:

`
ERROR:runner-local:Traceback (most recent call last):
File “/project/src/duckietown_challenges_runner/runner_local.py”, line 35, in runner_local_main
runner_local_main_()
File “/project/src/duckietown_challenges_runner/runner_local.py”, line 84, in runner_local_main_
client = check_docker_environment()
File “/project/src/duckietown-shell/lib/dt_shell/env_checks.py”, line 36, in check_docker_environment
check_user_in_group(‘docker’)
File “/project/src/duckietown-shell/lib/dt_shell/env_checks.py”, line 79, in check_user_in_group
active_groups = get_active_groups(username=None)
File “/project/src/duckietown-shell/lib/dt_shell/env_checks.py”, line 104, in get_active_groups
raise Exception(str(e))
Exception: Command ‘[‘groups’]’ returned non-zero exit status 1

INFO:dts:status: exited
ERROR:dts:The container exited.
INFO:dts:Logs saved at evaluator.log
`

evaluator.log is empty.

October 23, 2018 at 7:51 pm #28969

PolyProgrammist
Participant

Hello, my logs now are as following, but my username is in group docker

DEBUG:dts:checking docker environment
INFO:dts:The server URL is: https://challenges.duckietown.org/v3
INFO:dts:Updating container duckietown/dt-challenges-evaluator:v3
INFO:dts:Starting container local-evaluator with duckietown/dt-challenges-evaluator:v3
INFO:dts:Container command: dt-challenges-evaluate-local –output output
INFO:dts:status: running
INFO:dt-challenges-runner:dt-challenges-runner 3.1.41
INFO:duckietown-challenges:duckietown-challenges 3.2.3
INFO:dts:duckietown-shell 3.0.24
DEBUG:runner-local:Running in directory /home/polyprogrammist/Documents/duckietown/agent/lf1first
DEBUG:dts:Checking docker environment
ERROR:runner-local:Traceback (most recent call last):
File “/project/src/duckietown_challenges_runner/runner_local.py”, line 35, in runner_local_main
runner_local_main_()
File “/project/src/duckietown_challenges_runner/runner_local.py”, line 84, in runner_local_main_
client = check_docker_environment()
File “/project/src/duckietown-shell/lib/dt_shell/env_checks.py”, line 36, in check_docker_environment
check_user_in_group(‘docker’)
File “/project/src/duckietown-shell/lib/dt_shell/env_checks.py”, line 86, in check_user_in_group
raise InvalidEnvironment(msg)
InvalidEnvironment: The user is not in group “docker”.

It belongs to groups: root.

Note that when you add a user to a group, you need to login in and out.

October 23, 2018 at 10:37 pm #28970

PolyProgrammist
Participant

I did the troubleshooting. It helped. But for the second attempt it broke.
Step 10/10 : CMD python solution.py
—> Using cache
—> dd5a79cf7b39
Successfully built dd5a79cf7b39
Successfully tagged myimage:latest
INFO:runner-local:steps: [u’step1-simulation’, u’step2-scoring’]
INFO:runner-local:Now considering step “step1-simulation”
ERROR:runner-local:Traceback (most recent call last):
File “/home/polyprogrammist/.local/lib/python2.7/site-packages/duckietown_challenges_runner/runner_local.py”, line 35, in runner_local_main
runner_local_main_()
File “/home/polyprogrammist/.local/lib/python2.7/site-packages/duckietown_challenges_runner/runner_local.py”, line 129, in runner_local_main_
shutil.rmtree(wd)
File “/usr/lib/python2.7/shutil.py”, line 261, in rmtree
rmtree(fullname, ignore_errors, onerror)
File “/usr/lib/python2.7/shutil.py”, line 261, in rmtree
rmtree(fullname, ignore_errors, onerror)
File “/usr/lib/python2.7/shutil.py”, line 261, in rmtree
rmtree(fullname, ignore_errors, onerror)
File “/usr/lib/python2.7/shutil.py”, line 266, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File “/usr/lib/python2.7/shutil.py”, line 264, in rmtree
os.remove(fullname)
OSError: [Errno 13] Permission denied: ‘output-local-evaluation/step1-simulation.tmp/challenge-evaluation-output/episodes/ep000/log.bag’

October 27, 2018 at 6:02 pm #29097

PolyProgrammist
Participant

test

October 27, 2018 at 6:09 pm #29098

PolyProgrammist
Participant

From Anna Tsalapova. She couldn’t create a post here because of some issue of the forum
>

Hello
I’m facing a similar problem. When I first run dts challenges evaluate command (on the random template) it works fine. Then I change solution.py a bit and run dts challenges evaluate again and I failes with the problem

ERROR:runner-local:Traceback (most recent call last):
File “/project/src/duckietown_challenges_runner/runner_local.py”, line 35, in runner_local_main
runner_local_main_()
File “/project/src/duckietown_challenges_runner/runner_local.py”, line 124, in runner_local_main_
shutil.rmtree(wd_final)
File “/usr/lib/python2.7/shutil.py”, line 261, in rmtree
rmtree(fullname, ignore_errors, onerror)
File “/usr/lib/python2.7/shutil.py”, line 261, in rmtree
rmtree(fullname, ignore_errors, onerror)
File “/usr/lib/python2.7/shutil.py”, line 261, in rmtree
rmtree(fullname, ignore_errors, onerror)
File “/usr/lib/python2.7/shutil.py”, line 266, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File “/usr/lib/python2.7/shutil.py”, line 264, in rmtree
os.remove(fullname)
OSError: [Errno 13] Permission denied: ‘output/step1-simulation/challenge-evaluation-output/episodes/ep002/log.bag’

Full logs you can find here: https://drive.google.com/file/d/1iMuoBbhU9BeJugatU9AdE9Hqkm17cBjS/view?usp=sharing

October 29, 2018 at 12:34 pm #29113

Bhairav Mehta
Participant

It seems like you may have an older version of duckietown-shell

If you are running locally, can you do
sudo rm -rf output && dts update

If you’re running in Docker, just run the second command.

Try this and see if it works?

October 30, 2018 at 8:06 am #29127

anna.tsalapova
Participant

I’m running locally and it did not help, I got the same error.

October 31, 2018 at 2:37 am #29132

Bhairav Mehta
Participant

Could you explain what was changed in solution.py? Some lines inside are critical to correct scoring.

October 31, 2018 at 7:15 am #29181

anna.tsalapova
Participant

I just changed threshold value in our simple algorithm, just one number.

November 1, 2018 at 2:10 pm #29295

Andrea Censi
Keymaster

@polyprogrammist – can you remove the directory “output” and try again?

I think it’s a left over from before when we wrote the wrong permissions.

Alternatively use dts challenges evaluate --output DIR where DIR does not exist yet.

November 3, 2018 at 4:58 pm #29364

PolyProgrammist
Participant

Hello, It works with removing an output. As soon as I understood, we recalculate a lot of data doing this way? If so, it’s easier to send submission to server than evaluate locally.

We changed the rights of the following directory and that worked: output/step1-simulation/challenge-evaluation-output/episodes
Could you make the permissions for it, please?
Author

Posts

Viewing 15 posts - 1 through 15 (of 16 total)

1 2 →

You must be logged in to reply to this topic.