600 Timeout when submit

This topic contains 5 replies, has 3 voices, and was last updated by heyt0ny 7 months, 2 weeks ago.

Viewing 6 posts - 1 through 6 (of 6 total)

Author

Posts
November 8, 2018 at 6:27 pm #29618

heyt0ny
Participant

Hi!

I’m getting 600 timeout error – https://challenges.duckietown.org/v3/humans/submissions/832
What may this mean?

Thank you so much for answering the questions)

November 8, 2018 at 6:37 pm #29619

Bhairav Mehta
Participant

I believe this is the same issue as your other post I answered this morning.
Can you post a link to the solution.py you are using?

November 9, 2018 at 2:21 pm #29628

heyt0ny
Participant

So it’s really strange error and it’s not about solution file

So I have 2 models (A and B). They both return the same type and dimensions of action. The both run locally successfully.

But the model A is failed on step1-simulation. I dind’t change the architecture of model A for at least 1 week (I only changed weights). So I decided to use old weights from successful submits (~ 3 days ago) — and it still fails.

Then I decided to turn off weights load — and the model A is submitted successfully. But I’m 100% sure it’s not the problem with weights load, because model B uses the same code for weights load.

I’m really confused about it.

Is there any way I can download my submits to make sure I didn’t change anything.. ?

November 9, 2018 at 9:05 pm #29629

lavoiems
Participant

Hey Tony, so my extremly wild guess would be that the docker image that your are using is using python2 and your weight were saved, using pickle, on python3.

If this is the case, you can fix it by changing the first line of your Dockerfile to FROM pytorch/pytorch:0.4.1-cuda9-cudnn7-runtime

Cheers

November 10, 2018 at 9:45 pm #29872

heyt0ny
Participant

Thank you lavoiems, good point!

Tried it, but the same failure (

November 11, 2018 at 2:56 pm #29877
heyt0ny
Participant
seems like I was able to fix this, by adding frame skip:
```
    nested_break = False
    try:
        cis.info('Reset environment')
        observation = env.reset()
        while True:
            action = model.predict(observation)

            if debug:
                env.render()
            for i in range(config['frame_skip']):
                observation, reward, done, info = env.step(action)
                if 'simulation_done' in info:
                    cis.info('simulation_done received.')
                    nested_break = True
                    break
                if done:
                    cis.info('Episode done; calling reset()')
                    env.reset()
                    break
            if nested_break:
                break

    finally:
        # release CPU/GPU resources, let's be friendly with other users that may need them
        cis.info('Releasing resources')
        try:
            model.close()
        except Exception as e:
            cis.error(e)
            msg = 'Could not call model.close():\n%s' % traceback.format_exc()
            cis.error(msg)
    cis.info('Graceful exit of solve()')
```
magic !
Author

Posts

Viewing 6 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic.