Autograder [Thu Apr 30 17:10:06 2020]: Received job 11785-s20_hw4p1_14_carunach@andrew.cmu.edu:1209
Autograder [Thu Apr 30 17:10:39 2020]: Success: Autodriver returned normally
Autograder [Thu Apr 30 17:10:39 2020]: Here is the output from the autograder:
---
Autodriver: Job exited with status 0
mkdir -p handin
tar xf handin.tar -C handin
tar: training.ipynb: time stamp 2020-04-30 17:09:25 is 14349.668913288 s in the future
tar: predictions.npy: time stamp 2020-04-30 17:09:25 is 14349.65350398 s in the future
tar: generated.txt: time stamp 2020-04-30 17:09:25 is 14349.653396523 s in the future
tar: generated_logits.npy: time stamp 2020-04-30 17:09:25 is 14349.653342371 s in the future
tar xf autograde.tar
AUTOLAB=1 /usr/local/depot/anaconda3/bin/python3 autograde/runner.py --module-path=./handin/
Your mean NLL for generated sequences: 2.8740906715393066
.Your mean NLL for predicting a single word: 5.701335906982422
F
=================================== FAILURES ===================================
_______________________________ test_prediction ________________________________

    def test_prediction():
        fixture = np.load(fixture_path('prediction.npz'))
        inp = fixture['inp']
        targ = fixture['out']
        out = np.load(handin_path('predictions.npy'))
        assert out.shape[0] == targ.shape[0]
        vocab = np.load(fixture_path('vocab.npy'))
        assert out.shape[1] == vocab.shape[0]
        out = log_softmax(out, 1)
        nlls = out[np.arange(out.shape[0]), targ]
        nll = -np.mean(nlls)
        print("Your mean NLL for predicting a single word: {}".format(nll))
>       assert nll < 5.4
E       assert 5.7013359 < 5.4

autograde/tests/test_prediction.py:31: AssertionError
Run time:  15.691711902618408
{"scores": {"Generation": 50.0, "Prediction": 0.0}}