Autograder [Fri May  1 18:00:23 2020]: Received job 11785-s20_hw4p1_3_brianli@andrew.cmu.edu:1203
Autograder [Fri May  1 18:00:56 2020]: Success: Autodriver returned normally
Autograder [Fri May  1 18:00:56 2020]: Here is the output from the autograder:
---
Autodriver: Job exited with status 0
mkdir -p handin
tar xf handin.tar -C handin
tar xf autograde.tar
AUTOLAB=1 /usr/local/depot/anaconda3/bin/python3 autograde/runner.py --module-path=./handin/
Your mean NLL for generated sequences: 6.978278160095215
FYour mean NLL for predicting a single word: 5.180866718292236
.
=================================== FAILURES ===================================
_______________________________ test_generation ________________________________

    def test_generation():
        inp = np.load(fixture_path('generation.npy'))
        forward = 10
        n = inp.shape[0]
        t = inp.shape[1]
        pred = np.load(handin_path('generated_logits.npy'))
        assert pred.shape[0] == n
        assert pred.shape[1] == forward
    
        ninp = np.concatenate((inp, pred), axis=1)
        vocab = np.load(fixture_path('vocab.npy'))
        model = Model(vocab.shape[0], args=prediction_args)
        model.load_state_dict(load_from_numpy(read_chunks(fixture_path('model-00000099.tar.npy.{}'))))
        model.eval()
    
        state = model.zero_state(n)
        x = Variable(torch.from_numpy(ninp.T)).long()
        logits, rnn_hs = model(x, state)
        logits = logits.cpu().data.numpy()
        logits = log_softmax(logits, -1)
        logits = logits[t - 1:-1, :, :]
        logits = np.transpose(logits, (1, 0, 2))
        mg = np.meshgrid(np.arange(n), np.arange(forward), indexing='ij')
        nlls = logits[mg[0], mg[1], pred]
        nll = -np.mean(nlls)
    
        print("Your mean NLL for generated sequences: {}".format(nll))
    
>       assert nll < 3.
E       assert 6.9782782 < 3.0

autograde/tests/test_generation.py:245: AssertionError
Run time:  14.50851845741272
{"scores": {"Generation": 0.0, "Prediction": 50.0}}