PyTorch Transformers and Machine Translation

Here, I use PyTorch’s famous Transformers library to translate German sentence into English sentences. The model is trained on PyTorch’s Multi30k dataset.

Github Link: https://github.com/rukshar69/Transformers/blob/master/pytorch_transformers_machine_translation.ipynb

The model and training parameters are laid out just before the loop codes for training iterations. So, others can easily manipulate the models during their own training.

I record the losses and bleu score after each epoch which are used to plot graphs after the training is finished.

We see during training that the model moss is gradually going down.

Here the training loss is observed for every batch of data being used for training. The X-axis here refers to steps of batch data training.

The BLEU score hovers around between .4 and .43 from epoch 5 onwards.