Skip to content

Commit a42db9c

Browse files
NM512NM512Svetlana Karslioglu
committed
Update transformer_tutorial.py (#2451)
Co-authored-by: NM512 <morihira3513@gmailcom> Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
1 parent 97b3ead commit a42db9c

File tree

1 file changed

+5
-2
lines changed

1 file changed

+5
-2
lines changed

beginner_source/transformer_tutorial.py

+5-2
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
Language Modeling with ``nn.Transformer`` and torchtext
33
===============================================================
44
5-
This is a tutorial on training a sequence-to-sequence model that uses the
5+
This is a tutorial on training a model to predict the next word in a sequence using the
66
`nn.Transformer <https://pytorch.org/docs/stable/generated/torch.nn.Transformer.html>`__ module.
77
88
The PyTorch 1.2 release includes a standard transformer module based on the
@@ -29,7 +29,9 @@
2929

3030
######################################################################
3131
# In this tutorial, we train a ``nn.TransformerEncoder`` model on a
32-
# language modeling task. The language modeling task is to assign a
32+
# language modeling task. Please note that this tutorial does not cover
33+
# the training of `nn.TransformerDecoder <https://pytorch.org/docs/stable/generated/torch.nn.TransformerDecoder.html#torch.nn.TransformerDecoder>`__, as depicted in
34+
# the right half of the diagram above. The language modeling task is to assign a
3335
# probability for the likelihood of a given word (or a sequence of words)
3436
# to follow a sequence of words. A sequence of tokens are passed to the embedding
3537
# layer first, followed by a positional encoding layer to account for the order
@@ -130,6 +132,7 @@ def forward(self, x: Tensor) -> Tensor:
130132
# .. code-block:: bash
131133
#
132134
# %%bash
135+
# pip install portalocker
133136
# pip install torchdata
134137
#
135138
# The vocab object is built based on the train dataset and is used to numericalize

0 commit comments

Comments
 (0)