Update notebooks by re-syncing to transformers documentation (#118)

* Update notebooks

* Updates

* Final changes

Author: LysandreJik

transformers_doc/custom_datasets.ipynb

@@ -478,7 +478,7 @@
    "metadata": {},
    "source": [
     "We have seen the commands that will work for most cases (pad your batch to the length of the maximum sentence and\n",
-    "truncate to the maximum length the mode can accept). However, the API supports more strategies if you need them. The\n",
+    "truncate to the maximum length the model can accept). However, the API supports more strategies if you need them. The\n",
     "three arguments you need to know for this are `padding`, `truncation` and `max_length`.\n",
     "\n",
     "- `padding` controls the padding. It can be a boolean or a string which should be:\n",
@@ -493,15 +493,16 @@
     "\n",
     "- `truncation` controls the truncation. It can be a boolean or a string which should be:\n",
     "\n",
-    "    - `True` or `'only_first'` truncate to a maximum length specified by the `max_length` argument or\n",
+    "    - `True` or `'longest_first'` truncate to a maximum length specified by the `max_length` argument or\n",
     "      the maximum length accepted by the model if no `max_length` is provided (`max_length=None`). This will\n",
-    "      only truncate the first sentence of a pair if a pair of sequence (or a batch of pairs of sequences) is provided.\n",
+    "      truncate token by token, removing a token from the longest sequence in the pair until the proper length is\n",
+    "      reached.\n",
     "    - `'only_second'` truncate to a maximum length specified by the `max_length` argument or the maximum\n",
     "      length accepted by the model if no `max_length` is provided (`max_length=None`). This will only truncate\n",
     "      the second sentence of a pair if a pair of sequence (or a batch of pairs of sequences) is provided.\n",
-    "    - `'longest_first'` truncate to a maximum length specified by the `max_length` argument or the maximum\n",
-    "      length accepted by the model if no `max_length` is provided (`max_length=None`). This will truncate token\n",
-    "      by token, removing a token from the longest sequence in the pair until the proper length is reached.\n",
+    "    - `'only_first'` truncate to a maximum length specified by the `max_length` argument or the maximum\n",
+    "      length accepted by the model if no `max_length` is provided (`max_length=None`). This will only truncate\n",
+    "      the first sentence of a pair if a pair of sequence (or a batch of pairs of sequences) is provided.\n",
     "    - `False` or `'do_not_truncate'` to not truncate the sequences. As we have seen before, this is the\n",
     "      default behavior.\n",
     "\n",

transformers_doc/preprocessing.ipynb

@@ -436,7 +436,7 @@
    "metadata": {},
    "source": [
     "We have seen the commands that will work for most cases (pad your batch to the length of the maximum sentence and\n",
-    "truncate to the maximum length the mode can accept). However, the API supports more strategies if you need them. The\n",
+    "truncate to the maximum length the model can accept). However, the API supports more strategies if you need them. The\n",
     "three arguments you need to know for this are `padding`, `truncation` and `max_length`.\n",
     "\n",
     "- `padding` controls the padding. It can be a boolean or a string which should be:\n",
@@ -451,15 +451,16 @@
     "\n",
     "- `truncation` controls the truncation. It can be a boolean or a string which should be:\n",
     "\n",
-    "    - `True` or `'only_first'` truncate to a maximum length specified by the `max_length` argument or\n",
+    "    - `True` or `'longest_first'` truncate to a maximum length specified by the `max_length` argument or\n",
     "      the maximum length accepted by the model if no `max_length` is provided (`max_length=None`). This will\n",
-    "      only truncate the first sentence of a pair if a pair of sequence (or a batch of pairs of sequences) is provided.\n",
+    "      truncate token by token, removing a token from the longest sequence in the pair until the proper length is\n",
+    "      reached.\n",
     "    - `'only_second'` truncate to a maximum length specified by the `max_length` argument or the maximum\n",
     "      length accepted by the model if no `max_length` is provided (`max_length=None`). This will only truncate\n",
     "      the second sentence of a pair if a pair of sequence (or a batch of pairs of sequences) is provided.\n",
-    "    - `'longest_first'` truncate to a maximum length specified by the `max_length` argument or the maximum\n",
-    "      length accepted by the model if no `max_length` is provided (`max_length=None`). This will truncate token\n",
-    "      by token, removing a token from the longest sequence in the pair until the proper length is reached.\n",
+    "    - `'only_first'` truncate to a maximum length specified by the `max_length` argument or the maximum\n",
+    "      length accepted by the model if no `max_length` is provided (`max_length=None`). This will only truncate\n",
+    "      the first sentence of a pair if a pair of sequence (or a batch of pairs of sequences) is provided.\n",
     "    - `False` or `'do_not_truncate'` to not truncate the sequences. As we have seen before, this is the\n",
     "      default behavior.\n",
     "\n",

transformers_doc/pytorch/custom_datasets.ipynb

@@ -78,7 +78,18 @@
     "- Translation: translate a text in another language.\n",
     "- Feature extraction: return a tensor representation of the text.\n",
     "\n",
-    "Let's see how this work for sentiment analysis (the other tasks are all covered in the [task summary](https://huggingface.co/transformers/task_summary.html)):"
+    "Let's see how this work for sentiment analysis (the other tasks are all covered in the [task summary](https://huggingface.co/transformers/task_summary.html)):\n",
+    "\n",
+    "Install the following dependencies (if not already installed):"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "! pip install torch"
    ]
   },
   {
@@ -109,7 +120,7 @@
     {
      "data": {
       "text/plain": [
-       "[{'label': 'POSITIVE', 'score': 0.9997795224189758}]"
+       "[{'label': 'POSITIVE', 'score': 0.9998}]"
       ]
      },
      "execution_count": null,
@@ -125,8 +136,8 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "That's encouraging! You can use it on a list of sentences, which will be preprocessed then fed to the model as a\n",
-    "*batch*, returning a list of dictionaries like this one:"
+    "That's encouraging! You can use it on a list of sentences, which will be preprocessed then fed to the model, returning\n",
+    "a list of dictionaries like this one:"
    ]
   },
   {
@@ -157,6 +168,8 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
+    "To use with a large dataset, look at [iterating over a pipeline](https://huggingface.co/transformers/./main_classes/pipelines.html)\n",
+    "\n",
     "You can see the second sentence has been classified as negative (it needs to be positive or negative) but its score is\n",
     "fairly neutral.\n",
     "\n",
@@ -338,7 +351,8 @@
     {
      "data": {
       "text/plain": [
-       "{'input_ids': [101, 2057, 2024, 2200, 3407, 2000, 2265, 2017, 1996, 100, 19081, 3075, 1012, 102], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}"
+       "{'input_ids': [101, 2057, 2024, 2200, 3407, 2000, 2265, 2017, 1996, 100, 19081, 3075, 1012, 102],\n",
+       " 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}"
       ]
      },
      "execution_count": null,
@@ -453,7 +467,7 @@
      "data": {
       "text/plain": [
        "SequenceClassifierOutput(loss=None, logits=tensor([[-4.0833,  4.3364],\n",
-       "    [ 0.0818, -0.0418]], grad_fn=<AddmmBackward>), hidden_states=None, attentions=None)"
+       "        [ 0.0818, -0.0418]], grad_fn=<AddmmBackward>), hidden_states=None, attentions=None)"
       ]
      },
      "execution_count": null,
@@ -542,7 +556,7 @@
      "data": {
       "text/plain": [
        "SequenceClassifierOutput(loss=tensor(0.3167, grad_fn=<NllLossBackward>), logits=tensor([[-4.0833,  4.3364],\n",
-       "[ 0.0818, -0.0418]], grad_fn=<AddmmBackward>), hidden_states=None, attentions=None)"
+       "        [ 0.0818, -0.0418]], grad_fn=<AddmmBackward>), hidden_states=None, attentions=None)"
       ]
      },
      "execution_count": null,
@@ -588,8 +602,20 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "tokenizer.save_pretrained(save_directory)\n",
-    "model.save_pretrained(save_directory)"
+    "pt_save_directory = './pt_save_pretrained'\n",
+    "tokenizer.save_pretrained(pt_save_directory)\n",
+    "pt_model.save_pretrained(pt_save_directory)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tf_save_directory = './tf_save_pretrained'\n",
+    "tokenizer.save_pretrained(tf_save_directory)\n",
+    "tf_model.save_pretrained(tf_save_directory)"
    ]
   },
   {
@@ -609,8 +635,8 @@
    "outputs": [],
    "source": [
     "from transformers import TFAutoModel\n",
-    "tokenizer = AutoTokenizer.from_pretrained(save_directory)\n",
-    "model = TFAutoModel.from_pretrained(save_directory, from_pt=True)"
+    "tokenizer = AutoTokenizer.from_pretrained(pt_save_directory)\n",
+    "tf_model = TFAutoModel.from_pretrained(pt_save_directory, from_pt=True)"
    ]
   },
   {
@@ -627,8 +653,8 @@
    "outputs": [],
    "source": [
     "from transformers import AutoModel\n",
-    "tokenizer = AutoTokenizer.from_pretrained(save_directory)\n",
-    "model = AutoModel.from_pretrained(save_directory, from_tf=True)"
+    "tokenizer = AutoTokenizer.from_pretrained(tf_save_directory)\n",
+    "pt_model = AutoModel.from_pretrained(tf_save_directory, from_tf=True)"
    ]
   },
   {