Commits

Breadcrumbs

History for

onv4.10.3

User selector

All users

Datepicker

All time

Commit History

Commits on Aug 23, 2021

• Change how "additional_special_tokens" argument in the ".from_pretrained" method of the tokenizer is taken into account (#13056)

  Show description for 7223844
  SaulLu

  authored
  7223844

  Copy full SHA for 7223844

Commits on Jun 1, 2021

• Add regression tests for slow sentencepiece tokenizers. (#11737)

  Show description for fcad801
  PhilipMay

  authored
  fcad801

  Copy full SHA for fcad801

Commits on May 13, 2021

• Enable option for subword regularization in more tokenizers. (#11417)

  Show description for 37ed3ab
  PhilipMay

  authored
  37ed3ab

  Copy full SHA for 37ed3ab

Commits on May 4, 2021

• Enable added tokens (#11325)

  Show description for 09b0bcf
  LysandreJik

  authored
  09b0bcf

  Copy full SHA for 09b0bcf

Commits on Mar 16, 2021

• Flax testing should not run the full torch test suite (#10725)

  Show description for 9f8619c
  patrickvonplaten

  authored
  9f8619c

  Copy full SHA for 9f8619c

Commits on Feb 22, 2021

• Deprecate prepare_seq2seq_batch (#10287)

  Show description for 9e147d3
  

  3 peopleauthored
  9e147d3

  Copy full SHA for 9e147d3

Commits on Jan 6, 2021

• Fast transformers import part 1 (#9441)

  Show description for 0c96262
  sgugger

  authored
  0c96262

  Copy full SHA for 0c96262

Commits on Nov 10, 2020

• fix t5 token type ids (#8437)

  patrickvonplaten

  authored
  70708cc

  Copy full SHA for 70708cc

• fix t5 special tokens (#8435)

  patrickvonplaten

  authored
  b935694

  Copy full SHA for b935694

Commits on Oct 18, 2020

• [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659)

  Show description for ba8c4d0
  

  thomwolf

  and

  sgugger

  authored
  ba8c4d0

  Copy full SHA for ba8c4d0

Commits on Oct 9, 2020

• [pegasus] Faster tokenizer tests (#7672)

  stas00

  authored
  b0f05e0

  Copy full SHA for b0f05e0

Commits on Oct 8, 2020

• Adding Fast tokenizers for SentencePiece based tokenizers - Breaking: remove Transfo-XL fast tokenizer (#7141)

  Show description for 9aeacb5
  

  thomwolf

  and

  sgugger

  authored
  9aeacb5

  Copy full SHA for 9aeacb5

Commits on Sep 11, 2020

• [T5Tokenizer] remove prefix_tokens (#7078)

  patil-suraj

  authored
  0a8c17d

  Copy full SHA for 0a8c17d

Commits on Sep 10, 2020

• Add "Leveraging Pretrained Checkpoints for Generation" Seq2Seq models. (#6594)

  Show description for 7fd1feb
  patrickvonplaten

  authored
  7fd1feb

  Copy full SHA for 7fd1feb

Commits on Aug 30, 2020

• [tests] fix typos in inputs (#6818)

  stas00

  authored
  563485b

  Copy full SHA for 563485b

Commits on Aug 28, 2020

• t5 model should make decoder_attention_mask (#6800)

  sshleifer

  authored
  3cac867

  Copy full SHA for 3cac867

• prepare_seq2seq_batch makes labels/ decoder_input_ids made later. (#6654)

  Show description for 9336086
  sshleifer

  authored
  9336086

  Copy full SHA for 9336086

Commits on Aug 26, 2020

• Black 20 release

  LysandreJik

  committed
  a75c64d

  Copy full SHA for a75c64d

Commits on Aug 25, 2020

• T5Tokenizer adds EOS token if not already added (#5866)

  Show description for 6244957
  

  sshleifer

  and

  sgugger

  authored
  6244957

  Copy full SHA for 6244957

Commits on Aug 17, 2020

• [T5Tokenizer] add prepare_seq2seq_batch method (#6122)

  Show description for 407da12
  patil-suraj

  authored
  407da12

  Copy full SHA for 407da12

Commits on May 19, 2020

• [cleanup] test_tokenization_common.py (#4390)

  sshleifer

  authored
  07dd7c2

  Copy full SHA for 07dd7c2

Commits on Jan 15, 2020

• 💄 super

  julien-c

  committed
  83a41d3

  Copy full SHA for 83a41d3

Commits on Jan 6, 2020

• GPU text generation: mMoved the encoded_prompt to correct device

  

  alberduris

  authored and

  LysandreJik

  committed
  81d6841

  Copy full SHA for 81d6841

• Moved the encoded_prompts to correct device

  

  alberduris

  authored and

  LysandreJik

  committed
  dd4df80

  Copy full SHA for dd4df80

Commits on Dec 22, 2019

• Remove __future__ imports.

  aaugustin

  committed
  c824d15

  Copy full SHA for c824d15

• Replace CommonTestCases for tokenizers with a mixin.

  Show description for 00204f2
  aaugustin

  committed
  00204f2

  Copy full SHA for 00204f2

• Rename file for consistency.

  aaugustin

  committed
  a3c5883

  Copy full SHA for a3c5883

• Remove unittest.main() in test modules.

  Show description for 7e98e21
  aaugustin

  committed
  7e98e21

  Copy full SHA for 7e98e21

• Switch test files to the standard test_*.py scheme.

  aaugustin

  committed
  ced0a94

  Copy full SHA for ced0a94

Loading