This repository was archived by the owner on Jan 21, 2025. It is now read-only.

Commits

History for

Commits on Nov 17, 2023

Migrate references and remove legacy target tpu:tpu_estimator.

jam14j
authored and
Mesh TensorFlow Team
committed

Commits on Feb 18, 2022

#HyperPrompt Part 2 of HyperPrompt implementation: the actual computation of HyperPrompt inside self-attention layer.
Mesh TensorFlow Team
committed

Commits on Jan 19, 2022

fix typo in logging statement.
Mesh TensorFlow Team
committed

Commits on Sep 8, 2021

Expert Attention Fixes:
Mesh TensorFlow Team
committed

Commits on Sep 7, 2021

Expert Attention Fixes:
Mesh TensorFlow Team
committed

Commits on Aug 16, 2021

Fix bug in shared_kv attention for autoregressive decoding.
Mesh TensorFlow Team
committed

Commits on Aug 10, 2021

Add more options to Experts Attention. These options remove 1/3 of the all2all communication costs:
Mesh TensorFlow Team
committed

Commits on Aug 2, 2021

Minor changes to make Experts Attention work.
Mesh TensorFlow Team
committed

Commits on Jul 27, 2021

Add z_loss on all attention logits. This does not change model quality and can effectively decrease the attention logits by order of magnitude.
Mesh TensorFlow Team
committed

Commits on Jun 9, 2021

Unique variable names for ParallelLayer

William Fedus
authored and
Mesh TensorFlow Team
committed

Commits on Mar 5, 2021

[MTF] Maybe context.length_dim doesn't exist outside context.train?

afrozenator
authored and
Mesh TensorFlow Team
committed
[Mesh-TF] Add is_training as an arg to mtf.dropout

afrozenator
authored and
Mesh TensorFlow Team
committed

Commits on Feb 2, 2021

no public changes

nshazeer
authored and
Mesh TensorFlow Team
committed

Commits on Jan 19, 2021

Clarify names within Mesh Tensorflow

William Fedus
authored and
Mesh TensorFlow Team
committed

Commits on Jan 15, 2021

add ParallelLayer

nshazeer
authored and
Mesh TensorFlow Team
committed

Commits on Jan 5, 2021

Fix decoding bug for Funnel Transformer

hwchung27
authored and
Mesh TensorFlow Team
committed

Commits on Dec 7, 2020

Access T5 SentencePieceVocabulary directly from t5.data since t5.data.sentencepiece_vocabulary will soon be deleted.

adarob
authored and
Mesh TensorFlow Team
committed

Commits on Nov 9, 2020

Implement incremental decoding for various convolutional layers.

hwchung27
authored and
Mesh TensorFlow Team
committed
Implement state update mechanisms for convolution-based incremental decoding.

hwchung27
authored and
Mesh TensorFlow Team
committed

Commits on Nov 3, 2020

Make Conv1DLayer work with the same input and output dims.

hwchung27
authored and
Mesh TensorFlow Team
committed

Commits on Oct 22, 2020

Support custom subclasses of attention layers in the following ways:

nshazeer
authored and
Mesh TensorFlow Team
committed

Commits on Sep 10, 2020

Fixed NaNs in Synthesizer (Add fold_scaling_into_initializer into Synthesizer and fixed default gin in AT5 to set it to False.)
Mesh TensorFlow Team
committed

Commits on Sep 1, 2020

Create ExpertsAttention layer

William Fedus
authored and
Mesh TensorFlow Team
committed

Commits on Aug 19, 2020

Move layers for Evolved Transformer, Weighted Transformer, Lightweight convolutions and dynamic convolutions to OSS Mesh code.

sharannarang
authored and
Mesh TensorFlow Team
committed

Commits on Aug 11, 2020

Fixed some bugs for Synthesizer model.
Mesh TensorFlow Team
committed

Commits on Jun 24, 2020

Add Synthesizer Model.
Mesh TensorFlow Team
committed

Commits on Jun 19, 2020

Option to make transformer more identical to the original.

nshazeer
authored and
Mesh TensorFlow Team
committed

Commits on Jun 11, 2020

Transfomrer: automitically split the attention computation when using

nshazeer
authored and
Mesh TensorFlow Team
committed

Commits on Apr 14, 2020

Add use_bias argument to DenseReluDense.

adarob
authored and
Mesh TensorFlow Team
committed

Commits on Apr 6, 2020

Implement and describe the Unit-Scaling Convention:

nshazeer
authored and
Mesh TensorFlow Team
committed

Commits on Apr 1, 2020

Enable encoder-decoder relative attention for span-filling objectives.

nshazeer
authored and
Mesh TensorFlow Team
committed

Commits on Mar 27, 2020

Enable custom gradients in mesh-tensorflow.

nshazeer
authored and
Mesh TensorFlow Team
committed

Commits on Mar 25, 2020

Move the logic of the EncDecAttention layer into a function so that it can be reused in other layers + Add an option to keep the query heads dims in the output of AttentionParams.
Mesh TensorFlow Team
committed

Commits on Mar 18, 2020

Move the logic of the EncDecAttention layer into a function so that it can be reused in other layers + Add an option to keep the query heads dims in the output of AttentionParams.
Mesh TensorFlow Team
committed

Commits on Mar 17, 2020

Move the logic of the EncDecAttention layer into a function so that it can be reused in other layers + Add an option to keep the query heads dims in the output of AttentionParams.
Mesh TensorFlow Team
committed