Ongoing research training transformer models at scale

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Train transformer language models with reinforcement learning.

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

A General NeRF Acceleration Toolbox in PyTorch.

A collaboration friendly studio for NeRFs

Dr.Jit — A Just-In-Time-Compiler for Differentiable Rendering

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

Repo for external large-scale work

A framework for few-shot evaluation of autoregressive language models.

Plenoxels: Radiance Fields without Neural Networks, Code release WIP

Evaluate T0 with DeepSpeed

Task-based datasets, preprocessing, and evaluation for sequence models.

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

A distributed deep learning framework that supports flexible parallelization strategies.

Codebase for the engineering/scaling WG

Ongoing research training transformer language models at scale, including: BERT & GPT-2

🤗 The largest hub of ready-to-use NLP datasets for ML models with fast, easy-to-use and efficient data manipulation tools

🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.

Machine Learning applied to Natural Language Processing Toolkit used in the Lisbon Machine Learning Summer School

Toolkit for collecting and applying templates of prompting instances