wheels for TransformerEngine

llm-d enables high-performance distributed LLM inference on Kubernetes

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Scalable toolkit for efficient model reinforcement

A simple Python sandbox for helpful LLM data agents

Async RL Training at Scale

🚴 Call stack profiler for Python. Shows you why your code is slow!

Single File, Single GPU, From Scratch, Efficient, Full Parameter Tuning library for "RL for LLMs"

APPS: Automated Programming Progress Standard (NeurIPS 2021)

Realtime log viewer for containers. Supports Docker, Swarm and K8s.

A lightweight data processing framework built on DuckDB and 3FS.

Toolkit for linearizing PDFs for LLM datasets/training

Fast reinforcement learning 💨

A FlashAttention implementation for JAX with support for efficient document mask computation and context parallelism.

Environments for LLM Reinforcement Learning

A PyTorch native platform for training generative AI models

nanoGRPO is a lightweight implementation of Group Relative Policy Optimization (GRPO)

JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).

FlashMLA: Efficient Multi-head Latent Attention Kernels

Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation

Hardware-Accelerated Reinforcement Learning Algorithms in pure Jax!

[NeurIPS 2025 Spotlight] Reasoning Environments for Reinforcement Learning with Verifiable Rewards

🚀🚀 「大模型」2小时完全从0训练26M的小参数GPT！🌏 Train a 26M-parameter GPT from scratch in just 2h!

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

s1: Simple test-time scaling

Unikernel Linux