SGLang is a fast serving framework for large language models and vision language models.

A high-throughput and memory-efficient inference and serving engine for LLMs

AKShare is an elegant and simple financial data interface library for Python, built for human beings! 开源财经数据接口库

Nano vLLM

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.

High-quality single file implementation of Deep Reinforcement Learning algorithms with research-friendly features (PPO, DQN, C51, DDPG, TD3, SAC, PPG)

Production ready LLM model compression/quantization toolkit with accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.

Repository of LV-Eval Benchmark

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

巴菲特致股东的信1957-2024,巴菲特先生的信一直是价值投资的经典范本，超越时间，超越周期，站在资本与金融的顶峰，他和他的战友芒格先生事价值投资的坚定实践者。Warren E. Buffett's Letter,To the Stockholders of Berkshire Hathaway Inc

The Triton TensorRT-LLM Backend

Code Repository of Evaluating Quantized Large Language Models

Official github repo for C-Eval, a Chinese evaluation suite for foundation models [NeurIPS 2023]

CMMLU: Measuring massive multitask language understanding in Chinese

[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

An awesome & curated list of best LLMOps tools for developers

Public repo for HF blog posts

Making large AI models cheaper, faster and more accessible

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.