A high-throughput and memory-efficient inference and serving engine for LLMs

Fast and memory-efficient exact attention

Python library for Synthetic Data Generation

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Distributed Model Serving Framework

Large Language Model Text Generation Inference

🏎️ Accelerate training and inference of 🤗 Transformers with easy to use hardware optimization tools

The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)

Create, manipulate, apply and delete Kubernetes resource manifests at runtime

Fast Inference Solutions for BLOOM

KServe V2 Protocol Rest API Implementation Proxy

Serverless Inferencing on Kubernetes

Operator to install IBM Common Services

Unified runtime-adapter image of the sidecar containers which run in the modelmesh pods

The Java gRPC implementation. HTTP/2 based RPC

An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more

High-performance netty and thrift-based microservice RPC library for Java

Distributed reliable key-value store for the most critical data of a distributed system

Netty project - an event-driven asynchronous network application framework

A flexible, high-performance serving system for machine learning models

Argo Workflows: Get stuff done with Kubernetes.

Prometheus instrumentation library for JVM applications

Lightweight Kubernetes controllers as a service

Google core libraries for Java

a java statsd client library

Vert.x is a tool-kit for building reactive applications on the JVM

Mirror of Apache Curator

Java Agent for Memory Measurements

A ConcurrentLinkedHashMap for Java