|
1 | | -# M5Module-LLM_OpenAI_API |
| 1 | +# OpenAI Compatible API Server For StackFlow |
2 | 2 |
|
3 | 3 | ## Overview |
| 4 | +This server provides an OpenAI-compatible API interface supporting multiple AI model backends including LLMs, vision models, speech synthesis (TTS), and speech recognition (ASR). |
4 | 5 |
|
5 | | -M5Module-LLM_OpenAI_API is an open source Python framework that provides an OpenAI compatibility layer for StackFlow. The framework is compatible with core concepts, such as Text generation and prompting, Text to speech, and Speech to text. It supports direct calls from apps such as Chatbox. Quickly realize your AI needs. |
| 6 | +## Quick Start |
6 | 7 |
|
7 | | -## Related Link |
| 8 | +1. Install dependencies: |
| 9 | +```bash |
| 10 | +pip install -r requirements.txt |
| 11 | +``` |
8 | 12 |
|
9 | | -- [Document & Datasheet](https://docs.m5stack.com/en/module/Module-LLM) |
| 13 | +3. Start the server: |
| 14 | +```bash |
| 15 | +python main.py |
| 16 | +``` |
| 17 | + |
| 18 | +## Supported Endpoints |
| 19 | + |
| 20 | +### Chat Completions |
| 21 | +- **Endpoint**: `POST /v1/chat/completions` |
| 22 | +- **Request Format**: OpenAI-compatible chat completion request |
| 23 | +- **Streaming**: Supported |
| 24 | + |
| 25 | +### Text Completions |
| 26 | +- **Endpoint**: `POST /v1/completions` |
| 27 | +- **Request Format**: OpenAI-compatible completion request |
| 28 | +- **Streaming**: Supported |
| 29 | + |
| 30 | +### Speech Synthesis (TTS) |
| 31 | +- **Endpoint**: `POST /v1/audio/speech` |
| 32 | +- **Parameters**: |
| 33 | + - `model`: TTS model name |
| 34 | + - `input`: Text to synthesize |
| 35 | + - `voice`: Voice type |
| 36 | + - `response_format`: Audio format (mp3, wav, etc.) |
| 37 | + |
| 38 | +### Speech Recognition (ASR) |
| 39 | +- **Transcription**: `POST /v1/audio/transcriptions` |
| 40 | + - Converts speech to text in the same language |
| 41 | +- **Translation**: `POST /v1/audio/translations` |
| 42 | + - Converts speech to English text |
| 43 | +- **Parameters**: |
| 44 | + - `file`: Audio file |
| 45 | + - `model`: ASR model name |
| 46 | + - `language` (transcription only): Source language |
| 47 | + - `prompt`: Optional prompt |
| 48 | + |
| 49 | +### List Models |
| 50 | +- **Endpoint**: `GET /v1/models` |
| 51 | +- **Returns**: List of available models |
| 52 | + |
| 53 | +## FAQ |
| 54 | + |
| 55 | +### Q: Why am I getting "Unsupported model" errors? |
| 56 | +A: The model name must exactly match one of the configured models in your config file. |
| 57 | + |
| 58 | +### Q: How do I enable streaming responses? |
| 59 | +A: Set `"stream": true` in your request body for chat/completion endpoints. |
| 60 | + |
| 61 | +### Q: What audio formats are supported for ASR? |
| 62 | +A: The supported formats depend on your ASR backend implementation. |
| 63 | + |
| 64 | +### Q: How do I manage model memory usage? |
| 65 | +A: The server implements a pool system for LLM models - adjust `pool_size` in the config to control concurrent instances. |
| 66 | + |
| 67 | +## Troubleshooting |
| 68 | + |
| 69 | +- **Logs**: Check server logs for detailed error messages |
| 70 | +- **Model Initialization**: Verify all required backend services are running |
| 71 | +- **Configuration**: Double-check model names and parameters in config.yaml |
| 72 | + |
| 73 | +## Example Requests |
| 74 | + |
| 75 | +### Chat Completion |
| 76 | +```bash |
| 77 | +curl -X POST "http://localhost:8000/v1/chat/completions" \ |
| 78 | +-H "Content-Type: application/json" \ |
| 79 | +-H "Authorization: Bearer YOUR_KEY" \ |
| 80 | +-d '{ |
| 81 | + "model": "gpt-3.5-turbo", |
| 82 | + "messages": [{"role": "user", "content": "Hello!"}], |
| 83 | + "temperature": 0.7 |
| 84 | +}' |
| 85 | +``` |
| 86 | + |
| 87 | +### Speech Synthesis |
| 88 | +```bash |
| 89 | +curl -X POST "http://localhost:8000/v1/audio/speech" \ |
| 90 | +-H "Content-Type: application/json" \ |
| 91 | +-H "Authorization: Bearer YOUR_KEY" \ |
| 92 | +-d '{ |
| 93 | + "model": "tts-1", |
| 94 | + "input": "Hello world!", |
| 95 | + "voice": "alloy" |
| 96 | +}' |
| 97 | +``` |
10 | 98 |
|
11 | 99 | ## Required Libraries: |
12 | 100 |
|
|
0 commit comments