Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .prettierrc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{
"printWidth": 120,
"singleQuote": true,
"trailingComma": "all"
}
}
6 changes: 3 additions & 3 deletions README.en-US.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ You must have an Azure OpenAI account to use the Azure OpenAI Proxy.
2. Clone the code in the command line window.
3. Run `npm install` to install dependencies.
4. Run `npm start` to start the application.
5. Run the script below for testing, replacing `YOUR_RESOURCE_ID`, `YOUR_MODEL_DEPLOYMENT`, and `YOUR_API_KEY` before running it.
5. Run the script below for testing, replacing `YOUR_RESOURCE_ID`, `YOUR_MODEL_DEPLOYMENT`, and `YOUR_API_KEY` before running it, `AZURE_API_VERSION` is optional and the default value is 2023-03-15-preview..
```bash
curl -X "POST" "http://localhost:3000/v1/chat/completions" \
-H 'Authorization: YOUR_RESOURCE_ID:YOUR_MODEL_DEPLOYMENT:YOUR_API_KEY' \
-H 'Authorization: YOUR_RESOURCE_ID:YOUR_MODEL_DEPLOYMENT:YOUR_API_KEY:AZURE_API_VERSION' \
-H 'Content-Type: application/json; charset=utf-8' \
-d $'{
"messages": [
Expand Down Expand Up @@ -59,7 +59,7 @@ Q: How do I support GPT-4?

A: To use GPT-4, please use key format as follows:

`YOUR_RESOURCE_ID:gpt-3.5-turbo|YOUR_MODEL_DEPLOYMENT,gpt-4|YOUR_MODEL_DEPLOYMENT,gpt-4-32k|YOUR_MODEL_DEPLOYMENT:YOUR_API_KEY`
`YOUR_RESOURCE_ID:gpt-3.5-turbo|YOUR_MODEL_DEPLOYMENT,gpt-4|YOUR_MODEL_DEPLOYMENT,gpt-4-32k|YOUR_MODEL_DEPLOYMENT:YOUR_API_KEY:AZURE_API_VERSION`

# How To Contribute Code?

Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@
2. 克隆代码到命令行窗口。
3. 运行 `npm install` 安装依赖项。
4. 运行 `npm start` 启动应用程序。
5. 运行下面脚本测试,运行前需要把`YOUR_RESOURCE_ID`,`YOUR_MODEL_DEPLOYMENT`,`YOUR_API_KEY`替换
5. 运行下面脚本测试,运行前需要把`YOUR_RESOURCE_ID`,`YOUR_MODEL_DEPLOYMENT`,`YOUR_API_KEY`, `AZURE_API_VERSION`替换,`AZURE_API_VERSION`参数可选,目前默认是2023-03-15-preview
```bash
curl -X "POST" "http://localhost:3000/v1/chat/completions" \
-H 'Authorization: YOUR_RESOURCE_ID:YOUR_MODEL_DEPLOYMENT:YOUR_API_KEY' \
-H 'Authorization: YOUR_RESOURCE_ID:YOUR_MODEL_DEPLOYMENT:YOUR_API_KEY:AZURE_API_VERSION' \
-H 'Content-Type: application/json; charset=utf-8' \
-d $'{
"messages": [
Expand Down Expand Up @@ -59,7 +59,7 @@ Q: 如何支持GPT-4

A: 要使用GPT-4,请使用下列格式的key:

`YOUR_RESOURCE_ID:gpt-3.5-turbo|YOUR_MODEL_DEPLOYMENT,gpt-4|YOUR_MODEL_DEPLOYMENT,gpt-4-32k|YOUR_MODEL_DEPLOYMENT:YOUR_API_KEY`
`YOUR_RESOURCE_ID:gpt-3.5-turbo|YOUR_MODEL_DEPLOYMENT,gpt-4|YOUR_MODEL_DEPLOYMENT,gpt-4-32k|YOUR_MODEL_DEPLOYMENT:YOUR_API_KEY:AZURE_API_VERSION`

# 贡献代码方式

Expand Down
11 changes: 9 additions & 2 deletions src/app.controller.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Controller, Get, Post, Req, Res } from '@nestjs/common';
import { Controller, Get, Logger, Post, Req, Res } from '@nestjs/common';
import { AxiosHeaders } from 'axios';
import { Request, Response } from 'express';
import { AppService } from './app.service';
Expand All @@ -13,8 +13,11 @@ export class AppController {
}
}

const DEFAULT_API_VERSION = '2023-03-15-preview';

@Controller('chat')
export class ChatController {
private readonly logger = new Logger(ChatController.name);
constructor(private readonly appService: AppService) {}

@Get()
Expand All @@ -26,7 +29,10 @@ export class ChatController {
async completions(@Req() request: Request, @Res() res: Response) {
const auth = request.headers['authorization'];
const apiKey = auth.replace('Bearer ', '');
const [resource_id, deployment_id, azureApiKey] = apiKey.split(':');
const [resource_id, deployment_id, azureApiKey, apiVersion] = apiKey.split(':');
this.logger.debug(
`resource_id: ${resource_id}, deployment_id: ${deployment_id}, azureApiKey: ${azureApiKey}, apiVersion: ${apiVersion}`,
);
const endpoint = `https://${resource_id}.openai.azure.com`;
const stream = request.body['stream'];
const response = await this.appService.getCompletions(
Expand All @@ -35,6 +41,7 @@ export class ChatController {
azureApiKey,
request.body,
stream,
apiVersion || DEFAULT_API_VERSION,
);

// set response headers
Expand Down
17 changes: 8 additions & 9 deletions src/app.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,11 @@ export class AppService {
azureApiKey: string,
body: any,
stream: boolean,
apiVersion: string,
) {
const deployment_id = this.getDeploymentId(mapping, body['model']);
this.logger.debug(`deployment_id: ${deployment_id}`);
const url = `${endpoint}/openai/deployments/${deployment_id}/chat/completions?api-version=2023-03-15-preview`;
const url = `${endpoint}/openai/deployments/${deployment_id}/chat/completions?api-version=${apiVersion}`;
const headers = {
'api-key': azureApiKey,
'Content-Type': 'application/json',
Expand All @@ -46,14 +47,12 @@ export class AppService {
this.logger.debug(`mapping: ${mapping}, model: ${model}`);
if (mapping.includes(',')) {
let defaultDeploymentId = '';
const modelMapping = mapping
.split(',')
.reduce((acc: Record<string, string>, pair: string) => {
const [key, value] = pair.split('|');
if (defaultDeploymentId === '') defaultDeploymentId = value;
acc[key] = value;
return acc;
}, {});
const modelMapping = mapping.split(',').reduce((acc: Record<string, string>, pair: string) => {
const [key, value] = pair.split('|');
if (defaultDeploymentId === '') defaultDeploymentId = value;
acc[key] = value;
return acc;
}, {});
if (!model) {
return defaultDeploymentId;
}
Expand Down
2 changes: 1 addition & 1 deletion src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { AppModule } from './app.module';

async function bootstrap() {
const app = await NestFactory.create(AppModule, {
logger: ['log'],
logger: ['debug'],
});
app.setGlobalPrefix('v1');
await app.listen(3000);
Expand Down
Loading