Epic: AI Bridge

[dannykopping]

This is a parent ticket to track all the upcoming work on AI Bridge:

Phase 0 (Early Access):

• Implement in-memory aibridged server
• Implement delegated authentication to AI provider using session token
• Implement storage solution for prompts, tokens, tools usage
• Support OpenAI /v1/chat/completions streaming API
• Support OpenAI /v1/chat/completions non-streaming API
• Support Anthropic /v1/messages streaming API
• Support Anthropic /v1/messages non-streaming API
• Track user prompts
• Track token usage
• Track tool calls
• Support hosted MCP integrations using hardcoded auth token
• Ensure compatibility with major tools
  • Claude Code
  • Cursor
  • Windsurf
  • … others?
• Implement policy engine to evaluate rego at request/response time
  • Implement API to allow operators to CRUD rego policies (per org)
  • Allow/denylist for models
  • Allow/denylist for client-configured MCP tools
• Implement license entitlement

Phase 1 (Beta):

• Implement external aibridged server

• Handle all unsupported endpoints (i.e. OpenAI Responses API) by reverse-proxying

• Support Gemini streamGenerateContent API

• Support Gemini generateContent API

• Support hosted MCP integrations using external auth providers

• Expose key Prometheus metrics

  • Proxied requests counter (partitioned by provider)
  • Errors counter (partitioned by reason i.e. throttled, upstream error, etc)
  • Token use counter (partitioned by user, provider)
  • Prompts counter (partitioned by user, provider)
  • Tool use counter (partitioned by tool, user, provider, injected/builtin)

• UI: request log showing prompt, token, tool use
  
  

• UI: basic analytics view
  

• UI: configuration view (read-only)