Skip to content

Commit 415a614

Browse files
feat(providers): add OpenAI API package and migrate capabilities to mixins (withceleste#82)
* feat(providers): add OpenAI API package Add standalone provider package for OpenAI APIs with mixin pattern for capability-agnostic reuse across multiple endpoints. ## Images API (OpenAIImagesClient) - HTTP POST/streaming to /v1/images/generations endpoint - Supports DALL-E 2/3 (b64_json format) and gpt-image-1 (streaming) - Usage parsing: input_tokens, output_tokens, total_tokens - Content extraction from data array - Revised prompt handling in metadata ## Responses API (OpenAIResponsesClient) - HTTP POST/streaming to /v1/responses endpoint - Unified API for text generation capabilities - Usage parsing with cached and reasoning tokens support - Content extraction from output array - Finish reason mapping (completed status) ## Videos API (OpenAIVideosClient) - Async polling workflow for video generation - Phase 1: POST to /v1/videos to create job - Phase 2: Poll GET /v1/videos/{id} until completed/failed - Phase 3: GET /v1/videos/{id}/content to retrieve video - Usage parsing: billing units (seconds) - Multipart request support for input_reference images ## Audio API (OpenAIAudioClient) - HTTP POST to /v1/audio/speech endpoint - Binary audio response handling - Response format mapping (mp3, opus, aac, flac, wav, pcm) - MIME type conversion to AudioMimeType All clients follow the mixin pattern for reuse across capabilities. * fix(providers): add _parse_usage to GoogleVeoClient Add _parse_usage method to GoogleVeoClient to resolve mypy type checking error. GoogleVeoClient was missing this implementation, causing unsafe super() call in GoogleVideoGenerationClient. Google Veo API doesn't return usage data in the response, so this method returns an empty dict that capability clients can wrap in their Usage type. * refactor(speech-generation): rename RESPONSE_FORMAT to OUTPUT_FORMAT Standardize parameter naming across speech generation capability. Changes unified parameter name from RESPONSE_FORMAT to OUTPUT_FORMAT to match enum definition and be more consistent with other capabilities. Breaking change: Updates parameter name in: - SpeechGenerationParameter enum - SpeechGenerationParameters class - All ElevenLabs models (9 occurrences) - All OpenAI models (3 occurrences) - Provider parameter mappers * refactor(image-generation): migrate OpenAI and Google to provider mixins Migrate image generation capability clients to use provider package mixins, eliminating code duplication and centralizing API-specific logic. ## Changes - OpenAI client now inherits from OpenAIImagesClient mixin - Parameter mappers inherit from provider package mappers - Google client uses super()._parse_usage() pattern - Remove unused config.py file (config now in provider package) - Remove revised_prompt handling from provider mixin (handled in capability) ## Code Reduction - ~188 lines removed across client and parameter files - Significant deduplication of HTTP request logic * refactor(speech-generation): migrate OpenAI and ElevenLabs to provider mixins Migrate speech generation capability clients to use provider package mixins, eliminating code duplication and centralizing API-specific logic. ## Changes - OpenAI client now inherits from OpenAIAudioClient mixin - Parameter mappers inherit from provider package mappers - Add SpeechGenerationFinishReason type for consistency - Remove unused config.py file (config now in provider package) - Update _create_inputs to use parameters.get() pattern - Simplify VoiceConstraint docstring - Update tests to reflect new structure ## Code Reduction - ~187 lines removed across client and parameter files - Significant deduplication of HTTP request and parameter mapping logic * refactor(video-generation): migrate OpenAI and Google to provider mixins Migrate video generation capability clients to use provider package mixins, eliminating code duplication and centralizing API-specific logic. ## Changes - OpenAI client now inherits from OpenAIVideosClient mixin - Parameter mappers inherit from provider package mappers - Google client uses super()._parse_usage() pattern (after Commit 1 fix) - Remove unused config.py file (config now in provider package) - Remove async polling logic (now in provider mixin) - Simplify _parse_usage to use mixin's implementation ## Code Reduction - ~178 lines removed across client and parameter files - Significant deduplication of HTTP request and polling logic * refactor(text-generation): migrate OpenAI to provider mixins and remove config files Migrate text generation capability client to use provider package mixins, eliminating code duplication and centralizing API-specific logic. ## Changes - OpenAI client now inherits from OpenAIResponsesClient mixin - Parameter mappers inherit from provider package mappers - Streaming class inherits from OpenAIResponsesStream mixin - Remove unused config.py file (config now in provider package) - Simplify _parse_content to use mixin's output array parsing - Simplify _parse_finish_reason to use mixin's implementation ## Code Reduction - ~429 lines removed across client, parameters, and streaming files - Significant deduplication of HTTP request, streaming, and schema logic * fix(capabilities): add celeste-openai dependency to all capabilities Add celeste-openai to [tool.uv.sources] in all capability packages that now import from the OpenAI provider package after the mixin migration. ## Changes - image-generation: Add celeste-openai (imports celeste_openai.images) - speech-generation: Add celeste-openai (imports celeste_openai.audio) - text-generation: Add celeste-openai (imports celeste_openai.responses) - video-generation: Add celeste-openai (imports celeste_openai.videos) This ensures workspace dependencies are properly declared for the refactored capability clients that use OpenAI provider mixins.
1 parent b6a514d commit 415a614

File tree

51 files changed

+1383
-924
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

51 files changed

+1383
-924
lines changed

packages/capabilities/image-generation/pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ Issues = "https://github.com/withceleste/celeste-python/issues"
2929
celeste-ai = { workspace = true }
3030
celeste-bfl = { workspace = true }
3131
celeste-google = { workspace = true }
32+
celeste-openai = { workspace = true }
3233

3334
[project.entry-points."celeste.packages"]
3435
image-generation = "celeste_image_generation:register_package"

packages/capabilities/image-generation/src/celeste_image_generation/providers/google/imagen.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,8 +39,8 @@ def _init_request(self, inputs: ImageGenerationInput) -> dict[str, Any]:
3939

4040
def _parse_usage(self, response_data: dict[str, Any]) -> ImageGenerationUsage:
4141
"""Parse usage from response."""
42-
predictions = response_data.get("predictions", [])
43-
return ImageGenerationUsage(num_images=len(predictions))
42+
usage = super()._parse_usage(response_data)
43+
return ImageGenerationUsage(**usage)
4444

4545
def _parse_content(
4646
self,
Lines changed: 9 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,11 @@
11
"""OpenAI client implementation for image generation."""
22

33
import base64
4-
from collections.abc import AsyncIterator
54
from typing import Any, Unpack
65

7-
import httpx
6+
from celeste_openai.images.client import OpenAIImagesClient
87

98
from celeste.artifacts import ImageArtifact
10-
from celeste.mime_types import ApplicationMimeType
119
from celeste.parameters import ParameterMapper
1210
from celeste_image_generation.client import ImageGenerationClient
1311
from celeste_image_generation.io import (
@@ -17,12 +15,11 @@
1715
)
1816
from celeste_image_generation.parameters import ImageGenerationParameters
1917

20-
from . import config
2118
from .parameters import OPENAI_PARAMETER_MAPPERS
2219
from .streaming import OpenAIImageGenerationStream
2320

2421

25-
class OpenAIImageGenerationClient(ImageGenerationClient):
22+
class OpenAIImageGenerationClient(OpenAIImagesClient, ImageGenerationClient):
2623
"""OpenAI client for image generation."""
2724

2825
@classmethod
@@ -44,19 +41,17 @@ def _init_request(self, inputs: ImageGenerationInput) -> dict[str, Any]:
4441

4542
def _parse_usage(self, response_data: dict[str, Any]) -> ImageGenerationUsage:
4643
"""Parse usage from response."""
47-
return ImageGenerationUsage()
44+
usage = super()._parse_usage(response_data)
45+
return ImageGenerationUsage(**usage)
4846

4947
def _parse_content(
5048
self,
5149
response_data: dict[str, Any],
5250
**parameters: Unpack[ImageGenerationParameters],
5351
) -> ImageArtifact:
5452
"""Parse content from response."""
55-
data = response_data.get("data", [])
56-
if not data:
57-
msg = "No image data in response"
58-
raise ValueError(msg)
59-
53+
# Use mixin's _parse_content to get data array
54+
data = super()._parse_content(response_data)
6055
image_data = data[0]
6156

6257
b64_json = image_data.get("b64_json")
@@ -73,62 +68,13 @@ def _parse_content(
7368

7469
def _parse_finish_reason(
7570
self, response_data: dict[str, Any]
76-
) -> ImageGenerationFinishReason | None:
77-
"""Parse finish reason from response."""
78-
return None
79-
80-
def _build_metadata(self, response_data: dict[str, Any]) -> dict[str, Any]:
81-
"""Build metadata dictionary from response data."""
82-
metadata = super()._build_metadata(response_data)
83-
# Add provider-specific parsed fields
84-
if response_data.get("data") and response_data["data"]:
85-
revised_prompt = response_data["data"][0].get("revised_prompt")
86-
if revised_prompt:
87-
metadata["revised_prompt"] = revised_prompt
88-
return metadata
89-
90-
async def _make_request(
91-
self,
92-
request_body: dict[str, Any],
93-
**parameters: Unpack[ImageGenerationParameters],
94-
) -> httpx.Response:
95-
"""Make HTTP request(s) and return response object."""
96-
headers = {
97-
**self.auth.get_headers(),
98-
"Content-Type": ApplicationMimeType.JSON,
99-
}
100-
101-
return await self.http_client.post(
102-
f"{config.BASE_URL}{config.ENDPOINT}",
103-
headers=headers,
104-
json_body=request_body,
105-
)
71+
) -> ImageGenerationFinishReason:
72+
"""OpenAI Images API doesn't provide finish reasons."""
73+
return ImageGenerationFinishReason(reason=None)
10674

10775
def _stream_class(self) -> type[OpenAIImageGenerationStream]:
10876
"""Return the Stream class for this client."""
10977
return OpenAIImageGenerationStream
11078

111-
def _make_stream_request(
112-
self,
113-
request_body: dict[str, Any],
114-
**parameters: Unpack[ImageGenerationParameters],
115-
) -> AsyncIterator[dict[str, Any]]:
116-
"""Make HTTP streaming request and return async iterator of events."""
117-
request_body["stream"] = True
118-
119-
if "partial_images" not in request_body:
120-
request_body["partial_images"] = 1
121-
122-
headers = {
123-
**self.auth.get_headers(),
124-
"Content-Type": ApplicationMimeType.JSON,
125-
}
126-
127-
return self.http_client.stream_post(
128-
f"{config.BASE_URL}{config.STREAM_ENDPOINT}",
129-
headers=headers,
130-
json_body=request_body,
131-
)
132-
13379

13480
__all__ = ["OpenAIImageGenerationClient"]

packages/capabilities/image-generation/src/celeste_image_generation/providers/openai/config.py

Lines changed: 0 additions & 10 deletions
This file was deleted.

packages/capabilities/image-generation/src/celeste_image_generation/providers/openai/parameters.py

Lines changed: 14 additions & 97 deletions
Original file line numberDiff line numberDiff line change
@@ -1,113 +1,30 @@
1-
"""OpenAI parameter mappers for image generation."""
1+
"""OpenAI Images parameter mappers for image generation."""
2+
3+
from celeste_openai.images.parameters import (
4+
PartialImagesMapper as _PartialImagesMapper,
5+
)
6+
from celeste_openai.images.parameters import (
7+
QualityMapper as _QualityMapper,
8+
)
9+
from celeste_openai.images.parameters import (
10+
SizeMapper as _SizeMapper,
11+
)
212

3-
from typing import Any
4-
5-
from celeste import Model
613
from celeste.parameters import ParameterMapper
714
from celeste_image_generation.parameters import ImageGenerationParameter
815

916

10-
class AspectRatioMapper(ParameterMapper):
11-
"""Map aspect_ratio parameter to OpenAI's size parameter."""
12-
17+
class AspectRatioMapper(_SizeMapper):
1318
name = ImageGenerationParameter.ASPECT_RATIO
1419

15-
def map(
16-
self,
17-
request: dict[str, Any],
18-
value: object,
19-
model: Model,
20-
) -> dict[str, Any]:
21-
"""Transform aspect_ratio into provider request.
22-
23-
Maps unified aspect_ratio parameter to OpenAI's size format.
24-
Values are OpenAI's native size strings (e.g., "1024x1024", "1792x1024").
25-
Coercion from ratio format ("16:9") to size format can be added later.
26-
27-
Args:
28-
request: Provider request dictionary to modify.
29-
value: The aspect_ratio value (OpenAI size string).
30-
model: Model instance with parameter constraints.
31-
32-
Returns:
33-
Modified request dictionary with size parameter.
34-
"""
35-
validated_value = self._validate_value(value, model)
36-
if validated_value is None:
37-
return request
38-
39-
# Transform to provider-specific request format (size parameter)
40-
request["size"] = validated_value
41-
return request
42-
43-
44-
class PartialImagesMapper(ParameterMapper):
45-
"""Map partial_images parameter for streaming."""
4620

21+
class PartialImagesMapper(_PartialImagesMapper):
4722
name = ImageGenerationParameter.PARTIAL_IMAGES
4823

49-
def map(
50-
self,
51-
request: dict[str, Any],
52-
value: object,
53-
model: Model,
54-
) -> dict[str, Any]:
55-
"""Transform partial_images into provider request.
56-
57-
Controls number of partial images during streaming (0-3).
58-
59-
Args:
60-
request: Provider request dictionary to modify.
61-
value: The partial_images value (0-3).
62-
model: Model instance with parameter constraints.
63-
64-
Returns:
65-
Modified request dictionary with partial_images parameter.
66-
"""
67-
validated_value = self._validate_value(value, model)
68-
if validated_value is None:
69-
return request
70-
71-
# Transform to provider-specific request format (top-level field)
72-
request["partial_images"] = validated_value
73-
return request
74-
75-
76-
class QualityMapper(ParameterMapper):
77-
"""Map quality parameter"""
7824

25+
class QualityMapper(_QualityMapper):
7926
name = ImageGenerationParameter.QUALITY
8027

81-
def map(
82-
self,
83-
request: dict[str, Any],
84-
value: object,
85-
model: Model,
86-
) -> dict[str, Any]:
87-
"""Transform quality into provider request.
88-
89-
Controls image quality/detail level.
90-
- DALL-E 3: "standard" or "hd"
91-
- gpt-image-1: "low", "medium", "high", or "auto"
92-
- gpt-image-1-mini: "low", "medium", "high", or "auto"
93-
- DALL-E 2: Not supported (no constraint in model)
94-
95-
Args:
96-
request: Provider request dictionary to modify.
97-
value: The quality value.
98-
model: Model instance with parameter constraints.
99-
100-
Returns:
101-
Modified request dictionary with quality parameter.
102-
"""
103-
validated_value = self._validate_value(value, model)
104-
if validated_value is None:
105-
return request
106-
107-
# Transform to provider-specific request format (top-level field)
108-
request["quality"] = validated_value
109-
return request
110-
11128

11229
OPENAI_PARAMETER_MAPPERS: list[ParameterMapper] = [
11330
AspectRatioMapper(),
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
"""Integration tests for image generation capability."""
1+
"""Image generation integration test module."""

packages/capabilities/speech-generation/pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ Issues = "https://github.com/withceleste/celeste-python/issues"
2727

2828
[tool.uv.sources]
2929
celeste-ai = { workspace = true }
30+
celeste-openai = { workspace = true }
3031

3132
[project.entry-points."celeste.packages"]
3233
speech-generation = "celeste_speech_generation:register_package"

packages/capabilities/speech-generation/src/celeste_speech_generation/client.py

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -38,14 +38,12 @@ def _parse_content(
3838
"""Parse content from provider response."""
3939

4040
def _create_inputs(
41-
self,
42-
*args: str,
43-
text: str | None = None,
44-
**parameters: Unpack[SpeechGenerationParameters],
41+
self, *args: str, **parameters: Unpack[SpeechGenerationParameters]
4542
) -> SpeechGenerationInput:
4643
"""Map positional arguments to Input type."""
4744
if args:
4845
return SpeechGenerationInput(text=args[0])
46+
text: str | None = parameters.get("text")
4947
if text is None:
5048
msg = "text is required (either as positional argument or keyword argument)"
5149
raise ValidationError(msg)

packages/capabilities/speech-generation/src/celeste_speech_generation/constraints.py

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,7 @@
88

99

1010
class VoiceConstraint(Constraint):
11-
"""Voice constraint - value must be a valid voice ID or name from the provided voices.
12-
13-
Accepts both voice IDs and names. If a name is provided, returns the corresponding ID.
14-
"""
11+
"""Voice constraint - value must be a valid voice ID from the provided voices."""
1512

1613
voices: list[Voice] = Field(min_length=1)
1714

packages/capabilities/speech-generation/src/celeste_speech_generation/io.py

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
"""Input and output types for speech generation."""
22

33
from celeste.artifacts import AudioArtifact
4-
from celeste.io import Chunk, Input, Output, Usage
4+
from celeste.io import Chunk, FinishReason, Input, Output, Usage
55

66

77
class SpeechGenerationInput(Input):
@@ -17,23 +17,26 @@ class SpeechGenerationUsage(Usage):
1717
"""
1818

1919

20+
class SpeechGenerationFinishReason(FinishReason):
21+
"""Finish reason for speech generation."""
22+
23+
2024
class SpeechGenerationOutput(Output[AudioArtifact]):
2125
"""Output with audio artifact content."""
2226

2327

2428
class SpeechGenerationChunk(Chunk[bytes]):
2529
"""Typed chunk for speech generation streaming.
2630
27-
Note: Unlike TextGenerationChunk, this class intentionally omits a finish_reason
28-
field. TTS providers stream raw audio bytes without completion signals - the
29-
stream simply ends when audio generation is complete.
31+
Speech streaming sends raw bytes without finish_reason.
3032
"""
3133

3234
usage: SpeechGenerationUsage | None = None
3335

3436

3537
__all__ = [
3638
"SpeechGenerationChunk",
39+
"SpeechGenerationFinishReason",
3740
"SpeechGenerationInput",
3841
"SpeechGenerationOutput",
3942
"SpeechGenerationUsage",

0 commit comments

Comments
 (0)