models

optimus_dl.recipe.serve.models

ChatChoice

Bases: BaseModel

A single chat completion choice.

Attributes:

Name	Type	Description

Parameters:

Name	Type	Description	Default
index	int		required
message	ChatMessage		required
finish_reason	str | None		None

Source code in optimus_dl/recipe/serve/models.py

class ChatChoice(BaseModel):
    """A single chat completion choice.

    Attributes:
        index: The index of the choice in the list of choices.
        message: The generated message.
        finish_reason: The reason the model stopped generating tokens.
    """

    index: int
    message: ChatMessage
    finish_reason: str | None = None

ChatChunkChoice

Bases: BaseModel

A single chat completion chunk choice.

Attributes:

Name	Type	Description

Parameters:

Name	Type	Description	Default
index	int		required
delta	Delta		required
finish_reason	str | None		None

Source code in optimus_dl/recipe/serve/models.py

class ChatChunkChoice(BaseModel):
    """A single chat completion chunk choice.

    Attributes:
        index: The index of the choice in the list of choices.
        delta: The message delta.
        finish_reason: The reason the model stopped generating tokens.
    """

    index: int
    delta: Delta
    finish_reason: str | None = None

ChatCompletionChunk

Bases: BaseModel

Represents a streamed chunk of a chat completion response.

Attributes:

Name	Type	Description

Parameters:

Name	Type	Description	Default
id	str		required
object	Literal['chat.completion.chunk']		required
created	int		required
model	str		required
choices	list[ChatChunkChoice]		required

Source code in optimus_dl/recipe/serve/models.py

class ChatCompletionChunk(BaseModel):
    """Represents a streamed chunk of a chat completion response.

    Attributes:
        id: A unique identifier for the chat completion.
        object: The object type, which is always "chat.completion.chunk".
        created: The Unix timestamp (in seconds) of when the chat completion was created.
        model: The model used for the chat completion.
        choices: The list of chat completion choices.
    """

    id: str
    object: Literal["chat.completion.chunk"]
    created: int
    model: str
    choices: list[ChatChunkChoice]

ChatCompletionRequest

Bases: BaseModel

Request body for the chat completion API.

Attributes:

Name	Type	Description

Parameters:

Name	Type	Description	Default
model	str		'optimus-dl-model'
messages	list[dict]		required
max_tokens	int		50
temperature	float		1.0
top_k	int | None		None
stream	bool		False

Source code in optimus_dl/recipe/serve/models.py

class ChatCompletionRequest(BaseModel):
    """Request body for the chat completion API.

    Attributes:
        model: ID of the model to use.
        messages: A list of messages comprising the conversation so far.
        max_tokens: The maximum number of tokens to generate in the chat completion.
        temperature: What sampling temperature to use, between 0 and 2.
        top_k: The number of highest probability vocabulary tokens to keep for top-k-filtering.
        stream: If set, partial message deltas will be sent.
    """

    model: str = "optimus-dl-model"
    messages: list[dict]  # Use dict to allow flexibility or define strict message model
    max_tokens: int = Field(default=50, ge=1)
    temperature: float = Field(default=1.0, ge=0.0)
    top_k: int | None = Field(default=None, ge=1)
    stream: bool = False

ChatCompletionResponse

Bases: BaseModel

Response object for the chat completion API.

Attributes:

Name	Type	Description

Parameters:

Name	Type	Description	Default
id	str		required
object	Literal['chat.completion']		required
created	int		required
model	str		required
choices	list[ChatChoice]		required
usage	dict | None		None

Source code in optimus_dl/recipe/serve/models.py

class ChatCompletionResponse(BaseModel):
    """Response object for the chat completion API.

    Attributes:
        id: A unique identifier for the chat completion.
        object: The object type, which is always "chat.completion".
        created: The Unix timestamp (in seconds) of when the chat completion was created.
        model: The model used for the chat completion.
        choices: The list of chat completion choices.
        usage: Usage statistics for the completion request.
    """

    id: str
    object: Literal["chat.completion"]
    created: int
    model: str
    choices: list[ChatChoice]
    usage: dict | None = None

ChatMessage

Bases: BaseModel

A single message in a chat conversation.

Attributes:

Name	Type	Description

Parameters:

Name	Type	Description	Default
role	str | None		None
content	str | None		None

Source code in optimus_dl/recipe/serve/models.py

class ChatMessage(BaseModel):
    """A single message in a chat conversation.

    Attributes:
        role: The role of the message sender (e.g., 'user', 'assistant', 'system').
        content: The content of the message.
    """

    role: str | None = None
    content: str | None = None

Choice

Bases: BaseModel

A single completion choice.

Attributes:

Name	Type	Description

Parameters:

Name	Type	Description	Default
index	int		required
text	str		required
logprobs	dict | None		None
finish_reason	str | None		None

Source code in optimus_dl/recipe/serve/models.py

class Choice(BaseModel):
    """A single completion choice.

    Attributes:
        index: The index of the choice in the list of choices.
        text: The generated text.
        logprobs: Log probabilities of the token choices (optional).
        finish_reason: The reason the model stopped generating tokens.
    """

    index: int
    text: str
    logprobs: dict | None = None
    finish_reason: str | None = None

CompletionChunk

Bases: BaseModel

Represents a streamed chunk of a text completion response.

Attributes:

Name	Type	Description

Parameters:

Name	Type	Description	Default
id	str		required
object	Literal['text_completion']		required
created	int		required
model	str		required
choices	list[CompletionChunkChoice]		required

Source code in optimus_dl/recipe/serve/models.py

class CompletionChunk(BaseModel):
    """Represents a streamed chunk of a text completion response.

    Attributes:
        id: A unique identifier for the completion.
        object: The object type, which is always "text_completion".
        created: The Unix timestamp (in seconds) of when the completion was created.
        model: The model used for the completion.
        choices: The list of completion choices.
    """

    id: str
    object: Literal["text_completion"]
    created: int
    model: str
    choices: list[CompletionChunkChoice]

CompletionChunkChoice

Bases: BaseModel

A single text completion chunk choice.

Attributes:

Name	Type	Description

Parameters:

Name	Type	Description	Default
index	int		required
text	str		required
logprobs	dict | None		None
finish_reason	str | None		None

Source code in optimus_dl/recipe/serve/models.py

class CompletionChunkChoice(BaseModel):
    """A single text completion chunk choice.

    Attributes:
        index: The index of the choice in the list of choices.
        text: The text chunk.
        logprobs: Log probabilities of the token choices (optional).
        finish_reason: The reason the model stopped generating tokens.
    """

    index: int
    text: str
    logprobs: dict | None = None
    finish_reason: str | None = None

CompletionRequest

Bases: BaseModel

Request body for the text completion API.

Attributes:

Name	Type	Description

Parameters:

Name	Type	Description	Default
model	str		'optimus-dl-model'
prompt	str | list[str]		required
max_tokens	int		50
temperature	float		1.0
top_k	int | None		None
stream	bool		False

Source code in optimus_dl/recipe/serve/models.py

class CompletionRequest(BaseModel):
    """Request body for the text completion API.

    Attributes:
        model: ID of the model to use.
        prompt: The prompt(s) to generate completions for.
        max_tokens: The maximum number of tokens to generate in the completion.
        temperature: What sampling temperature to use, between 0 and 2.
        top_k: The number of highest probability vocabulary tokens to keep for top-k-filtering.
        stream: If set, partial message deltas will be sent.
    """

    model: str = "optimus-dl-model"
    prompt: str | list[str]
    max_tokens: int = Field(default=50, ge=1)
    temperature: float = Field(default=1.0, ge=0.0)
    top_k: int | None = Field(default=None, ge=1)
    stream: bool = False

CompletionResponse

Bases: BaseModel

Response object for the text completion API.

Attributes:

Name	Type	Description

Parameters:

Name	Type	Description	Default
id	str		required
object	Literal['text_completion']		required
created	int		required
model	str		required
choices	list[Choice]		required
usage	dict | None		None

Source code in optimus_dl/recipe/serve/models.py

class CompletionResponse(BaseModel):
    """Response object for the text completion API.

    Attributes:
        id: A unique identifier for the completion.
        object: The object type, which is always "text_completion".
        created: The Unix timestamp (in seconds) of when the completion was created.
        model: The model used for completion.
        choices: The list of completion choices.
        usage: Usage statistics for the completion request.
    """

    id: str
    object: Literal["text_completion"]
    created: int
    model: str
    choices: list[Choice]
    usage: dict | None = None

Delta

Bases: BaseModel

A partial message delta for streaming responses.

Attributes:

Name	Type	Description

Parameters:

Name	Type	Description	Default
role	str | None		None
content	str | None		None

Source code in optimus_dl/recipe/serve/models.py

class Delta(BaseModel):
    """A partial message delta for streaming responses.

    Attributes:
        role: The role of the message sender.
        content: The content of the message delta.
    """

    role: str | None = None
    content: str | None = None