LLM¶

The Llm protocol and its value types — the single interface every model call goes through. The PromptSolver uses it for text completions; the LlmJudge uses it for structured ones. LiteLlm is the litellm-backed implementation (requires the litellm extra); StubLlm is a fixed-reply implementation for tests.

llm ¶

The LLM seam: the Llm protocol, its value types, and the litellm-backed LiteLlm.

LiteLlm requires the litellm extra and is imported lazily; everything else has no optional dependency.

Completion `dataclass` ¶

Completion(parsed: T, usage: Usage)

Bases: Generic[T]

A successful LLM call: the parsed response_format instance plus its Usage.

Llm ¶

Bases: Protocol

Produces a parsed structured reply or a free-text reply for a prompt, or a typed error.

complete ¶

complete(
    prompt: str, *, response_format: type[T]
) -> Completion[T] | LlmError

Complete prompt, parsing the reply into response_format.

Parameters:

Name	Type	Description	Default
`prompt`	`str`	The user prompt to send.	required
`response_format`	`type[T]`	The Pydantic model the reply must validate against.	required

Returns:

Type	Description
`Completion[T] \| LlmError`	A `Completion` carrying the parsed model and telemetry, or an `LlmError` on an
`Completion[T] \| LlmError`	expected provider or parsing failure.

complete_text ¶

complete_text(prompt: str) -> TextCompletion | LlmError

Complete prompt, returning the reply as free text.

Parameters:

Name	Type	Description	Default
`prompt`	`str`	The user prompt to send.	required

Returns:

Type	Description
`TextCompletion \| LlmError`	A `TextCompletion` carrying the reply text and telemetry, or an `LlmError` on an
`TextCompletion \| LlmError`	expected provider failure.

StubLlm `dataclass` ¶

StubLlm(
    reply: str
    | BaseModel
    | LlmError
    | Callable[
        [str, type[BaseModel] | None],
        str | BaseModel | LlmError,
    ],
    prompts: list[str] = list(),
)

An in-memory Llm test double: returns a fixed reply and records every prompt.

reply is the canned outcome: a str (for complete_text), a BaseModel (for complete), an LlmError (returned as-is), or a callable resolving one of those from the prompt and the requested response_format (which is None for complete_text). Each call appends its prompt to prompts.

complete ¶

complete(
    prompt: str, *, response_format: type[T]
) -> Completion[T] | LlmError

Record prompt and return the resolved canned reply as a Completion.

Parameters:

Name	Type	Description	Default
`prompt`	`str`	The user prompt, appended to `prompts`.	required
`response_format`	`type[T]`	The Pydantic model the reply is presumed to match.	required

Returns:

Type	Description
`Completion[T] \| LlmError`	The configured `LlmError`, or a `Completion` wrapping the configured model with
`Completion[T] \| LlmError`	an empty `Usage`.

complete_text ¶

complete_text(prompt: str) -> TextCompletion | LlmError

Record prompt and return the resolved canned reply as a TextCompletion.

Parameters:

Name	Type	Description	Default
`prompt`	`str`	The user prompt, appended to `prompts`.	required

Returns:

Type	Description
`TextCompletion \| LlmError`	The configured `LlmError`, or a `TextCompletion` carrying the reply coerced to
`TextCompletion \| LlmError`	text with an empty `Usage`.

TextCompletion `dataclass` ¶

TextCompletion(text: str, usage: Usage)

A successful free-text LLM call: the raw reply text plus its Usage.

Usage `dataclass` ¶

Usage(
    prompt_tokens: int | None = None,
    completion_tokens: int | None = None,
    cost_usd: float | None = None,
    latency_seconds: float | None = None,
)

Telemetry for one LLM call: token counts, cost, and latency, each None when unknown.

LlmError ¶

Bases: Error

A typed, expected failure from an Llm.complete call.

LiteLlm ¶

LiteLlm(
    model: str,
    *,
    api_base: str | None = None,
    temperature: float | None = None,
    timeout: float | None = None,
)

An Llm backed by litellm.completion, returning a parsed reply or a typed LlmError.

Configure the backend.

Parameters:

Name	Type	Description	Default
`model`	`str`	The litellm model identifier (e.g. `"openai/gpt-4o-mini"`).	required
`api_base`	`str \| None`	An override base URL for the provider, or `None` for the default.	`None`
`temperature`	`float \| None`	Sampling temperature; `None` leaves the provider default.	`None`
`timeout`	`float \| None`	Per-request timeout in seconds.	`None`

complete ¶

complete(
    prompt: str, *, response_format: type[T]
) -> Completion[T] | LlmError

Complete prompt, parsing the reply into response_format.

Requires the model's provider to support native structured output: a model that does not yields a bad_request LlmError. Expected provider failures are mapped to a typed LlmError.

Parameters:

Name	Type	Description	Default
`prompt`	`str`	The user prompt to send.	required
`response_format`	`type[T]`	The Pydantic model the reply must validate against.	required

Returns:

Type	Description
`Completion[T] \| LlmError`	A `Completion` carrying the parsed model and telemetry, or an `LlmError`.

complete_text ¶

complete_text(prompt: str) -> TextCompletion | LlmError

Complete prompt, returning the reply as free text.

Sends the prompt verbatim with no structured-output mode and no JSON instruction. Expected provider failures are mapped to a typed LlmError.

Parameters:

Name	Type	Description	Default
`prompt`	`str`	The user prompt to send.	required

Returns:

Type	Description
`TextCompletion \| LlmError`	A `TextCompletion` carrying the reply text and telemetry, or an `LlmError`.

resolve_llm ¶

resolve_llm(
    model: str | Llm,
    *,
    temperature: float | None = None,
    timeout: float | None = None,
) -> Llm

Resolve model to an Llm, defaulting a model string to the litellm backend.

A string is wrapped in the litellm-backed LiteLlm; an existing Llm is returned unchanged. temperature and timeout apply only to the string path — they configure the constructed LiteLlm and are ignored when an Llm is passed.

Parameters:

Name	Type	Description	Default
`model`	`str \| Llm`	A litellm model identifier, or an `Llm` to use directly.	required
`temperature`	`float \| None`	Sampling temperature for the string path; `None` leaves the default.	`None`
`timeout`	`float \| None`	Per-request timeout in seconds for the string path.	`None`

Returns:

Type	Description
`Llm`	An `Llm`: the constructed `LiteLlm` for a string, or `model` unchanged.

Raises:

Type	Description
`ImportError`	If `model` is a string and the `litellm` extra is not installed.

LLM¶

llm ¶

Completion dataclass ¶

Llm ¶

complete ¶

complete_text ¶

StubLlm dataclass ¶

complete ¶

complete_text ¶

TextCompletion dataclass ¶

Usage dataclass ¶

LlmError ¶

LiteLlm ¶

complete ¶

complete_text ¶

resolve_llm ¶

Completion `dataclass` ¶

StubLlm `dataclass` ¶

TextCompletion `dataclass` ¶

Usage `dataclass` ¶