Budgets and token counting¶
Stable
Covered by the 1.0 API freeze (proposal 0029): no breaking change before 2.0.
Budget
dataclass
¶
Budget(tool_definition_target: int = 250, tool_definition_warn: int = 500, tool_definition_fail: int = 1000, startup_tokens: int = 4000, result_tokens: int = 2000, result_ceiling: int = 8000)
Token limits for tool definitions and results.
Attributes:
| Name | Type | Description |
|---|---|---|
tool_definition_target |
int
|
Ideal upper size for one tool definition. |
tool_definition_warn |
int
|
A tool definition above this size produces a warning. |
tool_definition_fail |
int
|
A tool definition above this size fails |
startup_tokens |
int
|
Total definition tokens above which the server warns (and, from M1, switches to search-based discovery). |
result_tokens |
int
|
Default maximum tokens for a concise result. |
result_ceiling |
int
|
Hard maximum for any result, including detailed ones. |
BudgetOverride
dataclass
¶
BudgetOverride(result_tokens: int | None = None, definition_tokens: int | None = None, reason: str | None = None)
A tool's own budget and the reason for it (spec BUD-4).
Attributes:
| Name | Type | Description |
|---|---|---|
result_tokens |
int | None
|
The tool's concise result budget, if it differs from the server's. |
definition_tokens |
int | None
|
The size above which this tool's definition fails
|
reason |
str | None
|
Why the tool needs it. |
TokenCounter ¶
Bases: ABC
Counts tokens. Subclass it, or use get_counter with an encoding or a callable.
A counter has a name that appears in every report, and a count method
that returns a non-negative integer. Shaping and truncation assume counts grow
with the text: a prefix of a text never has noticeably more tokens than the whole
text (a margin of a few tokens is allowed, as at a token boundary).
encoding_name
property
¶
The counter's name (kept for code written before BUD-6).
truncate_text ¶
Return the longest prefix of text that fits in max_tokens.
This default searches by prefix length and calls count about
log2(len(text)) times. Counters that can cut at token boundaries directly
should override it.
TiktokenCounter ¶
CallableCounter ¶
Bases: TokenCounter
Counts tokens with your own function, for example a model vendor's tokenizer.
fn takes a string and returns its token count as a non-negative integer.
name identifies the tokenizer in every report, so give it the model or
tokenizer and its version, such as "acme-tokenizer-2.1".
AnthropicCounter ¶
Bases: TokenCounter
Counts tokens with Anthropic's messages.count_tokens for one Claude model.
The endpoint counts a whole request, so each text is sent as one user message and
the fixed cost of that message framing is measured once and subtracted. client
is an anthropic.Anthropic instance; without one, a default client is created,
which reads credentials from the environment.
framing_tokens
property
¶
Tokens the endpoint adds for wrapping a text in one user message.
Provisional
AnthropicCounter follows Anthropic's token-counting API and may change in a minor release.
get_counter ¶
get_counter(tokenizer: str | TokenCounter | Callable[[str], int] = DEFAULT_ENCODING, *, name: str | None = None) -> TokenCounter
Return a counter for a tiktoken encoding name, a TokenCounter, or a callable.
name is required for a bare callable and not allowed otherwise.
measure_tool ¶
measure_tool(tool: Any, budget: Budget, counter: TokenCounter | None = None, output_schema_mode: str | None = None, override: BudgetOverride | None = None) -> ToolCost
Measure one tool definition. output_schema_mode is recorded in the result.
override is the tool's own budget (BUD-4); its definition_tokens replaces the
warn and fail thresholds for this tool.
measure_tools ¶
measure_tools(tools: Iterable[Any], budget: Budget, counter: TokenCounter | None = None, output_schema_modes: Mapping[str, str] | None = None, overrides: Mapping[str, BudgetOverride] | None = None) -> ServerCost
Measure a set of tool definitions.
output_schema_modes maps tool names to the output-schema mode each was
registered with (on, off or auto), for servers built with OunceMCP.
ServerCost
dataclass
¶
Token cost of everything a server exposes before the user's first message.
ToolCost
dataclass
¶
ToolCost(name: str, total: int, name_tokens: int, description: int, input_schema: int, output_schema: int, structure: int, status: Status, output_schema_advertised: bool = False, output_schema_mode: str | None = None, output_schema_cost: int = 0, override: dict[str, Any] | None = None)
Token cost of one tool definition, with a breakdown that sums to total.