Goal
Pass prior conversation turns as a structured messages array with a stable cache breakpoint after the system prompt + tool list, instead of jamming history into the system prompt as text.
Current state
_append_history_to_prompt concatenates prior turns into the system prompt string before sending to the provider. On Anthropic-shape providers this invalidates the prompt cache on every turn because the cache key is the system-prompt prefix and the prefix changes with each new turn. Result: every cognitive turn pays full input-token cost.
Acceptance
- Prior turns are sent as proper user/assistant message entries, not appended to the system prompt.
- Anthropic cache_creation/cache_read counters in
Usage show cache reuse across consecutive turns of the same agent.
- Equivalent behavior is preserved on stateless providers (the messages array is what they see anyway, no behavior change there).
Notes
- The cache breakpoint goes after
(system_prompt + tools) so those stay reused across turns even as messages grow.
Goal
Pass prior conversation turns as a structured
messagesarray with a stable cache breakpoint after the system prompt + tool list, instead of jamming history into the system prompt as text.Current state
_append_history_to_promptconcatenates prior turns into the system prompt string before sending to the provider. On Anthropic-shape providers this invalidates the prompt cache on every turn because the cache key is the system-prompt prefix and the prefix changes with each new turn. Result: every cognitive turn pays full input-token cost.Acceptance
Usageshow cache reuse across consecutive turns of the same agent.Notes
(system_prompt + tools)so those stay reused across turns even as messages grow.