Skip to content

openRouterDecider drops usage.cost, and decide() meta cannot surface the Decisions response id/provider #1456

Description

@daveycodez

Summary

openRouterDecider / createOpenRouterDecider (@tanstack/ai-openrouter, used with decide() from @tanstack/ai) drops data the OpenRouter Decisions API returns:

  1. usage.cost is never attached to result.meta.usage, even though the endpoint returns it on every response and the same package already has extractUsageCost() which the text/image adapters use.
  2. The response's id (OpenRouter generation id) and provider are not surfaced anywhere, because EvaluateAdapterResult / EvaluateResultMeta only carry model, answers and usage.

I'm billing per request off usage.cost and reconciling with GET /api/v1/generation?id=, so both are needed.

Environment

  • @tanstack/ai@0.58.0
  • @tanstack/ai-openrouter@0.19.16
  • @tanstack/ai-event-client@0.12.1
  • Node v24.20.0, server-side (no framework)

Reproduction

import { decide, boolean } from '@tanstack/ai'
import { openRouterDecider } from '@tanstack/ai-openrouter'

const result = await decide({
  adapter: openRouterDecider('typesafe/jev-1.13'),
  state: 'Customer says the invoice total is wrong after applying a coupon.',
  questions: {
    isBug: boolean({ instructions: 'Is this a software bug report?' }),
  },
})

console.log(result.meta.usage)
// { promptTokens: 30, completionTokens: 3, totalTokens: 33 }
// no `cost`, although the raw response body had usage.cost

console.log(result.meta)
// { model: 'typesafe/jev-1.13-20260917', usage: { ... } }
// no `id`, no `provider`

The raw POST https://openrouter.ai/api/alpha/decisions response (from OpenRouter's API reference) is:

{
  "answers": { "...": "..." },
  "id": "gen-dec-1789738314-X5e5eKGQdvR9rblyX250",
  "model": "typesafe/jev-1.13-20260917",
  "provider": "TypeSafe",
  "usage": { "cost": 0.000019992, "input_tokens": 476, "output_tokens": 70 }
}

Expected

  • result.meta.usage.cost is 0.000019992 (and costDetails when present), the same as chat() / generateImage() with the OpenRouter adapters. TokenUsage already declares cost?: number and costDetails?: UsageCostBreakdown (@tanstack/ai-event-client index.d.ts, TokenUsage).
  • result.meta exposes the response's id and provider (or a providerMetadata bag) so the request can be reconciled against GET /api/v1/generation?id=.

Actual

  • result.meta.usage has only promptTokens / completionTokens / totalTokens; cost is undefined.
  • result.meta has only model and usage; id and provider are unreachable from decide() without writing a custom adapter.

Where it happens

@tanstack/ai-openrouter/dist/esm/adapters/evaluate.js, mapUsage() (lines 51–65) reads only the token keys and calls buildBaseUsage with them:

function mapUsage(usage) {
	if (!isRecord(usage)) return buildBaseUsage({ promptTokens: 0, completionTokens: 0, totalTokens: 0 });
	const promptTokens = readNumber(usage, ["input_tokens", "prompt_tokens"]) ?? 0;
	const completionTokens = readNumber(usage, ["output_tokens", "completion_tokens"]) ?? 0;
	const totalTokens = readNumber(usage, ["total_tokens"]) ?? promptTokens + completionTokens;
	return buildBaseUsage({ promptTokens, completionTokens, totalTokens });
}

and evaluate() (lines 104–108) returns only model / answers / usage:

return {
	model: json.model ?? model,
	answers: json.answers,
	usage: mapUsage(json.usage)
};

extractUsageCost() lives in the same package (adapters/cost.js, exported with cost.d.ts) and is already spread into usage in adapters/text.js (lines 155, 386, 735), adapters/responses-text.js (133, 439, 1083) and adapters/image.js (121). evaluate.js does not import it.

On the core side, @tanstack/ai/dist/esm/activities/evaluate/adapter.d.ts lines 103–108:

export interface EvaluateAdapterResult {
    /** Resolved model id from the provider. */
    model: string;
    answers: Record<string, WireAnswer>;
    usage: TokenUsage;
}

and activities/evaluate/index.d.ts lines 50–54:

export interface EvaluateResultMeta {
    /** Resolved model id from the provider. */
    model: string;
    usage: TokenUsage;
}

so even a custom adapter has no typed place to put id / provider.

Suggested fix

  1. @tanstack/ai-openrouter adapters/evaluate.ts: spread the existing helper into the mapped usage, matching the other adapters:

    import { extractUsageCost } from './cost'
    
    return {
      ...buildBaseUsage({ promptTokens, completionTokens, totalTokens }),
      ...extractUsageCost(usage),
    }
  2. @tanstack/ai activities/evaluate: add optional id?: string and provider?: string (or providerMetadata?: Record<string, unknown>) to EvaluateAdapterResult and pass them through to EvaluateResultMeta in decide(). The OpenRouter adapter would then return id: json.id, provider: json.provider.

Both are additive; existing adapters and callers keep working.

I'm happy to open a PR for either or both if that's welcome.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions