TokenWise

Spend less, agent more.

Drop-in token optimization SDK for Agentic AI.
Reduce LLM API costs by 40–80% — no code rewrite needed.

---

The Problem

Agentic AI systems burn through tokens at alarming rates:

• Stateless repetition — 30-40% of tokens are redundant context resent every turn
• Tool bloat — 10-50 tool definitions sent per call, most irrelevant
• Wrong model for the job — expensive models used for simple tasks
• No visibility — you don't know where your tokens go until the bill arrives

How TokenWise Works

Your App ──→ TokenWise ──→ LLM API
                │
                ├── Context Differ     dedup messages, compress history
                ├── Skill Compressor   trim tool descriptions, filter irrelevant
                ├── Model Router       gpt-4o → gpt-4.1-nano for simple tasks
                └── Cost Tracker       log everything, report savings

TokenWise sits between your app and the LLM API. It optimizes every request transparently — your code sees the same responses, just cheaper.

Module	What it does	Savings
Context Differ	Deduplicates messages, compresses old turns, maximizes cache hits	20–40%
Skill Compressor	Removes filler from tool descriptions, filters irrelevant tools	30–50%
Model Router	Routes simple tasks to cheaper models automatically	50–90% per request
Cost Tracker	Real-time cost monitoring with savings dashboard	visibility

---

Quick Start

npm install @daino/tokenwise   # Node.js / TypeScript
pip install tokenwise-ai       # Python

Option 1: SDK — same API, automatic optimization

import { TokenWise } from '@daino/tokenwise';

const tw = new TokenWise({
  apiKey: process.env.OPENAI_API_KEY,
  verbose: true,
});

// Same API as OpenAI SDK — just cheaper
const response = await tw.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'What is 2+2?' }],
  tools: myTools,  // automatically compressed
});

console.log(tw.printSavings());

Option 2: Proxy — zero code change

Start the proxy, change one line:

npx tokenwise proxy --port 8787

// Your existing code — only change base_url
const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: 'http://localhost:8787/v1',  // ← this is it
});

// Everything else stays exactly the same
const response = await openai.chat.completions.create({ ... });

# Check savings anytime
curl http://localhost:8787/v1/tokenwise/report

---

Real Results

╔══════════════════════════════════════╗
║       TokenWise Cost Report          ║
╠══════════════════════════════════════╣
║  Requests:                      247 ║
║  Input tokens:            1,203,847 ║
║  Original:                2,891,203 ║
║  Saved tokens:            1,687,356 ║
╠══════════════════════════════════════╣
║  Actual cost:  $          2.4103    ║
║  Without TW:   $          8.6736    ║
║  You saved:    $          6.2633    ║
║  Savings:                   72.2%   ║
╚══════════════════════════════════════╝

---

Configuration

All modules are enabled by default. Toggle what you need:

const tw = new TokenWise({
  apiKey: process.env.OPENAI_API_KEY,

  contextDiffer: true,       // dedup & compress messages
  skillCompressor: true,     // trim tool definitions
  modelRouter: true,         // route to cheaper models
  trackCosts: true,          // cost monitoring

  // Custom routing rules
  routingRules: [
    { condition: 'simple', model: 'gpt-4.1-nano' },
    { condition: 'moderate', model: 'gpt-4.1-mini' },
    { condition: 'complex', model: 'gpt-4o' },
  ],

  verbose: true,
});

Option 3: Python SDK

from tokenwise import TokenWise

tw = TokenWise(api_key="sk-...")
response = tw.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(tw.print_savings())

Anthropic SDK

import { TokenWiseAnthropic } from '@daino/tokenwise';

const tw = new TokenWiseAnthropic({
  apiKey: process.env.ANTHROPIC_API_KEY,
  verbose: true,
});

const response = await tw.messages.create({
  model: 'claude-sonnet-4-6',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'Hello!' }],
});

Streaming

const stream = await tw.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Tell me a story' }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || '');
}

Web Dashboard

Start the proxy and visit http://localhost:8787/dashboard for a real-time cost analytics UI.

Advanced: Individual Modules

Use any module standalone:

import {
  SkillCompressor, ModelRouter, ContextDiffer,
  SharedStateStore, SmartWakeGate, OutputCompactor,
} from '@daino/tokenwise';

// Compress tools independently
const compressor = new SkillCompressor({ maxTools: 10 });
const { tools, stats } = compressor.optimize(myTools, userMessage);

// Route models independently
const router = new ModelRouter();
const { model, complexity } = router.route('gpt-4o', messages, tools);

// Share context across agents
const store = new SharedStateStore();
store.set('system-prompt', longSystemPrompt, 'agent-1');
const cached = store.get('system-prompt'); // reuse without resending

// Gate agent activation
const gate = new SmartWakeGate();
gate.register({ id: 'search', name: 'Search Agent', triggerKeywords: ['search', 'find'], toolNames: ['web_search'] });
const activeAgents = gate.evaluate('search for the latest news');

// Compact LLM output for downstream agents
const compactor = new OutputCompactor();
const { text } = compactor.compact(verboseLLMResponse);

---

Supported Models

Provider	Models
OpenAI	gpt-4o, gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, gpt-5, gpt-5-mini, o3, o4-mini
Anthropic	claude-opus-4-6, claude-sonnet-4-6, claude-haiku-4-5

---

Roadmap

• Context Differ — message dedup + compression
• Skill Compressor — tool description optimization
• Model Router — complexity-based routing
• Cost Tracker — real-time monitoring
• Proxy Server — zero code change mode
• Streaming support
• Anthropic API native support
• Shared State Store — cross-agent context sharing
• Smart Wake Gate — idle agent suppression
• Output Compactor — response format optimization
• Web dashboard for cost analytics
• Python SDK (pip install tokenwise-ai)

---

Contributing

PRs welcome. This project aims to make Agentic AI affordable for everyone.

git clone https://github.com/DainoJung/tokenwise.git
cd tokenwise
npm install
npm run build

License

MIT