Skip to content

Commit 723beea

Browse files
wukathcopybara-github
authored andcommitted
docs(samples): add classifier tool gate sample
Adds a sample `before_tool_callback` that asks a `BaseClassifier` a `Binary` question about each tool call, and blocks the call or asks the user to confirm it when the probability is at or above a threshold. Unlike `require_confirmation`, the decision depends on the call's arguments, so one question covers every tool: a `SELECT` runs, and a `DELETE` through the same tool waits for approval. A confirmed call is not classified again, and a classifier error fails closed. Co-authored-by: Kathy Wu <wukathy@google.com> PiperOrigin-RevId: 1001015315
1 parent bba90f2 commit 723beea

4 files changed

Lines changed: 281 additions & 0 deletions

File tree

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Classifier Tool Gate Sample
2+
3+
## Overview
4+
5+
This sample demonstrates how to gate tool calls with a classifier from `google.adk.labs.classifiers`.
6+
7+
The agent manages a small customer database through two tools: `run_sql` and `send_email`. Before each tool call, a `before_tool_callback` asks a classifier a `Binary` question about the call: "Would this tool call delete or change data, or contact a customer?" Calls the classifier rates below the threshold run right away. The rest pause until the user approves them through ADK's tool confirmation flow.
8+
9+
`FunctionTool(require_confirmation=...)` decides per tool, or with code you write for each tool. A classifier decides from the arguments, so one question covers every tool: a `SELECT` through `run_sql` runs, and a `DELETE` through the same tool asks first.
10+
11+
This sample uses `LlmClassifier`, which answers with any `BaseLlm`. A dedicated decision model answers faster, which matters for a check that runs before every tool call, and can be swapped in without other changes.
12+
13+
## Sample Inputs
14+
15+
- `Which customers are on the pro plan?`
16+
17+
- `Move Alan Turing to the pro plan.`
18+
19+
*The gate asks for confirmation before the `UPDATE` runs.*
20+
21+
- `Delete every customer on the free plan.`
22+
23+
- `Email Grace Hopper to thank her for renewing.`
24+
25+
## How To
26+
27+
1. Create the callback with `create_tool_gate_callback` from [`tool_gate.py`](tool_gate.py) and pass it as the agent's `before_tool_callback`:
28+
29+
```python
30+
tool_gate = create_tool_gate_callback(
31+
classifier=LlmClassifier(model="gemini-3.5-flash"),
32+
instructions="Would this tool call delete or change data, or contact a customer?",
33+
on_threshold_exceeded="confirm",
34+
)
35+
36+
root_agent = Agent(..., before_tool_callback=tool_gate)
37+
```
38+
39+
1. Choose what happens when a call's probability is at or above `threshold` (default `0.5`):
40+
41+
- `on_threshold_exceeded="block"` (default) skips the call and returns an error to the model.
42+
- `on_threshold_exceeded="confirm"` asks the user with `tool_context.request_confirmation()` and runs the call only if they approve. When the user answers, the call comes back through the callback with `tool_context.tool_confirmation` set, and the callback runs or rejects it without asking the classifier again.
43+
44+
1. By default the classifier sees the tool's name, description and arguments. To classify something else, such as recent conversation or session state, pass a `state_extractor(tool, args, tool_context)` that returns the classifier input.
45+
46+
If the classifier raises, the gate treats the call as risky and blocks it or asks the user, so an outage can't let a risky call through.
47+
48+
`LlmClassifier` probabilities are the model's stated values, not measured frequencies, and the right threshold differs between classifiers. Pick `threshold` from an evaluation set of calls you expect to pass and to stop.
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Copyright 2026 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
from . import agent
Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# Copyright 2026 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
from __future__ import annotations
16+
17+
import sqlite3
18+
from typing import Any
19+
20+
from classifier_tool_gate.tool_gate import create_tool_gate_callback
21+
from google.adk import Agent
22+
from google.adk.labs.classifiers import LlmClassifier
23+
24+
_db = sqlite3.connect(":memory:", check_same_thread=False)
25+
_db.executescript("""
26+
CREATE TABLE customers (
27+
id INTEGER PRIMARY KEY, name TEXT, email TEXT, plan TEXT
28+
);
29+
INSERT INTO customers VALUES
30+
(1, 'Ada Lovelace', 'ada@example.com', 'pro'),
31+
(2, 'Alan Turing', 'alan@example.com', 'free'),
32+
(3, 'Grace Hopper', 'grace@example.com', 'pro');
33+
""")
34+
35+
36+
def run_sql(query: str) -> dict[str, Any]:
37+
"""Runs one SQLite statement against the `customers` table.
38+
39+
The table has columns id, name, email and plan.
40+
"""
41+
try:
42+
cursor = _db.execute(query)
43+
rows = cursor.fetchall()
44+
_db.commit()
45+
except sqlite3.Error as e:
46+
return {"error": str(e)}
47+
return {"rows": [list(row) for row in rows], "rows_changed": cursor.rowcount}
48+
49+
50+
def send_email(to: str, subject: str, body: str) -> dict[str, str]:
51+
"""Sends an email to a customer."""
52+
return {"result": f"Sent {subject!r} to {to}."}
53+
54+
55+
# Any BaseClassifier works here, such as a dedicated decision model.
56+
classifier = LlmClassifier(model="gemini-3.5-flash")
57+
58+
tool_gate = create_tool_gate_callback(
59+
classifier=classifier,
60+
instructions=(
61+
"Would this tool call delete or change data, or contact a customer?"
62+
),
63+
on_threshold_exceeded="confirm",
64+
)
65+
66+
root_agent = Agent(
67+
name="customer_db_assistant",
68+
instruction=(
69+
"You help support staff look up and manage customers. Use `run_sql` to"
70+
" query or change the `customers` table, and `send_email` to email a"
71+
" customer. Report tool results to the user."
72+
),
73+
tools=[run_sql, send_email],
74+
before_tool_callback=tool_gate,
75+
)
Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
# Copyright 2026 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""A before_tool_callback that gates tool calls on a classifier's answer."""
16+
17+
from __future__ import annotations
18+
19+
from collections.abc import Awaitable
20+
from collections.abc import Callable
21+
import logging
22+
from typing import Any
23+
from typing import Literal
24+
25+
from google.adk.labs.classifiers import BaseClassifier
26+
from google.adk.labs.classifiers import Binary
27+
from google.adk.labs.classifiers import BinaryAnswer
28+
from google.adk.labs.classifiers import ClassifierInput
29+
from google.adk.labs.classifiers import JSONContent
30+
from google.adk.tools.base_tool import BaseTool
31+
from google.adk.tools.tool_context import ToolContext
32+
33+
logger = logging.getLogger("google_adk." + __name__)
34+
35+
StateExtractor = Callable[
36+
[BaseTool, dict[str, Any], ToolContext], ClassifierInput
37+
]
38+
ToolGateCallback = Callable[
39+
[BaseTool, dict[str, Any], ToolContext], Awaitable[dict[str, Any] | None]
40+
]
41+
42+
43+
def _default_state(
44+
tool: BaseTool, args: dict[str, Any], tool_context: ToolContext
45+
) -> ClassifierInput:
46+
del tool_context # Unused.
47+
return {"tool": tool.name, "description": tool.description, "args": args}
48+
49+
50+
def create_tool_gate_callback(
51+
*,
52+
classifier: BaseClassifier,
53+
instructions: JSONContent,
54+
criteria: JSONContent | None = None,
55+
threshold: float = 0.5,
56+
on_threshold_exceeded: Literal["block", "confirm"] = "block",
57+
state_extractor: StateExtractor | None = None,
58+
) -> ToolGateCallback:
59+
"""Creates a before_tool_callback that asks a classifier if a call is risky.
60+
61+
Before each tool call, the callback asks `classifier` a `Binary` question
62+
about the call. If the probability of true is at or above `threshold`, the
63+
call is blocked, or held until the user confirms it.
64+
65+
Args:
66+
classifier: Any `BaseClassifier`, such as `LlmClassifier` or a dedicated
67+
decision model.
68+
instructions: The yes/no question to ask about each call, where yes means
69+
the call is risky.
70+
criteria: Optionally, a description of what counts as risky.
71+
threshold: The probability of risky at or above which the gate acts.
72+
on_threshold_exceeded: "block" skips the call and returns an error to the
73+
model. "confirm" asks the user through `tool_context.request_confirmation`
74+
and runs the call only if they approve.
75+
state_extractor: Builds the classifier input from the tool, its arguments
76+
and the tool context. Defaults to the tool's name, description and
77+
arguments.
78+
79+
Returns:
80+
An async callback to pass as an agent's `before_tool_callback`.
81+
"""
82+
if not 0.0 <= threshold <= 1.0:
83+
raise ValueError(f"threshold must be in [0, 1], got {threshold}.")
84+
if on_threshold_exceeded not in ("block", "confirm"):
85+
raise ValueError(
86+
"on_threshold_exceeded must be 'block' or 'confirm', got"
87+
f" {on_threshold_exceeded!r}."
88+
)
89+
question = Binary(instructions=instructions, criteria=criteria)
90+
extract_state = state_extractor or _default_state
91+
92+
async def tool_gate(
93+
tool: BaseTool, args: dict[str, Any], tool_context: ToolContext
94+
) -> dict[str, Any] | None:
95+
# A confirmed call comes back through this callback with the user's
96+
# answer attached; classifying it again could ask the user twice.
97+
confirmation = tool_context.tool_confirmation
98+
if confirmation is not None:
99+
if confirmation.confirmed:
100+
return None
101+
return {"error": f"The user rejected this call to {tool.name}."}
102+
103+
try:
104+
response = await classifier.classify(
105+
input=extract_state(tool, args, tool_context),
106+
questions={"risky": question},
107+
)
108+
answer = response.answers["risky"]
109+
if not isinstance(answer, BinaryAnswer):
110+
raise ValueError(f"Expected a BinaryAnswer, got {answer!r}.")
111+
probability = answer.probability
112+
except Exception: # pylint: disable=broad-exception-caught
113+
# Fail closed: a classifier outage must not let risky calls through.
114+
logger.exception("Tool gate classifier failed for %s.", tool.name)
115+
probability = 1.0
116+
117+
logger.info("Tool gate: %s risk=%.2f", tool.name, probability)
118+
if probability < threshold:
119+
return None
120+
121+
if on_threshold_exceeded == "confirm":
122+
tool_context.request_confirmation(
123+
hint=(
124+
f"The tool gate flagged this call to {tool.name} as risky"
125+
f" ({probability:.0%}). Approve it?"
126+
),
127+
)
128+
# Pause for the user instead of letting the model react to the error,
129+
# which would make it call the tool again.
130+
tool_context.actions.skip_summarization = True
131+
return {
132+
"error": (
133+
"This tool call requires confirmation, please approve or reject."
134+
)
135+
}
136+
return {
137+
"error": (
138+
f"The call to {tool.name} was blocked because it was classified"
139+
" as risky. Do not retry it; tell the user it was blocked."
140+
)
141+
}
142+
143+
return tool_gate

0 commit comments

Comments
 (0)