-
Notifications
You must be signed in to change notification settings - Fork 48
feat: Add tracing channels support #193
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
logaretm
wants to merge
13
commits into
unjs:main
Choose a base branch
from
logaretm:awad/add-tracing-channels
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
5dd1cd3
feat: implement tracing for db0
logaretm 0ae9a84
feat: add dialect to the context data
logaretm 3ce185a
refactor: avoid explicit import of diag channel
logaretm c0b97d1
refactor: expose tracing as a subpath export
logaretm f858d89
docs: added info and examples to docs
logaretm d219d5e
feat: traced statement class
logaretm b36f567
fix: use prepared statement as a base instead
logaretm 7cdb895
fix: handle nested bind statements
logaretm c335701
fix: make sure to preserver dialect and disposed getters
logaretm b1244f6
chore: update lock
logaretm d356eee
Merge branch 'main' into awad/add-tracing-channels
pi0 c0a2791
refactor(tracing): normalize error events, skip work without subscribers
pi0 f854c44
fix(tracing): keep the wrapper transparent
pi0 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| --- | ||
| icon: ph:chart-bar-horizontal-duotone | ||
| --- | ||
|
|
||
| # Tracing | ||
|
|
||
| > db0 can emit tracing events for every query it runs. | ||
|
|
||
| ## Overview | ||
|
|
||
| db0 uses [tracing channels](https://nodejs.org/api/diagnostics_channel.html) of the diagnostics channel module to emit traceable actions for query operations. | ||
|
|
||
| Common use cases for tracing are: | ||
|
|
||
| - Query performance monitoring and profiling | ||
| - Logging | ||
| - Query debugging | ||
| - Error tracking | ||
|
|
||
| ## Enabling tracing | ||
|
|
||
| Tracing is opt-in. Wrap the database instance with `withTracing` from `db0/tracing` and use the returned instance: | ||
|
|
||
| ```ts | ||
| import { createDatabase } from "db0"; | ||
| import { withTracing } from "db0/tracing"; | ||
| import sqlite from "db0/connectors/better-sqlite3"; | ||
|
|
||
| const db = withTracing(createDatabase(sqlite({}))); | ||
|
|
||
| // Queries made with `db` now emit tracing events | ||
| const rows = await db.sql`SELECT * FROM users`; | ||
| ``` | ||
|
|
||
| Only the wrapped instance emits events, so tracing costs nothing for databases you do not wrap. Wrapping an already wrapped instance is a no-op. | ||
|
|
||
| Tracing is looked up through `process.getBuiltinModule("node:diagnostics_channel")`, which requires Node.js `20.16+` or `22.3+`. On older versions, or on runtimes where diagnostics channels are unavailable, `withTracing` returns the database unchanged and silently emits no events. | ||
|
|
||
| ## Query Tracing Channel | ||
|
|
||
| db0 uses `db0.query` for the tracing channel name. | ||
|
|
||
| To set up tracing, you need to subscribe to the tracing channel. Since query operations are asynchronous, you need to subscribe to the `asyncEnd` event as it signals the completion of the operation. | ||
|
|
||
| ```ts | ||
| import { tracingChannel } from "node:diagnostics_channel"; | ||
|
|
||
| const queryChannel = tracingChannel("db0.query"); | ||
|
|
||
| queryChannel.subscribe({ | ||
| start: (data) => { | ||
| console.log("start", data.query); | ||
| }, | ||
| asyncEnd: (data) => { | ||
| console.log("end", data.query, data.result); | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| The event payload contains several properties that can be used to track the query operation: | ||
|
|
||
| - `query`: The query string. | ||
| - `method`: The method used to execute the query (`exec`, `sql`, `prepare.all`, `prepare.run`, `prepare.get`). | ||
| - `connector`: The name of the connector (e.g. `sqlite`, `postgresql`). | ||
| - `dialect`: The dialect of the database connection (`sqlite`, `libsql`, `postgresql` or `mysql`). | ||
| - `result`: The result of the query (available on the `asyncStart` and `asyncEnd` events). | ||
| - `error`: The error that occurred during the query operation (available on the `error`, `asyncStart` and `asyncEnd` events). | ||
|
|
||
| The events are emitted in the following order: | ||
|
|
||
| - On success: `start` → `end` → `asyncStart` → `asyncEnd` | ||
| - On failure: `start` → `end` → `error` → `asyncStart` → `asyncEnd` | ||
|
|
||
| Queries always settle asynchronously, even with connectors that fail synchronously, so `asyncEnd` is emitted for every traced query. | ||
|
|
||
| ::note | ||
| Bound parameters are never included in the traced context. Note that the `query` string itself can still contain literal values, either from raw `db.exec()` strings or from static interpolations (`${}` inside `{}`) in the `sql` template. Diagnostics channels are process-wide, so any subscriber can read them. | ||
| :: | ||
|
|
||
| ## Next steps | ||
|
|
||
| :read-more{to="/connectors"} | ||
|
|
||
| :read-more{to="/integrations"} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| import { createDatabase } from "../../src"; | ||
| import { withTracing } from "../../src/tracing"; | ||
| import type { TraceContext } from "../../src/tracing"; | ||
| import { tracingChannel } from "node:diagnostics_channel"; | ||
|
|
||
| import sqlite from "../../src/connectors/better-sqlite3"; | ||
|
|
||
| async function main() { | ||
| const db = withTracing(createDatabase(sqlite({}))); | ||
|
|
||
| // Subscribe to tracing events | ||
| subscribeToTracing(); | ||
|
|
||
| await db.sql`create table if not exists users ( | ||
| id integer primary key autoincrement, | ||
| full_name text | ||
| )`; | ||
|
|
||
| const res = await db.sql`insert into users (full_name) values ('John Doe')`; | ||
|
|
||
| console.log({ res }); | ||
| } | ||
|
|
||
| function subscribeToTracing() { | ||
| const queryChannel = tracingChannel<TraceContext>("db0.query"); | ||
|
|
||
| queryChannel.subscribe({ | ||
| start: (data) => { | ||
| console.log("start", data.query); | ||
| }, | ||
| end: (message) => { | ||
| console.log("end", message.query); | ||
| }, | ||
| asyncStart: (data) => { | ||
| console.log("asyncStart", data.query); | ||
| }, | ||
| asyncEnd: (data) => { | ||
| console.log("asyncEnd", data.query, data.result); | ||
| }, | ||
| error: (data) => { | ||
| console.log("error", data.error); | ||
| }, | ||
| }); | ||
| } | ||
|
|
||
| // eslint-disable-next-line unicorn/prefer-top-level-await | ||
| main().catch((error) => { | ||
| console.error(error); | ||
| // eslint-disable-next-line unicorn/no-process-exit | ||
| process.exit(1); | ||
| }); | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| { | ||
| "name": "db0-with-tracing", | ||
| "private": true, | ||
| "scripts": { | ||
| "start": "jiti ./index.ts" | ||
| }, | ||
| "devDependencies": { | ||
| "jiti": "^1.21.0", | ||
| "db0": "latest" | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,148 @@ | ||
| import type { ConnectorName } from "./_connectors.ts"; | ||
| import type { | ||
| Connector, | ||
| Database, | ||
| Primitive, | ||
| SQLDialect, | ||
| Statement, | ||
| } from "./types.ts"; | ||
| import { sqlTemplate } from "./template.ts"; | ||
|
|
||
| export type TracedOperation = "query"; | ||
|
|
||
| const QUERY_OPERATION: TracedOperation = "query"; | ||
|
|
||
| export interface TraceContext { | ||
| query: string; | ||
| method: "exec" | "sql" | "prepare.all" | "prepare.run" | "prepare.get"; | ||
| connector: ConnectorName; | ||
| dialect: SQLDialect; | ||
| } | ||
|
|
||
| const TRACED: unique symbol = Symbol.for("db0.traced"); | ||
|
|
||
| type MaybeTracedDatabase<TConnector extends Connector = Connector> = | ||
| Database<TConnector> & { | ||
| [TRACED]?: boolean; | ||
| }; | ||
|
|
||
| /** | ||
| * Wrap a database instance with tracing functionality. | ||
| */ | ||
| export function withTracing<TConnector extends Connector = Connector>( | ||
| db: MaybeTracedDatabase<TConnector>, | ||
| ): Database<TConnector> { | ||
| // Avoids double patching | ||
| if (db[TRACED]) { | ||
| return db; | ||
| } | ||
|
|
||
| const { tracingChannel } = | ||
| globalThis.process?.getBuiltinModule?.("node:diagnostics_channel") || {}; | ||
| if (!tracingChannel) { | ||
| return db; | ||
| } | ||
|
|
||
| const queryChannel = tracingChannel<TraceContext>(`db0.${QUERY_OPERATION}`); | ||
|
|
||
| // The context is built lazily to avoid any work when nobody is subscribed. | ||
| // This is an async function so that subscribing never turns a rejection | ||
| // (e.g. an invalid `sql` template) into a synchronous throw. | ||
| async function tracePromise<T>( | ||
| exec: () => Promise<T>, | ||
| context: () => TraceContext, | ||
| ): Promise<T> { | ||
| if (!queryChannel.hasSubscribers) { | ||
| return exec(); | ||
| } | ||
| return queryChannel.tracePromise(exec, context()); | ||
| } | ||
|
|
||
| // Copy the property descriptors instead of spreading, so that getters like | ||
| // `dialect` and `disposed` stay getters. Spreading would evaluate them once, | ||
| // making `disposed` report the state at wrap-time forever after. | ||
| const tracedDb = Object.defineProperties( | ||
| {} as MaybeTracedDatabase<TConnector>, | ||
| Object.getOwnPropertyDescriptors(db), | ||
| ); | ||
| // Non-enumerable, so spreading or serializing a traced database does not leak it. | ||
| Object.defineProperty(tracedDb, TRACED, { value: true }); | ||
|
|
||
| // `exec` is wrapped in an async function so connectors throwing synchronously | ||
| // emit the same event sequence as connectors rejecting asynchronously. | ||
| tracedDb.exec = (query) => | ||
| tracePromise( | ||
| async () => db.exec(query), | ||
| () => ({ | ||
| query, | ||
| method: "exec", | ||
| connector: db.connector, | ||
| dialect: db.dialect, | ||
| }), | ||
| ); | ||
|
|
||
| tracedDb.sql = (strings, ...values) => | ||
| tracePromise( | ||
| async () => db.sql(strings, ...values), | ||
| () => ({ | ||
| query: sqlTemplate(strings, ...values)[0], | ||
| method: "sql", | ||
| connector: db.connector, | ||
| dialect: db.dialect, | ||
| }), | ||
| ); | ||
|
|
||
| class TracedStatement implements Statement { | ||
| #statement: Statement; | ||
| #query: string; | ||
|
|
||
| constructor(statement: Statement, query: string) { | ||
| this.#statement = statement; | ||
| this.#query = query; | ||
| } | ||
|
|
||
| #withTrace<T>( | ||
| fn: () => Promise<T>, | ||
| method: "prepare.all" | "prepare.run" | "prepare.get", | ||
| ) { | ||
| return tracePromise(fn, () => ({ | ||
| query: this.#query, | ||
| method, | ||
| connector: db.connector, | ||
| dialect: db.dialect, | ||
| })); | ||
| } | ||
|
|
||
| bind(...args: Primitive[]) { | ||
| return new TracedStatement(this.#statement.bind(...args), this.#query); | ||
| } | ||
|
|
||
| all(...args: Primitive[]) { | ||
| return this.#withTrace( | ||
| async () => this.#statement.all(...args), | ||
| "prepare.all", | ||
| ); | ||
| } | ||
|
|
||
| run(...args: Primitive[]) { | ||
| return this.#withTrace( | ||
| async () => this.#statement.run(...args), | ||
| "prepare.run", | ||
| ); | ||
| } | ||
|
|
||
| get(...args: Primitive[]) { | ||
| return this.#withTrace( | ||
| async () => this.#statement.get(...args), | ||
| "prepare.get", | ||
| ); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Prepare needs a special treatment because it returns a statement instance that needs to be patched. | ||
| */ | ||
| tracedDb.prepare = (query) => new TracedStatement(db.prepare(query), query); | ||
|
|
||
| return tracedDb; | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
Repository: unjs/db0
Length of output: 530
🏁 Script executed:
Repository: unjs/db0
Length of output: 5394
🏁 Script executed:
Repository: unjs/db0
Length of output: 683
Use the published
db0entry points hereThis example already declares
db0inexamples/tracing/package.json, so it should import fromdb0,db0/tracing, anddb0/connectors/better-sqlite3instead of../../src/*. That makes the example exercise the exported./tracingsubpath and match consumer usage.🤖 Prompt for AI Agents