diff --git a/doc/api/cli.md b/doc/api/cli.md index 80f2fa74818b92..8f7d6185ac464c 100644 --- a/doc/api/cli.md +++ b/doc/api/cli.md @@ -1413,6 +1413,23 @@ Enable module mocking in the test runner. This feature requires `--allow-worker` if used with the [Permission Model][]. +### `--experimental-test-tag-filter=` + + + +> Stability: 1.0 - Early development + +Run only tests whose tag set contains ``. Tests declare tags via the +`tags` option on `test()`, `it()`, `suite()`, or `describe()`; tags +inherit from suites to nested tests by union. Filtering is +case-insensitive. + +The flag may be specified more than once; tests must contain **every** +filter value to run. See [Test tags][] for details on declaring and +inheriting tags. + ### `--experimental-vm-modules` + +> Stability: 1.0 - Early development + +Tags annotate tests and suites with arbitrary string labels. The +[`--experimental-test-tag-filter`][] CLI flag (or the `testTagFilters` +option on [`run()`][]) selects tests whose tag set contains every +provided filter value. + +Tags are an alternative to encoding metadata into test names. They are +useful for cross-cutting axes such as subsystem, speed bucket, flakiness, +or environment, where a name pattern would be brittle. + +### Authoring tagged tests + +Pass a `tags` array on any of `test()`, `it()`, `suite()`, or `describe()`. +Tags inherit from a suite to its child tests by union—a test inside a +suite tagged `['db']` that declares its own `tags: ['integration']` +effectively has both tags. + +```mjs +import { describe, it } from 'node:test'; + +describe('database', { tags: ['db'] }, () => { + it('reads a row'); // tags: ['db'] + it('writes a row', { tags: ['integration'] }); // tags: ['db', 'integration'] + it('reconnects after disconnect', { tags: ['flaky'] }); // tags: ['db', 'flaky'] +}); +``` + +```cjs +const { describe, it } = require('node:test'); + +describe('database', { tags: ['db'] }, () => { + it('reads a row'); // tags: ['db'] + it('writes a row', { tags: ['integration'] }); // tags: ['db', 'integration'] + it('reconnects after disconnect', { tags: ['flaky'] }); // tags: ['db', 'flaky'] +}); +``` + +Tag values must be non-empty strings. Tags are matched case-insensitively; +the canonical form is lowercase. Duplicates within a single `tags` array +are collapsed on the lowercased form, preserving the first-seen +declaration order. + +Hooks (`before`, `after`, `beforeEach`, `afterEach`) do not declare their +own tags. They run as part of their owning suite, which carries the +suite's tags. + +### Filtering by tag + +Each [`--experimental-test-tag-filter`][] value is a literal tag name. A +test runs only when its tag set contains that name. The flag may be +specified more than once; tests must match **every** filter to run. The +same applies to the `testTagFilters` array on [`run()`][]. Filters are +case-insensitive and AND'd with [`--test-name-pattern`][], +[`--test-skip-pattern`][], and `.only` filtering. + +Untagged tests are excluded under any non-empty filter, since the filter +requires the tag to be present. + +### Reading tags from inside a test + +The [`TestContext`][] object exposes the test's tags as a frozen array +through [`context.tags`][], so tests can branch on their own metadata. + +### Errors + +A tag value that violates the validation rules above throws +`ERR_INVALID_ARG_VALUE` at the registration site, before any test runs. +A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`. + ## Extraneous asynchronous activity Once a test function finishes executing, the results are reported as quickly @@ -750,6 +826,8 @@ test runner functionality: * `--test` - Prevented to avoid recursive test execution * `--experimental-test-coverage` - Managed by the test runner +* `--experimental-test-tag-filter` - Filter values are validated by the parent + process and re-emitted to child processes * `--watch` - Watch mode is handled at the parent level * `--experimental-default-config-file` - Config file loading is handled by the parent * `--test-reporter` - Reporting is managed by the parent process @@ -1568,6 +1646,9 @@ added: - v18.9.0 - v16.19.0 changes: + - version: REPLACEME + pr-url: https://github.com/nodejs/node/pull/63221 + description: Added the `testTagFilters` option. - version: - v25.6.0 - v24.14.0 @@ -1656,6 +1737,10 @@ changes: For each test that is executed, any corresponding test hooks, such as `beforeEach()`, are also run. **Default:** `undefined`. + * `testTagFilters` {string|string\[]} A tag name, or an array of tag names, + used to filter tests by their declared tags. Tests must contain every + listed tag to run. Equivalent to passing [`--experimental-test-tag-filter`][] + on the command line. See [Test tags][]. **Default:** `undefined`. * `timeout` {number} A number of milliseconds the test execution will fail after. If unspecified, subtests inherit this value from their parent. @@ -1799,6 +1884,9 @@ added: - v18.0.0 - v16.17.0 changes: + - version: REPLACEME + pr-url: https://github.com/nodejs/node/pull/63221 + description: Added the `tags` option. - version: - v20.2.0 - v18.17.0 @@ -1842,6 +1930,10 @@ changes: * `skip` {boolean|string} If truthy, the test is skipped. If a string is provided, that string is displayed in the test results as the reason for skipping the test. **Default:** `false`. + * `tags` {string\[]} An array of string labels associated with the test. + Used together with [`--experimental-test-tag-filter`][] to filter which + tests run. Tags inherit from suites to nested tests by union. See + [Test tags][]. **Default:** `[]`. * `todo` {boolean|string} If truthy, the test marked as `TODO`. If a string is provided, that string is displayed in the test results as the reason why the test is `TODO`. **Default:** `false`. @@ -3430,6 +3522,9 @@ Emitted when code coverage is enabled and all tests have completed. `undefined` if the test was run through the REPL. * `name` {string} The test name. * `nesting` {number} The nesting level of the test. + * `tags` {string\[]} The flattened lowercased tags declared on the test + and its ancestor suites, in declaration order. Empty for untagged tests. + See [Test tags][]. * `testId` {number} A numeric identifier for this test instance, unique within the test file's process. Consistent across all events for the same test instance, enabling reliable correlation in custom reporters. @@ -3453,6 +3548,9 @@ The corresponding declaration ordered events are `'test:pass'` and `'test:fail'` `undefined` if the test was run through the REPL. * `name` {string} The test name. * `nesting` {number} The nesting level of the test. + * `tags` {string\[]} The flattened lowercased tags declared on the test + and its ancestor suites, in declaration order. Empty for untagged tests. + See [Test tags][]. * `testId` {number} A numeric identifier for this test instance, unique within the test file's process. Consistent across all events for the same test instance, enabling reliable correlation in custom reporters. @@ -3494,6 +3592,9 @@ defined. `undefined` if the test was run through the REPL. * `name` {string} The test name. * `nesting` {number} The nesting level of the test. + * `tags` {string\[]} The flattened lowercased tags declared on the test + and its ancestor suites, in declaration order. Empty for untagged tests. + See [Test tags][]. * `testId` {number} A numeric identifier for this test instance, unique within the test file's process. Consistent across all events for the same test instance, enabling reliable correlation in custom reporters. @@ -3520,6 +3621,9 @@ Emitted when a test is enqueued for execution. `undefined` if the test was run through the REPL. * `name` {string} The test name. * `nesting` {number} The nesting level of the test. + * `tags` {string\[]} The flattened lowercased tags declared on the test + and its ancestor suites, in declaration order. Empty for untagged tests. + See [Test tags][]. * `testId` {number} A numeric identifier for this test instance, unique within the test file's process. Consistent across all events for the same test instance, enabling reliable correlation in custom reporters. @@ -3579,6 +3683,9 @@ since the parent runner only knows about file-level tests. When using `undefined` if the test was run through the REPL. * `name` {string} The test name. * `nesting` {number} The nesting level of the test. + * `tags` {string\[]} The flattened lowercased tags declared on the test + and its ancestor suites, in declaration order. Empty for untagged tests. + See [Test tags][]. * `testId` {number} A numeric identifier for this test instance, unique within the test file's process. Consistent across all events for the same test instance, enabling reliable correlation in custom reporters. @@ -3618,6 +3725,9 @@ defined. `undefined` if the test was run through the REPL. * `name` {string} The test name. * `nesting` {number} The nesting level of the test. + * `tags` {string\[]} The flattened lowercased tags declared on the test + and its ancestor suites, in declaration order. Empty for untagged tests. + See [Test tags][]. * `testId` {number} A numeric identifier for this test instance, unique within the test file's process. Consistent across all events for the same test instance, enabling reliable correlation in custom reporters. @@ -4121,6 +4231,20 @@ The attempt number of the test. This value is zero-based, so the first attempt i the second attempt is `1`, and so on. This property is useful in conjunction with the `--test-rerun-failures` option to determine which attempt the test is currently running. +### `context.tags` + + + +> Stability: 1.0 - Early development + +* Type: {string\[]} + +A frozen array of the test's flattened lowercased tags, in declaration +order, including any tags inherited from ancestor suites. Empty when the +test has no tags. See [Test tags][]. + ### `context.workerId`