Skip to content

Latest commit

 

History

History
186 lines (145 loc) · 9.31 KB

readme.md

File metadata and controls

186 lines (145 loc) · 9.31 KB

Testing module

Receives FHIR TestReports - from TxTester, and from any other tool that produces them - and makes them available through a FHIR API and a set of web pages. It runs at /testing when modules.testing.enabled is true.

Submitting reports

POST /testing/TestReport
Content-Type: application/fhir+json
Authorization: Bearer <token>        (only if a token is configured)

R4 and R5 TestReports are treated the same. A report is accepted if it is JSON, has resourceType: TestReport, and has name, status, result, tester, issued (a valid dateTime) and at least one participant with a uri. score, if present, must be a number. Contained resources are not accepted. Nothing else is checked - but the TestReport schema in the OpenAPI description (below) says what a valid report looks like, and reports should conform to it.

The server gives the report a new id (any id sent is ignored), sets meta.lastUpdated to the time it was received, and drops meta.versionId: reports can't be changed once received, so there are no versions. Sending the same report twice stores it twice.

Responses follow the spec: 201 Created with a Location header, and the stored report, or nothing (Prefer: return=minimal), or an OperationOutcome (Prefer: return=OperationOutcome). Errors are OperationOutcomes: 400 (not JSON, or not an acceptable TestReport), 401 (token), 413 (too big), 415 (not JSON), 429 (rate limit).

The submitter's IP address is recorded, for tracing abuse. It is never returned by the API or shown on any page.

FHIRsmith's own tx test run writes its results as test-cases-report.json in the FHIRsmith root; node utilities/submit-test-report.js [-server url] [-token token] sends it to a /testing server (by default https://npm.fhir.org/testing).

Reading and searching

GET /testing/TestReport/{id}
GET /testing/TestReport?{params}
GET /testing/metadata

Both return HTML to a browser (an Accept that includes text/html); _format=json or _format=html overrides that.

None of the search parameters are formally defined yet, and the CapabilityStatement lists them without definitions. What they do:

Parameter Type Searches Notes
_id token id
name string name starts with, case-insensitive; :exact, :contains, :missing
tester string tester as for name
status token status :not, :missing
result token result :not, :missing
testscript uri testScript (R5 canonical, or R4 reference) :below (starts with), :missing
participant uri any participant.uri :below
score number score prefixes; implicit precision for eq (90 means 89.5 to 90.5); :missing
issued date issued prefixes; dates are ranges, so issued=2026-09 is all of September
_lastUpdated date when the report was received prefixes

Commas OR values, repeated parameters AND. _sort takes any of these except _id, comma separated, - for descending; the default is newest received first. _count defaults to 50 and is capped at 500; paging is by _offset, with first/previous/next/last links. total is always returned, and _summary=count returns only that.

Unknown parameters are ignored, with a warning OperationOutcome in the Bundle, unless the request has Prefer: handling=strict, in which case they're a 400. Values that can't be understood (a bad date or number) are always a 400.

OpenAPI description

The FHIR API (create, read, search, metadata) is described by an OpenAPI 3.1 spec:

URL
/testing/openapi Browsable reference (HTML), with a "try it" form for each GET operation
/testing/openapi.json The spec as JSON
/testing/openapi.yaml The hand-written part, as YAML (openapi.yaml)

Every response carries a Link: </testing/openapi.json>; rel="service-desc" header (RFC 8631), and the HTML pages carry the matching <link> element. Deleting reports and the administration pages are deliberately not described.

The spec is in two parts:

  • openapi.yaml - the paths, parameters and responses, written by hand
  • openapi-schemas.json - the FHIR schemas (TestReport, Bundle, OperationOutcome, the datatypes they use, and TestReportSearchBundle), generated from the R5 StructureDefinitions by library/fhir-openapi-schema.js, as configured in openapi-schemas.config.js. The loader merges them into the spec.

The generated schemas are closed (no properties FHIR doesn't define), include the _x siblings of primitive elements (so extensions on primitives, like _name, are allowed), and use the codes of required bindings as enums. Recursion is cut at two boundaries: an extension's value isn't described, and a resource inside another (Bundle.entry.resource) is "any resource" unless the config narrows it. contained is left out, so it isn't allowed. The config's overlay adds what this server requires beyond the base resource, and lets testScript be an R4 Reference.

After changing the config, regenerate (this needs hl7.fhir.r5.core#5.0.0 in the terminology cache):

node utilities/generate-openapi-schemas.js testing

tests/testing/openapi.test.js fails if the generated file is out of date, if a route or search parameter isn't described (or excluded), or if the TestReport schema doesn't require everything validateReport() requires. It also validates what the server actually returns (a stored report, a search Bundle, an OperationOutcome) against the schemas, using ajv.

Web pages

  • /testing - the list of reports, with filters (these map onto the search parameters above), sortable columns and paging
  • /testing/summary - a grid of test script x participant, with the latest report (by issued date) in each cell and a link to all the runs. ?by=tester makes the columns testers.
  • /testing/TestReport/{id} - the report: header, participants, setup / tests / teardown with each action's result and message, anything else in the report as JSON, and the raw JSON.

Everything in a report comes from whoever sent it, so all of it is escaped, and only http(s) URLs become links. The narrative (text.div) is never rendered as HTML - it appears only as text in the raw JSON.

What the pages show

The list has a column each for the test engine, client and server participants (the client column only appears once some report has a client; a participant of any other type is shown as a server). Tests is the number of entries in TestReport.test. Run Length is how long the run took, from the earliest test.period.start to the latest test.period.end; it is blank when the report's tests have no periods. Both are worked out when the report is received, and for reports received before they existed, the first time the server starts with this version.

Where a test script or participant URL, ignoring any |version, is one of the named links, its name is shown instead of the URL - linked to the named link's link, with the full URL in the tooltip and the version after the name.

Administration

/testing/login. The administrator logs in as admin with the adminPassword from the configuration, and manages the other users at /testing/admin/users. A user can have either or both of two rights:

  • edit named links - /testing/admin/links: the canonical URLs that are shown by name
  • delete reports - a Delete button on each report's page

Logins work as the publisher's do: a session cookie (set sessionSecret, and cookieSecure: false if the server is reached over plain http), bcrypt password hashes, and a rate limited login (loginAttemptsPerWindow, default 20 per 15 minutes). Rights are checked on every request, so changing or deleting a user takes effect at once. Every form post must come from the server's own pages: each form carries a per-session CSRF token (lusca) that the post must return, the cookie is SameSite=Lax, and a post whose Origin is another host is refused (behind a proxy, that needs the proxy to pass on Host or X-Forwarded-Host). The session and CSRF checks are only on the web pages: the FHIR API is authenticated by the header token, and sets no cookie.

Deleting

DELETE /testing/TestReport/{id}
Authorization: Bearer <adminToken>

Only when adminToken is configured; without it, nothing can be deleted through the API.

Configuration

"testing": {
  "enabled": true,
  "database": "testing.db",
  "token": "secret",
  "tokenHeader": "Authorization",
  "adminToken": "another-secret",
  "maxSize": 512000,
  "rateLimit": { "windowMinutes": 1, "max": 60 },
  "retentionDays": 365
}
  • database - relative paths are under the data folder's databases directory
  • token - if set, every POST must carry it in tokenHeader (a Bearer prefix is optional). If not set, anyone can submit.
  • tokenHeader - defaults to Authorization. adminToken uses the same header.
  • maxSize - bytes; default 500 KB. Note that the server's body parsers accept up to 50 MB before this module sees the request; a Content-Length over the limit is refused before the body is looked at.
  • rateLimit - POSTs per IP address per window; max: 0 turns it off. Default 60 a minute.
  • retentionDays - if set, reports received longer ago than this are deleted, at startup and daily. Default: keep everything.