hsg-jinks – hsg-jinks

Office of the Historian

This application was generated by TEI Publisher using the following profiles:

Only the non-standard bits are documented here. Full background: doc/uswds-profile-integration.md.

Workspace layout

This repo's tooling assumes its companion repos are flat siblings in one parent directory (<workspace> below) and reaches them by relative path (../../<sibling> from inside a worktree — see "Parallel worktrees"), never a hardcoded absolute path. That means <workspace> can live anywhere; only the sibling names matter:

<workspace>
├── profile-theme-uswds   USWDS theme profile (private releases) — see "USWDS profile baseline"
├── jinks                 the Jinks generator itself (Mode A) — baked into every devcontainer image
├── jinks-templates       companion templates library — also baked in
├── hsg-jinks-data        FRUS sample data, deployed as frus-data-jinks — see "Data"
├── frus-tei-guidelines   canonical FRUS TEI customization (schema, processing model, rendition vocabulary) — see "Tools"
├── hsg-project           pre-v10 reference implementation (hsg-shell) — see CLAUDE.md "Reference implementation"
└── hsg-jinks             this repo: `main` + parallel worktrees — see "Parallel worktrees"

Each sibling can be a plain clone, a symlink to a clone kept elsewhere, or its own git worktree of that repo — pick whatever fits how you're using it. jinks and jinks-templates don't need their own worktree unless you're actively changing them; a git worktree add is only worth it when, like profile-theme-uswds here, you want it parked on a different branch than whatever the original checkout has open.

secrets.env, worktree-ports.tsv, and the frozen baseline xars are not inside hsg-jinks/ despite belonging conceptually to this workspace — they're genuinely machine-global at ~/.config/hsg/ (see "Parallel worktrees" below), shared across every hsg-jinks worktree no matter where <workspace> itself lives.

Workflow

• All changes go through a branch + PR targeting main — in this repo and in profile-theme-uswds. No direct commits to either main.

• Page templates follow doc/templating.md: the page's markup lives in its template, XQuery supplies data and fragments.

• Check a work package's assignee in OpenProject before implementing it. If we decide to pick one up, assign it to the person actually doing the work, not the AI assistant — ask if it's unclear who that is.

• Reference work packages as OP#<id> in PR titles and descriptions — that is the token OpenProject's GitHub integration scans for, and how a PR gets linked to its work package. A bare #1217 means nothing to it (GitHub reads that as an issue in this repo), and a work package URL is a link for humans, not a reference. Example title:

  fix: content band on sidebar pages respects the page width (OP#1217)
  

  Linking is not closing: move the work package yourself when the PR merges. This board's workflow only permits New, In progress, Closed, On hold and Rejected — Closed is how it marks work done.

The devcontainer runs the generator

Unlike a stock TEI Publisher devcontainer (runtime library only), this one hosts Jinks itself: .devcontainer/Dockerfile.dev clones jinks from main, builds it, and copies in the USWDS profile xar. The .devcontainer/ files are therefore hand-edited away from the generated docker profile — do not let a sync overwrite them.

-Xss16m is set in JAVA_OPTS there to work around a Jinks generator StackOverflow on large profiles (generator:filter-conflicts recurses per file; USWDS ships ~2600). Remove when Jinks fixes it.

Baseline image

Dockerfile.dev builds on joewiz/hsg-project:v2, pinned to a digest, the image this app goes live with. It is a vector-capable eXist 7 (all-MiniLM-L6-v2 under /exist/onnx-models) whose database is baked in:

• hsg-jinks itself, which every worktree container replaces with its own build (see "Container initialization" below)
• frus-data-jinks with its vector index for semantic search
• the runtime libraries: tei-publisher-lib, jinks-templates, roaster, jwt, crypto, carousel
• the HistoryAtState data packages (see "External data packages" below)

So worktrees test new versions of hsg-jinks against the go-live baseline rather than against a set of versions pinned here. Dockerfile.dev only adds what development needs on top: ant, git and node for building, the jinks generator (built from main) and the USWDS theme baseline xar.

The image is pinned by digest (v2@sha256:99cbf7e1bb1c…, the v2 of 2026-10-15) in Dockerfile.dev and in the CI workflow; keep both the same. The v2 republished on 2026-10-16 is ~8 GB compressed and no longer fits the disk of a standard GitHub runner. To move to a newer image, replace the digest in both files (docker buildx imagetools inspect joewiz/hsg-project:v2 shows the current one), rebuild the container and run the full suite. After a rebuild, xst package list --versions shows what the image brought — xst package install has been seen reporting updated to X@<new> while the deployed collection stayed on the old version.

Parallel worktrees

Each work package gets its own git worktree with its own devcontainer/eXist instance. One command sets everything up:

tools/new-worktree.sh <slug> [branch]

Slug naming convention: <type>-<wp#>-<short-slug>, e.g. fix-1217-page-box-sidebar-pages, where:

• <type> is a conventional-commit-style prefix — feat, fix, chore, explore, docs, refactor, test, perf, build, or port — and sets the default branch's prefix (<type>/<rest>, e.g. fix/1217-page-box-sidebar-pages).
• <wp#> is the OpenProject work package number (see "Workflow" above, OP#<id> without the OP#); omit it for untracked/ad-hoc work, e.g. chore-parallel-worktrees → branch chore/parallel-worktrees.
• If <type> is absent or unrecognized, the slug is used as the branch name verbatim, with no prefix inserted (e.g. hsg-analysis).

The worktree folder itself is always <slug> (next paragraph) — the convention above is what makes that folder name double as the branch name, so git worktree list and ls .. stay legible together.

It creates <parent>/<slug> as a flat sibling of the main clone, nested one level under the repos root that also holds hsg-jinks-data and hsg-project — so from inside a worktree, ../../hsg-jinks-data and ../../hsg-project are the working relative references. Allocates the next free host port pair (8091+/8454+ upward; main keeps 8090/8453), writes the per-worktree config, copies in the baseline profile xar, builds + starts the container, and seeds data + app.

Machine-specific state lives in two gitignored files per worktree plus the shared ~/.config/hsg/ — never in tracked files:

location	contents
.env (worktree root)	EXISTDB_SERVER (this worktree's port), EXISTDB_USER/PASS, CYPRESS_BASE_URL
.devcontainer/.env	HSG_HTTP_PORT, HSG_HTTPS_PORT, COMPOSE_PROJECT_NAME (see env.template)
~/.config/hsg/worktree-ports.tsv	slug → port registry
~/.config/hsg/xars/	frozen baseline profile xar ("USWDS profile baseline" below)
~/.config/hsg/secrets.env	OP_TOKEN and OP_BASE_URL (the OpenProject instance) — source it before OpenProject curl calls; the token never appears in a repo or a command line

Port rules (unchanged in spirit from the old "Remapping ports" section):

• The host side comes from .devcontainer/.env via compose interpolation (${HSG_HTTP_PORT:-8090}:8080); compose publishes deterministically.
• eXist keeps listening on 8080/8443 inside every container — .devcontainer/init.sh and .existdb.json hardcode it; leave both alone.
• Host port 8080 is the read-only hsg-shell reference mirror (container hsg-shell-test, no volumes, restart: no). Reads only (guest) — never point xst/jinks at it, never REST-write to it. A Claude Code hook (.claude/hooks/protect-reference-mirror.sh) blocks write-shaped commands against it as a safety net. If it disappears after a reboot: docker run -d --name hsg-shell-test -p 8080:8080 joewiz/hsg-project:latest.

Teardown of a finished worktree: docker compose -f .devcontainer/docker-compose.yml down, git worktree remove <path>, delete its row from the port registry.

Container initialization (init.sh)

.devcontainer/init.sh is the devcontainer's postCreateCommand (see devcontainer.json): it runs every time the container is created, whether that's new-worktree.sh's first build or a later rebuild through the Dev Containers tooling. new-worktree.sh runs the exact same script itself (docker compose exec existdb sh .devcontainer/init.sh) rather than duplicating its steps, so there's one seed path, not two that can drift out of sync.

The dividing line: anything that seeds state living inside eXist's own database goes in init.sh, because that state does not survive container recreation (there's no volume — eXist's data directory is just part of the container's writable layer). Anything that's about the host, the git worktree, or the Docker image build stays a one-time step in new-worktree.sh, because it lives in git, in ~/.config/hsg/, or gets baked into the image before init.sh ever runs.

init.sh currently, using xst (pinned to 4.1.0 in Dockerfile.dev and the CI workflow) against the eXist on port 8080:

• adds admin to the tei group (security-manager state; without it a later jinks sync writes auth errors as file content instead of failing loudly)
• builds the app xar with ant (with PRODUCTION=true in the environment, ant -Dproduction=true, as CI does) and installs it with xst package install --force, which replaces any installed hsg-jinks, including the one the base image ships. xst undeploys and removes the old registration first; repo:install-and-deploy-from-db on top of a registered package silently drops files (observed: the roaster API-spec modules vanish, breaking the router). --force because the version number stays 1.0.0 between builds.

It exports EXISTDB_SERVER/EXISTDB_USER/EXISTDB_PASS itself: exported values win over the worktree's .env, which holds the host port. CI runs the same script on the runner, where 8080 is the published container port, with PRODUCTION=true: CI tests only the production build, and the full Cypress suite runs with --env PRODUCTION_BUILD=true. The development-only routes and CORS settings are covered by api/base-routes.cy.js when it runs without that flag, i.e. locally. Never run it on the host here: host port 8080 is the read-only mirror.

It does not install frus-data-jinks or the USWDS theme: the data is part of the base image's database and the theme autodeploys from the baseline xar baked into the image (see "Baseline image" above and "USWDS profile baseline" below), so both survive container recreation with no help from this script. The app xar it installs replaces the hsg-jinks that ships in the base image. It also does not run jinks create — this is a Mode B repo, so config.json's generated output is already committed to git; a plain xar install redeploys that same content through the same underlying mechanism jinks create itself uses, without the registration conflict above. Only run jinks update by hand, and only after actually editing config.json.

Known gap: postCreateCommand only fires through the Dev Containers CLI/VS Code. A plain docker compose down && up (e.g. after moving a worktree, as happened when header-cleanup got renamed) does not trigger it — the container comes back up with only whatever's baked into the image, silently missing anything init.sh would have seeded live. If you recreate a container that way, run it by hand afterward:

docker compose -f .devcontainer/docker-compose.yml exec existdb sh .devcontainer/init.sh

USWDS profile baseline

profile-theme-uswds is private, so neither docker.externalXar nor an anonymous download in Dockerfile.dev can reach its releases, and the base image doesn't carry it yet (HistoryAtState/hsg-project#64). The container build COPYs whatever xar sits in .devcontainer/ (gitignored). To keep parallel worktrees comparable they all use the same baseline: the theme's latest release that the app has picked up (profile-versions in context.json). How the theme plugs into Jinks: doc/uswds-profile-integration.md.

• The baseline is stored as ~/.config/hsg/xars/profile-theme-uswds-baseline.xar (profile-theme-uswds-BASELINE.txt beside it records release, commit and sha256). Fetch it with an authenticated gh:

  gh release download v<version> -R eeditiones/profile-theme-uswds \
      -p profile-theme-uswds.xar -O ~/.config/hsg/xars/profile-theme-uswds-baseline.xar

• tools/new-worktree.sh copies it into every new worktree — identical theme everywhere.

• Worktrees whose WP fixes the profile itself are the exception: create a worktree of the profile repo too, build there (ant), replace the xar in the app worktree's .devcontainer/ (only one may be there — Dockerfile COPYs by glob) and rebuild the container; or hot-update the running instance from the app worktree (its .env supplies the right port):

  xst package install --force <profile-worktree>/build/profile-theme-uswds-*.xar
  printf 'n\n' | jinks update hsg-jinks --server http://localhost:<port>/exist/apps/jinks

  (Pipe n: jinks update prompts on conflicts and dies without a TTY. The build.xml / expath-pkg.xml conflicts are app-specific — keep ours.)

• When the profile publishes a new release: download it as above (overwriting the old xar) and update profile-theme-uswds-BASELINE.txt. Other worktrees pick the new baseline up on their next container rebuild — nobody is forced to absorb it mid-work. context.json's profile-versions is sticky (only -r, which must never be used, recomputes it), so when the app picks up a release, edit the theme's entry there by hand along with the copied CSS.

Data

hsg-jinks-data (the legacy FRUS corpus, deployed as /db/apps/frus-data-jinks with volumes under data/frus-v2-data/ and guidelines apparatus under data/frus-tei-guidelines-resources/) comes with the base image (frus-data-jinks 0.4, with the vector index for semantic search). It is a curated multi-volume subset: every volume the landing page links plus representative era/structure samples — the authoritative list lives in hsg-jinks-data's build.xml. The full corpus (694 volumes, ~4.6 GB) is a separate infra decision.

To test against a different build of the data, install it into this worktree's own container (never port 8080):

cd ../../hsg-jinks-data && ant
xst --config <worktree>/.xstrc package install --force build/frus-data-jinks-*.xar

That is runtime-only: eXist's database lives in the container's writable layer, so recreating the container brings back the image's data.

External data packages (GitHub releases)

Beyond frus-data-jinks above, running the app also needs 12 data packages — the ones backing /departmenthistory, /countries, /conferences, /about, /tags, /milestones, /api/v1/catalog, and /historicaldocuments/frus-history — each maintained in its own repo under github.com/HistoryAtState and published as a versioned .xar GitHub release. The base image brings all of them; which release each one is at is the image's choice (xst package list --versions). When a count-style test breaks on a newer image, fix the test per "Cypress assertions on data-package counts" in CLAUDE.md, don't pin the data.

Repo	Latest release
conferences	https://github.com/HistoryAtState/conferences/releases/latest
frus-history	https://github.com/HistoryAtState/frus-history/releases/latest
gsh	https://github.com/HistoryAtState/gsh/releases/latest
milestones	https://github.com/HistoryAtState/milestones/releases/latest
other-publications	https://github.com/HistoryAtState/other-publications/releases/latest
pocom	https://github.com/HistoryAtState/pocom/releases/latest
rdcr	https://github.com/HistoryAtState/rdcr/releases/latest
tags	https://github.com/HistoryAtState/tags/releases/latest
travels	https://github.com/HistoryAtState/travels/releases/latest
visits	https://github.com/HistoryAtState/visits/releases/latest
wwdai	https://github.com/HistoryAtState/wwdai/releases/latest
hac	https://github.com/HistoryAtState/hac/releases/latest

tags (deployed as /db/apps/tags, taxonomy data at tags/taxonomy) also backs the OPDS catalog's browse/search-by-subject routes (OP#1105), on top of the /tags section itself.

To pick up a newer release without a full image rebuild, refresh one or all of them directly into this worktree's own devcontainer (never port 8080, see "Parallel worktrees & the read-only mirror" in CLAUDE.md):

for repo in conferences frus-history gsh milestones other-publications pocom rdcr tags travels visits wwdai hac; do
  url=$(gh release view --repo HistoryAtState/$repo --json assets -q '.assets[0].url')
  curl -sL -o "/tmp/${repo}.xar" "$url"
  xst --config .xstrc package install "/tmp/${repo}.xar"
done

That's a runtime-only change (eXist's database isn't persisted, see "Data" above) — it won't survive the next container recreation, which brings back the base image's releases.

These packages are append-only upstream — see CLAUDE.md ("Cypress assertions on data-package counts") before writing a test that counts anything rendered from them.

tags's own bespoke, non-TEI taxonomy.xml is deployed as-is at /db/apps/tags but never queried directly by the app — tools/convert- taxonomy.xql (OP#1177) re-encodes it as native TEI (tei:taxonomy/tei:category/tei:catDesc) into this app's own data/tags/taxonomy.xml, which is checked in and deploys with the rest of data/. Re-run that script (xst --config .xstrc execute -f tools/convert-taxonomy.xql) and commit the refreshed data/tags/taxonomy.xml whenever the tags package is refreshed to a newer release.

Production Docker image

The root Dockerfile builds a self-contained image: the app xar built from this checkout plus everything it needs at runtime — the EXPATH libraries (roaster, jinks-templates, tei-publisher-lib, pinned to the same versions as the devcontainer baseline), carousel, all 11 external data packages (each fetched from its repo's /releases/latest/, same policy as Dockerfile.dev), and frus-data-jinks. The USWDS theme is not needed at runtime — it's a generation-time profile whose output is already committed here and packed into the app xar.

Every build leaves the base profile's modules/lib/api.json out of the router in the built modules/lib/api.xql. The app serves only its own API specs (modules/*-api.json); any other path answers with the 404 page. To serve a base operation, declare its path in an app spec and name the operation (e.g. dapi:table-of-contents): api.xql still imports the base modules. The generated controller sends the ODD, lint, fix-odds and LaTeX routes to dedicated entry points that load lib/api.json themselves, so jinks run hsg-jinks fix-odds still works in development.

The same build step adds the modules/schema-constraints.xqm middleware to the router. Roaster itself only casts parameters by type; the middleware enforces pattern, maxLength, minLength and enum on path and query parameters (for arrays, on their items). A path parameter that fails answers 404, a query parameter 400. Declare these constraints in an app spec rather than checking an id by hand before it becomes part of a database path.

The image builds the app with ant -Dproduction=true. That build also rewrites the generated controller.xql so those requests go through api.xql as well, where they answer 404. It also empties the CORS origin whitelist ($config:origin-whitelist in modules/config.xqm), so no origin gets Access-Control-Allow-Credentials; every response carries Access-Control-Allow-Origin: *. Development keeps the localhost entries. All these rewrites fail the build if the generated file changes shape; see the app-routes, production-controller and production-cors targets in build.xml.

frus-data-jinks comes from the private, release-less hsg-jinks-data repo, so it must be staged at the repo root before building (the COPY fails loudly if it's missing; the xar is gitignored and excluded from the app xar by build.xml):

cp ~/.config/hsg/xars/frus-data-jinks-baseline.xar ./frus-data-jinks-0.2.xar
docker build -t hsg-jinks .

The container listens on 8081 (HTTP) and 8444 (HTTPS) directly (jetty.port/jetty.ssl.port are set via JDK_JAVA_OPTIONS), so these ports don't collide with the reference mirror on 8080 or the devcontainers on 8090+:

docker run -d --name hsg-jinks -p 8081:8081 -p 8444:8444 hsg-jinks
# → http://localhost:8081/exist/apps/hsg-jinks/index.html

To run the Cypress suite against it, point both the base URL and the API server at 8081 (see CLAUDE.md and the memory notes on suite invocation):

set -a; source .env; set +a
EXISTDB_SERVER=http://localhost:8081 \
CYPRESS_BASE_URL=http://localhost:8081/exist/apps/hsg-jinks \
npx cypress run > /tmp/cypress-docker.log 2>&1

Sync-down: files to exclude

xst get the app, then copy, never mirror (a mirror deletes .existdb.json, package-lock.json, .devcontainer/python-api.sh, which aren't in the DB).

The DB holds stale copies of these that will overwrite newer local versions — exclude them and git diff after:

• .devcontainer/ — the hand-edits above
• README.md, doc/ — this file and the doc
• .gitignore — DB copy predates the .devcontainer/*.xar rule
• test/cypress/support/ — emptied by the profile once; jinks update won't restore

Also note: Jinks' .jinks.json never prunes removed files — clean stale entries by hand.

Tools

• tools/inspect-page.xq — fetch a deployed page server-side, assert on rendered markup by CSS class (excluded from the app build; run with xst, see header).
• tools/sync-frus-guidelines.sh — vendor the canonical FRUS TEI artifacts from the frus-tei-guidelines sibling checkout (see "Workspace layout"). resources/odd/frus.odd is the live one: resources/odd/hsg-jinks.odd carries source="frus.odd", so TEI Publisher merges the canonical FRUS processing model with this app's overrides. frus.css, print.css and frus.rnc land in resources/frus-tei-guidelines/ for reference only and are excluded from the app build; frus.css in particular must never go into resources/odd/, because TEI Publisher would inline its bare TEI element selectors into the web stylesheet, where they match HTML elements. The vendored copies are kept byte-identical to canonical so drift is detectable. Recompile the transforms after a sync that touches frus.odd.

  tools/sync-frus-guidelines.sh          # copy + record provenance
  tools/sync-frus-guidelines.sh --check  # report drift, write nothing
  jinks run hsg-jinks fix-odds           # recompile transform/ after frus.odd changes
  

• tools/generate-frus-renditions.xq — regenerate resources/css/frus-renditions.css from the FRUS rendition vocabulary (the tei:rendition definitions in hsg-jinks-data's frus-tei-guidelines-resources/frus-encodingDesc.xml). That stylesheet is generated, not hand-maintained: re-run the script and commit the result whenever the vocabulary changes. xst --config .xstrc run -f tools/generate-frus-renditions.xq > resources/css/frus-renditions.css
• tools/measure-cell.sh <model> <dimension> <db|lucene> — measure one hybrid-search vector configuration in the worktree's own container: reindex frus-data-jinks, time five paraphrase queries in keyword and hybrid mode, and append the result to build/semantic-search-workload-results.json. Starts from the collection.xconf at $XCONF_SRC (default /tmp/frus-collection.xconf.xml).
• test/cypress/e2e/gui/breadcrumb.cy.js — run against the worktree's own container: dotenv -e .env -- npx cypress run --spec … (Cypress reads CYPRESS_BASE_URL from the environment, not from .env itself).

MCPs and plugins used

Get the community edition MCP from https://github.com/jtauschl/openproject-ce-mcp

Add it with

claude mcp add \
        --env OPENPROJECT_BASE_URL=$OP_BASE_URL \
        --env OPENPROJECT_API_TOKEN=$OP_TOKEN \
        --env OPENPROJECT_READ_PROJECTS=hsg-jinks \
        --env OPENPROJECT_WRITE_PROJECTS=hsg-jinks \
        --transport stdio \
        --scope local \
        openproject -- openproject-ce-mcp