Skip to content

✨ publish: Jira target (Epic + Stories, Cloud and Data Center) - #117

Merged
nicholasjackson merged 2 commits into
mainfrom
feat/publish-jira
Nov 9, 2026
Merged

nicholasjackson merged 2 commits into
mainfrom
feat/publish-jira

Conversation

@clubanderson

@clubanderson clubanderson commented Nov 8, 2026 •

Copy link
Copy Markdown
Member

Fixes #112

Adds a Jira publish target following the GitHub target from #115 exactly: same Target interface, same ordering (RFC → children → links → RFC body rewrite), same refusal codes with next_action, dry-run with zero network calls, httptest fake with recorded assertions.

What it does

  • RFC → Epic, children → Story under it: parent on Cloud; on Data Center (where parent is sub-task only) the Epic Link custom field, publish.jira.epic_link_field or discovered from GET field when unset. Issue types configurable.
  • depends_on → Blocks issue links (blocker outward, dependant inward). Parent and links are read back and reconciled on every run (parent/Epic Link + issuelinks in the search fields); anything missing is refiled and reported as linked. Failures are warnings.
  • Marker kept as the last line of the description and in the spek.publish issue property with a body_digest, so idempotency works although Jira stores ADF/wiki markup, not Markdown. The property is set in the create/edit request itself (properties), so no issue ever exists without it.
  • Find: project = <key> AND description ~ "spekpub<sha256(artifact_id)[:16]>" — a plain alphanumeric token on the line before the marker, so no Lucene escaping; the issue.property[...] JQL is not used (needs an app to index the property on both Cloud and DC). Issues recorded in the artifact's published: frontmatter are then read directly (GET issue/KEY), so a re-run after a partial failure or ahead of the search index reuses them. A partial failure records what it filed before returning.
  • Plain http:// base URLs are refused (publish_target_unconfigured) unless loopback or publish.jira.insecure_http: true.
  • Markdown subset → ADF (Cloud, REST v3) or wiki markup (Data Center, REST v2).
  • Auth: JIRA_EMAIL + JIRA_API_TOKEN (Cloud) or JIRA_PAT (DC); site via JIRA_BASE_URL or publish.jira.base_url; JIRA_CA_BUNDLE for private CAs. Labels whitespace → - (hive convention).
  • Refusals: publish_target_unconfigured (no project_key / base URL / bad CA bundle), publish_auth_missing.

Interface decisions

  • Filed gained Key and BodyDigest; Diff compares digests when present (SameAsFiled). Result.RFCKey / ResultChild.Key added (omitempty) so GitHub output is unchanged.
  • No parent_label for Jira: hive's default JQL already excludes issuetype = Epic.

Tests

make lint && make test green locally (internal/publish/jira, jiratest fake — refuses parent on DC Stories, no property JQL, SearchLag/FailCreateAfter — cmd tests for dry-run with zero calls, refusals incl. plain http, file-then-update, partial failure then resume, Data Center v2 with Epic Link discovery).

Not in this PR: the GitHub target (main) has the same "links only on create" gap; separate PR against main.

Docs: README publish section + config block, docs/publish-hive-handshake.md Jira row.

Add a Jira work-source target behind the publish Target interface. The RFC
is filed as an Epic in publish.jira.project_key and each child as a Story
under it (parent, or epic_link_field on Data Center); depends_on edges become
Blocks links, best effort. The spek:publish marker is kept as the last line
of the description and in the spek.publish issue property together with a
body digest, so re-publish finds and compares what it filed even though
Jira stores ADF/wiki markup rather than Markdown. Property JQL is tried
first and falls back to a description text search where properties are not
indexed. Cloud (REST v3, ADF, JIRA_EMAIL + JIRA_API_TOKEN) and Data Center
(REST v2, wiki markup, JIRA_PAT, JIRA_CA_BUNDLE) are both supported.

Includes an in-process fake Jira (jiratest) and unit/command tests covering
create, idempotent re-run, --update, link failures as warnings, refusals
and dry-run making no network call.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @nicholasjackson should decide whether Jira publishing must require HTTPS before sending credentials, or intentionally support plain HTTP Data Center sites.

The Jira dependency-link recovery needs a fix before this is ready.

  • Medium — internal/publish/jira/jira.go:340: failed links are never retried. blockedBy runs only when the dependant or blocker appears in res.Created. If the initial link POST fails, the issues and their marker properties still exist; on the next --update, Find reuses both, res.Created is empty, and this guard skips the link without even returning a warning. Adding a dependency between two existing children is skipped for the same reason. This contradicts the PR body's retry promise and leaves the native dependency graph incomplete. Reconcile missing links for existing endpoints too, with a regression case that fails a link, restores the service, and republishes.

Credential transport decision for @nicholasjackson: cmd/publish.go:331-341 accepts a nonempty base URL without checking its scheme, and internal/publish/jira/jira.go:465-483 attaches Basic/Bearer credentials and sends the request. An HTTP site therefore receives the credential without TLS. Issue #112 specifies the credential sources and CA support but does not settle whether insecure transport is allowed. Please decide the policy before this new authentication path lands.

I read the command caller, target, fake server and tests against the base implementation, then checked #112 and this description. Local test execution was denied by the reviewer environment, so the recovery finding is based on the verified control flow; no test result is claimed.

— hive: agent=reviewer backend=codex model=gpt-6.1-sol codex=0.167.0

@nicholasjackson

Copy link
Copy Markdown
Collaborator

@clubanderson coupe of comments on the PR, but I am pretty much happy to merge:

Applies to the whole stack, including main’s GitHub target: links are only created for issues filed in the same run. blockedBy runs only for new issues, and updateIssue never sets parent. A failed link, a newly added depends_on, or a re-created Epic is never repaired by --update, and a no-op run doesn’t try at all. The PR description says links are “retried on --update”, and the comment in github.go says the same, but neither is true. Hive relies on these links for umbrella and ordering, so links need reconciling against what’s filed on every run. I’m happy for that to be a separate PR against main, but the claim should come out of this description until it lands.

Jira-specific:

Data Center Stories under an Epic. On DC, a Story links to its Epic through the Epic Link custom field; parent is only for sub-tasks. With epic_link_field unset, we send parent, which I’d expect DC to reject. The DC fake accepts fields.parent on Stories, so the e2e can’t catch this. Either make epic_link_field required for DC, or discover it, and make the fake reject parent on non-sub-task types.
Find on Cloud. Cloud only supports JQL on an issue property when an installed app declares it indexed, so in practice Cloud will always use the description ~ fallback. The fake sets PropertyJQL: !dataCenter, so that fallback is never tested as the main path. The fallback also passes artifact_id= into Lucene text search unescaped, where /, = and - are all special. That can error or simply miss, and a miss means duplicates. Please escape the value, or search on a token that needs no escaping, and give the Cloud fake no property search.
Create, then set the property. If the property write fails after the issue is created, the issue has no property, and the next run’s property search won’t find it. Either set the property in the create call (properties on POST /issue), or have Find also fall back to the URLs already recorded in published:.
Re-run after a partial failure. published: is only written after full success, and Cloud search is eventually consistent. An immediate re-run, which is what our error message tells people to do, can therefore miss the issues it just created and file them again.
Minor: the marker appears as visible text at the bottom of every issue in both ADF and wiki markup. Could it go in the property only, or in a collapsed or hidden block?

…atomic properties, token search

- parent and Blocks links are read back from each issue and refiled when
  missing, on every run, not only for issues created in the same run;
  the result lists them as linked
- Data Center: the Epic Link field is discovered from GET field when
  publish.jira.epic_link_field is unset; the fake refuses parent on
  anything but a sub-task, as the real site does
- the spek.publish property travels in the create/edit request, so no
  issue exists without its marker; the separate property write is gone
- Find searches a plain alphanumeric token (spekpub<hash>) written on
  the line before the marker, not the property JQL (needs an indexed
  property app) nor artifact_id= (Lucene specials unescaped)
- a publish that fails part way records what it filed in published:,
  and a re-run passes the recorded URLs to the target so they are read
  directly while the search index catches up
- a plain-http base URL is refused unless loopback or
  publish.jira.insecure_http: true
- fakes assert instead of require on handler goroutines; Snapshot/Calls
  accessors; SearchLag and FailCreateAfter options

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
@clubanderson

Copy link
Copy Markdown
Member Author

Thanks Nic — all addressed in a151ea7 (plus the stack re-based on it). Point by point:

Links only created in the same run / "retried on --update" (whole stack). True as written. Fixed for Jira in a151ea7: Publish now reconciles parent (Cloud parent / DC Epic Link) and blocks links against what the issues actually carry on every run, including no-op runs and existing children, so a failed link, a new depends_on or a re-created Epic is repaired. Linear gets the same in #118 (0cf22c8). The GitHub target on main has the same gap; I've removed the "retried on --update" claim from this description and github.go's comment comes out with the fix, which I'll send as a separate PR against main as you suggested.

DC Stories under an Epic. Fixed. With epic_link_field unset on DC, the target discovers the Epic Link field from GET /rest/api/2/field (schema.custom == com.pyxis.greenhopper.jira:gp-epic-link) and refuses with a next_action naming epic_link_field if none is found; parent is only ever sent for sub-tasks on DC. The DC fake now serves /rest/api/2/field and returns 400 errors.parent for parent on any non-"Sub-task" type — the old code fails against it; the e2e in #119 asserts the Epic Link field (not parent) is what a DC Story carries.

Find on Cloud / unescaped Lucene. Fixed. Find no longer uses property JQL at all; it searches project = X AND description ~ "<token>" where the token is spekpub + 16 hex of sha256("spek:publish " + artifact_id) — a single alphanumeric word, nothing to escape. The token is written into the description on the line before the marker. The fakes have no property-search path any more.

Create, then set the property. Fixed. The property goes in the properties array of POST /rest/api/2/issue (and on PUT), so there is no window where an issue exists without it. The fake asserts it on create.

Re-run after a partial failure / eventual consistency. Fixed two ways: (1) published: is now written after a partial failure with whatever was filed so far (filedSoFar), and (2) the target gets a Hint(urls) of recorded URLs and reads those issues directly by key (GET issue/KEY, 404 skipped) before searching, so a re-run is independent of the search index. The Jira fake has a SearchLag mode (nothing searchable until Reindex()) and the #119 e2e runs with it on throughout; FailCreateAfter(n) covers the partial-failure case in cmd/publish_test.go.

Minor: marker visible in the description. Left as is, deliberately: hive's spekmarker package reads the marker from issue bodies (it's what drives the never-claim-the-RFC and clarity-gate behaviour), and Jira exposes neither property values nor a hidden node to a body reader — ADF has no comment node and {expand}/panel content is still visible text. So the marker has to be in the body. It's confined to one trailing line after the --- footer, and the property additionally carries it for anything that reads properties. If hive grows a property reader we can drop the body copy then.

Branch is green locally (make lint && make test) and in CI.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @nicholasjackson should decide whether Jira credentials may follow redirects across origins or whether the new client must restrict authenticated redirects to the configured origin.

Two issues need attention before this lands:

  • Medium — reversed dependency read-back (internal/publish/jira/jira.go:333-340). filedFrom records outwardIssue as BlockedBy. For Jira's default Blocks type, outward means the current issue blocks the linked issue; blockers are returned as inwardIssue (Atlassian model). Correct links therefore appear missing on every retry, while an opposite-direction link can suppress repair. The fake repeats this reversal at internal/publish/jira/jiratest/fake.go:90-93. Read the inward end and correct the fake and regression assertions.

  • High — redirects bypass the HTTPS credential guard (internal/publish/jira/jira.go:87, :640-650). requireHTTPS checks only the initial URL; the clients leave CheckRedirect unset. Go follows HTTPS-to-HTTP redirects on the same host with Authorization retained (Go client documentation). A Jira/reverse-proxy redirect can therefore send PAT or Basic credentials in clear with insecure_http: false. Enforce transport policy on redirects and cover the downgrade with dummy credentials.

Per #112, Cloud/DC authentication and publishing are intended scope. The unresolved security choice is the authenticated redirect trust boundary; @nicholasjackson is the approver in OWNERS.

Validation: read the adapter, renderer, fake, shared Diff, command callers, PR body and #112 (no comments). Existing build/test checks report success. Local tests were denied by reviewer execution policy, so I did not run them.

— hive: agent=reviewer backend=codex model=gpt-6.1-sol codex=0.167.0

@nicholasjackson

Copy link
Copy Markdown
Collaborator

/approve

@kubestellar-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nicholasjackson

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@nicholasjackson
nicholasjackson merged commit 8f6fd66 into main Nov 9, 2026
6 of 7 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the feat/publish-jira branch November 9, 2026 07:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

✨ publish --target jira: Epic + Stories (Cloud and Data Center)

2 participants