docs: add concise agent and PR readiness guidance - #2649
Conversation
|
Hey Rana Singh (@ranadeepsingh) 👋! We use semantic commit messages to streamline the release process. Examples of commit messages with semantic prefixes:
To test your commit locally, please follow our guild on building from source. |
## Summary`nReduce AGENTS.md from a long duplicated reference to a short repository decision guide with direct links to authoritative build, setup, review, codegen, testing, branch, and CI sources. ## Prompting Intent`nEnsure every instruction in the new AGENTS.md is helpful and terse, and replace copied detail with links and pointers to what coding agents actually need. ## Linked Sources`n- Pull request: https://github.com/microsoft/SynapseML/pull/2648`n- Contributor guide: CONTRIBUTING.md`n- Repository skills: .github/skills/`n- Build sources: build.sbt, environment.yml, pyproject.toml, pipeline.yaml ## Rationale`nAgents need high-signal boundaries and navigation, not a second copy of implementation examples. Keeping durable rules while linking to source files reduces staleness, token cost, and branch-sync conflicts without losing actionable guidance. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
f9e0086 to
a049ef7
Compare
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
There was a problem hiding this comment.
Pull request overview
This PR refactors AGENTS.md into a shorter, decision-focused agent guide, keeping durable repo-wide rules while pointing readers to authoritative sources for versions, setup, codegen, testing, branch practices, and CI.
Changes:
- Condenses
AGENTS.mdsubstantially while preserving key non-negotiable rules and branch-model guidance. - Replaces version-/example-heavy prose with links to source-of-truth files and repo Skills.
- Keeps a compact repository map and a checklist for Scala-first/codegen conventions and validation steps.
Show a summary per file
| File | Description |
|---|---|
AGENTS.md |
Rewrites the agent guide to be concise and link-driven while retaining core rules, branch model, and validation guidance. |
Review details
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- Files reviewed: 1/1 changed files
- Comments generated: 0
- Review effort level: Lite
AGENTS.md and CONTRIBUTING.md are required to be identical on every branch, but both enumerated the port branches by name -- three places in AGENTS.md and one in CONTRIBUTING.md. That makes adding a port branch an edit to a file that must then be re-synced everywhere, which is the exact churn the identical-everywhere rule exists to avoid. Describe the pattern instead of listing instances: port branches are named spark<version>, and `git branch -r` is the authoritative list. Same reason version numbers are read from build.sbt rather than restated -- an enumeration in prose is a copy that goes stale silently. State the boundary explicitly in "Keep this file useful", since it was implied rather than written: these two files may not name a Spark, Scala, Java or Python version, or a path containing one, while README, the website and module docs are free to be branch- and version-specific because nothing requires those to match across branches. Verified: no version-like token remains in either file, and every relative link still resolves. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
## Summary`nAdd a project skill for taking SynapseML issues and stale PRs through an evidence-based merge-readiness loop, with reusable readiness gates, CI triage, Spark performance guidance, and a live GitHub snapshot script. Link it tersely from AGENTS.md. ## Prompting Intent`nCapture the recurring 5/5 or 200%-ready development workflow from prior SynapseML sessions so agents consistently rebase, resolve all active and suppressed feedback, prove user value, add regression and end-to-end tests, protect compatibility and Spark performance, and iterate full CI to green. ## Linked Sources`n- Follow-up PR: https://github.com/microsoft/SynapseML/pull/2649`n- Original agent guide PR: https://github.com/microsoft/SynapseML/pull/2648`n- Project review skill: .github/skills/code-review/SKILL.md`n- Project local setup skill: .github/skills/synapseml-local-setup/SKILL.md`n- Agent Skills specification: https://agentskills.io/specification ## Rationale`nA dedicated skill provides repeatable progressive disclosure without bloating AGENTS.md. The workflow encodes evidence gates learned from real failures: stale targets, discarded conflict content, suppressed comments, helper-only tests, false-green skips, infrastructure failures, compatibility breaks, unshipped artifacts, and unmeasured Spark performance claims. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…idence Two statements in the new skill would send an agent down a path I measured as wrong while getting the spark4.0 and spark4.1 sync PRs green. "Push the exact validated head and comment /azp run" does not work for PRs targeting the port branches. The Azure Pipelines definition's pull-request trigger is defined in the pipeline UI with a branch filter of +master, and a UI-defined trigger overrides the pr: block in pipeline.yaml entirely, so the YAML listing the port branches has no effect. /azp run on such a PR silently does nothing -- no build queues and no error is reported -- which reads as "CI triggered" and then as "CI pending" forever. Replace it with: trigger, then confirm a build actually queued, and queue against refs/pull/N/merge when the target is not covered. Cite the build ID, since a comment is not evidence a build ran. CI triage described the four failure categories but not how to read a job result, and the mechanics are not binary. A filter of result -eq "succeeded" reports phantom failures, because succeededWithIssues is a normal outcome when a non-gating task -- usually dependency caching or TLS -- warns while every test passes. I hit exactly this and briefly reported a passing job as failed. It also cuts the other way: succeededWithIssues on a task that runs or publishes tests is a real failure. Add a section saying to identify the warning task and read published test results rather than trusting the badge in either direction. Same section records the harder lesson: compare per-test outcomes across builds. A fix that changes nothing leaves the same tests failing the same way, and a job-level summary hides that. Two changes I believed were fixes turned out to be placebos under that comparison. Also drop the remaining version numbers, matching the previous commit -- "spark4.x" becomes "spark<version>" and "Spark 4.1 compatibility" becomes "port-branch compatibility", so adding a branch does not require editing these files. Verified: no version token remains in the skill, and both sibling skill links resolve. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Review details
Suppressed comments (3)
Previously missed (2) — in code that hasn't changed since the last review.
.github/skills/synapseml-merge-ready/scripts/Get-PrReadiness.ps1:77
gh apicompare endpoint is built with$view.baseRefNameunescaped. If the base branch contains/(common for release/hotfix branches), this produces an invalid REST path and the script will fail even though the PR data is otherwise valid. URL-encode the base ref before constructing the compare URL.
$compareText = & gh api "repos/$Repo/compare/$($view.baseRefName)...$($view.headRefOid)"
.github/skills/synapseml-merge-ready/scripts/Get-PrReadiness.ps1:97
- Detection of suppressed Copilot review bodies is case-sensitive and requires the exact substring "Suppressed comments". GitHub review templates/messages can vary in casing, which will cause false negatives and silently drop the signal this script is meant to surface. Use a case-insensitive check.
$suppressed = @($review.reviews.nodes | Where-Object {
$_.body -and $_.body.Contains("Suppressed comments")
} | ForEach-Object {
.github/skills/synapseml-merge-ready/scripts/Get-PrReadiness.ps1:7
- The help text claims the script reports "active review threads" and "review bodies containing suppressed comments" without noting that the GraphQL query is capped (
reviewThreads(first: 100),comments(first: 20),reviews(last: 50)). For large PRs this can produce incomplete evidence without warning; the help should document these limits (or the script should paginate).
.DESCRIPTION
Reports head/base state, target divergence, checks, active review threads,
and review bodies containing suppressed comments. It does not make the
readiness decision; use the skill's evidence gates for that judgment.
- Files reviewed: 7/7 changed files
- Comments generated: 0 new
- Review effort level: Lite
Review feedback on the script, verified case by case rather than applied
wholesale. A readiness tool that under-reports is worse than no tool, so the
truncation issues were the ones worth fixing properly.
Pagination. The GraphQL query capped reviewThreads at 100, per-thread comments
at 20 and reviews at 50, with no pageInfo, so a large PR would silently report
fewer unresolved threads than it has -- reading as "review is clean" when it
is not. That is the false-green pattern the skill itself tells you to reject.
Both connections are now fully paginated through a shared helper, and the
emitted JSON carries a completeness object with page counts and any thread
whose comments were still truncated, so an incomplete snapshot is visible
instead of silent. The loop throws if a page claims hasNextPage without
returning a cursor rather than spinning forever.
Suppressed-comment detection was an exact case-sensitive match on
"Suppressed comments", which silently drops the signal the script exists to
surface if GitHub varies the wording. Now a case-insensitive match.
Repo validation accepted "owner/name/extra", because Split("/", 2) always
yields two parts when a slash is present. Now requires exactly two non-empty
segments and reports the offending value.
Renamed the helper's local $args to $ghArgs; $args is an automatic variable
inside a function and assigning to it is a trap for later edits.
Two review points I did not treat as defects:
statusCheckRollup was reported as an object whose checks live under .contexts.
That is the GraphQL shape, but `gh pr view --json` flattens it. Measured: it
returns Object[] of 13 CheckRun entries with name/status/conclusion. Verified
the existing filter against a PR that genuinely fails and it returned exactly
[Review Dependencies], matching `gh pr checks`. Left as is -- an all-green PR
would not have proven this either way.
The unescaped base ref in the compare URL was reported as breaking on branches
containing "/". It does not: the API accepts sync/spark4.1-with-master-2
unescaped and returns the same result as the encoded form. Kept the encoding
anyway as defensive, but it fixes no observed failure.
Verified: parses clean, and runs against three PRs producing counts that match
independent REST pagination.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
.github/skills/synapseml-merge-ready/references/ci-triage.md:28
- Grammar: this reads as though the issue itself has an owner. Consider using “owning issue” to mean the issue that tracks the baseline failure.
Action: collect comparable target/head evidence. Do not silently ignore it; link
the owner issue or repair it when tightly coupled.
- Files reviewed: 7/7 changed files
- Comments generated: 0 new
- Review effort level: Lite
## Summary Rename the reusable skill and its directory from synapseml-merge-ready to synapseml-pr-loop, then update the AGENTS.md activation link. ## Prompting Intent The engineer asked for a clearer, more understandable skill name that describes the recurring SynapseML pull-request remediation loop. ## Linked Sources - Pull request: #2649 - Follow-up context: #2648 ## Rationale SynapseML PR loop communicates the skill's repeatable issue/PR workflow more directly than the outcome-oriented merge-ready name while preserving all existing progressive-disclosure content and behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
## Summary Add the remaining concise instructions needed for repeatable PR remediation: audit historical review feedback, verify published artifacts ship the capability, update public documentation without editing generated files, and safely validate external services. ## Prompting Intent The engineer asked for the SynapseML PR loop to contain all recurring instructions and checklists so future merge-readiness requests do not require repeated guidance, while keeping the skill terse. ## Linked Sources - Pull request: #2649 - Original guide: #2648 ## Rationale Keep the main workflow at 114 lines and place detailed exit, CI, and Spark checks in focused references. The added bullets close material workflow gaps without duplicating those references. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
.github/skills/synapseml-pr-loop/scripts/Get-PrReadiness.ps1:109
- The loop guard comment says it protects against a cursor that "never advances", but the current check only handles a missing cursor. If the GraphQL API ever returns the same endCursor with hasNextPage=true, this can still loop forever. Capture the previous cursor and also throw when the cursor does not advance.
# Guard against a cursor that never advances rather than looping forever.
if ($hasNext -and -not $cursor) {
throw "$Description reported more pages but returned no cursor for PR #$Number"
}
- Files reviewed: 7/7 changed files
- Comments generated: 0 new
- Review effort level: Lite
## Summary Add a concise synapseml-branch skill with focused master, spark3.5, spark4.0, spark4.1, and fallback references. Wire AGENTS.md and the PR loop to resolve context from the PR base branch and add historical false-confidence gates. ## Prompting Intent The engineer asked to mine prior SynapseML PRs and Copilot sessions for durable lessons, make agent responsibilities and repeated checks explicit, provide branch-specific shared context, and ensure agents fall back to it without bloating the main PR loop. ## Linked Sources - Pull request: #2649 - Branch CI coverage: #2644 - Spark 4.1 synchronization: #2617 - Compatibility replay fix: #2611 - Compatibility identity fix: #2608 - Orphaned test-suite coverage: #2622 ## Rationale Use .github/skills because branch-local .agents guidance identifies it as authoritative. Keep common decision logic in one small skill, isolate volatile branch facts in references, derive context from the PR base rather than feature-branch names, and require live build/CI verification so historical notes cannot become stale authority. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
## Summary Handle GraphQL errors and missing data explicitly, reject non-advancing cursors, classify stale checks as failures, emit a stable JSON array shape, and warn that review content must remain local or be redacted. ## Prompting Intent The engineer asked agents to double- and triple-check the reusable PR loop. The loop's own final snapshot surfaced active and suppressed Copilot findings that needed to be fixed before the skill could be considered reliable. ## Linked Sources - Pull request: #2649 - GraphQL error review: #2649 (comment) - Stale-check review: #2649 (comment) - Snapshot privacy review: #2649 (comment) ## Rationale A readiness collector must fail closed and preserve a stable machine-readable contract. Explicit errors, cursor progress checks, stale-signal blocking, array output, and local/redacted evidence prevent false-green or accidental disclosure outcomes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Final review follow-up on
Validated live against one and two PRs, including complete pagination and stable JSON shape. |
## Summary Refactor the Spark 4 branch references into a shared template plus concise spark4.0 and spark4.1 overlays derived from PRs 2646 and 2645. Preserve their core toolchain, codegen, R, Databricks, Fabric, CI, failure-triage, and porting differences. ## Prompting Intent The engineer asked to validate that the new branch-specific skills are templatized versions of PRs 2645 and 2646, while retaining the important differences so those sync PRs can be updated later. ## Linked Sources - Pull request: #2649 - Spark 4.1 sync/context: #2645 - Spark 4.0 sync/context: #2646 ## Rationale Extract shared Spark 4 responsibilities once, keep branch-only facts in small overlays, and add a reusable branch-reference template. This retains the operational knowledge from the long branch manuals without duplicating hundreds of lines or treating snapshot values as permanent truth. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
.github/skills/synapseml-pr-loop/scripts/Get-PrReadiness.ps1:31
$Repoisn’t trimmed before splitting/validation, so inputs like"microsoft/SynapseML "(trailing space) will pass the currentowner/nameshape check but then fail later with a confusing API error. Trimming early makes the validation actually enforce the intended contract and improves diagnostics.
$repoParts = $Repo.Split("/")
if ($repoParts.Count -ne 2 -or -not $repoParts[0] -or -not $repoParts[1]) {
throw "Repo must use owner/name format; got '$Repo'."
}
- Files reviewed: 15/15 changed files
- Comments generated: 0 new
- Review effort level: Lite
## Summary Rename the skill to synapseml-branches, normalize references to branch-spark3p5/branch-spark4p0/branch-spark4p1, and explicitly map both master and spark3.5 to the Spark 3.5 context while preserving their different sync policies. ## Prompting Intent The engineer requested branch-oriented filenames without dots, compliant Agent Skills naming/frontmatter, and a clear mapping in the skill showing that master currently uses the Spark 3.5 baseline. ## Linked Sources - Pull request: #2649 - Spark 4.1 context source: #2645 - Spark 4.0 context source: #2646 ## Rationale Use predictable branch-<runtime> filenames, pluralize the routing skill because it covers multiple targets, and share one Spark 3.5 reference while distinguishing canonical master development from the shared spark3.5 release branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| - Record *why* a divergence exists at the point it is introduced — in a comment | ||
| next to the change and, if it is durable, in `AGENTS_<branch>.md`. A pin with | ||
| no rationale gets "helpfully" reverted by the next sync. | ||
| Black is pinned in [pyproject.toml](pyproject.toml); use that version. |
I documented the opposite earlier today, and the pipeline trigger has since been fixed, so the guidance is now wrong in the direction that costs the most: an agent reading it would skip /azp run entirely and hand-queue every build. Measured before changing the text. Commenting /azp run on the two port-branch PRs produced builds 231455958 and 231455959, both recording reason=pullRequest and requestedFor=GitHub, where every build queued by hand beforehand recorded reason=manual under a personal account. The definition's pullRequest trigger filter now reads +master | +spark3.5 | +spark4.0 | +spark4.1. That reason field is the part worth writing down. A trigger-driven build and a hand-queued one are otherwise indistinguishable in the UI, so it is the cheapest way to answer "did my comment actually do anything" -- and the skill already insists a comment is not evidence a build ran. The UI-overrides-YAML explanation stays, demoted from cause to diagnostic: it remains the first thing to check when a comment produces no build, because that failure mode is completely silent -- no error, no build, no feedback anywhere. The merge-ref fallback stays for that case, along with the warning against queueing refs/heads/<branch>, which validates the branch rather than the merge result. The branch reference previously hedged with "historically did not queue ... verify live behavior". Now that it has been verified, it states the current filter and the date, so the next reader does not have to re-derive it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
.github/skills/synapseml-pr-loop/SKILL.md:39
- The instruction/link text suggests running
scripts/Get-PrReadiness.ps1, but there is no repository-rootscripts/Get-PrReadiness.ps1(the script lives under.github/skills/synapseml-pr-loop/scripts/). As written, readers following the doc from the repo root will get a "file not found" and may assume the readiness tooling is missing.
- Run
[scripts/Get-PrReadiness.ps1](scripts/Get-PrReadiness.ps1)
with `-PullRequest <numbers>` and retain its JSON locally as the initial
snapshot. It can contain review text; redact it before public sharing.
- Files reviewed: 14/14 changed files
- Comments generated: 0 new
- Review effort level: Lite
…ills Brings in #2649, which rewrote AGENTS.md and CONTRIBUTING.md and added the synapseml-branches and synapseml-pr-loop skills. AGENTS.md conflicted because both sides rewrote it. Resolved by taking master wholesale rather than merging line by line: the file is required to stay byte-identical on every branch, so any blended result would be wrong by construction. Verified by blob hash after resolving, not by reading the diff. Nothing was lost in that resolution. The branch material this file used to carry -- the branch table, the sync-and-conflict rules -- now lives in the synapseml-branches skill, which arrives in this same merge. CONTRIBUTING.md merged cleanly and is likewise identical to master. All twelve new skill files arrived unchanged. synapseml-local-setup still differs from master, and should. #2649 never touched it; the divergence predates this merge and is version-driven -- JDK 17 and Scala 2.13 against master's 11 and 2.12. Taking master's side would have handed this branch a toolchain it does not use. Confirmed by diffing the merge base against master for that path and finding it empty. AGENTS_spark4.0.md is untouched here and its future is a separate question: master's new model routes branch facts to the branch skill, and the condensed branch-spark4p0.md reference was derived from this very file, so the two now overlap. Consolidating is a content decision, not a merge decision, and is deliberately not bundled into a sync commit. Docs only. No Scala, Python, build, or pipeline changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ills Brings in #2649, which rewrote AGENTS.md and CONTRIBUTING.md and added the synapseml-branches and synapseml-pr-loop skills. AGENTS.md conflicted because both sides rewrote it. Resolved by taking master wholesale rather than merging line by line: the file is required to stay byte-identical on every branch, so any blended result would be wrong by construction. Verified by blob hash after resolving, not by reading the diff. The spark4.0 sync resolved the same conflict the same way, and both branches now carry the identical blob. Nothing was lost in that resolution. The branch material this file used to carry now lives in the synapseml-branches skill, which arrives in this merge. synapseml-local-setup still differs from master, and should. #2649 never touched it; the divergence predates this merge and is version-driven, so taking master's side would have handed this branch a toolchain it does not use. AGENTS_spark4.1.md is untouched. Master's new model routes branch facts to the branch skill, and branch-spark4p1.md overlaps this file, so consolidation is a content decision and is deliberately kept out of a sync commit. Docs only. No Scala, Python, build, or pipeline changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: add the Spark 4 branch details the condensed skill dropped The branch references in #2649 were condensed from the branch guides in #2645 and #2646, and the condensation kept the conclusions but dropped a few facts that are only useful in their specific form. Adding those back before the source guides are removed from the port branches, so nothing is lost when they go. pyarrow and mlflow move together. The pinned MLflow requires pyarrow<20, so raising pyarrow alone breaks the environment solve -- a failure that surfaces during dependency resolution, far from the pin that caused it. Both comments already say so in environment.yml; the skill said only "preserve dependency comments", which does not tell you the two are coupled. RCodegenSuite exists on both Spark 4 branches, not just 4.1 where the reference happened to mention it. That asymmetry mattered: I had it recorded as missing from 4.0 and worth back-porting, and checking the tree rather than trusting the note showed it was already there. It turns an R failure into a unit test instead of a full pipeline run, so it belongs in the shared file where both branches see it. DatabricksCPUStreamingTests is unscheduled on both branches -- confirmed absent from both pipeline.yaml files, not merely undocumented. Only the 4.1 reference mentioned it, which reads as a 4.1 quirk rather than a coverage gap both branches carry. The 4.0 reference also claimed the branch "historically had no PR checks even when master contained corrected filters". That was true when written and is now misleading: the ADO trigger filter was widened on 2026-08-17 and /azp run queues this target, verified by build 231455959 recording reason=pullRequest. The mechanics live in the common file, so this now points there and keeps only the instruction that still matters -- confirm a build actually queued. Finally, "two of four GPU notebooks failing" is recorded as the gap's expected shape, because checking the count distinguishes the known gap from a new regression where red/green cannot. The petastorm shim is marked explicitly unproven so it stops being repeated as the cause. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: correct the Petastorm shim as a pyarrow gap, not a Python 3.13 one Copilot flagged that the spark4.0 GPU bullet read as if spark4.1's shims caused spark4.0's failure. Checking the trees to reword it turned up something larger than the wording. _petastorm_compat.py has no Python version gating anywhere in its 510 lines. What it does is reimplement pyarrow APIs that Petastorm still calls and pyarrow no longer ships -- ParquetDataset, pyarrow.filesystem, pyarrow.hdfs, dataset pieces and partitions. Both Spark 4 branches pin pyarrow==18.0.0. So this is a library-version problem, and spark4.0 is not exempt from it by being on an older Python. Both references said otherwise. spark4.0's told readers not to copy "Python 3.13 petastorm/cloudpickle shims" and listed them as do-not-port; spark4.1's grouped them with genuine 3.13 concerns. That framing makes the layer look inapplicable to 4.0 for a reason that is not true of it, which is worse than saying nothing: it forecloses the question. The concrete divergence is now stated plainly. spark4.0 has _horovod.py but not _petastorm_compat.py, so it uses the plain Horovod SparkBackend where 4.1 substitutes a Petastorm-compatible subclass. Its unit tests do not cover this. Without a usable Horovod the estimators fall back to stubs, so the Petastorm path never executes -- which is why the branch can look healthy while missing the layer. That also sharpens the GPU note Copilot was reading. The wheel is the first blocker and it masks this one, so fixing the wheel alone should not be expected to turn those notebooks green. The shims are removed from spark4.0's do-not-port list and marked a back-port candidate instead, explicitly requiring validation on real 4.0 rather than adoption on suspicion. Docs only. Whether to port the layer is a separate change with its own validation, deliberately not bundled here. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: link the cross-references instead of naming them in prose Both new pointers to branch-spark4-common.md were plain text while the file's own "Read ... first" line at the top is a link, so the same target was styled two ways in one document. Made them links. Verified all four relative targets in the references directory resolve on disk rather than assuming a same-directory filename is safe. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: say pins the same pyarrow version, not runs the same pyarrow "Runs the same pyarrow" left it ambiguous whether the comparison was the version, the runtime, or the branch. The claim only holds because both branches declare pyarrow==18.0.0, so the sentence now says pins, and names the version as the thing being compared. Raised as a suppressed review comment rather than a posted one, so it had no thread to reply to. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: make suppressed comments findable and gate review on the current head The PR loop already said to read suppressed comments, three separate times, and I still missed one. Saying it again would not have helped, because the skill never said where they are. Suppressed comments are not review threads. They live inside a collapsed section of the review body, so querying reviewThreads returns zero while they exist, and an agent doing exactly what the skill asked gets a clean result and moves on. They also have no thread to reply to or resolve, which the skill never mentioned either -- so the natural next step after finding one silently fails. The skill now says all of that and directs the fix into the follow-up commit message or a PR comment. The second failure is timing. Automated review re-runs per commit and is asynchronous, so auditing immediately after a push reads the review of the previous head. It reports zero findings for code nobody has looked at yet, and that is indistinguishable from being genuinely clean. Get-PrReadiness.ps1 now makes that distinction visible rather than leaving it to judgment: automatedReviewCoversHead compares the newest automated review's commit against the current head and gates `complete`, and suppressedReviewBodiesForHead narrows suppressed feedback to that head so stale entries from earlier commits stop reading as outstanding. Running it against the live PRs immediately justified the change. #2650 reported one suppressed body but zero for the current head -- correctly stale, already fixed. #2646 reported automatedReviewCoversHead false with no automated review commit at all, which is the real state: that PR has zero reviews. Its clean thread count was never evidence of anything. Under the old output both PRs looked equally clean. The readiness gates now say suppressed feedback must be read from the review body for the current head, and that coverage is compared by commit rather than by recency, since a review produced before the last push never saw that code. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: match the review bot by login instead of by substring Review caught a real defect in the coverage gate I had just added. Automated reviews were identified with `-imatch 'copilot'`, a substring test against the author login, so any account whose name merely contains the word counted as automated coverage. That is the exact failure the gate exists to prevent. A review from `copilotfan` would have set automatedReviewCoversHead true and marked the PR complete while the actual reviewer had never seen the head commit -- a false all-clear produced by the check meant to catch false all-clears. Matching is now an exact comparison against a configurable login list, with the `[bot]` suffix normalised so both `copilot-pull-request-reviewer` and `copilot-pull-request-reviewer[bot]` match the same entry. The list is exposed as -AutomatedReviewer so a repo using a different reviewer does not have to edit the script. The matched login is also emitted as latestAutomatedReviewAuthor, so a wrong match is visible in the output rather than hidden behind a boolean. Verified against the three real logins and four adversarial ones: `copilotfan`, `my-copilot-bot` and `not-copilot` all matched under the old test and none match now, while the genuine bot still does with and without the suffix. Live output for #2650, #2646 and #2645 is unchanged, each attributed to copilot-pull-request-reviewer on its current head. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: emit a readiness snapshot instead of throwing under StrictMode Suppressed review comment on ebe6377 flagged that $latestAutomated can be null when a PR has no automated reviews, and that reading .commit.oid off it would throw and stop the script emitting a snapshot at all. The stated failure does not reproduce. The script sets no StrictMode, so null property access coalesces rather than throws; run against a PR with no matching automated review it emits covered=False, complete=False and exits cleanly. That path was already exercised for real, when #2646 had zero reviews. The underlying concern is real though, just not where it was reported. With StrictMode enabled by the caller -- which propagates into this script's scope -- it does throw, and the first failure is not this line. It is $response.errors in the GraphQL error check, then .state, with the null review fields never reached. It also throws on the completely normal path with a real review present, so this was never about null reviews: the script has never been StrictMode-safe anywhere. Fixing that one reported line would have produced a partial fix wearing the appearance of a complete one, so the contract is now explicit instead. The script pins Set-StrictMode -Off for its own scope, so its behaviour no longer depends on the caller's session state, and it emits a snapshot describing what is missing rather than failing to report at all -- which is the outcome the comment asked for, and now holds on the normal path too. The three review fields are still made explicitly null-safe. They no longer rely on null-coalescing to do the right thing, so the intent is readable rather than incidental. Verified in both modes: StrictMode 3.0 with a forced-null reviewer gives covered=False complete=False, StrictMode 3.0 on the normal path gives covered=True complete=True attributed to copilot-pull-request-reviewer, and both threw before this change. Default-mode output for #2650, #2646 and #2645 is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: ground the pyarrow bound and the streaming gap in checkable facts Reviewing this PR's own claims against the branch trees turned up two that were weaker than they read. The pyarrow/mlflow bullet said "the pinned MLflow requires pyarrow<20" and told the reader to check the inline comments in environment.yml. Those comments disagree: spark4.0 says pyarrow<20, spark4.1 says pyarrow<19. mlflow 2.21.3's published metadata declares pyarrow<20,>=4.0.0, so spark4.0 is right and 4.1's comment is wrong -- and the guidance as written would have sent someone on 4.1 to a comment that misinforms them. It now states the bound from the package metadata and flags the stale comment instead of pointing at it. Fixing 4.1's comment belongs on that branch, not here. The streaming bullet recorded that DatabricksCPUStreamingTests is unscheduled and that scheduling it needs pool capacity and a notebook fix, which is faithful to the note it came from but stops at status. Verified the mechanism: the class is real in DatabricksCPUTests.scala, pipeline.yaml names its CPU legs explicitly as DatabricksCPUTests1..5 plus DatabricksGPUTests and never lists it, and it is a separate class because the streaming notebook's server.stop() cancels concurrent SparkContext jobs, so it needs its own cluster rather than a slot on an existing leg. That is why capacity is the blocker, which the bullet asserted without explaining. The first draft of that sentence generalised the in-repo comment from "Spark 4.0" to "Spark 4". Both branches carry the same comment naming 4.0, and nothing re-confirms it on 4.1 -- which is the same overgeneralisation that produced the Petastorm error this PR exists to correct. It now reports what the comment says and that 4.1 is unverified. Also verified and left unchanged: RCodegenSuite.scala is present on both branches, and all relative links still resolve. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: wait for the automated review of the current head instead of sampling once The loop kept reporting a clean PR seconds after a push, before review of that push existed. Nothing was wrong with the query; it answered honestly about a head nobody had reviewed yet. The gap was that the result was read as clean, so findings surfaced only when a human pointed out that comments had appeared. Review is triggered automatically on push, so the fix is not to ask for one. It is to wait for it. -WaitForReview polls until the newest automated review's commit matches the current head, with -TimeoutMinutes and -PollSeconds to bound it, and on timeout it warns that findings may still be pending rather than returning a quiet all-clear. The per-PR body moved into Get-PrSnapshot so the wait re-queries real state each time instead of re-reading a stale snapshot. This puts the waiting inside the tool. It was previously prose in the skill, and prose asking an agent to hand-roll a polling loop is exactly what got skipped -- the same loop has now been written by hand three times, which is evidence the instruction does not survive contact. -RequestReview stays but is demoted to a fallback and documented as one. It exists because #2646 was found with zero reviews, where waiting alone would block until timeout, and the REST handle is the only path that works: the GraphQL requestReviews mutation rejects the reviewer's Bot node id and `gh pr edit --add-reviewer` cannot resolve the bot login. The reviewer never appears in requested_reviewers afterwards either, so a request cannot be confirmed by reading that list back -- only by waiting for coverage, which is now what the script does. Verified: without -RequestReview no request is sent; with it, and coverage false, one is. Both StrictMode and default-mode paths still emit a snapshot, and output for #2650/#2646/#2645 is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: recover the last branch facts and bound the pyarrow claim correctly A gap audit of both AGENTS_<branch>.md files against the skill references, run specifically to find what deleting them would destroy, came back with four durable facts that exist in no reference file. Each is verified against the branch trees rather than copied on the strength of the note it came from. cyber/utils/spark_utils.py differs between the branches -- 4.0 builds its indexed frame with rdd.toDF(schema), 4.1 with spark.createDataFrame(rdd, schema) -- and toDF was measured working on both 4.0.1 and 4.1.1. Confirmed both forms in the tree, and confirmed that df.rdd.zipWithIndex() still precedes both, which is what makes the back-port low-value: it removes one monkey-patched RDD call and leaves another. Without this, the difference reads as a version requirement and gets preserved forever or "fixed" for no gain. The r-base=4.4 pin was recorded on neither reference, although the sparklyr 1.9.5 pin next to it was. Both branches pin r-base=4.4, and the 69/69 RTests result was measured for the pair, so carrying one half invites someone to move the other. Fabric re-enablement kept the intent but lost the specifics: the payload lives in the Fabric test package's FabricOperations.scala, hardcodes 'SparkVersion': '3.5', and must request '4.1'. Verified that master, spark4.0 and spark4.1 all carry that same hardcoded value, which is the point worth recording -- it means editing it on the port branch cannot affect master, so an agent has no reason to refuse the change as cross-branch. The sempy-integration-region capacity requirement is the one prerequisite that cannot be discovered from code at all. The sparklyr 1.9.3 failure signature is now written down. The reference already said to read the backtrace and not to blame a dead session; it did not say what to look for. The frame chain through dbplyr:::select.tbl_lazy, sparklyr:::tidyselect_data_proxy.tbl_spark and simulate_vars_spark surfaces as invoke_static/hive_context on NULL, which is exactly why it reads like a dead session. Separately, a suppressed comment on b61db32 was right about the lockstep bullet. "Raising pyarrow alone breaks the environment solve" overstates it: the declared bound is pyarrow<20, so 18 to 19 is fine and only crossing the bound without moving mlflow breaks. Now bounded as stated rather than as remembered. Deliberately not carried over, per the audit: the claim that RCodegenSuite does not exist on spark4.0, which is false and contradicted by the tree, and the framing of the Petastorm shims as Python 3.13 workarounds, which is the error this PR exists to correct. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: state the pyarrow/mlflow rule instead of a branch state that is not live yet Review caught the lockstep bullet asserting `mlflow==2.21.3` with `pyarrow<20` as though it described both Spark 4 branches. It does not describe either one as they exist today. I read those pins out of the sync worktree, which is the result of #2646 rather than the live branch. The live pins are not close to what the bullet implied: master has pyarrow 10.0.1 with mlflow 2.21.3, spark4.1 has 18.0.0 with 2.21.3, and spark4.0 has pyarrow 22.0.0 with mlflow 1.26.1 -- a different MLflow major with a different bound, and a pyarrow above the number the bullet said could not be crossed. On master, where this file lands, the claim would have been wrong for every branch a reader might check. That is the third time in this work that a derived tree has been written down as current fact, so the bullet no longer states pins at all. It states the rule -- the bound comes from the pinned MLflow, check that version's own metadata, read both live values on the branch being edited -- and keeps 2.21.3's pyarrow<20 as a worked example rather than a claim about the world. That stays true after #2646 merges and after the next pin bump, which the previous wording would not have. Also fixes a suppressed comment on the same head: $automatedLogins rejected empty strings but accepted whitespace-only ones, since ' ' is truthy in PowerShell. That produced a non-empty login list guaranteed never to match, leaving automatedReviewCoversHead false forever with no error to explain why -- a silent hang in the gate that exists to stop silent passes. Entries are now trimmed and blank ones dropped before the emptiness check. Verified: ' ' and @(' ','') both throw with a clear message, @(' ','Copilot') is accepted, and default behaviour is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: correct branch claims verified against live refs Addresses the suppressed Copilot comment on branch-spark4-common.md:69 and a wider audit it prompted. Every branch claim below was re-verified with `git show ms/<branch>:<path>` / `git grep <pat> ms/<branch>` against the live refs rather than against local sync worktrees, which are PR *results* and do not reflect current branch state. That contamination is what produced the wrong claims being fixed here. - DatabricksCPUStreamingTests: was "unscheduled on both Spark 4 branches". Live spark4.0 schedules it (job databricks-cpu-streaming); the class does not exist on master at all. Rewritten to send readers to pipeline.yaml on their own branch, and to note that dropping the leg while leaving the class defined is silent coverage loss. - RCodegenSuite: was "on both Spark 4 branches". Live spark4.0 does not have it; spark4.1 does. - sparklyr: was "both branches pair that pin with r-base=4.4". Live spark4.0 is on 1.9.3, spark4.1 on 1.9.5, so the 1.9.3 backtrace signature documented below it is a live concern on 4.0, not history. r-base=4.4 on both is correct and is retained. - pyarrow: branch-spark4p0.md still claimed "both branches pin the same pyarrow" - the same error already corrected in the common doc. Live spark4.0 is 22.0.0, spark4.1 is 18.0.0. The Petastorm conclusion is unchanged and in fact strengthened: 4.0 resolves to the newer pyarrow, so the APIs the shim restores are at least as absent there. - NumPy: was "Keep NumPy 1.26.4 pinned", and separately listed "unpinned NumPy" as 4.1-only. NumPy is unpinned on both live branches; the pin described post-merge state. Restated as a conditional rule. Also retracts an earlier unverified suspicion of mine: LongOffset is present on live spark4.0 (DistributedHTTPSource.scala, HTTPSource.scala). The empty grep that prompted it was a case-sensitivity error, not a defect. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: preserve branch operating detail; fix review-coverage by commit Addresses the suppressed Copilot comment on Get-PrReadiness.ps1:251 and closes the content-loss gaps found by auditing the skill against the files it is meant to replace. Review coverage: automatedReviewCoversHead was computed from the newest automated review overall, so a force-push back to an already-reviewed commit reported a reviewed head as uncovered and would send -WaitForReview into a pointless wait. Coverage is now true when *any* automated review targets the head, and the reported review prefers one that covers the head, falling back to the newest for diagnostics. Knowledge preservation: AGENTS_spark4.0.md and AGENTS_spark4.1.md exist only on the two sync branches; neither live spark4.0 nor live spark4.1 carries any branch documentation. Deleting them therefore destroys that knowledge unless this skill carries it, so the condensation was re-audited line by line against both sources. Recovered here: - generated Python lives under target/scala-2.13, not master's scala-2.12 - the Java 17 surface: Dockerfile JAVA_HOME, pr-validation.yml JDK, and the exact CMS flags whose restoration stops the JVM booting - why environment.yml pins move (pip too old, first torch/torchvision releases, interpreter-specific wheel URLs) and why their comments matter - the Spark 4 adaptations by name: UnboundRowEncoder and the SAR case class, DetectAmbiguousSelfJoin and the qualified join column, safeGetDefault, and why the classifier fixture dropped NaN rather than weakening the assertion - OpenAIPromptPythonOverrides.scala and the NameError that zero-arg super() avoids - the __init__.py rationale and its guard tests (test_http_package.py, test_package_exports.py) - rLoadLine/new_ml_pipeline_stage: signature stable v1.8.0-v1.9.5, four param types share one implementation, keep the three assertions in step, and it is alignment rather than a proven fix - the sibling-diff habit with its command and its track record - GPU pool name, its 1 x 3 worker shape, and the sibling branch as a free control; CPU pool name on 4.1 - areLibrariesInstalled == false is a ~10 minute timeout (60 * 10 attempts) and inverts expectations: a genuine FAILED throws instead - the torchvision 0.17.0 / torch 2.2.0 downgrade instance - the fine-tune measurements showing the wheel swap was alignment, not a fix - Fabric on 4.0 may never be possible; supersession by 4.1 is likelier Also corrects branch-spark4p1.md, which still claimed spark4.0 "pins the same pyarrow version". It does not - it pins a newer one, which strengthens rather than weakens the point that spark4.0 is not exempt. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: scope Spark 4 claims to sync-PR state, correct GPU pool A third pass of verifying claims against live refs found that most of AGENTS_spark4.0.md describes the branch *after* #2646, not as it stands. The guides were written on the sync branches, so they describe the merged result; importing them as present-tense fact overstates what live spark4.0 has. Measured against ms/spark4.0 (git ls-tree / git grep, not a worktree): - GPU pool is synapseml-build-17.3-gpu, not the shared synapseml-build-14.3-gpu. #2646 switches it to the shared pool, so "the pool is shared, use the sibling branch as a control" is a post-merge property, not a current one. - OpenAIPromptPythonOverrides.scala: absent - test_http_package.py / test_package_exports.py: absent, so the __init__.py policy is currently unenforced there - new_ml_pipeline_stage in generated R: absent Changes: - the header now states the guides describe the branches as of #2645/#2646, lists what live spark4.0 lacks, and names the worktree-vs-live mistake explicitly so the next reader does not repeat it - the GPU bullet no longer hardcodes a pool name; pool names differ by branch and the sharing-derived advice is scoped to where sharing actually holds - the guard-test, OpenAIPrompt and rLoadLine bullets say where they apply - branch-spark4p0.md records the live-vs-#2646 delta directly Verified correct and left as-is: CMS flags absent on both Spark 4 branches and present on master; spark4.1's CPU pool synapseml-build-18.0 and shared 14.3-gpu; GpuConcurrentRuns; areLibrariesInstalled. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: scope StrictMode to functions; narrow automated-reviewer default Addresses both suppressed Copilot comments on 7577faa. Get-PrReadiness.ps1:40 - the default reviewer list contained "copilot" and "github-copilot". Matching is exact, but those shorter logins are registerable by humans, so an exact match could count a human review as automated coverage and set complete=true on an unreviewed head. The default is now this repo's reviewer bot alone (copilot-pull-request-reviewer); -AutomatedReviewer still extends it for other repos. Get-PrReadiness.ps1:69 - Set-StrictMode -Off sat at script scope, so dot-sourcing disabled StrictMode for the caller's whole session. Rather than document that hazard, it is now scoped to the two functions that actually read sparse GraphQL fields positionally (Invoke-PagedQuery, Get-PrSnapshot). Script scope only validates parameters and defines here-strings, so it does not need the relaxation. That move initially introduced a silent defect worth recording: Set-StrictMode was placed *before* each function's param() block. PowerShell requires param() to be the first statement in a function body, and when it is not, param(...) parses cleanly as a command invocation and fails only at runtime. The file parsed clean while being broken, so parse-checking was not sufficient verification here. Verified by running, not by inspection: - normal invocation reports covers=True, attributed to copilot-pull-request-reviewer, so narrowing the default did not break detection - a caller with Set-StrictMode -Version 3.0 still gets a snapshot, which is the original defect this relaxation exists for - after dot-sourcing, a missing-property access in the caller still throws, confirming StrictMode is no longer disabled session-wide Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: count only submitted reviews as head coverage Addresses the review comment on Get-PrReadiness.ps1:259. $reviewsForHead matched on commit.oid alone. A pending review already carries the head's commit oid but has a null submittedAt, so an in-progress review would have satisfied automatedReviewCoversHead - reporting complete=true and releasing -WaitForReview while the review was still being written. That is the same premature all-clear this gate exists to prevent, so the filter now requires submittedAt as well as a matching oid. This is the third defect of this class found in this gate: substring login matching, then coverage decided by recency instead of commit, now coverage satisfied by an unsubmitted review. Each one made the check answer "yes" when the honest answer was "not yet". Verified: a live run still reports covers=True on the real head, and a simulated pending review carrying the head oid with a null submittedAt yields coversHead=False. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: make complete mean complete; request a review at most once Addresses the review comment on Get-PrReadiness.ps1:293, plus a worse defect that comment exposed. The reported defect: under -WaitForReview, Get-PrSnapshot runs once per poll, so -RequestReview re-POSTed the reviewer request every cycle until coverage arrived - notification spam and a rate-limit risk. The request is now attempted at most once per PR per invocation, with the outcome recorded and reused. The defect that surfaced alongside it: completeness.complete tested $truncatedThreadComments, which counts comment-pagination truncation, not unresolved review threads. The two were conflated, so the field reported complete=true on this very PR while a review thread was open - the headline readiness signal giving an all-clear over outstanding review feedback. It now requires all of: no pagination truncation, an automated review covering the head, and zero unresolved threads, suppressed-for-head items, failed checks and pending checks. Pending is treated as unknown rather than passing. Verified against live #2650 with one thread open: the same input that previously produced complete=true now produces complete=false. readiness-gates.md documents what the flag now means, and says to trust the individual fields over the summary when they disagree - that flag has now been wrong in both directions. This is the fourth false all-clear found in this gate (substring login matching, coverage by recency, unsubmitted reviews counting as coverage, and now unresolved threads not counting at all). Every one resolved in the direction of declaring readiness that had not been established, which is worth recording as a property of the tool rather than four separate slips. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: restore the remaining searchable specifics from the branch guides Knowledge transfer was measured rather than assumed: 40 distinctive terms from AGENTS_spark4.0.md and AGENTS_spark4.1.md were tested against the skill, and six were absent. Five are restored here because each is the string an engineer would actually search for when hitting the failure: - spark.sql.ansi.enabled / spark.sql.ansi.doubleQuotedIdentifiers, and PARSE_SYNTAX_ERROR as the symptom when the second flag is missing - AdbGpuRuntime as the field that must not be bumped to DBR 18, with the reason (17.3 LTS ML ships Spark 4.0, 18.0 ML ships 4.1, so bumping makes the suite green by no longer testing this branch) - refs/pull/<N>/merge as the literal ADO fallback target, and why refs/heads/<branch> fails (service-connection authorization) - ImageFeaturizerSuite as the ONNX OOM site, and the conda HTTP 403 in RTests vw The sixth, cloudpickle, is deliberately not restored. Both guides describe the Petastorm shim as working around "cloudpickle/petastorm breakage under Python 3.13". Reading the file disproved that: it contains no Python-version gating at all and reimplements pyarrow APIs Petastorm still calls. Restoring the term would restore the error, and the corrected framing is already recorded. Coverage after this change: 39/40, with the one omission reasoned above. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: gate readiness on required checks being present, not just green A head whose Azure Pipelines build never queued reported failed=0 and pending=0 and scored complete, because an absent check is neither. On this pull request the GitHub Actions checks started 12s after the push and all passed, while the ADO build only appeared 3m17s later and only because a human commented `/azp run`. Between those points every mechanical gate read clean on a head that had no CI on it. Add `missingRequiredChecks` (prefix-matched, default `microsoft.SynapseML`) to the snapshot and to `complete`, extend `-WaitForReview` to wait for required checks as well as the review, and add `-RunPipeline` to post `/azp run` when the build is missing -- guarded to at most one comment per PR per invocation, mirroring `-RequestReview`, since the wait loop calls the snapshot every poll. Document that the build does not re-queue itself on a push, so every push needs its own comment, and that green Actions checks are not CI passing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: make the spark4.0 GPU pool claim self-verifying Addresses both suppressed review comments on 0f2f9bd, which reported the `synapseml-build-17.3-gpu` claim as stale. Measured instead of taken on trust, and the claim holds -- `git show` of core/src/test/scala/com/microsoft/azure/synapse/ml/nbtest/DatabricksUtilities.scala: ms/master 14.3-gpu / 14.3.x-gpu-ml-scala2.12 ms/spark4.0 (live) 17.3-gpu / 17.3.x-gpu-ml-scala2.13 ms/spark4.1 14.3-gpu / 18.0.x-gpu-ml-scala2.13 ms/sync/spark4.0-with-master 14.3-gpu / 17.3.x-gpu-ml-scala2.13 The reviewer read the sync branch as live `spark4.0` -- the same worktree-for-branch confusion this paragraph exists to warn about, made against the paragraph itself. So the wording was not wrong, it was not checkable: cite the file and the value, and the next reader can repeat the check in one command rather than deriving it from whichever tree is open. Record what #2646 actually changes here, since it is operating knowledge: the GPU pool moves to master's shared `14.3-gpu` resolved by `getPoolIdByNameAndNodeType`, while the CPU pool stays 17.3 and the GPU runtime stays 17.3.x. `spark4.1` already pairs `14.3-gpu` with an 18.0.x runtime, so the pool name identifies a warm node pool, not a DBR version. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: SynapseML CI <synapseml-ci@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Summary
AGENTS.mdsynapseml-pr-loopfor repeatable 5/5 merge-readiness worksynapseml-brancheswith focused context formaster,spark3.5,spark4.0,spark4.1, and unlisted targetsmasterandspark3.5tobranch-spark3p5.md, with separate sync rulesWhy
Recent SynapseML PRs showed that green aggregate CI is insufficient. Agents must resolve context from the PR base branch, verify the target branch's live build and CI definitions, prove the public shipped path, inspect every review surface, and recheck context before implementation, validation, and final push.
Detailed checklists remain in focused references so the always-loaded guidance stays terse.
Validation
mastergit diff --checkpassesHistorical inputs include #2608, #2611, #2617, #2622, #2627, #2631-#2638, and #2644.
Follow-up to #2648.