fix: resolve current model metadata deterministically

[Rana Singh (ranadeepsingh)]

Summary

Fixes #1697.

ComputeModelStatistics now resolves scored-model metadata deterministically after repeated training/evaluation in the same DataFrame lineage. It ignores orphaned entries, collapses semantically equivalent duplicates, rejects conflicting metadata independently of column order, honors complete explicit settings, and reports actionable missing/ambiguous column errors instead of reaching col(null).

Key changes

• Build a one-pass immutable index of score-model metadata rather than selecting the first schema/metadata entry.
• Require compatible label + prediction metadata and valid classification/regression kind.
• De-duplicate equivalent model IDs by semantic column signature and sort all diagnostics.
• Detect conflicting label/prediction/raw-score/probability roles deterministically.
• Preserve explicit classification/regression columns when they fully define evaluation.
• Resolve columns with Spark case-sensitivity semantics and validate them before expression construction.
• Keep classification score lookup lazy for score-independent metrics.
• Preserve uid/params through copy, pipelines, and serialization via defaultCopy.
• Replay merged feat(core): add precision-recall AUC metric #2635 as a Spark 4.1 release-compatibility prerequisite.

Coverage

Added regression and negative coverage for:

• regressor → classifier reuse with stale lineage metadata;
• explicit columns versus unrelated/ambiguous metadata;
• equivalent duplicates and conflicting roles;
• schema column-order independence;
• missing/default/invalid columns and lazy score validation;
• classification and regression behavior;
• copy, PipelineModel, serialization, getters, codegen, and Python wrapper surface;
• feat(core): add precision-recall AUC metric #2635 binary metric/schema ordering interactions.

Validation

• JDK 11 VerifyComputeModelStatistics: 32/32 passed.
• JDK 11 VerifyComputePerInstanceStatistics: 5/5 passed.
• JDK 11 core compile + test compile: passed.
• Scala style + test style: passed.
• Serialization fuzzing + getters/setters: passed.
• core/codegen: passed; generated ComputeModelStatistics.py compiles.
• Black 22.3.0: 189 files unchanged.
• CI pytest tools/ci/tests/test_pipeline_yaml.py: 59/59 passed.
• Exact Spark 4.1 replay with configured prerequisites and JDK 17 test:compile: passed.

No existing public method signature was removed or changed; the original MetricUtils.getSchemaInfo and ComputeModelStatistics.copy signatures remain intact.

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[github-actions]

Hey Rana Singh (@ranadeepsingh) 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

• fix: Fix LightGBM crashes with empty partitions
• feat: Make HTTP on Spark back-offs configurable
• docs: Update Spark Serving usage
• build: Add codecov support
• perf: improve LightGBM memory usage
• refactor: make python code generation rely on classes
• style: Remove nulls from CNTKModel
• test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Pull request overview

This PR fixes non-deterministic scored-model metadata resolution in the core metrics pipeline, preventing ComputeModelStatistics from selecting stale model metadata across repeated training/evaluation within the same DataFrame lineage (as reported in #1697).

Changes:

• Make MetricUtils.getSchemaInfo deterministically enumerate, de-duplicate, and sort scored-model metadata candidates, filtering to complete (label + valid kind + prediction) candidates and rejecting ambiguity with guidance.
• Add early, actionable column validation in ComputeModelStatistics to avoid constructing Spark expressions with unresolved/null columns, while keeping classification score resolution lazy for score-irrelevant metrics.
• Add targeted regression tests covering stale metadata scenarios, explicit override precedence, deterministic ambiguity rejection, and improved error cases.

Show a summary per file

File	Description
core/src/main/scala/com/microsoft/azure/synapse/ml/core/metrics/MetricUtils.scala	Deterministic scored-model candidate discovery/selection and improved ambiguity/missing-metadata handling.
core/src/main/scala/com/microsoft/azure/synapse/ml/train/ComputeModelStatistics.scala	Validates resolved label/prediction/score columns up-front with actionable errors; keeps score validation lazy for non-score metrics.
core/src/test/scala/com/microsoft/azure/synapse/ml/train/VerifyComputeModelStatistics.scala	Adds focused tests for stale lineage metadata, explicit overrides, deterministic ambiguity failure, and score-column validation behavior.

Review details

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

• Files reviewed: 3/3 changed files
• Comments generated: 1
• Review effort level: Lite

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Review details

• Files reviewed: 3/3 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Codecov Comments Bot (codecov-commenter)]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.63%. Comparing base (60cf93c) to head (5cdf9ed).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2632      +/-   ##
==========================================
+ Coverage   88.60%   88.63%   +0.03%     
==========================================
  Files         331      331              
  Lines       18531    18556      +25     
  Branches     1752     1740      -12     
==========================================
+ Hits        16419    16447      +28     
+ Misses       2112     2109       -3     

Files with missing lines	Coverage Δ
...ft/azure/synapse/ml/core/metrics/MetricUtils.scala	97.50% <100.00%> (+7.84%)	⬆️
...zure/synapse/ml/train/ComputeModelStatistics.scala	86.38% <100.00%> (+0.95%)	⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Rana Singh (ranadeepsingh)]

CI follow-up for latest SHA 5cdf9ed5ac47cd562582787502a4a8271c1c1641:

• GitHub Actions and the code-relevant Azure UnitTests train job passed.
• The initial Azure run (#231234591) had an unrelated transient Maven resolution failure in UnitTests lightgbm2; I retriggered /azp run, and that job passed on retry.
• Retry run #231242691 completed with two unrelated live-service failures: UnitTests misc and UnitTests search1 both received HTTP 429 from mmlspark-azure-search because the shared service had reached its 50-index quota. The failing test was IndexSchemaLiveRoundTripSuite plus Azure Search writer suites; no changed metric/train code appears in either failure.

I did not delete shared Azure Search indexes to make the PR green. Diagnostic logs and the local JDK 11 validation evidence are retained in the requested issue artifact directory.

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Lite

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

core/src/main/scala/com/microsoft/azure/synapse/ml/core/metrics/MetricUtils.scala:196

• The ambiguity error suggests only setting labelCol and evaluationMetric, but callers may already have both set and still hit this when multiple candidates share the same label/kind (e.g., differing prediction/raw columns). The message should also guide users toward specifying columns explicitly (or clearing stale metadata) so the next action is clear.

        throw new IllegalArgumentException(
          "Ambiguous scored-model metadata. Multiple complete candidates match: " +
            candidates.map(_.description).mkString("[", ", ", "]. ") +
            "Set labelCol and evaluationMetric to select one candidate; remove stale score metadata if needed.")

• Files reviewed: 6/6 changed files
• Comments generated: 1
• Review effort level: Lite

[Rana Singh (ranadeepsingh)]

The suppressed Copilot observation in review 4945931887 is also addressed by 0564a2b: ambiguous metadata guidance now recommends explicit scoredLabelsCol/scoresCol selection when labelCol and evaluationMetric are insufficient, or removal of stale metadata.

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Review details

• Files reviewed: 7/7 changed files
• Comments generated: 1
• Review effort level: Lite

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Review details

Suppressed comments (1)

pipeline.yaml:1219

• Same as above: LATER_PARENT=$(git rev-parse ...) runs under set -e and can terminate the release-compat job with an unstructured git error if a later prerequisite is invalid. Wrap this rev-parse in an if ! ...; then with a ##vso[task.logissue type=error] message so failures are actionable and consistent.

                  LATER_PREREQUISITE="${CONFIGURED_PREREQUISITES[$later_index]}"
                  LATER_PARENT=$(git rev-parse "$LATER_PREREQUISITE^1")

• Files reviewed: 7/7 changed files
• Comments generated: 2
• Review effort level: Lite

[Rana Singh (ranadeepsingh)]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Review details

• Files reviewed: 7/7 changed files
• Comments generated: 0 new
• Review effort level: Lite