You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Training-spectrum loss is not enough to rank ARTS2D reconstructions under finite angular coverage. The measurement sees a limited set of Radon projections and their susceptibility transforms, so an EDF can improve on observed pixels while moving farther from truth in the unobserved/null subspace.
MLflow provides a concrete counterexample. In experiment 164974, run 08c639c2bb1040939b26ba0c1af468b4 reached spectrum MSE 1.45e-5, while EDF L2 improved to 0.02199 at step 354 and then worsened to 0.03830 by step 1000 as spectral loss continued to fall.
In experiment 188374 (controlled rotated suite, git 8ce493b07ec976b4567c93efc4a540fdb1e8d49b), median spectrum losses over five common truths were:
arbitrary 2-D: 5.37e-4;
NN spherical harmonics: 1.35e-3;
arbitrary-coefficient SH: 2.07e-3;
Mora-Yahi SH: 2.19e-3.
These rankings are useful for numerical expressivity, but the current benchmark is close to an inverse crime and does not measure held-out prediction, physical moments, or observable-vs-null-space recovery. Some run names/logged parameters also describe the forced truth model rather than the instantiated fit model.
This complements #124: that issue restores executable ARTS tests; this issue establishes a scientific inverse-problem regression.
Acceptance criteria
Add deterministic synthetic truth families, realistic noise levels, finite-angle masks, and seeds; use differing generation/inversion grids or an independent reference forward path.
Compare arbitrary 2-D, NN-SH, and coefficient/low-order angular bases with identical observed angles, nuisance parameters, initialization policy, and compute budget.
Split observed and withheld angles/detector wedges and log noise-whitened train and held-out spectral residuals.
Log full EDF error plus normalization, flow/current, energy/temperature, pressure anisotropy, and any targeted heat-flux moment.
Compute the local noise-weighted detector Jacobian and report an SVD/Fisher visible/null-space decomposition; score EDF error separately in those subspaces.
Persist actual instantiated parameterization, active leaves, angular mask, seed, config hash, git SHA, and forward-resolution settings; prevent truth configuration from being mislabeled as the fit model.
Add a CPU smoke test and a documented full GPU benchmark over multiple seeds, with uncertainty intervals rather than a single best run.
Related: #124 and the separate likelihood/regularization issue.
Motivation
Training-spectrum loss is not enough to rank ARTS2D reconstructions under finite angular coverage. The measurement sees a limited set of Radon projections and their susceptibility transforms, so an EDF can improve on observed pixels while moving farther from truth in the unobserved/null subspace.
MLflow provides a concrete counterexample. In experiment
164974, run08c639c2bb1040939b26ba0c1af468b4reached spectrum MSE1.45e-5, while EDF L2 improved to0.02199at step 354 and then worsened to0.03830by step 1000 as spectral loss continued to fall.In experiment
188374(controlled rotated suite, git8ce493b07ec976b4567c93efc4a540fdb1e8d49b), median spectrum losses over five common truths were:5.37e-4;1.35e-3;2.07e-3;2.19e-3.These rankings are useful for numerical expressivity, but the current benchmark is close to an inverse crime and does not measure held-out prediction, physical moments, or observable-vs-null-space recovery. Some run names/logged parameters also describe the forced truth model rather than the instantiated fit model.
This complements #124: that issue restores executable ARTS tests; this issue establishes a scientific inverse-problem regression.
Acceptance criteria
Related: #124 and the separate likelihood/regularization issue.