BUG: Smooth structure tensor with an isotropic Gaussian kernel

[physwkim]

StructureTensorImageFilter smoothed the tensor field with RecursiveGaussianImageFilter, which filters one axis only. Fixing it invalidates all 16 AnisotropicDiffusionLBR baselines, so this is a draft pending a maintainer decision. Addresses B22 of #6575.

Root cause

itkStructureTensorImageFilter.hxx:124 builds the feature-scale (K_rho) smoother as

using GaussianFilterType = RecursiveGaussianImageFilter<TensorImageType>;

RecursiveGaussianImageFilter derives from RecursiveSeparableImageFilter and filters along a single direction — m_Direction, which defaults to 0 and is never set here. The structure tensor K_rho * (grad(u_sigma) (x) grad(u_sigma)) requires an isotropic kernel, i.e. smoothing on every axis. The same mistake appears at :75 for the noise-scale (K_sigma) smoother, on the m_UseGradientRecursiveGaussianImageFilter == false branch (currently unreachable — the flag has no setter — but the same defect, so it is fixed too). Both now use SmoothingRecursiveGaussianImageFilter, which chains one RecursiveGaussianImageFilter per axis. GradientRecursiveGaussianImageFilter, used elsewhere in the same file, already loops SetDirection() over every axis and is correct.

⚠ This changes filter output — 16 baselines are invalidated

Every AnisotropicDiffusionLBR baseline was generated against the axis-0-only smoothing, so all 16 image-comparison tests now fail with a nonzero ImageError (2 to 1.95e6 depending on the case):

PacMan_cEED / cCED / I, FingerPrint_cEED / cCED / I, Cos3D_cCED, mrbrain_cEED, Lena_Detail_cCED_2 / cEED_2 / I_2, VectorField_Circle_cEED, Triangle_cEED / Triangle_EED, Oscillations1_cCED / Oscillations1_CED.

The baselines have not been regenerated here — that is a maintainer call, and the new bytes need an ExternalData upload I cannot perform. Two questions for the maintainers:

1. Is the single-axis smoothing genuinely a bug (the reading here), or is it load-bearing behavior of the LBR implementation that downstream users depend on?
2. If it is a bug, would you like regenerated baselines uploaded to ExternalData, and do you want the content links updated in this PR or a follow-up?

The PR stays in draft until that is settled.

Local validation

New GoogleTest itkStructureTensorImageFilterGTest.cxx, registered in AnisotropicDiffusionLBRGTests. Input is a step edge varying only along y and constant along x, so axis-0-only smoothing is a mathematical no-op on it and any response to the feature scale proves cross-axis smoothing.

• StructureTensorImageFilter.FeatureScaleSmoothsAlongAllAxes — fail-before: largeScaleTrace - smallScaleTrace was exactly 0. Pass-after: rho=1 gives 5.97e-4, rho=8 gives 88.65.
• ITKSmoothing added to the module's DEPENDS for the new include.

AI assistance

• Tool: Claude Code
• Role: implementation and regression-test authoring from the B22 analysis in BUG: Findings collected while porting ~60 filters — reported together in one issue #6575
• All code was reviewed, built, and tested locally before committing

[hjmjohnson]

Status of this draft: blocked on baselines and now on a rebase conflict, and a high-effort automated review found two confirmed behavioral regressions in the smoothing swap.

Blockers

• The Pixi-Cxx failures on all three platforms are the CoherenceEnhancing/EdgeEnhancing example tests (PacMan_*, FingerPrint_*, Cos3D_*, Lena_*) failing against their committed baselines — the behavior change requires regenerated baselines (or tolerances) before CI can pass.
• The branch no longer rebases cleanly onto main: a same-named itkStructureTensorImageFilterGTest.cxx was added there, so the test needs merging with the upstream one.

Review findings (confirmed)

1. SmoothingRecursiveGaussianImageFilter throws for images with <4 pixels along any axis, where the old axis-0-only recursion tolerated thin images — previously-working pipelines on thin/single-slice inputs now hard-fail.
2. The swap silently smooths through NumericTraits<PixelType>::FloatType (float for the default double tensor components) — a precision downgrade unrelated to the axis bug.
3. The Dispatch<false> fix site is dead code: m_UseGradientRecursiveGaussianImageFilter defaults to true with no setter, so that branch is unreachable and unexercised by the new GTest.
4. Minor: the heavier gaussianFilter is now constructed unconditionally per component but only used in the unreachable branch; the test's EXPECT_GT(delta, 1.0) floor is ~3 orders of magnitude below the true delta and would miss magnitude errors.