fix: latent diffusion models train the denoiser on the raw input, never on the vae latent

[ooples]

Summary

LatentDiffusionModelBase<T> never trains its denoiser on a VAE latent. Only UpscaleAVideoModel overrides PrepareTrainingSample to encode the target first. The other latent models inherit DiffusionModelBase.PrepareTrainingSample, which returns input unchanged, the raw first argument of Train. It is not the target and not a latent.

Latent diffusion (Rombach et al. 2022, arXiv 2112.10752, section 3.3) trains the denoiser on z = E(x), the scaled latent of the image. Sampling here is in latent space: Generate denoises a latent, then calls DecodeFromLatent. A caller who passes images to Train therefore trains a denoiser on pixels, while inference runs it on latents.

Evidence

• src/Diffusion/DiffusionModelBase.cs: Train calls PrepareTrainingSample(input, expectedOutput). The default is => input ("Unconditional diffusion models train directly on input"), which is then noised with _scheduler.AddNoise and scored with PredictTrainingNoise.
• src/Diffusion/LatentDiffusionModelBase.cs: overrides neither hook. EncodeToLatent (VAE encode + ScaleLatent) exists but training never reaches it.
• grep "override Tensor<T> PrepareTrainingSample" src finds one match: SuperResolution/UpscaleAVideoModel.cs.
• There are 168 : LatentDiffusionModelBase<T> subclasses, including StableDiffusion 1.5/2/3, SDXL, the ControlNet family and the inpainting models.

Why the tests don't catch it

DiffusionModelTestBase trains with InputShape/OutputShape of [1, 4], which is already latent-shaped. The generated fixtures never pass an image-shaped target, so a pixel-space denoiser is never exercised.

Proposed fix

In LatentDiffusionModelBase.PrepareTrainingSample:

• When the target has the VAE's input channel count, encode it with EncodeToLatent (posterior sample, frozen first stage).
• Leave an already-latent target (LatentChannels channels) alone, so existing callers keep working.

Then add a test that trains on an image-shaped target and checks that the noised sample has latent shape. Run the full diffusion family, since this changes training for every latent model.

Found while fixing DiffusionAutoMLModel for #2155 (PR #2136). That model is being moved onto this base, and it trains on the latent explicitly.