A review of Sohl-Dickstein et al.'s Deep Unsupervised Learning using Nonequilibrium Thermodynamics (ICML 2015, arXiv:1503.03585), read from an implementer's seat after auditing a broken reimplementation of it.
A 329M-parameter MNIST diffusion reimplementation's validation loss plateaued at 0.9425 -- exactly the analytic NLL of the best constant two-scalar Gaussian. The forward posterior was missing from the image path entirely; after fixing it, the denoiser demonstrably recovers digits, but full unconditional MNIST generation still doesn't work.
A detector reported 0.91 near-train mAP, a GAN reported a non-standard FID of 0.24, and a diffusion model's validation loss plateaued at 0.9425 -- the exact NLL of a two-scalar constant baseline. All three numbers were easy to overinterpret until their pipelines were audited.
A reusable protocol for auditing and reporting FID: pin the extractor, input pipeline, sampling, seeds, checkpoint, controls, and sample-size limits, while recording missing provenance honestly.
After a discriminator-balance sweep did not help, one DiffAugment run lowered the same-N subset FID estimate from 163 to 118.5 and showed no comparable late degradation through the last evaluated checkpoint.
Our text-to-image GAN peaked at epoch 20 then degraded. Four single-seed configurations using EMA, separate learning rates, label smoothing, and fewer discriminator updates did not robustly improve the same-N FID estimate.
A structural map of MS-CLIP-GAN: CLIP text embeddings, conditioning augmentation, a 64→128→256 generator, stage-wise discriminators, and why that architecture made the discriminator-balance hypothesis worth testing.
A text-to-image GAN reported FID 0.2423 because pytorch-ignite's default used 1000-d logits, not the canonical 2048-d pool3 features. Re-evaluation produced a much larger small-N estimate, with important sample-size limits.
A reusable SQL pattern for aggregating entities within groups and returning all ties at the top rank.
Position-aware ESM2 features, frozen embedding caching, and honest evaluation caveats for a supervised GOF/LOF protein variant classifier.