hope this question is in the right place. I’m working with a toy diffusion model to generate points e.g learning a Swiss roll which to me is a basic use case that I wanted to start with.
My model is generally sensible and I’ve implemented both a score matching and denoising approach, which borrows code from other (working) use cases.
I wanted to start by overfitting a sample with my model and achieve zero validation loss. This has been going wrong, and further, the model does not train. For score matching I know my loss is correct. I’m not able to provide code details, but are there any places to look when debugging besides playing with hyperparameters?
