> But, the neural networks are pretty small, so you could train a "hypernetwork" that predicts μ, Σ, θ for a given puzzle, and even predicts how to train these parameters.

Kurt Godel (or maybe Douglas Hofstadter, rather) would raise an eyebrow. :)