There's nothing "simply" here. The weights in question are a particular configuration of several gigabytes worth of data. They're not random. Getting anything comparable by randomly generating a number this long is a "total atoms in the universe to the power of total atoms in the universe" kind of a deal.
In abstract terms, those weights are by far the most dense form of meaning we've ever dealt with.
I think the point is more that the primary edge that they (Meta, OpenAI, etc.) have is compute capacity, since the models are easy to implement (and the most of the training data is out there). Once you have the weights, you can do with it as you please.
This is probably why they hate/fight these leaks. Once the model weights are available, they lose much of their competitive advantage.
could ai, just basically brute force better weights? say instead of doing training to get their own weights, maybe have 10 training inputs, test a new set of weights, have gpt4 rate the quality of the output from each set of weights and then try another and maybe infer a pattern to how weights could maybe potentially be created without all the work?
I feel like soon, ai will be it's own researcher and go far beyond what we ever could do on our own.
In abstract terms, those weights are by far the most dense form of meaning we've ever dealt with.