> Ban offline training/pretraining. Models must train from scratch after submission
Previously this was considered impossible so rule. My model shows this is possible
Guarantees no synthetic data can be used
It makes the comparison fair across differet models. Otherwise some models like LLMs can benchmaxx ARC by using ungodly amounts of offline training. (Since the benchmark has been around a long time, many ARC-like datasets have been created)
I'm not an ML researcher, so YMMV, but... how could a model learn to answer these ARC-AGI questions without training beforehand?
Predicate logic is trivially realized by linear transformations (I.e., matrices), and these matrices are easily discovered via gradient descent with appropriate reward functions.
> I don't think a baby or even average kid could solve these
The reward functions of a typical baby or kid is not 'get a huge dopamine boost when you solve a logic puzzle' (or whatever neurotransmitter, I don't know).
I'm not an ML researcher, so YMMV, but... how could a model learn to answer these ARC-AGI questions without training beforehand?