Hi! As someone who spent the last month pouring myself into the ARC challenge (which has been lots of fun, thanks so much for creating it), I'm happy to see it made harder, but please make it harder by requiring more reasoning, not by requiring more human-like visual perception! ARC is almost perfect as a benchmark for analogical reasoning, except for the need for lots of image processing as well. [Edit: however, I've realised that perception is representation, so requiring it is a good thing.]
Any plan for more validation data to match the new harder testset?
I had never thought about how close perception and reasoning are from a computational point of view, the parts of ARC that we call "reasoning" seem to just be operations that the human brain is not predisposed to solve easily.
A very interesting corollary is that the first AGIs might be way better thinkers than humans by default because of how they can seamlessly integrate new programs into their cognition in a perfect symbiosis with computers.
Perception is the representation of raw inputs into a form useful for further processing, but it is not a feed-forward computation. You repeatedly re-represent what you see as you keep looking. Particularly something like an ARC puzzle where you have to find a representation that reveals the pattern. That's what my ARC solver is about (I did not finish it for the deadline).
> A very interesting corollary is that the first AGIs might be way better thinkers than humans by default
I agree at least this far. Human System 2 cognition has some very severe limitations (especially working memory, speed, and error rate) which an AGI probably would not have. Beyond fixing those limitations, I agree with François that we shouldn't assume there aren't diminishing intelligence returns to better mental architectures.
Any plan for more validation data to match the new harder testset?