Your metaphor fails to indicate the dependence of AI on data. AI can't appear in a void. There must be an environment producing its data, then AI development is conditioned on the environment. We forget about that when we just focus on the learned model. But without all the external support structure there is no AI.
Let's take a simple case - AlphaZero. Its environment was just a Go board, and rules simple. Knowing who won the game easy to test at the end of the game. So this led to a massive number of self-play games. In a few days AZ beat humans, and it was our own game, with 2 millennia head start.
This is how much the environment and the external support structure influences AI development. "Maximally likely output for a given input" depends on it.
"Your metaphor fails to indicate the dependence of AI on data." I'm not sure how you conclude that; the post you're responding to says exactly "...conditioned on some historical data." No historical data, no conditioning, and therefore no AI, just random bits.
And fwiw, you could say much the same about human beings. There must be an environment producing our oxygen, water, food. Indeed, it's not clear what a single human raised in isolation would be like, even assuming the oxygen, water, food. Not very intelligent seeming, I would guess; perhaps not even with much of a language. Fortunately, no one has performed the experiment, but we've come close: deaf people raised by non-deaf parents in a non-deaf community appear to have a sparse sort of sign language. Only when they're put together with other deaf people does a more typical, fluent sign language arise, with a full grammar. (This we know from the deaf people in Nicaragua.)
Let's take a simple case - AlphaZero. Its environment was just a Go board, and rules simple. Knowing who won the game easy to test at the end of the game. So this led to a massive number of self-play games. In a few days AZ beat humans, and it was our own game, with 2 millennia head start.
This is how much the environment and the external support structure influences AI development. "Maximally likely output for a given input" depends on it.