> But scaling isn't what got them there. It was tons of labeled data, both from rules based implementations and shadowing human drivers.
Isn't the second half of that saying that scaling up the dataset was what got them where they are?
When I hear about "scaling up" models, I think there are two parts of it: 1) use the same architecture, but bigger 2) make the training data bigger to make use of all those new parameters.
So when I hear about something that's not scaling, it would require some sort of fundamental change to architecture or algorithms.
Isn't the second half of that saying that scaling up the dataset was what got them where they are?
When I hear about "scaling up" models, I think there are two parts of it: 1) use the same architecture, but bigger 2) make the training data bigger to make use of all those new parameters.
So when I hear about something that's not scaling, it would require some sort of fundamental change to architecture or algorithms.