Without changing the architecture in a significant manner/using more training data, there does come a point where adding more parameters will result in no gain (and sometimes even worse results).

There's also a consequence of performance by adding more params. The inference time will be longer and even just training the model will take longer and won't be able to run as many epochs in an efficient manner.

Probably not much at all. They used a pretty straightforward and easy to run model, EfficientNetV2. Something that doesn't give out the best results, but will offer very quick inference and train time.