Especially because there is no way to MITM a connection with perfect-forward-secrecy only if it ends up serving a self-signed certificate, because the connection first negotiates an ephemeral key with which everything, including the certificate, will be encrypted.
This means that with eSNI and at least one CA-signed cert on the IP, any attacker runs the risk of having to spoof the CA-signed certificate.
A sophisticated attacker might know that you were going to connect to a self-signed site, though. Interestingly though, private DNS (DoH, etc.) might help further shroud this fact from the attacker.
All in all, I'd say that the browser should still throw up a full-page warning because of the implications of TOFU, but it can be one where the "continue to site" option is clearly shown even to a naïve user, and not hidden behind a spoiler.
This means that with eSNI and at least one CA-signed cert on the IP, any attacker runs the risk of having to spoof the CA-signed certificate.