You solve that by forwarding/decrypting/adding noise between servers, enough to cover metadata traffic you generate. The only data you reveal is anyone listening know you might have used it at some point. See https://vuvuzela.io/ I suspect it is named so because it uses a lot of bandwidth.

It's of course possible to mitigate, but that's somewhat more involved than "just" sprinkling e2e encryption on top ;)

Signal does it just fine.