This is by far the biggest culprit OP, look into this.
You can try to come up with imposters representing these far away dragons, or simple LoD levels. Some games do use particles to represent far away and repeated "meshes" (Ghost of Tsushima does these for soldiers far away).
Lot's of techniques in this area ranging from simple to bananas. LoD levels alone can get you pretty far! Of course, this is at the cost of having more different draw calls, so it is a balancing game.
Think about the topology too, hope these old gems helps getting a grasp on the cost of this:
Yeah, I use LODs already but as you say, even my lowest lod far away is too many vertices. Imposter rendering seems very interesting but also completely bonkers (viewing angle, lighting)!
have you considered using meshlets technique like Unreal Engine Nanite or Assassin's Creed? It could potentially open the door for better culling and more effective depth prepass.
You can try to come up with imposters representing these far away dragons, or simple LoD levels. Some games do use particles to represent far away and repeated "meshes" (Ghost of Tsushima does these for soldiers far away).
Lot's of techniques in this area ranging from simple to bananas. LoD levels alone can get you pretty far! Of course, this is at the cost of having more different draw calls, so it is a balancing game.
Think about the topology too, hope these old gems helps getting a grasp on the cost of this:
https://www.humus.name/index.php?page=Comments&ID=228
https://www.g-truc.net/post-0662.html