Having retested not I have local access to that filesystem again, the issue seems to be when used remote via RDC, even over a local wifi network. Running properly local the delay goes away in even the pathological case.

As SpaceSniffer has no similar bad slow-down on display updates, I assume that means how WizTree draws interacts badly with the remoting protocol: either it is drawing in small steps and RDC is trying to send each update out individually so many small updates are going over the wire, or the 3D-ish look is causing issues, or both.