I wondered about this too. But even balloons are maneuverable in some sense, by changing their altitude. So maybe the meaning is more like "didn't maneuver in response to our presence".
You can use propellers to move up and down too. The buoyancy of a fixed amount of helium doesn't drop/rise much with altitude changes. It isnt like submarines. The needed vertical thrust to move a few thousand feet is minimal.
