I think some kind of disposable multi-layered lens cover that peels off automatically would work well. Pretty cheap, and easily replaceable. Effective against more than just ice.
I'm assuming that they only need the video from when they get close to landing so the lens cap can be ejected once the rocket is low enough in the atmosphere that icing is not a problem.