Compressing 640x64@660 takes exactly the same amount of work as compressing 1280x720@30. You could even use hardware h.264 encoder as long as you can debayer raw 10bit in software fast enough (or convince Broadcom employee 6by9 to make internal Videocore firmware debayering resolution agnostic, its already fast enough keeping up with twice the amount of data when using camera in 1920x1080@25 mode).
But you dont even need fancy compression, at those framerates things change really slowly so simple delta encoding on raw data will do wonders.
Compressing 660fps in real time is a much different task than 30 frames per second, notably because your latency budget is a lot lower. At 30fps you have 33ms to load the frame, compress it and send it off to the destination, at 660fps you have 1.5ms for the same task, which does not necessarily get exponentially faster with lowered resolution.
You have exactly as much time as at 30fps at 22x higher resolution. You dont need to constantly reuse the same 52KB buffer. Data comes at a steady snail 35MB/s pace. Pee 3 should even be able to save it on USB2 HDD in real time with no compression.
You do not need any special heavy compression. Just deltas between consecutive frames should result in 1:100 or more compression rate at such high frame rate. This is assuming you can keep both pivot and current frames in processor's SRAM.
