Can you share any of the techniques you use to avoid being blocked for scraping?

Interesting to hear that storage was too expensive. I'm sure you could find personal sponsors here who would help with funding in exchange for access to the data - even with a 'no commercial use' type of restriction!