I spent a month setting up serverless endpoint for a custom model last year with Runpod. It was expensive and unreliable, in addition to long cold boot times. The product was unusable even as a prototype, to cover the costs, I'd have to raise money first.
In a different product, I was given some Google Cloud credits, which unlocked me to put the product in front of customer. This one also needed GPU but not as expensive as the previous. It works reliably and it's fast.
Personally, I had two use cases for GPU providers in past 3 months.
I think there's definitely demand for reliability and better pricing. Not sure Fly will be able to touch that market though as it's not known for both (stability & developer friendly pricing).
P.S If anyone is working on a serverless provider and want me to test their product, reach me out :)
Ironically GCP and AWS GPUs are so overpriced that getting even half the number of credits from Runpod is like a 4x increase in "GPU runway", especially with .44/hr A40s.
Yeah but the quality on Runpod is not reliable enough for productionizing it. Do you know a product that works reliably and it's built with Runpod serverless?
In a different product, I was given some Google Cloud credits, which unlocked me to put the product in front of customer. This one also needed GPU but not as expensive as the previous. It works reliably and it's fast.
Personally, I had two use cases for GPU providers in past 3 months.
I think there's definitely demand for reliability and better pricing. Not sure Fly will be able to touch that market though as it's not known for both (stability & developer friendly pricing).
P.S If anyone is working on a serverless provider and want me to test their product, reach me out :)