Sub-50ms on text-to-speech running on an LLM is commendable.
We recently shipped text-to-speech and speech-to-text support inside Finsight (Maxint). We tapped into the platform’s native speech capabilities, which were integrated with the user’s preferred LLM inference endpoint (including local on-device models). This approach enabled us to eliminate latency and preserve user’s privacy, delivering human-like conversational experience.
Since your approach involves running the model directly, did you run into memory bandwidth contention or audio buffer underruns during bursty generation, when both the LLM and TTS models are executing concurrently?
How does your VM handle local state and performance, when the user attempts to fetch, join and render data from multiple endpoints?
We use Dart and Go (front/backend) to allow users to render widgets/mini-apps on Maxint (Finsight). And hence, the speed (virtually instant) and overall quality of the output mostly depends on the LLM/inference endpoint that they choose.
In general, most users are happy with the existing features, however more advanced users have a playground to ship enhancements on-the-fly.
We just released Crosspay, a cross-platform in-app subscriptions SDK for iOS, Android, macOS, Linux, Windows, and Web apps, enabling users to purchase subscriptions once, and use anywhere. As this ruling becomes effective, we will also enable users to choose their payment method on any platform, instead of being tied to Apple App Store.
However, apps that charge $1 or less per transaction will continue to pay over 30% in fees (e.g. Stripe charges 2.9% + 30 cents per transaction)
I vouched this comment because I found it genuinely helpful, I was looking for cross payment service for my apps that work across all platforms (they're in Flutter, do you support that?).
Be careful with treasuries, during times when the Government is heading towards default otherwise pick the highest yield and lowest fee option, SGOV offers the highest yield right now, here are the latest rankings, based on yield and fees: https://blog.maxint.com/treasury-bill-etf/ unless you don't want to pay any fee and do it yourself via https://www.treasurydirect.gov/
There are a lot of money market funds, make sure you read their prospectus to understand where they invest your money. VMRXX offers the highest yield right now, here are a few more: https://blog.maxint.com/best-money-market-funds/ which are also a good alternative when the Government is heading towards default.
Prioritize based on your needs and financial circumstances, starting with MMFs, treasury ETFs and lastly consider bank accounts for a small % that you need to have available on-sight. This is not financial advice.
there's no need to worry about a non-technical default at all. if there is a substantive default on Treasurys the world is already over and it doesn't matter where you had your money.
We recently shipped text-to-speech and speech-to-text support inside Finsight (Maxint). We tapped into the platform’s native speech capabilities, which were integrated with the user’s preferred LLM inference endpoint (including local on-device models). This approach enabled us to eliminate latency and preserve user’s privacy, delivering human-like conversational experience.
Since your approach involves running the model directly, did you run into memory bandwidth contention or audio buffer underruns during bursty generation, when both the LLM and TTS models are executing concurrently?
reply