> MCP tools don't really work for financial data at scale. One tool call for five years of daily prices dumps tens of thousands of tokens into the context window.
I maintain an OSS SDK for Databento market data. A year ago, I naively wrapped the API and certainly felt this pain. Having an API call drop a firehose of structured data into the context window was not very helpful. The tool there was get_range and the data was lost to the context.
Recently I updated the MCP server [1] to download the Databento market data into Parquet files onto the local filesystem and track those with DuckDB. So the MCP tool calls are fetch_range to fill the cache along with list_cache and query_cache to run SQL queries on it.
I haven't promoted it at all, but it would probably pair well with a platform like this. I'd be interested in how people might use this and I'm trying to understand how this approach might generally work with LLMs and DuckLake.
> A year ago, I naively wrapped the API and certainly felt this pain.
Most people, before being confronted to it, have no idea how big market data feeds really are: I certainly had no idea what I was getting into. There's a reason all these subscriptions are that pricey.
Here's an example of the pricing for the OPRA feed for Databento you mentioned:
We're talking about feeds that sustain 25+ Gb/s and can have spikes at twice or even three times that. And that's only for options market data.
I mean: even people with 25 GB/s fiber (which we can all agree ain't the most common and that's an understatement) at home still can't dream of getting the entire feed.
Having a bandwith big enough, storing, analyzing such amount of data: everything becomes problematic as such scales.
As to me I'm reusing my brokers' feeds (as I already pay for them): it's not a panacea but I get the info I need (plus balances/orders/etc. tied to my accounts).
I used the word firehose, which is typically reserved for streaming data and is a whole other very interesting problem space! Big Data vs Fast Data.
I’m just noting for interest that firms are applying transformers and other networks at this streaming microstructure level, but specially trained for feature extraction. HRT + Nvidia have some nice videos about it
I will also note that it is insane how much better all the LLMs are at calling MCP tools after just a year, especially the local ones.
One of the reasons I like DuckDB is the scale flexibility. I started with grabbing data and playing on my laptop, then I jumped to a server with high cores and a NAS.
While I’ve had tremendous success with Golang projects and Typescript Web Apps, when I tried to use Metal Mesh Shaders in January, both Codex and Claude both had issues getting it right.
That sort of GPU code has a lot of concepts and machinery, it’s not just a syntax to express, and everything has to be just right or you will get a blank screen. I also use them differently than most examples; I use it for data viz (turning data into meshes) and most samples are about level of detail. So a double whammy.
But once I pointed either LLM at my own previous work — the code from months of my prior personal exploration and battles for understanding, then they both worked much better. Not great, but we could make progress.
I also needed to make more mini-harnesses / scaffolds for it to work through; in other words isolating its focus, kind of like test-driven development.
Huh?? You are replying to a comment about a Free Speaker who was murdered point blank on a US street for protecting another Free Speaker who was being pepper sprayed for exercising their Free Speech!
Wikipedia: "The Last of Us is an action-adventure video game series and media franchise created by Naughty Dog and published by Sony Interactive Entertainment.[a] The series is set in a post-apocalyptic United States ravaged by cannibalistic humans infected by a mutated fungus in the genus Cordyceps."
We've got these sessions stored in ~/.claude ~/.codex ~/.kimi ~/.gemini ...
When you resume a session, it's reading from those folders... restoring the context.
Change something in the session, you change the agent's behavior without the user really realizing it. This is exacerbated by the YOLO and VIBE attitudes.
I don't think we are protecting those folders enough.
I think the main technological limitation is that other browsers cannot just-in-time compile (JIT) JavaScript or any other embedded language. Except in the EU.
ETA: your link includes JIT; I’m pointing out that that’s why they don’t exist outside of the EU. Non-JIT browsers would just not be very performant.
Watching that again reminds me that this guy is so poorly educated and not learned. Sure he was an engineering student at UIUC and wrote a neato browser, but wow he does not know history or philosophy well.
But he does the grifter trick of making coy observations like that 1920s bs. And he had so much time and wealth to improve those insights!
Or, he knows but this narrative makes him feel better about his behavior.
I'm not so sure he is grifting, do you have any clips where he slips and shows himself to be knowledgeable of his society and its history?
I have a similar view of Musk, Thiel and Karp, and some others I can't be bothered to remind myself of. Perhaps the Mandelsons belong to this crowd, but I suspect the Kochs have a bit more of an intellectual slant.
It's known to take a lot of discipline to keep your mind sharp and study when your power grows and other people make your life convenient and shielded from the misery of the masses. Marcus Aurelius makes this a core issue of his Meditations, the strife needed to make true and confident measurements of oneself and not get lost in superstition. Perhaps it is an impossible ideal but the ambition is at least interesting, in contrast to whatever Andreessen and his ilk are doing to themselves.
It's a grift tactic (even the exact type of laugh when he brings up Freud), but I agree with you that he's not literally grifting, more like a white-washing.
Thanks for bringing up Meditations -- it was gifted to me three years ago. Although I roughly knew Stoicism, it was my first time reading it. It helped reaffirm my focus on OSS as a channel for personal improvement and contribution to society. With so much world chaos, probably a good time re-read it.
If that's your thing, maybe complement with Augustine's Confessions. He had a somewhat complicated relation to stoicism but was one of the most important patristic writers to make use of stoic ideas.
It's a document that details an interesting person, who takes both ethics and himself very seriously, much more seriously than is common in the contemporary occident.
My mind-blown moment was when I was doing work like this and Claude started positioning the camera itself to get better looks at areas it wanted to improve.
I gave a similar presentation in January which covers the AI features that emerged in 2025 that culminated in the step-function in capability in Nov'25 and where I went from there.... (certainly my GitHub activity is bright green since)
The presentation was created with Claude Code to prove itself; never going back to Keynote/PowerPoint. Press 'X' key to disable "safe mode". Prompts are in the repo.
I maintain an OSS SDK for Databento market data. A year ago, I naively wrapped the API and certainly felt this pain. Having an API call drop a firehose of structured data into the context window was not very helpful. The tool there was get_range and the data was lost to the context.
Recently I updated the MCP server [1] to download the Databento market data into Parquet files onto the local filesystem and track those with DuckDB. So the MCP tool calls are fetch_range to fill the cache along with list_cache and query_cache to run SQL queries on it.
I haven't promoted it at all, but it would probably pair well with a platform like this. I'd be interested in how people might use this and I'm trying to understand how this approach might generally work with LLMs and DuckLake.
[1] https://github.com/NimbleMarkets/dbn-go/blob/main/cmd/dbn-go...
reply