I'd be interested to try this out. I'm especially keen on AI tools that implement a native RAG workflow. I've given Cursor documentation links, populated my codebase with relevant READMEs and diagram files that I'm hoping might provide useful context, and yet when I ask it to assist on some refactoring task it often spends 10-20 minutes simply grepping for various symbol names and reading through file matches before attempting to generate a response. This doesn't seem like an efficient way for an LLM to navigate a medium-sized codebase. And for an IDE with first-class LLM tooling, it is a bit surprising that it doesn't seem to provide powerful vector-based querying capabilities out of the box — if implemented well, a Google-like search interface to one's codebase could be useful to humans as well as to LLMs.
What does this flow look like in Brokk? Do models still need to resort to using obsolete terminal-based CLI tools in order to find stuff?
We implemented a multi-step process to find the required context:
1. Quick Context
Shows the most relevant files based on a pagerank algorithm (static analysis) and semantic embeddings (JLama inference engine). The input are the instructions and the AI workspace fragments (i.e. files).
2. Deep Scan
A richer LLM receives the summaries of the AI workspace files (+instructions) and returns a recommendation of files and tests. It also recommends the type of inclusion (editable, read-only, summary/skeleton).
3. Agentic Search
The AI has access to a set of tools for finding the required files. But the tools are not limited to grep/rg. Instead you can:
- find symbols (classes, methods, ...) in the project
- ask for summaries/skeletons of files
- provide class or method implementations
- find usages of symbols (where is x used?)
- call sites (in/out)
...
I have such a love/hate relationship with dunders. On the one hand, I get warm fuzzies from simplifying a module's interface with `__call__`. More than once I've seized an opportunity to rewrite some ugly conditional logic with a partial ordering by leveraging `__gt__`, `__lte__` and friends. `__setattr__` remains an interesting alternative to using decorators to execute some pre- or post-call behavior to a class method. (IIRC it can also be more efficient than wrapping a method with a decorator, since the `__setattr__` override gets precompiled into Python bytecode along with the rest of the class def.) Probably some other sleights of hand along these lines that I'm forgetting. I'm sure they were fun to write and I probably felt clever doing it.
Unfortunately in almost every case, dunder methods made it harder for me to collaborate with other people on the same codebase. And I get it. Tinkering with `__setattr__` leads to recursion errors if you're not careful, `__call__` introduces state in an unexpected place (wait, foo is a class instance? But but it behaves just like a function...). It's one of the only "stupid Python tricks" I can think of that lacks a clear analog in other programming languages, so polyglots without a strong Python background tend to hate them. I've tried to make the case on two separate occasions that dunder methods represent a more object-oriented approach to dealing with systemic complexity than type dispatch, and I stand by this. But I concede that the benefits of making the codebase nicer and more ergonomic in some places invariably requires writing class definitions that look horrendous in other places. So it's not so much that dunder methods reduce that complexity, so much as try to contain it and hopefully prevent it from spilling out into the main execution logic.
P.S. - Trey, if you're reading this, hello from a former TA!
> Trey, if you're reading this, hello from a former TA!
Hi former TA! It's been a long time.
> Unfortunately in almost every case, dunder methods made it harder for me to collaborate with other people on the same codebase.
I see dunder methods as methods that should usually (with the exception of __init__, __repr__, and __eq__) be used much more often than they're read.
I typically use dunder methods when implementing a library or framework that's meant to be used by others who would be looking at its documentation but not necessarily its code.
For example the dramatic library I published over the weekend relies on __enter__, __exit__, and __del__ (which is a very uncommon one typically): https://pypi.org/project/dramatic/
Users of that code shouldn't need to look at the code.
For code that's modified by newer Python users often, using dunder methods might be more difficult to justify. Though you might be able to get away with making a library which that could could use.
There’s also a risk that you could spend a long time making some nice behavior possible with dunders, but then people using your object don’t know about it because a lot of the new behaviors don’t get surfaced in typical IDEs.
Python has a bunch of facilities for poking holes in the "normal" execution flow (most "dunder" methods), but true enlightenment comes from understanding that it's all just dictionaries, type descriptors, functions, and a particularly gnarly execution order all the way down. The CPython interpreter has some pretty interesting C code too.
> In the following, we show that [taking cosine similarity between two features in a learned embedding] can lead to arbitrary results, and they may not even be unique.
Was uniqueness ever a guarantee? It's a distance metric. It's reasonable to assume that two features can be equidistant to the ideal solution to a linear system of equations. Maybe I'm missing something.
It's not even a distance metric, it doesn't obey the triangle inequality (hence the not-technically-meaningful name "similarity", like "collection" as opposed to "set").
Consider the geometric definition of the dot product of two vectors,
a.b = |a||b|cos theta.
This means you get cos of the angle between the two vectors by just dividing the dot product by the product of their magnitudes. You don't actually take cos of the angle to get cosine similarity (for one because you don't know the angle) you just use "cos theta" (calculated as above) as a proxy for how narrow the angle is and therefore how close the two embeddings are.
The paper in TFA shows that if you construct an embedding space such that that angle isn't meaningfully measuring similarity then a low angle doesn't mean two things are very similar. I have a similar paper measuring bears and woods but I haven't got around to typesetting it for publication yet.
I sure hope noone claimed that. You’re doing potentially huge dimensionality reduction, uniqueness would be like saying you cannot have md5 collisions.
If you have 1000 points and want to preserve their squared distances to within an error of 1%, the Johnson-Lindenstrauss construction suggests an embedding dimension of 8(ln 1000)/(0.01²) > 552620. If your points start out in a lower-dimensional space than that to begin with, it's obviously pointless.
The crossover point where the number of dimensions falls below the number of points is at 1113868. If you're willing to tolerate 10% error, it's at 7094.
I think maybe it's poorly phrased. As far as I can tell, their linear regression example for eq. 2 has an unique solution, but I think they state I that when optimizing for cosine similarity you can find non-unique solutions. But I haven't read in detail.
Then again, you could argue whether that is a problem when considering very high dimensional embeddings. Their conclusions seem to point in that direction but I would not agree on that.
I don’t think bitcoin is some sort of intelligence thing, Occam’s razor, it is probably just a dude. I mean, what was his big feat of secrecy? Writing a paper needn’t necessarily leave a big footprint.
But! Bitcoins are too clunky to threaten the federal reserve really. And, a system that is widely understood by laymen to be anonymous, but is actually pseudonymous and inherently traceable, seems like Christmas for intelligence and police organizations.
The funny thing is, crypto is actually good for the dollar.
USDT/USDC are far easier to acquire than actual USD in third world countries. There’s a separate economy built up in Africa, Phillippines, etc. that operates entirely on USDT.
You can see this in the price of USDT vs USD in these countries - USDT commands a premium.
Crypto via coins like USDT means that Nigerians increasingly want to work for USDT, not Naira. Bad for Naira, good for the USD
Tether has to be backed by USD-denominated reserves for that to work. If it's actually backed by something else (or nothing) then all that value is going somewhere other than the dollar
Silver (#8 asset in the world) is also "clunky" but it is close to being surpassed in market cap by Bitcoin (#10 asset in the world).
A large part of that value is derived from a market cap based upon the stability created by the dormant coins, like gold sitting dormant in fort knox used to stabilize the USD value.
It really does seem like an intelligent design passes Occam's razor more than "oops i conveniently formatted that harddrive then disappeared"
Hypothetically, getting those who hate "the feds" to record their transactions on a public ledger would be criminal intelligence coup even bigger than when FBI's "Encrypted Phone" platform became popular with criminals[1]. In this hypothetical, the FBI would hack/subvert/operate their own mixer service and eliminate uncooperative services, so that all money-flows are transparent to investigators.
Not all cryptocurrencies use public ledgers. Bitcoin was the first, but it was always intended to add privacy tech - see MimbleWimble.
Zero knowledge proofs and other strong privacy protections are available on more modern projects. Some ledgers are entirely dark, granting a decent anonymity set.
How many of those cryptocurrency were created by Satoshi Nakamoto, the subject of the thread's speculation? I was speculating on the why a 3LA would have created Bitcoin, specifically.
Others like Monero also have mysterious origins. I don’t think Satoshi was behind the design even in part, but it’s one theory.
I’m pointing out that even if Bitcoin was the creation, the introduction of this concept and class of data structure is much more important and will have more lasting impact.
Surely any government agency would have considered the feasibility of new alternatives with similar designs.
It’s no threat at all for the foreseeable future.
Make no mistake though, over the next decades, the core idea is an existential threat.
If a currency like Monero with further developed scaling and privacy features is able to gain a foothold in developing nations, and enough off-ramps are set up via decentralized exchanges like Bisq and through direct acceptance for goods and services, then it is difficult to see the spread stopping between nations with reasonably free Internet, especially as factions within each government will likely have some direct interest.
The US dollar will be least threatened, the longest.
Privacy features in a currency are a double-edged sword. It means that the money is very difficult or impossible to recover in the event of theft, fraud, or even user errors. It isn't a risk most people want to be exposed to.
Physical cash is typically used in face-to-face transactions, which somewhat reduces the risks. The scenarios in which a digital version of physical cash might make sense are few and far between. To suggest that such a currency could one day threaten the US dollar dominance is absurd, in my opinion.
I completely agree with you. And you're assuming that a crypto network even exists that could support that kind of transaction volume, so the reality is even more absurd.
Monero for instance is optionally transparent using view keys, which allow read access to incoming but not outgoing transactions.
As cryptocurrency already requires the safeguarding of a private key or a custodial partner doing so, and generally do not accept rollbacks on the base level, there isn’t that much added drawback to making strong privacy default.
Spitballing here… it could have been a test to see how easy they could get a digital currency into common use. The idea of a digital currency offers the Fed a lot of advantages… no money laundering, traceability, etc. it could actually advance them if well executed, not threaten them.
Does the fed employ a lot of cryptographers? I feel like people expect the U.S. gov to be omnipotent when the last 20 years have been just fumble after another
While not omnipotent, the NSA does hire a huge amount of mathematicians and has a budget in the tens of billions. Most of what they do is also behind (extremely) closed doors.
The Bretton-Woods paradigm was already in collapse at the advent of bitcoin due to the corresponding growth of the US trade deficit as the US stopped producing/exporting and began massive consumption from the east.
The USA is in a debt spiral spending $659B on interest on debt annually (13% of tax revenue) while running larger and larger deficits. This cannot really be escaped, and it's impact on inflation is becoming unavoidable.
Bitcoin? No. But if you're asking whether digital currencies (which share a lot of the same underlying characteristics) might transform the global monetary landscape, well, they already have: 11 countries have issued CBDCs, and another 130 are actively exploring them as a more convenient alternative to USD for international transaction settlements. Several of those are in advanced pilot stages. No one with serious ties to the US financial system finds this to be a laughing matter, I assure you. The dollar is by far the currency of choice in trade invoicing (more than 50% of total trade) and foreign exchange transaction volume (almost 90% of the total) globally (Moronoti, 2022). This also means that US settlement authorities and financial institutions are involved in finalising most global transactions. If two countries have CBDCs, then they in principle would have the ability to settle transactions between themselves with near-instant finality, potentially bypassing the current dollar-based system.
I think we can safely expect at least one major CBDC-based cross-border payment system to launch by the end of the year. Soramitsu is the most promising candidate IMHO. A prevailing theory is that foreign corporations that operate domestically within a country will need to create accounts with a domestic central bank for CBDC payments to work efficiently. If this becomes a reality, the status of the dollar's "exorbitant privilege" will be up for immediate dispute. Its geopolitical hegemony over global finance won't be swept away overnight, but it will suffer a major blow. Only time will tell how serious.
You don't say why a CBDC would be a more convenient alternative to USD for international transaction settlements. A CBDC is simply a digital version of an existing currency. It isn't nothing new, since bank deposits already allow digital transactions with any currency.
You're missing the point. The reason the dollar is the global currency of choice is because it offers the infrastructure for any two parties to settle a transaction. The existence of CDBCs for wholesale purposes has the potential to fundamentally change that. Central banks could directly settle transactions between themselves in local currencies via dedicated corridors that bypass the dollar settlement system. That would mean more diversification of currency pairs, with increased liquidity for currency pairs that do not include USD.
> It isn't nothing new,
It is though. The infrastructure to support cross-border payments with CBDCs is bleeding edge stuff. The term floated around in obscurity for a while, but it's only been in use since 2019 or so. See: https://en.wikipedia.org/wiki/History_of_CBDCs_by_country
The USD doesn't have a technological monopoly on cross-border payments. Before the introduction of the euro, European nations were trading with each other using their local currencies just fine.
Moreover, CBDCs are ill-suited for internation trade, or for any kind of trade, because they're cash-like. They're intended as a substitute for physical cash.
The speculative currency crises in Europe during the 90s helped drive the adoption of the Euro in the first place. It's a bit outside my ken, but I do wonder if Italians of a certain age would agree that the older system worked "just fine."
I think reasonable minds can disagree about whether CBDCs are any more or less suitable than the alternatives (which seem worse to me in some respects, and certainly are worse in others) but either way, the world's central banks are singing a similar tune in unison right now. Like it or not, the macroeconomic tailwinds favor a more decentralized approach to cross-border settlements and we'll soon have infrastructure to enable this at massive scale.
Again, that's simply not true. You can't point to a single instance where CBDC infrastructure has enabled or improved wholesale international payments that were impossible or otherwise costly to do before.
Give it until the end of the year. These systems are new. Legal and logistical frameworks are being created around them. Southeast Asia will have a streamlined import/export relationship with Japan by the end of the year. Where it goes is from there is anyone's guess, but there's a lot of momentum for this to be the first of several major events over the next 2-3 years.
I was in the market a couple months ago (US). If you haven't given the 2023 Id.4 a look, you might consider it. They're practically giving them away right now due to surplus inventory. The lease deals are especially attractive. My monthly is about what I was quoted for a Honda Accord or a Prius.
Software is atrocious. Everything else, extremely happy. While everyone else queues up in an endless Costco line to save $10 off their $100 gas bill, I just relax and order sushi while my car chugs free Level 1 electricity outside.
I've joked that LinkedIn is like a reverse dating site: Instead of attractive women dealing with loads of men whose messages they're uninterested in, it's men dealing with loads of attractive women whose messages they're uninterested in. :D
The point is that if you're "choosing the next token based on a list of valid next tokens," it's not surprising that you'll only generate valid output, since absolutely any choice mechanism will suffice.
My experience was similar. I've built far more impactful things since then, but at the end of the day, the software that I'm most proud of was a calculator, of all things, that I made in Clojurescript. It was loosely based on the final chapter of SICP, and implemented an hilariously metacircular approach to transitioning the state of the register machine. Since the implementation was way more interesting than the actual application, I used Processing to visualize the register machine as if it were a rudimentary ALU (which it basically was).
If it hadn't been for Rich Hickey, or Clojure, I'm not sure I'd have the slightest inkling of how joyful and creative it can be to write software. It was an A+ experience, even if I'd be swiftly fired for trying to push logic like that into production.
I've been programming in Python for most of the past 10 years, and I've never experienced a regression in backwards-compatibility between minor releases. What problems have you had?
The syntax of the language changes every version in non-backwards-compatible ways.
If you have a couple scripts, sure, maybe you're not affected. But when you buy a company that shat out 90kloc of python and then all the employees quit, it's not a happy day.
And sure... I shouldn't have used those features. I get it. I'm the one who's bad because I'm calling your baby ugly. Even though I wasn't the one that originally wrote that code.
Though I did write some code that used package variables. And then the syntax for package variables changed, but that was an easy fix. And then the scope of package variables changed to be class variables, which is totally fine, but harder to find. Then the syntax changed again, but in a way that made it harder to fine. And then the debugger stopped working if you enabled async io for a few versions.
Which syntactic features have changed in ways that aren't backwards compatible? I've had some minor headaches, don't get me wrong, but in each case those headaches are a result of interface changes to common objects, like Exception-types. Python has added some syntactic sugar between minor releases, sure, but never at the expense of backwards compatibility.
Wow, didn't realize folks were still actively developing on Python 2.x. FWIW, the Python steering committee takes backwards-compatibility pretty seriously these days. Here's a recent mailing list discussion on it's decision to reject a popular PEP on the grounds that it would have broken Pydantic: https://mail.python.org/archives/list/python-dev@python.org/...
What does this flow look like in Brokk? Do models still need to resort to using obsolete terminal-based CLI tools in order to find stuff?