As a model is trained previously hidden memories sporadically return. Essentially a model's memory is time dependent to when you sample.
Study was:
1. Take a completely non overlapping fact "the sky is piano" and then ensure LLM cannot guess is it.
2. Train it one or more shots on this
3. Continue training on c4 without this fact.
4. The effect is that the random fact is forgotten but not linerally. Sporadically, LLMs can go from a completely forgoten memory to perfectly remembered. A type of internal self reinforcement without training data.
A rare but reproducible effect (1/15 training runs self reinforce). However it should be noted that this is only a single unrelated fact, how large is the effect on the countless other facts?
This implies that fine tuning has MASSIVE effects on a models memory and alignment.
Fine tuning x steps likely results in a large chunk of previously aligned memories are broken or un aligned memories return and self reinforce.
Memory is a facinating and very misunderstoof part of AI.
> Orgmode got me through college, research, and at work, it really is the perfected markup language that can do a lot more than just being a markup language. The extensibility and out of the box export to other formats makes it immediately useful for at least 80% of common tasks.
With regards to your 80% claim, do you happen to know an extension that works well with pasting images from clipboard?
Over the years, my professional note taking has become extremely reliant on quickly pasting images (most often screenshots from papers or quick-and-dirty plots I made myself) from clipboard directly into the notes. The friction of doing this in vanilla org-mode is the only reason I'm not doing everything in org-mode.
An example of the prompt engineering phenomenon: my wife and I were recently discussing a financial decision. I'd offered my arguments in favor of one choice and she was mostly persuaded but decided to check in with ChatGPT to help reassure herself that I was right. She asked the financial question in layman's terms and got the opposite answer that I had given.
She showed me the result and I immediately saw the logical flaws and pointed them out to her. She pressed the model on it and it of course apologized and corrected itself. Out of curiosity I tried the prompt again, this time using financial jargon that I was familiar with and my wife was not. The intended meaning of the words was the same, the only difference is that my prompt sounded like it came from someone who knew finance. The result was that the model got it right and gave an explanation for the reasoning in exacting detail.
It was an interesting result to me because it shows that experts in a field are not only more likely to recognize when a model is giving incorrect answers but they're also more likely to get correct answers because they are able to tap into a set of weights that are populated by text that knew what it was talking about. Lay people trying to use an LLM to understand an unfamiliar field are vulnerable to accidentally tapping into the "amateur" weights and ending up with an answer learned from random Reddit threads or SEO marketing blog posts, whereas experts can use jargon correctly in order to tap into answers learned from other experts.
A clear case where LLMs exceed humans is in identifying solutions to disparate shallow constraints involving what would normally require very wide searches of more knowledge than any of us will ever have.
A simple case I have found, is looking for existing or creating new terms. If I have a series of concepts, which I have names for which have a nice linguistic pattern to emphasize their close relationship, except for one. I can describe the regularly named concepts, then ask for suggestions for the remaining concept.
The LLM pulls from virtually every topic with domain terminology, repurposable languages (Greek, Roman), words from fiction, all the way to creative construction of new words, tenses, etc to come up with great proposals in seconds.
I could imagine that crafting persuasive wording would be a similar challenge. Choosing the right words, right phrasing, etc. to carry as much positive connotation, implication of solidity, avoiding anything sounding challenging or controlling, etc. from all of human language and its huge space of emotional constraints and composites.
Very shallow but very wide reasoning/searching/balancing done in very little time.
And with an ability to avoid giving any unnecessary purchase for disagreement, being informed of all the myriad of typical and idiosyncratic ways people get hung up on failed persuasions. Whether in general or specific topic related.
LLM generated writing can be stereotypical.
But the more constraints put on requested material, the more their ability to construct really very original high quality, or even cleverly unique, prose in real time shines.
My headcanon is that it’s an interaction between cavitation from the object dragging through the fluid and recoil from the particles flinging off of the surface. If you move fast enough then the momentum imparted by the ram surface onto the molecules are large enough to overcome the external pressure trying to fill the cavitation, so you get boundary layer separation.
I'm using Qwen3-30B-A3B locally and it's very impressive. Feels like the GPT-4 killer we were waiting for for two years. I'm getting 70 tok/s on an M3 Max, which is pushing it into the "very usable" quadrant.
What was even more impressive is the 0.6B model which made the sub 1B actually useful for non-trivial tasks.
Overall very impressed. I am evaluating how it can integrate with my current setup and will probably report somewhere about that.
If you’re using a debit card anyway, Privacy dot com is what you should use to solve this problem, and many others relating to your authority to control access to your funds. (You can pick when the generated numbers stop working, pause them at will, limit amounts etc.)
I don’t use it extensively because I’m a credit card points nerd and the only fee-free way to use it is to pull from checking with ACH. But I do use it when I’m suspicious that a business will make it hard for me to cancel.
As a bonus, you don’t need to use a real name and address — it’ll pass those checks as correct with any name or address you make up.
Note: I’m not a paid endorser or anything - I do use a free account personally.
It's really a bummer to see this marketed as 'AI Discovers Something New'. The authors in the actual paper carried out an enormous amount of work, the vast majority of which is relatively standard biochemistry and cell biology - nothing to do with computational techniques. The AlphaFold3 analysis (the AI contribution) literally accounts for a few panels in a supplementary figure - it didn't even help guide their choice of small molecule inhibitors since those were already known. AlphaFold (among other related tools) is absolutely a game changer in structural biology and biophysics, but this is a pretty severe case of AI hype overshadowing the real value of the work.
Strengths are weaknesses because they create a bias to use the strength rather than developing a weak alternate, and you only get better at what you do - creating a virtuous cycle that can quickly turn vicious.
This will happen whenever growth is mediated mainly by feedback loops. (Think hard about that!)
The solution is instead to have a model of what you're trying to grow, whether it's a company or a positive presence in the world, and be willing to sacrifice to make that happen.
For any given thing or category of thing, a tiny minority of the human population will be enthusiasts of that thing, but those enthusiasts will have an outsize effect in determining everyone else's taste for that thing. For example, very few people have any real interest in driving a car at 200 MPH, but Ferraris, Lamborghinis and Porsches are widely understood as desirable cars, because the people who are into cars like those marques.
If you're designing a consumer-oriented web service like Netflix or Spotify or Instagram, you will probably add in some user analytics service, and use the insights from that analysis to inform future development. However, that analysis will aggregate its results over all your users, and won't pick out the enthusiasts, who will shape discourse and public opinion about your service. Consequently, your results will be dominated by people who don't really have an opinion, and just take whatever they're given.
Think about web browsers. The first popular browser was Netscape Navigator; then, Internet Explorer came onto the scene. Mozilla Firefox clawed back a fair chunk of market share, and then Google Chrome came along and ate everyone's lunch. In all of these changes, most of the userbase didn't really care what browser they were using: the change was driven by enthusiasts recommending the latest and greatest to their less-technically-inclined friends and family.
So if you develop your product by following your analytics, you'll inevitably converge on something that just shoves content into the faces of an indiscriminating userbase, because that's what the median user of any given service wants. (This isn't to say that most people are tasteless blobs; I think everyone is a connoisseur of something, it's just that for any given individual, that something probably isn't your product.) But who knows - maybe that really is the most profitable way to run a tech business.
There is a lot of way and the most common is shiny (https://shiny.posit.co/) but with a biais towards data app. Not having a Django-like or others web stack python may have talks more about the users of R than the language per se. Its background was to replace S which was a proprietary statistics language not to enter competition with Perl used in CGI and early web. R is very powerful and is Lisp in disguise coupled with the same infrastructure that let you use C under the hood like python for most libraries/packages.
Why not mix R and Python in interactive analysis workflows:
1) Download positron: https://github.com/posit-dev/positron
2) Set up a quarto (.qmd) notebook
3) Set up R and Python code chunks in tour quarto document
4a) Use reticulate to spawn a Python session inside R and exchange objects beween both languages (https://github.com/posit-dev/positron/pull/4603)
4b) Write a few helper functions that pass objects between R and Python by reading/writing a temporary file.
Anyone who wants to demystify ML should read: The StatQuest Illustrated Guide to Machine Learning [0] By Josh Starmer.
To this day I haven't found a teacher who could express complex ideas as clearly and concisely as Starmer does. It's written in an almost children's book like format that is very easy to read and understand. He also just published a book on NN that is just as good. Highly recommend even if you are already an expert as it will give you great ways to teach and communicate complex ideas in ML.
I'll save everyone hours of headache. The only politician's disclosures that are worth anything are Pelosi's. And by Pelosi, it is actually her husband that trades but that she has to disclose those trades as a politician. You can also copy Pelosi's trades exactly if you are into speculation and you can experience first hand how many of his trades end up -30% or more, but over the course of the year they have a chance of gaining. They gain not so much because there is any meaningful insider information, but because he holds the position for 1 year and we are still in a bull market (even as of tonight).
I think it's amusing that folks think that politicians have any sort of capability to be able to consistently beat the S&P 500.
Looking at the sparse documentation of openrsync does not create any confidence for me that it can be an acceptable substitute for rsync.
In my opinion, any program that is supposed to copy files, but which is not able to make perfect copies, i.e. copies that do not lose any bit of data or metadata that was present in the original file, is just unusable garbage.
Unfortunately, most copying programs available in UNIX-like operating systems (and also many archiving programs) do not make perfect file copies with their default options and many of them are never able to make perfect copies, regardless what options are used.
I have not looked recently at the scp command of ssh, but at least until a few years ago it was not possible to make perfect file copies with scp, especially when the copies were done between different operating systems and file systems. That is why I never use scp, but only rsync over ssh.
Rsync is the only program that I have seen, which is able (with the right options) to make perfect file copies even between different operating systems and file systems (for instance between FreeBSD with UFS and Linux with XFS), preserving also metadata like extended file attributes, access control lists and high-precision file timestamps (some copying programs and archiving programs truncate high-precision timestamps).
The current documentation of openrsync does not make any guarantee that it can make complete file copies, so by default I assume that it cannot, so for now it is a program that I consider useless.
Beside rsync for copying, one of the few Linux archiving programs that can archive perfect file copies is bsdtar (when using the pax file format; the ancient tar and cpio file formats cannot store all modern file metadata).
(FYI: I always alias rsync to '/usr/bin/rsync --archive --xattrs --acls --hard-links --progress --rsh="ssh -p XXX -l YYYYYYY"')
(With the right CLI options, "cp" from coreutils can make perfect file copies, but only if it has been compiled with appropriate options; some Linux distributions compile coreutils with wrong options, e.g. without extended file attributes support, in which case "cp" makes only partial file copies, without giving any warnings or errors.)
> Preview cannot save files in the Graphics Interchange Format (GIF).
It can, actually, you just need to hold done ⌥ while clicking on the file format picker. (Perhaps Apple thinks demonstrating the true capabilities of Core Image would blow the brains of Preview’s users.)
I looked for resources a few years ago and found a post by a software engineer talking about "Fearless Salary Negotiation" [1]. The engineer said: "it's a small book, cost me around 40 bucks which were immediately compensated by the fact that I managed to negotiate a higher salary".
I wanted to ask for a raise: I had been working in that startup for 4 years and had never had a raise. I thought "if that book helps me get a raise of a few bucks per month, that will pay for it".
It didn't go as planned: I followed the instructions in the book, my boss spent an hour bullshitting me and I didn't get a raise. So I sent my resignation the next day. The boss called me back, and I got a substantial raise and a bonus (to compensate for the shitty salary I had been having before, it was not a miracle).
All that to say that this book did not make me a pro negotiator. But it made me understand how salary negotiation works (reading it, I felt like a child: it only says common sense stuff, but I had been doing everything wrong my whole life). And it gave me the confidence to actually ask for a raise.
I think this article is not "martingales for laypeople". It is "martingales for math nerds" where the application is solving certain types of math problems to mathematical standards of rigor. For that rigor, you do need the sigma algebras. I would say also, the target reader of the article already knows what a sigma algebra is, so the formal definition is given only for review purposes, plus to motivate something the author does about finding a disjoing minimal cover of the sample space. I haven't yet read the article carefully enough to understand why he does that.
Sigma algebras are useful even in a finite setting where we don't have to worry about pathologies. Think of them as modelling the lack of complete information in a probabilistic setting. If I know exactly which sample point represents my state, then I know the exact value of any random variable. If instead I only know that my state belongs to a given member of my sigma algebra, then I have some information, but not enough to necessarily pinpoint the value of a random variable.
In fact, the familiar tools of measure theory can take this intuition further. If a random variable is measurable with respect to a sigma algebra, then knowing which element of that sigma algebra my state is in actually is sufficient to pinpoint the value of a random variable.
Maybe to make this more concrete:
Let's say I'm going to do two coinflips. My probability space is {HH, HT, TH, TT}. You can check for yourself that the sigma algebra generated by {{HH, HT}}, {TT, TH}} is not the trivial one- this is the sigma algebra that represents "Knowing the value of the first flip, but not the second".
If we let X_first and X_second be 1 or 0 if the first or second flip is H or T respectively, then X_first is measurable with respect to this sigma algebra, but X_second is not.
With Martingales and other stochastic processes, we don't generally have just one sigma algebra, but a sequence of sigma algebras called a "filtration", where each sigma algebra is finer than the last (ie, contains more sets, therefore gives you more measurable random variables). This filtration sort of defines the stochastic process- it's encoding the slow drip of extra information as the stochastic process evolves over time.
The fact that favorites are a hidden feature and hard to use made me think the quality of the curation signal would be even better. Also the fact the HackerNews API, Angolia don't expose this feature made it even more interesting to aggregate.
Hope you like it and as it was a pain gathering the data, I put it in an observable notebook so hopefully we don't need to gather that dataset again for a while.
The learning resources were the most useful thing to me. I have seen most of those links whoosh by me when reading HN, but this list has made me revisit some of them that were lost to the sands of time. In particular the bash shell resource [1] is something I have been trying to re-find for ages!
I found an effect that explains this.
LLM memory isn't linearly lost or updated.
As a model is trained previously hidden memories sporadically return. Essentially a model's memory is time dependent to when you sample.
Study was: 1. Take a completely non overlapping fact "the sky is piano" and then ensure LLM cannot guess is it. 2. Train it one or more shots on this 3. Continue training on c4 without this fact. 4. The effect is that the random fact is forgotten but not linerally. Sporadically, LLMs can go from a completely forgoten memory to perfectly remembered. A type of internal self reinforcement without training data.
A rare but reproducible effect (1/15 training runs self reinforce). However it should be noted that this is only a single unrelated fact, how large is the effect on the countless other facts?
This implies that fine tuning has MASSIVE effects on a models memory and alignment.
Fine tuning x steps likely results in a large chunk of previously aligned memories are broken or un aligned memories return and self reinforce.
Memory is a facinating and very misunderstoof part of AI.