Personally, I think Orwell's version is a lot easier to read. I read the Ecclesiastes verse several times and can't really get it: "time and chance happens to everyone"? Okay?
While phrases like "contemporary phenomena" and "tendency to be commensurate" are a bit over-the-top, summarily the picture is conveyed much more clearly imo: Luck plays a factor, no matter how good you are.
I don't see how "nor yet favor to men of skill" is more natural or clearer, at least to the modern reader.
Not well informed on the topic- but the title made me think of the recently launched Roman Space Telescope.
The difference: LUX-ZEPLIN, which is underground, is waiting to detect a dark matter particle itself. On the other hand, NGRST seeks to observe the effects of dark matter.
It is useful to know the level of how "hands-off" to specify directions for each task. With some experience one can learn when to specify the high level requirements, and when to stop and think about the problem at hand.
For example, with frontend, I can get an LLM to design half a page 80% to my satisfaction with two paragraphs worth of a prompt. "It should look like so, and have a text box here, and room for a demo there".
With ML training or backend or user-facing code, I might instead spend a paragraph thinking out my design intentions for a single function or even a single line, more for myself than the LLM. A harness generates a plan based on that paragraph, which one can then comment on and review the pseudocode it provided, ensuring it aligns with expectations.
Lastly have the LLM output some sort of documentation and "here's what I did" after each change. Your final step is to handwrite (paraphrasing what it gave) into any docs or commit messages, and ensure your commits are small enough to keep this maintainable. Paraphrasing the LLM, rather than the LLM paraphrasing you, is helpful to ensure the commit messages make sense to you three months from now.
I think it's interesting how the same books recommended for rebuilding yourself after a breakup can also be "manuals of statecraft". E.g. Marcus Aurelius' Meditations
In my opinion, whether it was actually by Ilya or not is not worthy of debate. Many of them are widely recognized for being good pedagogical resources (e.g. annotated transformer, unreasonable effectiveness of RNNs, understanding LSTM networks), and others are landmark papers which anyone interested in the field would benefit from reading:
- Krizhevsky et al. (2012) introduced AlexNet
- Bahdanau et al. (2014) introduced attention
- He et al. (2015) introduced ResNet
- Vaswani et al. (2017) introduced the Transformer
Other papers are more specialized. Of them, I think Kaplan et al. (2020) by OpenAI is probably most important.
Even if Ilya didn't really create this list I have a very good opinion about every paper on this page that I've read (most of them) so I think it's a great resource.
Lately during my off time I want to do something related to AI research (which I am already doing full time atm so I need something light) and I am for sure going to read through this.
It does not seem that the author cites the source of the control theory map. It was created by Brian Douglas [1], an engineer whose YouTube videos [2] are great for learning core topics.
Also useful is Steve Brunton's channel [3]. He has a freely available book [4] co-authored with Nathan Kutz that ties machine learning and control.
> Maybe it works because the sequences are short and the dimension is high and there's plenty of room for interesting results to fit in the merged key/value space.
In fact, on the second last page of the paper, they discuss this very problem. There is a clear correlation between performance and increasing sequence lengths for the Q-K=V model. While limited to a tight n=3 sample between 512, 1024, 2048 lengths, the degradation decreases from 5.4% to 2.2% as context is increased, suggesting that it is unlikely shorter sequences are the reason K=V performs acceptably.
While phrases like "contemporary phenomena" and "tendency to be commensurate" are a bit over-the-top, summarily the picture is conveyed much more clearly imo: Luck plays a factor, no matter how good you are.
I don't see how "nor yet favor to men of skill" is more natural or clearer, at least to the modern reader.
reply