How is that any different from the pre-llm days, when Jim was using stackoverflow to build the largest crypto exchange in the world? Where's stackoverflow accountability?
At least for me, the answer is that despite the mistakes and the sheer annoyance the prose causes me, they are unbelievably useful. I accomplished multiple major achievements in the last two years that most probably wouldn't be possible at all, surely not within that timeframe.
The idea is that by the time you will have time and remember the clothes might be smelly and wrinkled. The issue is with the genius product manager that decided the washing machine should have the most annoying beep possible, repeating every minute whether you like it or not, until turned off. Luckily, some manufacturers do employ better product manager.
You have an answer on your page regarding "Should I pick 26B-A4B or 31B?", but can you please clarify if, assuming 24GB vRAM, I should pick a full precision smaller model or 4 bit larger model?
Try 26B first.
31B seems to have very heavy KV cache (maybe bugged in llama.cpp at the moment; 16K takes up 4.9GB).
edit: 31B cache is not bugged, there's static SWA cost of 3.6GB.. so IQ4_XS at 15.2GB seems like reasonable pair, but even then barely enough for 64K for 24GB VRAM. Maybe 8 bit KV quantization is fine now after https://github.com/ggml-org/llama.cpp/pull/21038 got merged, so 100K+ is possible.
> I should pick a full precision smaller model or 4 bit larger model?
4 bit larger model. You have to use quant either way -- even if by full precision you mean 8 bit, it's gonna be 26GB + overhead + chat context.
For the best quality reply, I used the Gemma-4 31B UD-Q8_K_XL quant with Unsloth Studio to summarize the URL with web search. It produced 4.9 tok/s (including web search) on an MacBook Pro M1 Max with 64GB.
Here an excerpt of it's own words:
Unsloth Dynamic 2.0 Quantization
Dynamic 2.0 is not just a "bit-reduction" but an intelligent, per-layer optimization strategy.
- Selective Layer Quantization: Instead of making every layer 4-bit, Dynamic 2.0 analyzes every single layer and selectively adjusts the quantization type. Some critical layers may be kept at higher precision, while less critical layers are compressed more.
- Model-Specific Tailoring: The quantization scheme is custom-built for each model. For example, the layers selected for quantization in Gemma 3 are completely different from those in Llama 4.
- High-Quality Calibration: They use a hand-curated calibration dataset of >1.5M tokens specifically designed to enhance conversational chat performance, rather than just optimizing for Wikipedia-style text.
- Architecture Agnostic: While previous versions were mostly effective for MoE (Mixture of Experts) models, Dynamic 2.0 works for all architectures (both MoE and non-MoE).
This is one of the more confusing aspects of experimenting with local models as a noob. Given my GPU, which model should I use, which quantization of that model should I pick (unsloth tends to offer over a dozen!) and what context size should I use? Overestimate any of these, and the model just won't load and you have to trial-and-error your way to finding a good combination. The red/yellow/green indicators on huggingface.co are kind of nice, but you only know for sure when you try to load the model and allocate context.
It should, opkssh just creates ssh public keys. The integration tests don't current cover that case so I created an issue to add that to the integration tests:
One can still hold that it's a better link, it's a matter of preference. Anyways, if you scroll the original article to the very end it does contain an impressive set of photos. It is beautiful.
I can't find any viable alternative. Keyboard is much faster than those click and release interfaces. Keyboards also have repeat keys, when you press a character for a long time you can actually press and depress the shift key and see the change in the line of characters input. This is extremely useful feature in games, graphic design software and other applications.
Generation of keystroke based on the up event, beside been incompatible with repeating keys for long strokes, will slow down typing significantly, as it requires tracking timing pressing keys for longer duration. I'm pretty sure that this isn't only effect of me being used to track keypress timing on the way down, but an unavoidable result of the duration of the action.
Waiting for up event on contemporary GUI, when the contempt UI is a sluggish fit-to-nothing dirty touchscreen in a public kiosk is sensible. When you know an interface will yield more errors than intended input it is only sensible to assume that any input is a mistake unless the user is making an effort to validate it.
Keyboard repeat is only useful in ANSI terminal games on Unix, and games on some old 8 bit home computers that didn't have up and down events (Apple II+, ...).
A game written for an IBM PC and everything after that can know exactly which keys are being held down and when they are released by intercepting the "scan codes" (or abstract keyboard events in a GUI event loop).
All that is missing is synthesizer-like velocity and pressure info. :)
> will slow down typing significantly
Only for people who have to look at the screen. :)
The hunt-and-peck beginners who look at their fingers are not effect, and neither are those who can look at something else or close their eyes.
A serious concern affecting even those people is that using the release event could reorder things, causing mistakes. Like say if someone types the sequece wh (involving two hands) such that they release the w later than h.
While I didn't find the joke funny, it does thematically match the piece - the hacker who supposedly see the possibility to get free internet as a viable opportunity. Later in the piece the author does distance himself from that image, revealing the tone in the opening was merely a stylistic choice, a writer's device, as clearly he is not the kind of a person who will in practice exploit the airline systems.
Assuming you are a man, I'm not sure you would say the same thing if the arbitrary lingual fact was opposite, and on your door it would be written in bold chairwoman, or if you were been introduced as a policewoman.
I'm not sure I'm for changing the language all over, but I don't think dismissing issues that disturb a group that you don't belong to is a manner that fits a gentleman.
reply