The Qwen family of models are REALLY impressive. I would encourage anyone who hasn't paid them any attention to at least add them to your mental list of LLMs worth knowing about.
QwQ is the Qwen team's exploration of the o1-style of model that has built in chain-of-thought. It's absolutely fascinating, partly because if you ask it a question in English it will often think in Chinese before spitting out an answer in English. My notes on that one here: https://simonwillison.net/2024/Nov/27/qwq/
Most of the Qwen models are Apache 2 licensed, which makes them more open than many of the other open weights models (Llama etc).
(Unsurprisingly they all get quite stubborn if you ask them about topics like Tiananmen Square)
Thanks for the summary. I have been testing QwQ on my M1 (via ollama). I tried a couple double-slit quantum thought experiments, and also found the reasoning mode absolutely fascinating. Occasionally a few logographs appear, but they so far they were not in the way.
The funniest was asking for an ascii graphics depiction of a minecraft watch recipe, and I was actually feeling quite sorry for it, 'wait that can't be right' 'let me try' 'still not right' round and round it went, at least a few pages at which point it decided to try the second recipe I'd asked about to see if that helped with the first.
I didnt know about the other models, 'coder' is downloading now, and fingers crossed it fits in 32GB and knows a bit about Zig.
It sounds like you got the vision one running locally on your M2, nice. I'm running Asahi Linux and not tried anything AI/SD/graphical orientated yet. But nice that you got some SVG out of coder, I never thought of using a coding model in that way.
QwQ often spits out Chinese characters smack dab in the middle of a sentence. Weirdly, it doesn’t break up the coherence or logic, there’s just symbols added.
I haven’t seen the architecture of QwQ but I just assumed it learns languages insofar as to pick up relationships between words. It must mean it picks up logic across languages. Huh
Browser use is very easy. Can even do that headless. That way, you can also do bulk processing. For a client, I did some 16k websites with a simple LLM agent. With “computer use” how long would that take, and what would it cost? For me, it was ~$20 (I used Gemini for this task).
Agree. It is amazing that you can run an o1 style model on a Mac. I was able to run QwQ on my 24GB M3 MacBook Air, though results on complex reasoning on domain specific tasks did not work well, and I saw the Chinese 'thinking' too (they don't work well in o1 either). It opens up experimentation which is great, and the reasoning traces for domain specific tasks for RL is where the next improvements are going to come from
Why does this model think in Chinese and o1 think in English? Is this because chain-of-thought is achieved by training these models on examples of what “thinking” looks like, which have been constructed by their respective model developers, as opposed to being a more generic feature?
I noticed that too, but I haven't seen it think in numbers in Chinese like most bilingual chinese speakers prefers. Or at least I haven't been able to trigger it.
Recently I was scrolling through HF to try a very small model. Fired up Qwen 0.5 B and surprisingly for my purposes it did between that even Lllama 2 7B. That was very surprising to me.
That seems to be just for LLMs, not visual. I'm wanting to go from images of maths notation (photos, scans, digital handwriting) to formulas in Latex or MathML or something. Qwen2-VL can do it, but it's pretty heavyweight for just that.
Qwen2-VL is a decent vision model. You can try it out online here: https://huggingface.co/spaces/GanymedeNil/Qwen2-VL-7B - I got great results from it for OCR against handwritten text: https://simonwillison.net/2024/Sep/4/qwen2-vl/
Qwen2.5-Coder-32B is an excellent (I'd say even GPT-4 class) model at generating code which I can run on a 64GB M2 MacBook Pro: https://simonwillison.net/2024/Nov/12/qwen25-coder/
QwQ is the Qwen team's exploration of the o1-style of model that has built in chain-of-thought. It's absolutely fascinating, partly because if you ask it a question in English it will often think in Chinese before spitting out an answer in English. My notes on that one here: https://simonwillison.net/2024/Nov/27/qwq/
Most of the Qwen models are Apache 2 licensed, which makes them more open than many of the other open weights models (Llama etc).
(Unsurprisingly they all get quite stubborn if you ask them about topics like Tiananmen Square)