Not the GP but the new Qwen-Coder-Next release feels like a step change, at 60 tokens per second on a single 96GB Blackwell. And that's at full 8-bit quantization and 256K context, which I wasn't sure was going to work at all.
It is probably enough to handle a lot of what people use the big-3 closed models for. Somewhat slower and somewhat dumber, granted, but still extraordinarily capable. It punches way above its weight class for an 80B model.
Agree, these new models are a game changer. I switched from Claude to Qwen3-Coder-Next for day-to-day on dev projects and don't see a big difference. Just use Claude when I need comprehensive planning or review. Running Qwen3-Coder-Next-Q8 with 256K context.
"Single 96GB Blackwell" is still $15K+ worth of hardware. You'd have to use it at full capacity for 5-10 years to break even when compared to "Max" plans from OpenAI/Anthropic/Google. And you'd still get nowhere near the quality of something like Opus. Yes there are plenty of valid arguments in favor of self hosting, but at the moment value simply isn't one of them.
Eh, they can be found in the $8K neighborhood, $9K at most. As zozbot234 suggests, a much cheaper card would probably be fine for this particular model.
I need to do more testing before I can agree that it is performing at a Sonnet-equivalent level (it was never claimed to be Opus-class.) But it is pretty cool to get beaten in a programming contest by my own video card. For those who get it, no explanation is necessary; for those who don't, no explanation is possible.
And unlike the hosted models, the ones you run locally will still work just as well several years from now. No ads, no spying, no additional censorship, no additional usage limits or restrictions. You'll get no such guarantee from Google, OpenAI and the other major players.
IIRC, that new Qwen model has 3B active parameters so it's going to run well enough even on far less than 96GB VRAM. (Though more VRAM may of course help wrt. enabling the full available context length.) Very impressive work from the Qwen folks.
The brand new Qwen3-Coder-Next runs at 300Tok/s PP and 40Tok/s on M1 64GB with 4-bit MLX quant. Together with Qwen Code (fork of Gemini) it is actually pretty capable.
Before that I used Qwen3-30B which is good enough for some quick javascript or Python, like 'add a new endpoint /api/foobar which does foobaz'. Also very decent for a quick summary of code.
It is 530Tok/s PP and 50Tok/s TG. If you have it spit out lots of the code that is just copy of the input, then it does 200Tok/s, i.e. 'add a new endpoint /api/foobar which does foobaz and return the whole file'
I personally have just 20% of my net worth in stocks, as they seem very expensive right now. A crash would allow me to increase my allocation at reasonable prices.
With stock prices divorced from reality, the ones who benefit are the having the funds to buy in volume, the gamblers, and the ones hyping the stocks and creating the illusion of profitability and growth. Years ago it would have been unthinkable to have so many unprofitable companies with unclear path to profitablility having such a high valuation, but we have normalized frenzied gambling as a society.
The current absolute balloon of a market is about to pop, and sadly, the people who hyped the stocks are also the ones knowing when to jump ship, while the hapless schmucks who believed the hype will most likely lose their money, along with a lot of folks whose retirement investment funds either didn't due their diligence or were outright greedy.
In a way, as a society we deserve this upcoming crash, because we allow charlatans and con people like Musk, Zuck and Sam to sell us snake oil.
I suppose if you’re operating on the assumption that tech stocks are vastly overinflated then this makes sense. Otherwise I would expect the people that are regularly buying these securities would be happy that they’re increasing in value, no?
No. The 'goal' of investing (e.g. regularly buying) means attempting to own as many shares as possible. That is achieved by buying low and selling high. Buyers benefit from lower prices, not higher.
So many investors get this concept wrong. I suppose they get excited because what they bought went up in value and they have a sense of being enriched. But, that is backwards. That is what they want 20-40 years from now when it will almost certainly be the case that prices are not just higher, but much higher, than today. But, when they are buying shares, the goal is to pay the lowest price possible. If I am 20 years old, I am screaming: crash and burn baby! Crash and burn! Gimme those shares at 50% off yesterday's price.
> I am screaming: crash and burn baby! Crash and burn! Gimme those shares at 50% off yesterday's price.
Sure, but once you reach the point where you have a lot of money in the market you probably won't enjoy watching 50% of it disappear, even if it means your next auto investment is for a nice bargain price.
Also, when the stock market crashes usually bad things accompany it. Like a depressed economy and job losses.
>>Sure, but once you reach the point where you have a lot of money in the market you probably won't enjoy watching 50% of it disappear, even if it means your next auto investment is for a nice bargain price.
I assume I am investing to build wealth. That means my goal is to never spend down my wealth. When I retire, I am withdrawing a maximum 4% a year and expect my portfolio to average >6% per year. When I die I will own the largest number of shares I ever owned in my lifetime (assuming for simplicity sake I own a total stock index fund as my sole investment).
So, my goal remains to celebrate buying low since I never intend to sell shares (how this is managed upon retirement is a slightly more complex subject, probably involving 'buckets' of assets to cover withrawals so a 50% crash doesn't change the overall thinking that the price of shares is irrelevant to stock that will never be sold).
edit: speaking theory when I say "when I retire" because I've already been retired for almost a decade. My portfolio continues to grow (highest ever literally at yesterday's close).
Never spend it, but you're ok with it being taken from you? You're a special case retiree if you can watch the market take half of your life savings and cheer it on. Especially with a 4% withdrawal rate, which fails a lot of 30 year backtests.
> I am withdrawing a maximum 4% a year and expect my portfolio to average >6% per year. When I die I will own the largest number of shares I ever owned in my lifetime.
A lot of successful backtests end with significantly fewer shares, too. There are no guarantees here.
There has to be some floor where you stop cheering on a market crash, because if it drops low enough for long enough then you are screwed and so are your heirs.
> Also, when the stock market crashes usually bad things accompany it. Like a depressed economy and job losses.
It's our own fault for tying the stock market performance to our economy's performance. Why would I, a train worker, should have my pension affected by Sam a
Altman's bad decision making or by Enron's lies and deception.
It's our own fault that the stock market is so volatile and that we tie so much of our economy to a financial gambling machine that's become increasingly divorced from reality in the last couple of decades. Like you are putting money on a stock that trades at 1000 on a company that is 10 years away from being profitable? You deserve your money to go poof.
> Like you are putting money on a stock that trades at 1000 on a company that is 10 years away from being profitable? You deserve your money to go poof.
Who is suggesting that?
NVDA trades at 57x earnings, MSFT 37, GOOG 22. The article is about META and they are 27x. These are the big companies that dominate the s&p that we're talking about.
I don't think anyone is suggesting to put their life savings into Anthropic. They can't anyway, it's not public.
The s&p PE is 30, which is high, but still lower than it was in 2020 before the AI "bubble" started.
The ponzu scheme of SPY is great until it stops. 10% of America’s payroll gets lumped into it each month and generational wisdom is you get a 10% ROI despite the economy growing 2%.
At some point that will collapse, and it won’t be pretty.
Valuation of companies tied to their real current profits! If a company is unprofitable now, it doesn't make sense and is wholly wrong that its stock is trading 1000x more than other companies which actually turn a profit.
The difference from public ownership to public gambling is huge in its impact to society, especially when the markets crashes.
So rather that someone auto-investing a slice of their paycheck into a s&p fund in their 401k, they should instead learn how to evaluate company financials so they can pick winners from a non tax advantaged account?
This is a losing strategy for the large majority, and it's been demonstrated repeatedly that even professional investors can't beat the market especially after considering fees.
Is that not the point of carbon offsets, at least in principle. If you emit X and sequester Y then it would be correct to consider your net emissions to be X - Y.
P.S. I don't believe it's "sequester Y", but something more like "promise we wouldn't burn Y that we also promise was otherwise definitely going to be burned". I could be off here, feel free to double check.
The news article you’ve linked seems to suggest that these scam offset companies may soon be losing their ability to sell their credits.
Carbon offset startups are booming right now, not all of them will succeed, but there are viable technologies already being scaled up. For example, Heirloom Carbon has just made front page news today with a 200 million dollar deal they signed with Microsoft.
All this is to say, I agree that there are some bad actors in the space, and it’s great that they’re being called out for it. To suggest they’re all scams is an unfair, and largely inaccurate statement.
> To suggest they’re all scams is an unfair, and largely inaccurate statement.
It's jarring to read this when earlier you believed carbon offsets represent actual sequestration, and I pointed out that wasn't the case.
> Carbon offset startups are booming right now, not all of them will succeed [...] For example, Heirloom Carbon has just made front page news today with a 200 million dollar deal they signed with Microsoft.
Are you aware of how plastic recycling was a scam? or do you believe that is a "largely inaccurate statement" too? Businesses "booming" or signing massive deals with each other in no way tells you anything about the planet benefiting as a result. If anything, it tells you they've found a very effective way to spend money on PR. Historically that hasn't frequently been a good sign for the planet.
> but there are viable technologies already being scaled up
There were viable plastic recycling technologies already being scaled up too.
* In terms of cooking stability - saturated fats > monounsaturated fatty acids > polyunsaturated fatty acids.
* The less stable the cooking oil, the more damage (oxidation?) occurs while cooking, the more inflammation they cause
* Seed oils are generally to be avoided, especially ones that advertise a high smoke point. This is due to a highly industrialized refinement process that damages the oils (see previous point)
* High smoke point avocado oil is included in the above category (avoid)
* Extra virgin avocado oil is good (but don’t cook with it)
* saturated fats are healthy
* high carbohydrate intake is bad for your arteries
He recommends the following for cooking:
* ghee
* butter
* coconut oil
* olive oil
I’ll second this.
It really shines when learning new technologies in a field in which you already have some expertise. Being able to apply the smell test, or quickly validate the output is important.
I haven’t tested this, but I would imagine the type of information you’re requesting dictates the quality of response you’ll get. In my case I use it mainly as interactive documentation for different tooling. Asking it to explain the process for synthesizing new hypothetical polymers likely will not yield useful results.
It’s worth noting that my experiences are with GPT 4.
I've been using it a lot while working through two very math-y books right now - Princeton's Companion to Mathematics and Data-Driven Science and Engineering. To your point, I have some background in undergraduate level mostly applied math, but there are a lot of things where I don't understand the argument or immediately have an intuition for what's going on, and I've found gpt-4 to be actually really helpful with that. Sometimes it is wrong, but it can't totally BS me, because I'm sitting there looking at the correct equations and proofs, so if it's wrong I always know what's right. But a lot of times it really helps me with intuitions on things that if I was purely just self-learning, outside a class with a professor and TA, I'd probably just accept the poorer level of understanding and move on, instead of investing the time in trying to find what I'm looking for on the internet.
“Crypto” is a technology that is functional and has proven utility.
Those using it as a tool to skirt regulations, defraud others, or to commit outright theft are the scammers.
I would agree that as a concept it is generally overhyped, usually by those who seek to profit from doing so. But I think it’s important to not conflate the unscrupulous gold miners looking to get rich quick with the utility of the precious metal itself.
Not really because the assets can totally be frozen by the exchange. The people who have money in crypto that disappears don't care if that was due to a government action or due to whomever is in charge of enough servers.
