I once worked in a meatball factory. I touched almost every single meatball with nitrile gloves. There's probably a lot of earlier process steps where humans are touching the food with gloves.
Is there an error in the visualization? It shows that every vector is rotated the same amount. My understanding was that they are randomized with different values, which results in a predictable distribution, which is easier to quantize.
That's actually correct and intentional. TurboQuant applies the same rotation matrix to every vector. The key insight is that any unit vector, when multiplied by a random orthogonal matrix, produces coordinates with a known distribution (Beta/arcsine in 2D, near-Gaussian in high-d). The randomness is in the matrix itself (generated once from a seed), not per-vector. Since the distribution is the same regardless of the input vector, a single precomputed quantization grid works for everything. I've updated the description to make this clearer.
Thanks. However, from this visualization it's not clear how the random rotation is beneficial. I guess it makes more sense on higher dimensional vectors.
I believe they are all rotated by the same random matrix, the purpose being (IIUC) to distribute the signal evenly across all dimensions. So effectively it drowns any structure that might be present in noise. That's essential for data efficiency in addition to avoiding bias related issues during the initial quantization step. However there are still some other issues due to bias that are addressed by a second quantization step involving the residual.
That said, I don't believe the visualization is correct. The grid for one doesn't seem to match what's described in the paper.
Also it's entirely possible I've misunderstood or neglected to notice key details.
I think electric motors are just barely suitable for mobile / humanoid robots. Compared to human muscles, they're heavy and they overheat quickly. Current humanoid robots are spending most of their energy in carrying around just themselves, which is a huge amount of dead weight, while being unsafe for anyone to be around.
If someone invents a new type of 'artificial muscle' which has low inertia, high force/torque density, and can work without overheating, that would instantly kill all other robotics companies.
Electric motors are heavy, but they have better energy efficiency than muscles.
The reason why muscles do not overheat as much as electric motors, despite lower efficiency, is because they have good liquid cooling, by blood. The cooling system of animals has outstanding reliability, due to self repair. Liquid cooling is also possible for electric motors, but it is usually avoided due to high cost and low reliability.
The weight problem of the electric motors is solved by keeping them in the body of the robot and transmitting the movement towards the moving parts that need low inertia by various means (ropes, cables, levers, hydraulic/pneumatic fluids).
This is also done with muscles, which frequently are far away from the bone that they are moving and the movement is transmitted by long thin tendons, to reduce the inertia of the moving limb.
Electric motors can have quite a low efficiency in slow speed / high torque regime, where robots operate, because torque scales linearly with current, and power loss (resistance -> heat) scales quadratically to current (P=UI, U=RI -> P=RI^2). In addition, when the motor is statically loaded, when it's just holding a position, it wastes a lot of power (depending on the torque) with zero efficiency, if it doesn't have mechanical brakes.
I think only 1X is using tendon drives for all (?) joints in their robots, but most robots use them just for hands. 1X uses tendons just for moving the motor one link upwards the chain, not through multiple joints to the body (elbow motor is in shoulder etc.). Transmitting power from the body to remote joints with tendons is quite hard problem, and I'm not aware of anyone doing it this way. One problem is that there is no obvious way to decouple the motion of tendons when they go through a joint, e.g. if the elbow joint moves then it affects all tendons which go through the elbow. Also, it's rather complex and there is friction, stretching and vibration and so on, which might be hard to model / simulate.
Hydraulics might work, but Boston Dynamics gave up on it, so I think it's probably not worth it. Hydraulics could in theory be very good because it needs just one motor for the pump and the fluid can distribute the power to many actuators.
Moving the motors to the body is a good way to solve the mass / inertia problem, but no one has really figured out how to do it. Whatever 1X is doing might be the sweet spot.
Is this a joke? Growth is caused by consumer demand, and answering to it in the best way possible. It's not an 'obsession' of the capitalist businesses or countries. Capitalism is simply an algorithm which tries to allocate resources such that a growing demand can be answered, and that everyone has their needs satisfied.
There are plenty of people who don't have enough food, enough housing, or other basic needs. The only way to solve this is growth. People in developing countries upgrading their lifestyle to modern standards is a huge source of growth. Are you saying that all these people should stop their obsession with growth?
Edit:
I've got to add that it's true that economic policies of countries are 'obsessed' with GDP growth. It's a problem of central planning, not markets.
Inflationary monetary policies are specifically designed to accelerate consumption, and as a consequence a lot of economic activity is allocated serving this fake demand, resulting in cancerous growth, i.e. consumption for the sake of consumption, while in real terms people are poorer than before.
Businesses need, or at the very least strongly want, to increase profits. There’s not like there’s any end to that supposed algorithm. And it gets harder and harder as time goes on.
And as usual the cart is put before the horse. Businesses just don’t answer to consumer demand. They very strongly set the terms for it. And if consumer demands go down? That’s what the marketing industry is for. To create new demands.
Some Hank Hill person isn’t the one who designed e.g. America to be car dependent. The car lobby did. But typically Hank Hill gets blamed when he chooses to live two hours from his workplace because home prices are too high, taking the bus takes twice as long and train does not exist so he uses a car to commute, and he consumes beer in his freetime to unwind from the job he chooses to work at, and he isn’t great at recycling. (This might have deviated a little from the real-life Hank Hill.)
A lot of growth activity in businesses is zero sum, if it isn't increasing the efficiency of production. Businesses can't create demand from thin air with marketing.
Total productivity of the world is number_of_people * productivity_of_human. There's growth whenever these terms grow. People want to produce at least as much as they want to consume. So, growth is caused by more people being born, or people adopting more efficient methods of production, up to a limit where their all needs are met.
> Businesses can't create demand from thin air with marketing.
I would never suggest ex nihilo demand. There’s always some seed.[1]
Women back in the day didn’t smoke. Untapped market. A marketing campaign convinced a lot of women that smoking was something that liberated women did.
I’d say convincing people that replacing clean air with carginogenic fumes gets pretty close to a manufactured want.
[1] For cigarettes: maybe stress relief from nicotine.
I mean, doesn't that fall out of pricing externalities sort of naturally? What's the difference if you're told all new cars cost $500k, vs being told not to buy a new car? You can't reasonably afford it either way.
Yes, prices would limit consumption of stuff that has externalities priced in, but also push producers to reduce their externalities, if they're littering the world with poisons etc. It's not $500K, but something reasonable like $5K (for cars) which reduces demand.
Is it really worth having separate GPU and NE? Seems redundant and weird compared to what Nvidia is doing, i.e. "GPUs are good NEs", or is that not really true?
No, GPUs are not what you'd design for neural networks from first principles. They were adopted for that because they offered far more parallelism than general purpose cpus, not because they're ideal. That's why Google et all designed TPUs that have a very different internal structure.
Most TPU designs have been based around systolic arrays, which for matrix ops have a quadratic speedup. A typical design is a 128x128 array of MAC units. You shift weights along one dimension, parameters along the other. It takes 128 cycles to shift a full matrix input in, then 128 cycles to shift the answer back out, but during those 256 cycles you got 16,384 MAC operations done, for a factor of 64 speedup.
The other big appeal of this design is it's way simpler than GPUs. The memory access patterns are predictable, there's no threads or thread divergence, etc. So it can be way more efficient in silicon, not just in area but especially in power efficiency.
There's other ideas for architectures besides this basic systolic array idea. If you want to learn about them, a good place would be the HotChips presentations of the last few years: https://hc2025.hotchips.org and similar domain names for prior years.
When you already have a GPU in a system, adding tensor cores to it is much more efficient than adding a separate NPU which needs to replicate all the data transfer pipelines and storage buffers that the GPU already has. Besides, Nvidia's tensor cores are systolic.
You might want to take magnesium with Vitamin D, because taking vitamin D depletes magnesium. Not sure if it's strictly necessary, but if you're already low on magnesium it might be an issue. I once got tinnitus (which lasted a few months) when supplementing Vitamin D, the only explanation I've figured is that my magnesium was very low, which can cause tinnitus. Might be something else too, who knows.
Meta uses dark patterns to inflate Threads DAU. If you install Threads, it starts sending notifications for suggested posts every day. It also sends notifications on Instagram, and when you click it, it opens a random post on Threads. I don't follow anyone on Threads.
reply