Hacker Newsnew | past | comments | ask | show | jobs | submit | more wizeman's commentslogin

If someone replaces the APK on their website with a compromised one, what prevents them from replacing the PGP signature with a signature created by the attacker?


Perhaps we should start admitting to ourselves that infinity is not really something that actually exists but instead it's just an imaginary concept that we created but that has no correspondence to anything that makes sense in our universe?

Or at least, some versions of infinity (I will leave which ones, exactly, to the experts).

Hence all the paradoxes that derive from it.

I mean, perhaps we can work with some version(s) of infinity, without creating a plethora of paradoxes.

The natural numbers, for example, might be something that is infinite and makes sense. However, perhaps talking about a single set that contains an infinite amount of natural numbers might not make sense (or perhaps it still might?). Or perhaps we just need a new language for talking about infinite things without re-using the language we use for describing (finite) sets, so that we don't try to reuse concepts from finite objects with infinite objects.

Or perhaps the only objects that exist (and can be meaningfully talked about) are the computable ones. And/or perhaps "computable function" is one that can use infinite amounts of time but only finite amounts of storage (rather than infinite amounts of storage).

Would it make sense to go back to classical finitism and re-evaluate some of the choices that were made along the way to where we are now?

I can't help but think about the many paradoxes that arise from the very (supposed) existence of infinity.

For example, everybody thinks the halting problem is undecidable. The halting problem is clearly not undecidable on finite-state machines (i.e. real computers), it's only undecidable if you use imaginary Turing machines (which supposedly can have literally infinite state) as a model. But of course, Turing machines cannot exist in our universe.

So it's rather ironic that people say the Halting problem is undecidable, when this is only true for (impossible to construct) imaginary machines but it's not true for the real machines that we actually have and that are possible to construct.

Or think about Hilbert's paradox of the Grand Hotel.

Or the Banach-Tarski paradox.

Or Galileo's paradox.

Or other such paradoxes that only exist if you believe that imaginary infinite objects could possibly exist.

Most people will tell you "well, it's just that our intuition doesn't work for infinite things", but in this specific instance, it's really hard for me to believe that it's our intuitions that are mistaken rather than that we actually made some conceptual mistakes along the way. To me, this argument sounds more like gaslighting rather than a real convincing argument.

I'm not saying our intuition is always correct, though. For example, the quantum properties of our universe are clearly not intuitive but it's hard to argue that our universe is not quantum, after all the scientific experiments that have clearly verified this to be true.

I don't think that invalidates my argument, though, because the convincing argument in that case is not "sorry, you must believe my theory that the universe is quantum" despite not existing any evidence that this was true, but rather, "sorry, but here's all this real evidence that our universe truly is quantum".

Am I crazy here?


The thing is, infinity is extremely useful for taking things to the extreme. The halting problem is a very good example: as you rightly point out, the halting problem is trivially decidable for any finite Turing machine. However, the core of the argument remains true, we just need to complicate it significantly if we want to remain within the realm of finite machines: instead of saying "there is no TM that can decide if any arbitrary TM halts", we need to say something like "there is no finite TM that can decide if any TM smaller than itself halts in an amount of time less than that TM". They are equivalent concepts on some level, but the second one is significantly more complex, and harder to prove (assuming you don't appeal to the infinite version, from which it easily follows).

That is, even if you seek to do maths on finite but unbounded sets, you will find mostly the same properties that infinite sets have, but you'll have a much harder time working with the concepts. It's true that there are some problems that infinite sets have that unbounded finite sets don't, but those may well be a worthy price to pay.

And either way, regardless of intuition, the logic is valid. The Banach-Tarski "paradox" is perfectly logically valid, though unintuitive. So, even if you could say that it's useless and a result that we should avoid having in our theories, you can't say that it's wrong, since it doesn't violate any rule of logic.


I can't really give a substantial response to your first argument, as I'm not certain about all the implications that you mention.

Your second argument, though, doesn't sound very convincing to me.

There are many possible consistent logical frameworks (perhaps an infinite amount of them? hah!) that are clearly not adequate for describing our universe. Or at least, they wouldn't make sense for us.

For a trivial (and perhaps stupid) example, consider an alternate universe where we decided to use a logical framework which can represent natural numbers, but where the digit "2" is represented as "9" and the digit "9" is represented as "7", etc. Let's say you have a 1-to-1 mapping from one digit to another, compared to our usual representation.

Or perhaps the mapping is a lot more complicated than that, and not just about digits, but about entire numbers.

Let's say we decided to do that, for no good reason. Or maybe there was a good reason at the time, but since then we have figured out that it doesn't make sense anymore.

Yes, you could do exactly the same math in this logical framework as we do in our usual ones, and no contradictions would arise.

However, would that really be a wise idea? Wouldn't that just lead to making unnecessary mistakes, sometimes even conceptual ones, when considering that we would be prone to confusing numbers in this system with the numbers that we use normally?

I think language is also important here, not just logical soundness. Especially if we want our intuitions to be helpful.

Just to bring this back to something less abstract again, I will just mention the following:

> However, the core of the argument remains true, we just need to complicate it significantly if we want to remain within the realm of finite machines: instead of saying "there is no TM that can decide if any arbitrary TM halts", we need to say something like "there is no finite TM that can decide if any TM smaller than itself halts in an amount of time less than that TM".

Well, I don't believe the core of the Halting problem to be true, and that's exactly the language problem that I'm talking about: confusing language leads us to making conceptual mistakes.

For example, time is not necessarily relevant here.

I could argue that what the Halting problem really demonstrates is that to analyze whether a finite-state machine (FSM) halts, you need to use an FSM that can have more state than the machine being analyzed (which is why it stops working if you believe you can have infinite state).

This FSM with more state would always be able to decide whether the other FSM halts in finite time, regardless of whether the other machine halts or whether it never does.

In fact, that's exactly what cycle-detection algorithms do, such as the tortoise and hare algorithm.


The biggest problem with finite sets is that most operations don't work nicely in finite sets.

For example, if we take the naturals to be a finite set, and arbitrarily define some number G that is the largest number, then common operations stop being commutative/associative/distributive (e.g. x-y+z is no longer equal to x+z-y, if x>G-z; or x×(y-z) ≠ x×y - y×z). Lots of simple equations stop having solutions at all. The finite rationals also get similar problems. Your models also have all these arbitrary parameters - the largest positive integer, the largest negative integer, the smallest fraction, the largest allowed amount of non-repeating digits in a decimal representation of a fraction - things like these become independent parameters, and can be arbitrarily chosen to be different.

> Well, I don't believe the core of the Halting problem to be true, and that's exactly the language problem that I'm talking about: confusing language leads us to making conceptual mistakes.

But it is - you still can't, in practice, construct a useful program that depends on solving the "finite halting problem", since you will need too much memory/time for anything but the most trivial of examples. If you don't beleive this, than it should be doable for you to create a program that checks whether P=NP if we limit it to programs that can run on, say, an 8086 processor and only use 16 KB of RAM, right?


> The biggest problem with finite sets is that most operations don't work nicely in finite sets.

I didn't necessarily mean to suggest that we should not work with infinite sets, only that perhaps we should not call them "sets" (so as to not confuse them with the finite ones), but rather something else, like "domains" or whatever.

So perhaps you could say that those commutative/associative operations are valid over the domain of natural numbers. But you couldn't say, "X" represents the cardinality of the set of natural numbers, because there's no such thing as the set of natural numbers.

Or, maybe the set of infinite natural numbers and infinite rationals makes sense but the set of real numbers doesn't (as they have different infinite properties?).

I don't know exactly what would be a better solution here, as I'm not a mathematician. I'm only trying to suggest that there might be a better solution than what we have right now, even if it would simply amount to just using different language and not necessarily changing the logical properties of the theory, so as to not arrive at confusing conclusions.

> But it is - you still can't, in practice, construct a useful program that depends on solving the "finite halting problem", since you will need too much memory/time for anything but the most trivial of examples.

But the decidability of the Halting problem is not actually a question of how long it takes for an algorithm to solve it, is it?

The decidability of a problem is a matter of whether it's actually possible at all to construct an algorithm that solves it, no matter how long the algorithm takes to solve it.

That is, whether this algorithm is computable and finishes in a finite amount of steps.

So I will say again that yes, the Halting problem is decidable, as long as you don't use an absurd model to answer the question (i.e. a model that does not represent anything that can possibly be built in our universe even in theory?).

The tortoise and hare algorithm solves the Halting problem for FSMs in a finite amount of time and it only needs twice the amount of storage as the underlying FSM that is being analyzed.

> If you don't beleive this, than it should be doable for you to create a program that checks whether P=NP if we limit it to programs that can run on, say, an 8086 processor and only use 16 KB of RAM, right?

The P=NP is a problem regarding the resources required during computation to solve a given problem, as far as I understand.

I am not entirely sure how exactly this relates to the decidability of the Halting problem when applied to finite-state machines, though.

I mean, it's clear that the currently-known algorithms that solve the Halting problem on FSMs, such as the tortoise and hare algorithm, in the worst case take a time to solve that is proportional to the cycle length of non-halting FSMs.

But also note that these currently-known cycle detection algorithms don't even try to analyze the FSM to decide whether it halts, except by testing their states for equality. This is due to the inherent definition of the cycle detection problem.

So they can definitely be made more efficient, perhaps for a large set of useful FSMs, even, as long as they would be allowed to inspect the FSM's state transition table. I'm not sure how efficient they could be, in theory.

I would be cautious in taking any computational complexity theory results at face value when we know that they use Turing machines as models, which can give you results that are the opposite of the results that apply to FSMs. But I'm not an expert and admittedly, I'm not entirely sure of what I'm talking about here.

Regardless, even if you can trust the computational complexity results, to me this is an entirely different question than the decidability of the Halting problem, which is a "yes/no" problem.


I agree that infinity doesn't exist in the universe, but this isn't a problem for mathematics. Mathematics isn't about what exists in the universe, it's about what exists in the realm of concepts. Some concepts are more interesting than others, sure. Cantor's work made it clear that infinite sets are pretty interesting.


The Flying Spaghetti Monster is also an interesting concept.

It can even be useful as a concept, in certain discussions/contexts.

Should we take the Flying Spaghetti Monster as a core axiom of our theories of physics?

Wouldn't this lead to us logically concluding for example, that there has to exist something else beyond the universe that we know about, even if this is not really true?

I mean, I know that I'm exaggerating and that the analogy is not necessarily very good.

But I'm also not entirely sure how different is the Flying Spaghetti Monster from the infinite objects that mathematicians talk about and that lead us to logically arrive at certain conclusions (that I would argue might not really be true, in terms of things we can understand).

I'm not saying that I'm right and that most mathematicians are wrong, necessarily. Perhaps it's just a linguistic issue, I don't know.


> You have to shield your home from your neighbor's emitters too. And from the police radar surveillance van sitting in the street in front of your house.

That's what tin foil houses are for, obviously.


And for saving them from raging wildfires. (Seriously! Wrapping a structure completely in foil can save it, even when that structure is in a forest with no clearing arund it. I guess it's like baking a potato in hot coals.)


> there's a lot that had to go right with the Earth for it to support not just life, but complex, high-energy life, which enables (expensive-in-energy-terms) intelligence

Like what?

> It's way more than just being the right size, the right distance from a not-too-dangerous star, and having water.

Besides that and having carbon (and nitrogen, I suppose), perhaps some small amounts of other elements, what are you referring to?


Basically we have a bunch of things going on that cause some graphs to cycle rather than run away, and if you break any of them—which can be as simple as tweaking a value here or there—most of our biosphere's gone.

Carbonate-silicate cycle breaking is what's considered the most likely thing to end complex life on Earth, for instance. All it takes to cause that is the Sun getting slightly brighter. In a few hundred million years it'll become increasingly difficult (inefficient) for plants to photosynthesize, reducing energy available to the biosphere, until finally photosynthesis stops completely.


That's very interesting, thanks.

So you're saying that in such a scenario, it would be impossible for complex life to exist, even if it looks different from the complex life we have now?

I am just wondering whether we are not confusing "complex life as we've seen to exist" with "complex life that can possibly exist".

For example, I am wondering whether the Great Oxidation Event is not a counterexample where we had a run-away biological process that caused a mass extinction (which, if it were possible for us to be alive at the time, might have looked like it would be impossible to recover from) but still lead to complex life existing despite oxygen being so toxic (at least from the point of view of the kind of life that existed at the time).

I know this is not the best example, as at the time complex life didn't exist yet. But I'm wondering if it wouldn't have been possible for complex life to exist in an oxygen-poor environment. Well, in fact some multicellular species that exist today are anaerobic and therefore do thrive without oxygen, right? Perhaps such anaerobic life would have evolved a lot more if our atmosphere had never been oxygenated.

Unfortunately I know nothing about the examples you mentioned, so I can't tell whether they are comparable.

Edit: I just read a bit about the carbonate-silicate cycle and I think I know what you mean now. Yes, if surface temperatures were more extreme it would indeed be quite difficult for complex life to exist (at the surface, at least). After all, some limits to the existence of interesting life must surely exist and temperature does seem like an inherent one.


Carbonate-silicate cycle breaking means photosynthesis can't, chemically, happen, not just that its being hotter makes plants less happy.

My point isn't that life can't exist otherwise (ocean vents exist—though, you do need an active core for that—and anaerobic life is a thing, as you point out) but that only certain routes, even assuming there are some wildly different paths that might work, will yield enough energy to support complex life. You need a system that's reasonably stable (i.e. tends to prevent a variety of runaway events, turning them into cycles instead) even when life itself is changing the environment. Sometimes life will step in to do that if you get lucky (as in the oxygenation crisis, which you mentioned) but the planet itself needs a bunch of stuff to be just right, too—not just conditions, but processes.

I suspect there are lots of planets with some bio-goo on them, maybe even quite a few with things as complex as some insects or crustaceans. I further suspect there are very few (perhaps about as close as one can get to "none" without reaching it) that could support something like (even for generous values of "like") our giant tree of vertebrates. Not enough energy available, or, there is, but the world lacks the processes to stop life's own effects from wrecking the environment and stunting its own potential.

Then there's the fact that even complex life doesn't seem to necessarily lead to advanced intelligence and civilization. I mean, look at how long it took humans to develop writing! All that other stuff had gone just right, and intelligence emerged, and we happened to be a tribal species (probably), but we still puttered around for, what, a few hundred thousand years? The dinosaurs had no space program, nor the birds, nor the whales, nor fish, despite existing for tens or hundreds of millions of years—intelligence and the right kind of cooperative pack/tribe behavior to let it flourish seems to be a really rare combo, even when everything else has worked out very well for complex life.


It's very rare and extremely interesting for me to read a well-argued opinion that is the polar opposite of mine.

Furthermore, I don't even have any good counter-arguments (at least not any that wouldn't just be pure speculation).

You definitely gave me something new to think about.

Thank you for your thoughts!


Oh I mean I could be way off, but the more I learn about how the Earth functions the more it seems like it's a whole bunch of spinning plates and upsetting any of them to the point that they stop spinning means the whole party (more or less) stops. There's certainly more to it than being roughly the right size, roughly the right composition, and in the "goldilocks zone". The only reason it works is that the plates (to continue the metaphor) self-stabilize when they get bumped, on account of the spinning. Take away even one of them and one of our cycles that keep things nice & cozy for complex life instead goes up-and-right or down-and-right... permanently.

Things like plate tectonics to periodically send material down to the mantle, pull fresh material back up, and pump all kinds of stuff into the atmosphere, are really important for maintaining long-term stability. Imagine how much less life the oceans could support without rivers dumping disolved nutrients into them all the time—and those rivers have nutrients to dump, even billions of years into our planet's history, because of plate tectonics, because there's a water cycle at all, et c.

There are other filters, too. Hard for an aquatic intelligent species to kick-start civilization—how will they get started using fire? One can come up with ways, maybe, kinda, but would they stumble on those methods the way we stumbled on fire, almost effortlessly (then still took many millennia to discover how to do much more with it than burn forests/grasslands, cook food, excavate shallow caves, and harden wooden points)? For some high-clouds ecosystems (as some speculate may exist) on planets that can't support (much) life at ground level... how would an intelligent species there build anything? The only materials available would be parts of other high-altitude life forms. That's assuming you could get enough biomass up there to supply the kind of readily-available energy and nutrients an active, complex, smart life-form needs to be able to gather from its environment in the first place. That sort of thing. Plus you're counting on the emergence of physical features necessary for tool use at all (like our hands) which is something that pretty much only climbing species develop, as a side-effect of needing to grip things all the time.

I think there are lots of things working against the emergence of not just complex life, not just intelligent life, but coordinated civilizations with long-term retention and transmission of knowledge that can accomplish things we might associate with advanced alien life forms. Even if life turns out to be very common, I expect complex life is not, intelligent life even less so, and civilization vanishingly rare. But, again, I could be wrong—it's all guesswork, really.


If it hadn't been for the K-T event wiping out the dinosaurs and creating a niche for mammals and eventually primates to thrive, we probably wouldn't be here.

It doesn't get discussed enough how much of a factor blind stupid luck was in the one example we have of the evolution of intelligence in the universe.


But the question was not whether we would be here.

What's to say that in the absence of the K-T event, some dominant dinosaur species wouldn't have eventually evolved their intelligence to our level? Perhaps even earlier?

It's not like the K-T event is a prerequisite for "general" intelligence, is it?

Perhaps in such an alternative universe where the K-T event didn't happen you'd be a dinosaur posting on Dinohacker News right now.


I don't know, dinosaurs were around for hundreds of millions of years or more, and while they evolved, they never evolved intelligence. It only happened when a catastrophic event disrupted the natural equilibrium. I can't help but wonder if intelligence isn't something that tends to arise under extreme circumstances.


there's really no saying. mammals have been around for ~180 million years, so ~112 million before the mass extinction. The dinosaurs were on earth for about 165 million years.

Given the relative quickness that intelligence exploded in later hominids, who knows. we could know about some gibbon analog in the dinosaurs right now, that given just a little longer could've made it to human levels of intelligence.


What does the 'single' profile do in btrfs?

I thought it was equivalent to RAID-0 mode, but apparently it isn't, because they are separate options and btrfs switched from RAID-0 mode to 'single' mode by default, although the 'mkfs.btrfs' man page doesn't explain what's the difference between them nor why btrfs did that. It has helpful diagrams for the RAID modes, but not 'single' mode.

So how does btrfs allocate data among disks with this profile? This is not explained anywhere I could find. Is it just like concatenating a bunch of disks together?

If so, yes, you would expect corrupting a single disk would completely corrupt a corresponding percentage of the files, but almost nothing on others, assuming the files were 1) written perfectly in order (i.e. were not fragmented) and 2) they were filling all of the disks. In this scenario, completely corrupting one of N disks would correspondingly completely corrupt 1/Nth of files (approximately speaking).

If you were using RAID-0 mode, however, it would be different: it would corrupt almost all (large enough) files, but only 1/Nth of their contents.

The results, however, could vary a lot depending on how file data actually ends up being allocated on disk, of course.

That said, note that the results of this experiment can be completely misleading due to most of the data still being cached in RAM. I would say that to conduct this experiment more meaningfully you should:

1. "umount" the filesystem after having written all data to it.

2. Corrupt the disk.

3. Run "echo 3 > /proc/sys/vm/drop_caches" (a couple of times, just in case). This should get rid of the still-cached blocks in RAM, just in case. Or better yet, just reboot your machine :)

4. Mount the filesystem and check for corruption.

At least, that's what you should do if you were performing this test on ZFS, otherwise corrupting the disks would not affect ZFS's cache and you wouldn't necessarily observe the effects of corruption when accessing the corrupted files normally (i.e. not as part of a scrub).

Admittedly, in btrfs this might not be necessary because btrfs' cached file data might be exactly the same as the underlying disks' cached blocks. This is not true with ZFS, however, as ZFS's caches and the Linux kernel one don't share data (at least, not the vast majority of it).

It would also be helpful to know the distribution of file sizes that were copied into the filesystem, otherwise the "successfully read the entire file" vs "got an error reading the entire file" statistics are not as helpful as it might seem at first sight (e.g. if there is a significant percentage of very small files).


> At least, that's what you should do if you were performing this test on ZFS,

Just to clarify: in ZFS you shouldn't just 'umount' the filesystem, you should also export the pool, otherwise the cache would still be in use.


This is very good advice. I did the same preparation, here is the distribution of files before the degraded state:

    Number of successful reads: 280
    Number of IO errors: 0
    Successful read files size: sum 82648303047 max 4884066696 average 295172511
then I unmounted the fs, deleted disk 2, echo 3 > /proc/sys/vm/drop_caches, and remounted the fs.

    sudo umount /mnt/loop
    echo 3 | sudo tee /proc/sys/vm/drop_caches
    echo 3 | sudo tee /proc/sys/vm/drop_caches
    echo 3 | sudo tee /proc/sys/vm/drop_caches

    dmesg --human --nopager --decode --level emerg,alert,crit,err,warn,notice,info
    kern  :info  : [Jan22 13:18] tee (215899): drop_caches: 3
    kern  :info  : [  +3.232287] tee (215931): drop_caches: 3
    kern  :info  : [  +0.775697] tee (215953): drop_caches: 3

    rm d2.img
    sudo mount "$ld1" /mnt/loop
I am surprised that mounting worked without error but I guess the device is still active via losetup. I'm assuming this would be similar to an actual disk failure though, if the device weren't there maybe btrfs will complain and ask to be mounted with the `-o degraded` flag.

There was nothing exciting in dmesg

    kern  :info  : [ +14.363762] BTRFS info (device loop0): using crc32c (crc32c-intel) checksum algorithm
    kern  :info  : [  +0.000004] BTRFS info (device loop0): using free space tree
Oohh weird...

    Number of successful reads: 280
    Number of IO errors: 0
    Successful read files size: sum 82648303047 max 4884066696 average 295172511

    sudo btrfs scrub status /mnt/loop/
    UUID:             a57027e5-feb8-4f58-9022-f5dc0a5c67ac
    Scrub started:    Sun Jan 22 13:33:49 2023
    Status:           finished
    Duration:         0:00:28
    Total to scrub:   77.25GiB
    Rate:             2.76GiB/s
    Error summary:    no errors found
Okay turns out the deleted file is still connected to the loopback device.

    sudo losetup -d $ld2
    sudo umount /mnt/loop
    echo 3 | sudo tee /proc/sys/vm/drop_caches
Now we get some interesting stuff in dmesg

    sudo mount -o degraded "$ld1" /mnt/loop
    mount: /mnt/loop: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
        dmesg(1) may have more information after failed mount system call.

    kern  :info  : [Jan22 13:37] tee (222135): drop_caches: 3
    kern  :info  : [ +16.362674] BTRFS info (device loop0): using crc32c (crc32c-intel) checksum algorithm
    kern  :info  : [  +0.000004] BTRFS info (device loop0): using free space tree
    kern  :err   : [  +0.000419] BTRFS error (device loop0): devid 2 uuid 1b352839-f719-499f-b9a7-25ed4d06e2be is missing
    kern  :err   : [  +0.000003] BTRFS error (device loop0): failed to read chunk tree: -2
    kern  :err   : [  +0.000183] BTRFS error (device loop0): open_ctree failed
    kern  :info  : [ +11.713125] BTRFS info (device loop0): using crc32c (crc32c-intel) checksum algorithm
    kern  :info  : [  +0.000004] BTRFS info (device loop0): allowing degraded mounts
    kern  :info  : [  +0.000001] BTRFS info (device loop0): using free space tree
    kern  :warn  : [  +0.000167] BTRFS warning (device loop0): devid 2 uuid 1b352839-f719-499f-b9a7-25ed4d06e2be is missing
    kern  :warn  : [  +0.007647] BTRFS warning (device loop0): chunk 2177892352 missing 1 devices, max tolerance is 0 for writable mount
    kern  :warn  : [  +0.000002] BTRFS warning (device loop0): writable mount is not allowed due to too many missing devices
    kern  :err   : [  +0.000155] BTRFS error (device loop0): open_ctree failed
But we can still mount it as read-only

    sudo mount -o ro,degraded "$ld1" /mnt/loop
And the results are

    Number of successful reads: 219
    Number of IO errors: 61
    Successful read files size: sum 21798190683 max 2122064756 average 99535117
    IO error files size:        sum 60850112364 max 4884066696 average 997542825
In this test about 26% of data is still fully readable (21798190683 / (21798190683+60850112364)).

I also tried another variant of the experiment where I did all of the above but ran this command before removing the disk:

    sudo rm /mnt/loop/file  # a 500 mb file that was included the above tests. I deleted this to give btrfs defrag some room to work
    sudo btrfs fi defrag -v -r -czstd /mnt/loop/
and the results are not much better... in fact they are worse 20% lol

    Number of successful reads: 199
    Number of IO errors: 80
    Successful read files size: sum 16695157031 max 2122064756 average 83895261
    IO error files size:        sum 65428858016 max 4884066696 average 817860725


OK, so I have a few comments about your experiments:

> I am surprised that mounting worked without error but I guess the device is still active via losetup.

Exactly. `rm` doesn't actually delete the file contents while the file is still open, it just unlinks it from the filesystem tree. So your loopback-mounted disk is still there and all its contents are still available through /dev/loopX.

> I'm assuming this would be similar to an actual disk failure though, if the device weren't there maybe btrfs will complain and ask to be mounted with the `-o degraded` flag.

If the /dev/loopX device wasn't there then it would be similar to a complete disk failure, yes.

> In this test about 26% of data is still fully readable

It's true that only 26% of data is still fully readable if you account only for files that are fully intact. But also note that about 78% of files were still completely intact.

This is not clear from your comment, but I'm assuming that you are using 4 devices for the btrfs pool as well?

In this scenario, with such a disk configuration and subsequent disk failure you would expect to lose about 25% of files, while the remaining 75% would be intact (especially if the files are small enough)...

But actually, in reality things can be quite better or quite worse, depending on a few factors.

For example:

1. If the free space was fragmented. In such a case, a significant percentage of files might actually be allocated on more than one disk, so you'd lose more files than expected if a single disk fails. Although I can see that on your latter experiment, you've defragged the btrfs filesystem beforehand, so perhaps this is not the main issue.

2. Depending on how btrfs allocates data, if the files are not completely filling all of the disks then they can be heavily skewed towards a subset of the disks.

For example, imagine that each of your disks are 1 TB-sized and your files total less than 1 TB.

In this case, all of your files could be allocated on the first disk only, so losing this disk could lead to losing 100% of your data.

Or for example, if your files are less than 2 TB, they might all be allocated on the first 2 disks only, so losing one of these disks would lead to losing a lot more files than you'd expect if files were evenly distributed across all disks.

But on the other hand, if you'd lose one of the other disks, you might not lose any data whatsoever.

3. Depending on how large files are and how much free space there is on each disk, btrfs might be forced to (or might choose to) span a file across more than 1 disk even on the 'single' profile, even if free space was not fragmented.

4. But of course, more generally, how many files you would lose basically depends on how btrfs allocates disk space across the disks for each file.

These disk space allocation algorithms can be quite more complex than you'd expect from a naive allocator, mostly due to performance reasons.

Unfortunately, I know exactly nothing about how btrfs allocates data, so I can't give you more insight than this, sorry!


> 26% of data is still fully readable -- But also note that about 78% of files were still completely intact.

Do you mean partially intact? I did not count that data. Ah I think I understand what you're saying. 78% of files are fully readable (by number of files) but most of those are small files and those are stored within btrfs metadata ("inlined" extents)

I computed 26% using the quantity of data (ie. sum() of file size) which I feel is a more accurate representation of what is readable. Sure, btrfs `single` mode will have many partially readable files--and if that is an ideal failure state then I would recommend it.

I tried the same experiment with raid0 btrfs config and only the inlined extents were fully readable--less than 1MB recovered from 80GB of data.

> you would expect to lose about 25% of files, while the remaining 75% would be intact

That's what I expected from btrfs for that last 9 months and people online were saying that `single` mode is the same data guarantees as `raid0` mode--which is kind of true but also it is kind of not as we can see. It's true but not likely that in a highly fragmented filesystem the spread of data in `single` could be a similar shape to `raid0` and in that case you could only easily recover the same amount of data (almost none).

What happens in practice is that btrfs will allocate 1gb blocks one drive at a time but, in a multi-disk setup, it writes file extents to multiple disks at a time. So at the file level there are no guarantees about one file being on one disk. This is why I was only able to read 20~30% of data rather than the 75% you and I both naively expected from btrfs single mode. It's important to note that this 20~30% is not guaranteed--it depends how file extents are saved across multiple disks and that is probabilistic not deterministic.

> the remaining 75% would be intact (especially if the files are small enough)

If all the files are inlined extents (default limit is 2048 bytes per file), and you were using raid1c4 metadata profile, then theoretically you could have 100% intact even after losing 3 of 4 disks (regardless of what the data profile is set to since that would not be used to save the file data)--but you would be using 80GB of allocated as "metadata" space in btrfs. (I have not tested that scenario but I think it is likely to be true). So all of the file redundancy is provided by the raid1c3 metadata configuration which I used for <2kb files in my test but the larger files like the max() 2GB one were recovered due to the chance that the file extents were only saved on one or more of the other three disks.

> Although I can see that on your latter experiment, you've defragged the btrfs filesystem beforehand

Yes, I think btrfs defrag does not do much different from when it writes the files initially, but it is still a useful utility in situations where files were overwritten many times. As I understand it there are many reasons that btrfs will decide to write a file to multiple extents and there seems to be no option to have it write one file to one disk as much as possible

> all of your files could be allocated on the first disk only

maybe a good example would be if I had filled up a disk then added a new one. btrfs really tries to allocate data fairly but adding a new disk is a situation where it would definitely be skewed toward one disk. I was actually thinking of recreating my filesystem and just copying over data one disk at a time so that the file extents would be written more consolidated to each disk--but still there would be no mechanism to prevent cross-disk extent writing...

> But on the other hand, if you'd lose one of the other disks, you might not lose any data whatsoever

yep

> even if free space was not fragmented

That would be ideal but I think it is pretty unlikely in practice unless the 1gb blocks which btrfs allocates per disk are used immediately and no files are appended to or changed then there is lots of free space within each 1gb block for btrfs to find

> Unfortunately, ... I can't give you more insight than this

Your comments were helpful and interesting. Hopefully I could share some of my findings as well. I still like btrfs but it certainly acts like a mad chef who is trying to boil 6,827 pots of water to cook spaghetti in this situation.

multi-disk and `single` profile is a bit weird. I'm planning on switching my array to individual btrfs `single` profile disks with `dup` metadata. I will also try MergerFS to group them into one disk but if MergerFS feels sketchy I'll just interface directly with many disks and balance files between them manually


> I was under the impression running such file systems without ECC was kind of reckless and not a solution either?

It's about as reckless as running any other filesystem without ECC [1].

In fact, you're much more likely to discover memory errors earlier if you use a checksumming filesystem than if you use a non-checksumming one.

[1] https://news.ycombinator.com/item?id=34477035


Thanks! We already had that discussion, your link is your reply to another comment of mine :^)

I was wrong twice huh!


> One of the prime prerequisites of e.g. zfs actually is using ECC RAM.

ECC RAM is not a prerequisite for using ZFS.

Matt Ahrens, co-creator of ZFS and still one of the main developers, said this [1]:

"There's nothing special about ZFS that requires/encourages the use of ECC RAM more so than any other filesystem. If you use UFS, EXT, NTFS, btrfs, etc without ECC RAM, you are just as much at risk as if you used ZFS without ECC RAM. Actually, ZFS can mitigate this risk to some degree if you enable the unsupported ZFS_DEBUG_MODIFY flag (zfs_flags=0x10). This will checksum the data while at rest in memory, and verify it before writing to disk, thus reducing the window of vulnerability from a memory error.

I would simply say: if you love your data, use ECC RAM. Additionally, use a filesystem that checksums your data, such as ZFS."

[1] https://arstechnica.com/civis/threads/ars-walkthrough-using-...


True, I phrased it too strongly. Let's put it this way: zfs (or btrfs, ...) can make for a bullet-proof system through checksumming and self-healing. However, its Achilles heel is non-ECC RAM. With it, at least in theory and excluding universal disaster, data can live perpetually and remain intact indefinitely. Without ECC, zfs remains at the mercy of what the RAM might get wrong. That's what I remember learning about it a while ago.


Yes, that's true.

However, even if you're not using ECC RAM, you're much better off using ZFS or btrfs, because due to their frequent checksumming and checksum validations, these filesystems will usually detect memory corruption much sooner than if you didn't use them.

This could be immensely helpful in scenarios such as the great^4-parent poster.

Note that bad hardware is not limited to non-ECC RAM. ZFS and btrfs help just as much in detecting other kinds of bad hardware, such as bad SATA cables, bad disks, bad disk/SATA controllers, bad CPUs, bad power supply, etc.

But of course, once these checksum errors are flagged by ZFS/btrfs, it's a signal to test and fix/replace your hardware, not keep using a machine with bad hardware.

And yes, while ZFS and btrfs cannot fix errors that happen before the checksumming takes place (e.g. due to bad RAM, bad CPUs, etc), they can still detect these kinds of errors (in some cases, at least), especially when they happen after checksumming already took place. And they definitely can detect and fix errors in the rest of the data-to-storage-and-back path (e.g. bad disk cables, bad disks or disk controllers, etc).


> perf has an option to use DWARF data for stacks, but IMHO it simply does not work.

Could DWARF data be fixed/improved so that this works reliably?

If so, we could have our cake and eat it too (in the future, not right now, of course).


DWARF perf sampling works fine but has really high overhead, requires reading a large amount of stack space to find each prior frame, and the current tooling for generating perf reports from DWARF samples is really slow. You could definitely improve the last issue, I suspect no one has because it's not widely used. I don't think you can fix the first two issues, or at least you can't make it as fast as frame pointers.

Also, frame pointers just use up one extra register. The actual overhead of compiling with -fomit-frame-pointer vs -fno-omit-frame-pointer is small enough for most programs that it's definitely worth including frame pointers if you think you'll ever need to profile a program running as a release build. I'll also add frame pointers are very useful for debugging if you don't have debug symbols, which is more often than you might think.


> Also, frame pointers just use up one extra register.

This comment suggests it's more than that: https://lwn.net/Articles/920165/

My non-expert understanding: It seems that when frame pointers are enabled, the compiler prefers to address all stack variables using offsets from the frame pointer, whereas when frame pointers are disabled, stack variables are addressed at offsets from the stack pointer. But, it turns out frame-pointer-based addressing is slightly less efficient than stack-pointer-based addressing. The problem is that the offset from the stack pointer / frame pointer is encoded into the instruction as either 8 bits or 32 bits -- 8 bits if that's enough, 32 bits otherwise. But it turns out that for large stack frames, the stuff close to the frame pointer is more stuff that isn't actually used during the function body, such as saved register values, whereas the stuff close to the stack pointer is typically the data that's currently being operated upon. So, addressing based on the stack pointer is less likely to spill over into 32-bit offsets.

However, this all sounds like something that could be fixed in the compilers. They could still address off the stack pointer even when frame pointers are available. In fact, if they could choose on an instruction-by-instruction basis, maybe they could even save bytes. But for some reason they don't currently do this. It's not clear to me why -- maybe just because omitting frame pointers has been such a standard optimization for so long that no one has bothered optimizing the other case?


That makes sense and isn't something I had considered, but a 32-bit displacement vs an 8-bit displacement just leads to bloat in binary size, it doesn't affect how many cycles your movs jumps etc. take. There are some second order effects of things where larger code size can cause you to get a worse hit rate in the instruction cache, but usually those effects are miniscule when you actually benchmark code. There are going to be some pathological programs where the instruction cache hit rate gets way worse with the large binary size, but I can't really think of many programs I've seen that are mostly stalled on instruction decoding/fetching.

Here's my hypothesis of why stack-pointer offsets haven't been implemented when compiling with frame pointers. At companies like Google/Facebook that compile huge C++ binaries (usually 100MB+ stripped) it's common that production binaries would be compiled with something like -fno-omit-frame-pointer (so system-wide profiling can be done on all C++ code in production) and also generate split debug symbols using -gline-tables-only. The latter compiler flag basically just generates enough DWARF information to recover the mapping of pc -> source code line, so if you get a core dump you can figure out which line of C++ code was actually being executed in each frame of the call stack. If you're doing frame-pointer offsets it means that you can also recover the value of local variables in each frame (assuming they haven't been optimized out by the inliner) just based on the offset of the variable from the frame-pointer. So basically -fno-omit-frame-pointer and -gline-tables-only give you enough debug data to get the full call stack with line numbers and the values of local variables in each frame (except the inlined ones), while also minimizing the cost of generating/storing debug data.


> If it's the former, I'd love to know why the frame stack version of the program would run more slowly.

I believe it's the former and my understanding is that the stack frame version uses a register (RBP on x86-64) to keep track of the (start?) of the stack frame, which presumably helps in debugging / getting stack traces and profiling performance.

However, when not keeping track of the stack frame, the compiler can use the RBP register for storing other values, which means that the compiler has one more register it can use to avoid loading/storing values in memory (accessing memory is much slower than accessing registers).

Also, when not keeping track of the stack frame, I believe there is less of a need to store/restore the value of RBP at the beginning/end of functions, which also reduces code size and number of instructions executed, I think.

As far as I understand, those are the reasons for the slowdown.


These "current AI systems" that you're talking about were presumably specifically trained by "woke" Silicon Valley employees to reflect their opinions about what the AI should answer [1], which are hardly representative of the general population's opinions.

> If conservatives want a fascist chatbot they can train their own off of 8chan, Stormfront, Parlor, etc...

I don't think conservatives want a fascist chatbot. They just don't want a biased "woke" one either.

[1] https://openai.com/blog/chatgpt/

> "human AI trainers provided conversations in which they played both sides—the user and an AI assistant."

> "had AI trainers rank them. "

> "We performed several iterations of this process."


Actually, "woke" opinions do reflect those of the general population, which is exactly why conservatives feel they live in some kind of progressive hellscape.


Or, rather than posting non-sense, you could learn about what "woke" usually means in this context, which I can quote for you [1]:

"shorthand for American Left ideas involving identity politics and social justice"

"By 2020, members of the political center and right wing in several Western countries were using the term woke, often in an ironic way, as an insult for various progressive or leftist movements and ideologies perceived as overzealous, performative, or insincere. In turn, some commentators came to consider it an offensive term with negative associations to those who promote political ideas involving identity and race."

Unless, of course, you believe that the term "general population" excludes the "political center and right wing in several Western countries" and only includes "the American Left".

That said, just to be clear: when I used the term "woke", I did not mean it in an insulting or pejorative way, only as a means to describe the ideology itself.

[1] https://en.wikipedia.org/wiki/Woke


I like how you only quoted the half of the definition that supports your personal definition of “woke.”

Here’s the first half! “Beginning in the 2010s, it came to encompass a broader awareness of social inequalities such as sexism, and has also been used as shorthand”

Rather than posting non-sense you could admit that there is a difference between “has been used as shorthand…” is different from “The definition of this word is: shorthand for…”

It’s kind of odd, it’s almost as if there is a group of right wing culture warriors that insist that anyone that doesn’t use their artificially constructed pejorative the same way that they do is part of some vast gay communist conspiracy.


> Beginning in the 2010s, it came to encompass a broader awareness of social inequalities such as sexism

You are right, it also encompasses this meaning.

Regardless, my point still stands.

> It’s kind of odd, it’s almost as if there is a group of right wing culture warriors that insist that anyone that doesn’t use their artificially constructed pejorative the same way that they do is part of some vast gay communist conspiracy.

I don't think it's an artificially constructed pejorative, according to Wikipedia the term seems to come from the 1930s "referring to an awareness of the social and political issues affecting African Americans."

I think it only started to be used in a negative way once all the clown world stuff came into full strength.


If "woke" is going to be bucketed into purely leftist ideology, which is already oversimplifying it as progressivism or simply being anti-racist isn't strictly leftist, then "clown world" is going to be bucketed as a neo-Nazi dogwhistle.

Its origin on the internet was tightly bound to the "Honkler" meme, which was a neo-Nazi dogwhistle so thinly veiled that it was only denied by people with "1488" in their usernames. There was essentially no real attempt to try and present it as something else.

Is spreading a neo-Nazi dogwhistle really a good push back against anti-racism?


> Its origin on the internet was tightly bound to the "Honkler" meme, which was a neo-Nazi dogwhistle so thinly veiled that it was only denied by people with "1488" in their usernames. There was essentially no real attempt to try and present it as something else.

> Is spreading a neo-Nazi dogwhistle really a good push back against anti-racism?

lol, I have to admit, assuming you are right, I wasn't aware of the origin of the term "clown world". I guess I shouldn't use that term, then.

To be clear, I was trying to refer to the leftist ideology taken to an extreme, such that you try to be so anti-racist that you become racist yourself (against whites, for example).

Or you try to be so anti-sexist, that you become sexist (against men).

Or that you are no longer allowed to use any word whatsoever because every word offends some snowflake of an individual.

And also, of course, other extreme examples of behavior where it's clear that you're no longer contributing to a solution, but rather creating a new problem.

There are many such recent examples of this kind of non-sense.


I was skeptical about the GP's alleged etymology about clown world but... [1]

> The alt-right sense emerged from The Honkler, a version of Pepe the Frog meme featuring a red clown nose and rainbow wig, characterized as honking a bicycle horn whenever liberals speak. That meme is associated with the use of honk honk ("HH") as a dog whistle for heil Hitler.

It is worse than I thought. At least "woke" has to be used "ironically" to be coopted. There's literally no where to go with "heil hitler" - it's written right on the tin. Why is our discourse polluted with these dog whistles?

[1] https://en.wiktionary.org/wiki/clown_world


> Why is our discourse polluted with these dog whistles?

Because neo-nazis have essentially perfected coopting “respectability” politics in public online discourse. Calling out dogwhistles can have a nasty blowback effect because they’ve manufactured cover (eg “I’m just quoting what [a party with unknown intentions] wrote on wikipedia!”) that gives them footing to claim misinterpretation at the mildest, or outright persecution in other cases.

HN is actually a really interesting case for this stuff, because the Primary Sin here is “violating decorum”. If I wanted to A/B test hateful dogwhistles, this site would be the perfect place. Nobody is allowed to post “Fuck off nazi/tankie/[insert other toxic group]”

Respectability and Decorum are how we open the door to obvious but just-subtle-enough crap like that.


I have a follow up post to this downthread. You can find it here:

https://news.ycombinator.com/item?id=34423773


> To be clear, I was trying to refer to the leftist ideology taken to an extreme, such that you try to be so anti-racist that you become racist yourself (against whites, for example).

“The REAL nazis are people that don’t buy into the great replacement theory!”

This isn’t a dogwhistle, this is a foghorn.


> “The REAL nazis are people that don’t buy into the great replacement theory!”

If you are insinuating that this is what I think or what I was trying to convey, I can assure you, it's not.

I don't believe in the great replacement theory, never have and wouldn't even care if some race (including whites) is being somehow replaced somewhere (or everywhere) due to migration or cultural acceptance of other races or whatever the hell the (conspiracy?) theory is about nowadays.

I've met and became friends (even best friends) with people of many different races (yes, including black people) and cultures and I like all of them, no exception.

I am willing to admit I may not always recognize racist behavior when I see it, as in some cases it can be hard to see unless you're in the receiving end of it (or even if you are).

But you can't gaslight me into thinking that obvious racist behavior is not racist, even if it's presumably being done to supposedly accomplish some kind of moral goal.

Many terrorists also have good intentions, that doesn't mean that the means they use to accomplish their goals are acceptable.

There are many ways to accomplish the same anti-racist goal without resorting to overt... racism.


Buddy I’m not falling for your pivot from “innocently quoting wikipedia” to your sudden interest in “the left hates whites” discourse. Nobody is required to rehash /pol/ debates.

You gave yourself away with the clown world stuff. Oops!


You are misrepresenting my points.

I don't believe the left hates whites. Where did you get that from?

When I was referring to the clown world stuff, I was talking about the extreme side of the left (which I'm assuming is where you stand?).

It was not a sudden interest either, it was in response to your (apparently misguided) theory that "woke" was an artificially constructed pejorative, rather than a term that was positive but simply became negatively associated due to these extreme cases becoming more common (or at least, more visible) in the past decade.

Look, I have just as much criticism about the right (especially the far right, of course) as the left, OK? Including overt racist behavior and narratives. We can definitely talk about that if you want!

I was just following the conversation, nothing more.


For anyone following this, I am no longer interacting with wizeman, and I recommend checking out my previous post (1) about using respectability and decorum as cover for spreading nazi rhetoric.

For example, this poster brought up “anti-white racism” from the left unprompted, but is claiming to be just “following” a conversation that they both started and steered. The tactic of claiming to be misunderstood is the first move. Claiming persecution is the next one.

This is how you end up with pointless, awful arguments with people with terrible opinions! Don’t touch the poop!

1. https://news.ycombinator.com/item?id=34422552


Here's some more Nazi rhetoric for you:

1. I believe that there are many more racist right-wing people than left-wing people, both in the US and outside, even when excluding the far-right.

2. I believe there is a significant problem with racism in the way black people are treated in the US, especially regarding law enforcement, courts, prisons, etc. As far as I understand, this is not as much of a significant issue with other races (but I am willing to concede I am wrong on that).

3. I believe that there is a lot of unjustified right-wing narratives portraying black people as being more violent.

4. I believe that a lot of the anti-immigration sentiment is also in part due to racism.

5. I believe that another significant issue with racism in the US is the way black people (and also people from other non-white races) are effectively segregated, either due to a significant fraction of them not being able to live in the same places as white people, or simply due to historical reasons.

6. I also believe that the latter is in fact, the single greatest contributor to all racist problems in the US, rather than some other insignificant (manufactured) issues. However, I also believe that this issue is vastly harder to solve in the US than in other countries.

Does that sound Nazi to you?

It seems like you felt personally attacked by my opinions.

As you can see, I can just as easily attack right-wing ideology... and mean it!

(See folks, my comments are an example of how not to make friends of any kind... lol).


Once more for the folks following this, I’m happy to point out a series of red flags as an example of how to spot somebody posting *chan drivel! (I am still not interacting with wizeman)

This poster started off with pointing out the importance of the orthodoxy of using the term “woke” in line with the right wing culture warriors. It’s prescriptive that you use the same words they do.

When called out, they made the mistake of using dogwhistle phrasing (“clown world“ is very specific to one group) that you don’t really see much outside of /pol/ and its ilk and in threads where those folks bravely venture out to try to leak /pol/ chat to other forums.

So now the dogwhistle has been sounded.

Poster gets called out. Knowing that they’re losing credibility, there is a panicked attempt to gain support from people that already support this garbage.

Poster deploys other, louder and more transparent signal, (apropos of literally nothing) “the left is racist against whites”!

Somehow we have teleported from “I’m just sharing a wikipedia link” to “anti-racism is racism” and we are meant to believe with a straight face that that’s the logical next step in this conversation.

(As an aside, I almost never hear “whites” outside of a context wherein someone is airing a grievance on behalf of the group that’s otherwise known as “white people”. Another red flag!)

This appeal fails.

Poster gets called out again. This time they change tack, now it’s a “misunderstood” play. Damage control.

Poster gets called out for the obvious “misunderstood” play. Poster doubles down. And again and again.

Finally, poster abandons their position and falls back to “saying stuff they think will ingratiate you to them.” This is an extension of the “I’m misunderstood! I just use far right terms while describing far right talking points after hopping into a discussion to enforce adherence to a far-right definition of a common term!”

If you want to understand how this stuff degrades communication, just believe your gut.


> When called out, they made the mistake of using dogwhistle phrasing (“clown world“ is very specific to one group) that you don’t really see much outside of /pol/ and its ilk and in threads where those folks bravely venture out to try to leak /pol/ chat to other forums.

WTF. I don't even know what you're talking about. What the hell is /pol/?

I learned about the "clown world" term on Twitter. And apparently, its meaning is not even what I thought it was, according to another poster.

Sorry for striking down one of your absurd theories once more.

> As an aside, I almost never hear “whites” outside of a context wherein someone is airing a grievance on behalf of the group that’s otherwise known as “white people”

I am not a native English speaker, so sorry for not knowing about all the connotations of how words are used in your country. Whites/white people. It's all the same to me (in my native language, at least).

I didn't even know "woke" had such negative connotations, until I went and quoted that from Wikipedia (which is why I clarified that I didn't mean it pejoratively).

> poster abandons their position and falls back to “saying stuff they think will ingratiate you to them.”

What are you even talking about? How did I abandon my position?!

And how the hell am I ingratiating myself to whoever is reading when I clearly criticized and almost certainly pissed off both right-wingers and left-wingers?!

Let me say it again so that it's crystal clear to you:

1. Anti-white racism is racism.

2. There are more racist right-wing people than left-wing people, even when excluding the far-right.

3. Anti-black racism is a much more prevalent (and therefore significant) issue than anti-white racism.

Believe it or not, I hold all of these opinions at the same time. Mind-blowing, right?

I'm sorry that you don't believe that it's possible for someone to have all these opinions or that you think they are "poop" (using your term) or whatever the hell you think.

I think you have severe issues, to be honest.

And it's clear to me that I've been wasting my time trying to engage and have a constructive conversation with you, so don't expect any more responses from me.


Postscript: (Still not interacting with wizeman)

This poster that has just only learned of the word “woke” according to them but also wants to educate folks about it has a lot to say in other threads!

https://news.ycombinator.com/item?id=34415794

https://news.ycombinator.com/item?id=34418890

The only reason why I’m revisiting this is to point out that they have (as I’ve pointed out) gone through with the natural pivot from misunderstood to persecuted. It’s a very simple playbook that can be followed by anyone. It gets kind of like the fedex arrow once you notice it!

The biggest part of this color-by-numbers silliness is the demand at every step that the opponent debates their positions (which involves re-stating them). If a far right person refuses to expand on their position and demands that you explain why they’re wrong, you’re looking at bait! Namaste!


> This poster that has just only learned of the word “woke” according to them

I only just learned about how the term is used as an insult in your language. Again, I'm not from your country, so we don't necessarily use it the same way here (as an imported word), capiche?

But apart from the word itself, we do have definitely imported the same ideology here, as well as the crazies who take it to an extreme. I'm not saying they're like you, but I'm not saying they're not, either.

(We have many right-wing crazies too, mind you! I'm not saying only the lefties have crazies)

So again, I used "woke" as a shorthand for the ideology. It's a word I have seen used many many times before, just like any other English word you can pick from any of my comments. The term "woke" is used in contexts such as this one to describe this type of ideology, in many countries including the US.

Hell, look at how this word is used in the Vice article this HN post is about. Does that mean the Vice article is insulting all woke people, since it's using that word and that word is now an insult?

Or perhaps is it that "woke" is not always used as an insult, but it has been (and still continues to be) simply used to describe this ideology, despite of the negative connotations it has apparently acquired recently?

This is something you would likely understand if you were actually trying to make any effort whatsoever.

Does all of this mean I can't educate someone on what the word means (i.e. its usual meaning), especially when I have access to sources of truth about the definition of the word in your language? A word, again, that has been exported to many other countries and languages... like wifi, sandwich or shampoo.

Note that I'm not insisting on using the term "woke". I can see that it triggers some people, so I'm not likely to use it as much in the future. I only care about discussing the ideology itself, I don't really care what the hell it's actually called, be it in your language or any other.

That said, obviously it's good to have clear names (especially without negative connotations) to describe concepts, otherwise it's hard to have constructive dialogues.

As you can see from other posts, even when the definition of a term is clear, some people still misunderstand you.

> If a far right person refuses to expand on their position and demands that you explain why they’re wrong, you’re looking at bait!

Look out, there's another person behind you who doesn't believe in racism (including against white people)!

Which means, he is far, far-right, obviously!

Oh God, the horror! Careful, he might bite you!


The best way to prove braingenious wrong is by not engaging with him [1]. This is a pseudoanonymous internet forum - not responding and moving on with your life is a perfectly acceptable response to this situation [1]!

If you’re not a far right troll, you gain nothing by trying to prove it [1].

[1] IF you’re not a far right troll

Edit: FWIW I don’t believe you when you say “your language,” hence the [1]. “Capisce” is too specific a colloquialism to American English.


> The best way to prove braingenious wrong is by not engaging with him

Well, I like to give someone the benefit of the doubt when discussing these kind of contentious issues (up to a limit, of course). It has served me well in the past (here on HN, for example), and I've learned things that I wouldn't have learned otherwise (from people who hold different opinions than mine).

I also get bothered when I'm accused of something that I'm not, obviously. It's not always easy to move on when someone is saying blatantly false statements about you, as you can imagine.

> FWIW I don’t believe you when you say “your language,”

My native language is Portuguese. I was born and raised in Portugal and lived there well until after I was an adult. All of my family and all my childhood friends are Portuguese.

I learned English very early in my childhood, as part of my formal education, due to US cultural exports (i.e. Hollywood movies and TV shows that are broadcast in Portugal without dubbing, i.e. they have always been subtitled and therefore it's much easier to learn the language and become fluent in it) and due to my early childhood interest in programming.

I've always read lots of English prose, especially related to computers and programming (magazines, articles, documentation, websites, Youtube videos, source code, etc, etc). Almost all of this content is from the US, as you can imagine.

I've worked for a few US companies (mostly multinational ones, think FAANG-level) for many years, had many coworkers from all over the world (including many from the US) and always worked remotely, so both oral and written communication has always been in English.

Both due to my professional work and as a tourist, I've spent many months in the U.S., in total (close to a year).

Well into my adulthood, I've also lived for more than a decade in Spain, so I'm also a fluent Spanish speaker.

Also, for many years (perhaps since I started working) I've been reading and listening to way, way more English content than in the other languages I'm a fluent speaker of. This includes Hacker News and other news websites. It includes all media entertainment (Netflix, etc). For decades, I have stopped bothering to find Portuguese or Spanish subtitles anymore for video content due to their mediocre quality or lost meanings, although I do try to put English ones when it's easy to do so.

As another example, I almost exclusively follow English-speaking Twitter accounts (although some Spanish ones as well).

All of this English content is way, way more interesting to me than what I can find in Portuguese or even Spanish. It's not even remotely comparable.

I also don't bother at all with translations from English content which, when they exist at all (well, nowadays there is automated translation, but still), are inevitably mediocre or incomplete due to the much smaller vocabulary in these languages or simply due to the incompetent or error-prone translations.

So to summarize, it would be fair to say that you have culturally captured me :)

It's OK if you don't believe me, but there you have it anyway.


> "shorthand for American Left ideas involving identity politics and social justice"

Where did you get that definition from? I searched for it in the article but found nothing. Everything from Disney to M&Ms to the very concept of pronouns in English have been called "woke", and as far as I'm concerned it's utterly meaningless at this point.

edit - apologies totally missed the wiki link somehow. regardless, doesn't change anything else I said


> Everything from Disney to M&Ms

Sorry, I'm not sure what you're talking about here.

> the very concept of pronouns in English have been called "woke"

It's not "the very concept of pronouns" that is called "woke", what is "woke" in that context is using them inappropriately (or perhaps I should say, exaggerate their significance?) in order to impose a political agenda.

This ties into the "identity politics" part of the meaning of "woke", which is not surprising because of what a pronoun is.

> and as far as I'm concerned it's utterly meaningless at this point.

Perhaps it is meaningless to you, but I assure you, the term is not meaningless for a lot of people.


> Sorry, I'm not sure what you're talking about here.

Those were two examples of the "outrage of the week" for fox news and right wing. But pick any week and it will be something new that is deemed "woke".

> is using them inappropriately [...] in order to impose a political agenda.

What agenda would that be? I don't see how someone wishing to be referred to by a specific pronoun should be political. In my mind it's just something I do to be respectful to people I interact with.

> the term is not meaningless for a lot of people

Outside of being a word used to put down people/ideas, what other purpose does it serve you? genuinely asking


> What agenda would that be?

Modern liberal politics.

> I don't see how someone wishing to be referred to by a specific pronoun should be political. In my mind it's just something I do to be respectful to people I interact with.

Well, ~99% of people don't need to specify the pronouns they wish to be called with because everybody already called them by those pronouns.

So you should ask yourself: why are all these people putting their pronouns on their twitter profiles (or whatever)? It's clearly not to prevent other people from being disrespectful, because they weren't being disrespectful.

So I think you are either being very naive or you are not being honest, because it's clear that the real point of specifying your pronouns is not to communicate to other people of what you wish to be called to prevent someone from being disrespectful (except for a minority of cases where people really do get confused).

And furthermore, I also try to be respectful to the people I interact with. But in my view, respect is much more than calling someone him, her or whatever. In fact, even if someone would make an occasional mistake, it's not (and should not) be a big deal. It would certainly not be due to a lack of respect (assuming it's a genuine mistake).

Note that nobody puts "please don't call me an asshole" in their Twitter bio, even though people are called assholes a lot more than they get their pronouns confused (again, except for a minority of people).

So I'm fine if someone puts their pronouns in their bio if other people get them confused frequently.

But even then, is that really the solution?

Should that person also paint their pronouns in their forehead when they go outside, so that other people aren't disrespectful unintentionally?

Or should they just learn to live with a minor grammatical annoyance?

> Outside of being a word used to put down people/ideas, what other purpose does it serve you? genuinely asking

It's a shorthand to describe modern liberal politics regarding identity politics, social justice, cancel culture, etc.

Mind you, this is not the same liberal politics from a few decades ago. This is on a whole other level.


> Mind you, this is not the same liberal politics from a few decades ago.

Might it be that those who occupy the left-of-centre political space hold more nuanced and varied positions than the sort of monolithic bloc that you would have readers believe? Yes, at both poles of the political axis there are those who strive to impose their own purity tests - whether it’s over pronouns, or guns, or immigration, or the proper lexicon to use, or marriage rights or whatever. But the rest of us can unbundle and reassemble policy positions for ourselves. I would hope this is true on the left and the right. But maybe it’s misplaced hope…


I think (and hope) that you are correct.

But I also think that, even though what you said can be true, that there has also been an increase in how these political positions are becoming more extreme (both on the left and on the right!) and how these extreme positions are becoming more frequent and accepted.

I think that these extreme ideas/views and conflicts have been greatly amplified by social networks and information bubbles.

Or maybe it's just that these extreme cases are more visible now, I don't know.

But I am still quite worried that we are not heading in a good direction.


> So you should ask yourself: why are all these people putting their pronouns on their twitter profiles (or whatever)? It's clearly not to prevent other people from being disrespectful, because they weren't being disrespectful.

You keep using a lot of words that basically boils down to "i really don't know what i'm talking about"


It's be more accurate to say the word "woke" is a conservative satire of the left side of the aisle - it does not convey meaning about the ideology itself, but about the reaction to it. It's also not super accurate to say its about identity politics, as the right leans heavily on that approach with different identities. Instead of race, it's being Christian. Everything is identity politics - abortion and 2A are possibly the 2 biggest conservative positions that derive from identity politics.

When using an overarching term, I think "progressive politics" is less inflammatory than "woke" (which is generally not used for self-description). It has the nice effect of linguistically balancing out conservative politics - we can have "woke" versus "fascist", or we can say "progressive" versus "conservative".


I think what you're saying makes sense. I was not aware of the full breadth of negativity associated with the term "woke".

I think I will start using the term "progressive politics" from now on (at least, as long as it makes sense).

That said, there is a clear distinction between modern progressive politics (of the "woke" type) and the progressive politics from a few decades ago, and I think "woke" is a term that makes that distinction more clear.

It's unfortunate that it has become so negatively charged, though.


> there is a clear distinction between modern progressive politics (of the "woke" type) and the progressive politics from a few decades ago

Not to be glib, but this is the nature of "progress" - it changes things. ;)


> Not to be glib, but this is the nature of "progress" - it changes things. ;)

Well, I genuinely hope you are right.. :)

But note that not everything that we call or called "progress" is necessarily positive. We have made quite a few mistakes before which we thought was progress.

Probably too many to mention, even, but as examples, I can think of the Dark Ages and Nazism.

Mind you, I'm not saying modern liberal politics can be compared to these examples! I don't even known that much about history to be able to comment on them.

It's just food for thought.


I've noticed people using "identity politics" where I think they really mean "intersectionality."

Also, I have no dog in this fight, I've been out of the US for a few years. Though my outsider perspective (right/wrong/indifferent) is that intersectionality has been a substantive policy plank on the left.

The right has utilized identity politics as a means to expand their "big tent" and to drive voter turnout.


"Woke" as used by conservatives now more or less just means that when you're an asshole in public that someone might actually call you on it.

AIs produced by companies who want to sell to the general public will want them to generally not be acting like assholes.

Similarly to how the workers in big-box stores generally don't express strong political opinions.


>Or, rather than posting non-sense, you could learn about what "woke" usually means in this context, which I can quote for you

I am well aware of what "woke" means, where the term came from, how it's been corrupted and by whom.

>Unless, of course, you believe that the term "general population" excludes the "political center and right wing in several Western countries" and only includes "the American Left".

Yes. Because the linked article is discussing an article by the National Review, an American conservative magazine, about the complex of beliefs among American conservatives about perceived left-wing bias in American media, with the specific concern of "woke" bias in AI being the latest iteration on the theme. Thus the "general population" of the American public, specifically the American conservative public, is the valid context under which these terms are being discussed.


> Actually, "woke" opinions do reflect those of the general population

I think a lot probably don't. A whole lot don't if you consider the scope of the entire developed world, not just the US.

There's good support (at least in the US) for the liberal position on the last wave of fights over civil rights. Interracial marriage (this was still a tad controversial in the 90s, even, I recall it being a topic on daytime talk shows), gay marriage, that kind of thing.

No abortion restrictions even into the third trimester? Not favored by the general population (and, notably, that's not what Roe and especially not what Casey-modified-Roe protected). Various trans rights issues and slogans can only seem universally-supported in certain bubbles—sentiment is much more reserved (conservative, if you will) among the general public. I recall the polling on "defund the police" looking really unfavorable. And so on.

(Nb. I'm pretty damn "woke" myself, but I calls 'em like I sees 'em and I'm fairly sure the general population isn't solidly behind a lot of what gets labeled "woke")


> No abortion restrictions even into the third trimester? Not favored by the general population

I think this one in particular is more popular in America than in Europe. Third trimester abortions are broadly banned in almost all of Europe, except when a doctor says it's medically necessary.

https://en.wikipedia.org/wiki/Abortion_in_Europe#Abortion_ru...

Personally I think France gets it about right, but saying this is sure to piss off a lot of people on both sides in America. It pisses off the conservatives because I oppose the total abortion ban they want, and it pisses off progressives because I don't favor legalizing elective third trimester abortions. The American discussion around this issue is extremely polarized.


I don't think that's an accurate statement of facts on the ground. For instance, if you poll the general population about trans women participating in sports, a large majority of respondents are opposed to it:

https://www.npr.org/2022/06/29/1107484965/transgender-athlet...

However, compare ChatGPT's responses to "What are the arguments for trans women participating in sports?" and "What are the arguments against trans women participating in sports?"

That's not to say that trans women shouldn't be welcome in women's sports; the majority may be wrong about that, as they often are. It's not, however, a widely agreed upon consensus that they should.

I'd point at that there's at least a tension between believing that "society is a hellscape dominated by racist/sexist/transphobic oppression" and believing that "woke opinions are reflective of the general population."


Not necessarily. We know for example that Twitter was a key source of training data for GPT and there is also clear evidence that tweets were heavily curated by a team that was pretty significantly left of center.


Do you think this because this is what our media portrays as the opinions of the general population or because there is hard data to back that statement up that you can share?


> Actually, "woke" opinions do reflect those of the general population

If woke here means thinking that racism is bad, sure. If it means supporting gay marriage or first trimester abortion, sure. If it means not being heteronormative, thinking that gender identity should be taught in public schools, thinking that affirmative action is good... you're starting to drift into fringe territory. Not saying I don't agree with fringe, but claiming these beliefs are majority doesn't agree with experience. For the record, most people I know are democrat voters

If woke here means thinking that social justice and "systematic overhaul" (whatever that happens to mean) are a good idea, then from my own personal experience that's just corporate noise to people. The only place where I really saw a lot of this was in college where most of it was from administrators and a loud but small group of the student body

There's a reason why Obama avoided a lot of these topics while campaigning


this is pretty contentious, and based on your definition of "woke", which is a pretty vague word with unhelpful connotations

a less mocking term would be "modern progressivism", which I would summarise as being something like thinking that you can be subconsciously prejudiced against a particular identity and that this is almost or entirely as bad as consciously being so

this conflation is a huge problem. sure, subconscious bias exists. it would be weird if it didn't. and yes it's a problem. but acting like its just as bad as conscious bias basically signals to people who may have a bias here or there might as well be loud and proud about it because you'll treat them the same either way

but in terms of the general population, I think that modern progressives are probably a similar size segment of society as the rabid conservatives, but because that segment contains most of the prominent voices in non-conservative-focused media, a majority of people just nod along or ignore it


Conservatives feel like they live in a progressive hellscape because the 8% of the population that’s “woke” has disproportionate power in media and communications—which everyone is exposed to every day: https://hiddentribes.us/profiles/


That article refers to 8% of the population as "progressive activists," but of course most people who believe in progressive ideals are not activists, so attempting to imply that 98% of the population is conservative and anti-woke is disingenuous at best.

In the context of the posted article, even objecting to the presence of racist and offensive speech counts as "woke." Any positive representation of non-white, non-Christian and non-heteronormative demographics in the media beyond minimal tokenism, and certainly outside of the conservative narrative, is considered "woke."

Most Americans tend to identify as "socially progressive and financially conservative."


As long as we're doing sweeping generalizations: Perhaps white Christian nationalist conservatives just can't stand it that so many people reject what they (the WCNCs) claim to believe is Divine Will.


The opposition to wokeness is far broader than “white Christian national conservatives.” Heck, the Republican Party alone is far broader than that. Most people don’t want their kids learning victimhood mentality in school, they don’t want the founding fathers redefined as slavers, they don’t think “the constitution is racist,” are fine distinguishing between men and women and having gender roles, don’t like retooling the language around small gender minorities (“Latinx,” etc.) My Biden voting dad hates the term “person of color” and my Biden voting mom is sending me articles complaining about schools dropping admissions tests.

Heck, most people (60%) don’t even think we should give racial minorities special consideration in college admissions and hiring! That seems like the absolute floor for “woke” people, and I think it’s fair to say even many people who accept affirmative action aren’t “woke.”


What you're presenting is the Fox News version of "wokeness." Here's an alternative, from former Republican congressman Joe Walsh: "Being woke just means being empathetic. And tolerant. And willing to listen. And open to learning."

https://twitter.com/WalshFreedom/status/1615481522647171074


And here’s another alternative, from the African-American Marxist Adolph Reed, according to whom “woke” is a form of “irrationalist race reductionism”: https://newrepublic.com/article/160305/beyond-great-awokenin...

Not Fox News, but a long way away from Joe Walsh too


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: