Japan's initiative is focused on "green open access" which is different from pay-to-publish. I recommend the section titled "Green OA" in the submitted article. Relevant quote:
"Japan’s move to greater access to its research is focusing on 'green OA' — in which authors make the author-accepted, but unfinalized, versions of papers available in the digital repositories, says Seiichi.
Seiichi says that gold OA — in which the final copyedited and polished version of a paper is made freely available on the journal site — is not feasible on a wide scale. That’s because the cost to make every paper free to read would be too high for universities."
I think it applies to most STEM fields. I'm a reviewer for several journals in a STEM field (not AI/ML specifically, but some manuscripts do try to apply AI/ML to this field) and the vast majority of authors seem to upload their preprints to arxiv etc.
Social sciences may be behind though as you say, I do not know as I'm not in that field.
As someone with graduate degrees in both STEM (math) and social science (psychology) fields, it's true that social science is way behind STEM in terms of preprints to digital archives. It's possible there's momentum here in the last 5 or so years that I'm unaware of though.
That matters for a few reasons:
(1) the average person more frequently encounters psychological, social and medical issues more than they do math problems. And since the research in those fields tends to be pay walled, people are at the mercy of things like SEO spam medical and health sites.
(2) wrong ideas in medicine or psychology can (and have in the past) damaged entire generations of people. So in that sense their blast radius can be very large. This means that peer review is especially important and that there's a potential negative externality to posting preprints and drafts before they're finalized. I suspect we'd have to solve the peer review and quality problem before STEM-style preprint archives become the norms in all fields.
For Bioengineering it definetly is, but a lot of Medicine is still locked behind high impact Wiley and Sage publications, and for a lot of that research it's fairly easy to pay the $3-4k to make the article open access.
The truth is that OA is a childish illusion that got “absorbed” by the adults in the room who tapped the kids on their back and said “no worries, we’ll take it from here”. Then they turned traditional publishing, which was already an elaborate expensive ruse, into OA which is an even more expensive (but less elaborate) ruse. Now everyone is happy, except someone trying to do actual research and having to read 1000 meaningless papers a day.
Green OA is just as meaningless as—-or worse than—-Gold OA. You pay the publisher a large sum of money for the “right” to self-publish the preprint, and they still paywall it. The vast majority of people will find the paywalled version before yours, and anyway there is no guarantee that your preprint is accurate with the final, published version, so most people will still trust the paywalled version more than your PDF. Especially when performing systematic literature reviews where you need to document the sources of your references.
The current implementation of OA (any of them) is basically a self-fulfilling prophecy: we convinced ourselves that “publishers are evil” and impossible to get rid of, and now we are paying them so that they don’t have to do their job. We basically retired publishers early with an extra pension, all because everyone “wants to believe” in open access. But guess what? This is not disruptive. OA is just as “capitalist evil” as the usual publishing, or even more so. Do you want to be disruptive? The disrupt. Get rid of the publishers. Or at least constrain funding only to not-for-profits for example.
> You pay the publisher a large sum of money for the “right” to self-publish the preprint, and they still paywall it
There's no APC with Green OA, so what money are you talking about? Green OA is regular publishing, but with self archiving. There will be a version freely available, and the publishers aren't paid for that privilege.
If you want a route to the death of publishers, green OA is a promising one.
(I think the headline ought to emphasise that this is pushing green OA, which is the interesting bit)
Ah, I somewhat see the confusion. Green OA doesn't require the publisher to publish other versions for free, it just means you are allowed to publish them. Typically you'd publish them via an institutional repository, preprint server (often discipline specific), or in one of a number of free online services.
"there is no guarantee that your preprint is accurate with the final, published version, so most people will still trust the paywalled version more than your PDF"
I think this is backwards. The definitive version that should be cited is the freely-available one, since that is the version that everyone can read. No one should cite the paywalled version.
"Japan’s move to greater access to its research is focusing on 'green OA' — in which authors make the author-accepted, but unfinalized, versions of papers available in the digital repositories, says Seiichi.
Seiichi says that gold OA — in which the final copyedited and polished version of a paper is made freely available on the journal site — is not feasible on a wide scale. That’s because the cost to make every paper free to read would be too high for universities."