What happens when there is the next Codehaus-like shutdown, and so much source code just won't work? Or when a bad actor takes control of a domain that commonly hosted packages (perhaps through completely legitimate means, such as registration expiration), can get a completely legitimate SSL certificate for it, and responds to requests for packages with malicious code? I think the abstraction, and to some degree the centralization, of package management is generally a good thing
URL-based imports aren't less secure. They just make an existing attack vector more obvious. Is NPM really keeping you safe? What happens if a package maintainer is compromised and the attacker adds malicious code?
The fact that URL-based imports make you uncomfortable is good. Let that discomfort guide you to adopt some extra security measures to protect yourself from third-party dependency exploits
I would argue that NPM and other centralized package managers have the ability to add security: if npm made 2FA a requirement (or publicized the status of a package maintainer having 2FA enabled like GitHub does, which is perhaps a security concern itself), there would be some assurance that a maintainer is not compromised.
If we are using URL-based imports, the scope of security assurances are much broader: an SSL certificate doesn't say anything about whether the current owner of a domain was the owner at the time of a dependency being safe. There is no one authority who we can expect to take swift (or even slow) action to remove malicious packages from being available on the myriad hosting protocols that are accessible through an environment like Deno
Every time you fetch a package from NPM you are accessing that code from a URL (at npmjs.com) and then caching that locally. Deno is just eliminating the middleman. If you still trust npmjs.com for source code delivery you could continue to do that.
What it isn't eliminating is the ability to define "local" caches. Just because you are importing everything from URLs doesn't mean that they can't all be URLs that you directly control. You don't have to use CDN-like URLs in Production, you can entirely copy and paste all the modules you need onto a server you control and URL scheme that you maintain.
There will still possibly be good uses of caching automation in Deno (once people get bored with xcopy/robocopy/cp -r/rsync scripts), but it will likely seem a return to more bower-like tools rather than necessarily full blown packager npm-like tools. (ETA: Or proxy services like GitHub Artifacts that pull an upstream source for you and rehost it on controlled URLs with some additional security checks.)
a note for many: yarn v2 provides '0-config' fully cachable dependencies (zips in lfs). This makes it possible to fully vet dependencies and enforce change approval and analysis in CI/CD.
They don't do a great job of advertising the changes; take a look under features/zero-installs.
The common practice is to still use yarnv1 as your global yarn, just do "yarn set version berry" inside your project to use v2. 0-config can be a breaking change - though I haven't had problems in a long time.
I wish there was something like Docker Hub's automated builds in the Node world because the way NPM works right now, what comes from NPM is an unknown. The only thing you know is if you download a specific version once, you'll always get that same version again, unless it's invalidated. Otherwise, whatever the package author wants to upload and include, that's what you get and you can't know that what you're seeing in some Git commit is what's running in your application. I wish that was the state of the art.
THIS! I cannot believe that these is still no auto hash validator thing between git and npm. Feels like npm should force a commit containing hash for the current version on every publish or something. How can we make this happen?
It would have to be some kind of integrated CI build system thing that builds the product in a container. Seems like they have no incentive to offer that given that they totally own JS packages.
Also: Nobody prevents you from using a package manager anyway. Just because you can use urls in imports doesn't mean you have to. But it is very convenient that deno downloads exactly the code that is imported. A package manager will always just throw everything at you. Some packages in node.js try to fix this by splitting the project into submodules like @babel/core, @babel/env, .... But that made installing dependencies worse. Just let deno figure out what is required is way more elegant IMO.
Not sure I understand, are you implying Deno does automatic tree-shaking on package imports? If not, how does "deno download exactly the code that is imported" and not just a whole package?
Also, from your link:
"In Node this is done by checking node_modules into source control. In Deno this is done by pointing $DENO_DIR to some project-local directory at runtime, and similarly checking that into source control:"
I disagree with this. In my opinion, this is done by using a pull-through cache that caches every package developers request and so inherently has a cache of the packages that will go to production.
Is it possible to do this in deno today? I don't really get that sense.
> Not sure I understand, are you implying Deno does automatic tree-shaking on package imports? If not, how does "deno download exactly the code that is imported" and not just a whole package?
npm install copies in to your node_modules an entire zip/tarball package. Deno uses ES2015+ module syntax and it's only going to grab the JS files that are imported (or imported by imports). So it "naturally" tree shakes at the file level, and doesn't have a concept of a "package" in the same way. It's not directly going to tree shake inside of single file boundaries in the way that a major bundler/packager might (though the V8 JIT should still sort of indirectly compensate).
So yeah, if the package is published as just a single (M)JS file it will still get entirely downloaded by Deno, but if the modules are split across multiple files, Deno will only download the ones that are directly or indirectly imported (and will have no idea of the remainder of the files in the "package").
> I disagree with this. In my opinion, this is done by using a pull-through cache that caches every package developers request and so inherently has a cache of the packages that will go to production.
> Is it possible to do this in deno today? I don't really get that sense.
Yes, because URLs are just URLs, you could always have a Proxy service running at a URL that knows how to request the packages from upstream. https://jsproxy.mydomain.example.com/lodash/v1.1/module.js could be simple caching proxy that knows how to get lodash from upstream if it doesn't have the requested version cached (or sends a 404 error if it isn't allowed to cache a specific version or whatever).
Thanks for the package / module / import explanation.
Re: URL proxying, this all feels ad-hoc and overly decentralized. I agree with your assessment that "rolling it yourself" looks simple enough at first glance, but after having so much success with package managers and associated tooling I can feel the doubt in my mind that a new approach won't just reskin the same problems. I see they've done some reasonably smart decision-making along the way though, so let's hope they walked down this logic tree and are happy with the likely paths
> Re: URL proxying, this all feels ad-hoc and overly decentralized
Well, I started from the "full paranoia" mode where people would want it to be ad hoc and as decentralized as possible. It's very easy to imagine that there would still be trusted/trustable 3rd parties creating central proxies for stuff like this. Just as unpkg today provides one way to pull individual JS files from npm, you could imagine unpkg as an option for Deno. Similarly, there are lots of artifacts repositories with upstream pulling options already in the wild such as GitHub Artifacts or Azure Artifacts or jFrog Artifactory as three easy examples to mind (among many). It's not hard to imagine those artifact libraries also supporting Deno style URLs (in a similar manner to what unpkg does with npm) as Deno becomes more popular.
Ultimately it is likely to be a huge spectrum from people that want a JS proxy they entirely control/run/manage themselves, to those that want one they trust from a 3rd party, to those that are fine with whatever CDNs their libraries suggest. That's already a spectrum that exists in the npm world: people have private npm servers, people use npm artifact libraries like GitHub Artifacts, people use unpkg directly to pull in JS files from npm, people still go to Bootstrap or JQuery in 2021 and just copy and paste whatever CDN is mentioned in the "Getting Started" section of the appropriate docs. That spectrum is still likely to exist for Deno libraries, and while it might make it harder as a user/developer to choose which part of that spectrum is best for your own projects, Deno having little to no "out of the box" opinion on which part of the spectrum your project falls into (as opposed to Node defaulting to npm these days and the two ever more seemingly inseparable) isn't necessarily a bad thing for the ecosystem's health as a whole.
>"deno download exactly the code that is imported" and not just a whole package?
In a fairly simple and elegant manner. You specify a URL to the specific file FOO you want to import and FOO gets downloaded. Then deno looks at FOO to see what files FOO depends on, and downloads those, so on so forth...
That's very different from how NPM works where you have to download the entire package, including parts you may never use, along with every dependency the package depends on, even if you never end up using those dependencies either.
NPM's approach, which frankly worked well given the circumstances, has the downside of not providing end-users much benefit of writing code in a modular fashion, so there's no advantage to breaking a package up into multiple files versus distributing a single and large minified file.
Deno's approach promotes breaking up packages into multiple files so that an end-user only downloads the files and dependencies that are actually needed.
Thanks for the explanation. I would be a bit scared to have a recursive solver install my dependencies, but it's ultimately not that different from packages I suppose.
I will be interested to see how the tooling & community resulting from this decision look. Hopefully good.
Unfortunately this won't help those who add dependencies based on shared snippets, without the context of a project. Or the innocent mistake (or choice?) of not checking the lock file into version control. But yes good point, existing projects that have checked in the integrity file will be safe against future malicious modifications of the resource
There is some degree of assurance that this dependency won't last long in the Maven central repo, or any other user configured repository, if it contained malicious code. Obviously it is not foolproof and incidents happen, but without a centralized authority for package management, there is much less assurance that a package is not malicious
if you can magically copy and paste in code that also includes a dependency then you might have just screwed yourself if you didn't read the code of said dep (or even if you did, maybe you missed something) if it just looks like a comment then maybe your team missed it in review. its harder to reason about deps that live in deep modules.
Yes, i think recreating an npm kind of central place for your project dependencies seems almost required if you want to avoid depending on a different version of a lib in each file of your project.
I cannot understand the benefit of this scheme honestly. I would have preferred they fix something npm is lacking, like adding the ability to sign packages
What happens when there is the next Codehaus-like shutdown, and so much source code just won't work? Or when a bad actor takes control of a domain that commonly hosted packages (perhaps through completely legitimate means, such as registration expiration), can get a completely legitimate SSL certificate for it, and responds to requests for packages with malicious code? I think the abstraction, and to some degree the centralization, of package management is generally a good thing