Ok, so I gotta ask - why package it at all? I'd never recommend it for a public tool, but in an internal, trusted environment, why not post a "curl ... | sh" command? Or point them at the master/main.zip of the repo where that script lives in gitlab/hub/something?
If this is aimed for developers, going through all of that to provide an installable and clickable thing feels a bit unnecessary...
Alternatively, since you're already using nixpkgs, that contains some wrappers and examples for signing Darwin binaries. May be easier than doing it from scratch.
> Ok, so I gotta ask - why package it at all? I'd never recommend it for a public tool, but in an internal, trusted environment, why not post a "curl ... | sh" command?
Because I don't want to install some tools on my PC just to build _your tool_. A packaged application is less complicated for the final user to use it.
In this, Windows and some Linux distribution have managed to create a good experience (MSI, AppImage, Flatpack)
The author wrote the following concerning a curl option:
"""So, what can we do with all of this? We have a few options, but none of them are particularly appealing. ...
Download the file with curl. Unix command-line tools like curl and tar don’t add the quarantine bit to the files they create on macOS, so it’s pretty easy to run software downloaded with them.
Unfortunately, the repository where this tool is developed is private, so unauthenticated curl downloads won’t work. I’m seriously considering asking the CTO if we can make the repository public solely so we can use curl as a distribution mechanism. I could even write a platform-detecting shell script!
"""
The "curl ... | sh" technique could get pass through the authentication information for a subsequent curl call, but then authentication becomes a facade.
That's why I added the bit about downloading master/main.zip. You can do that from the logged in browser and it's still around the same number of steps as installing a custom package and running it. (Login. Download the package/script. Install the package or unzip the script. Run it.)
> Unfortunately, the repository where this tool is developed is private
Have you thought about publishing the install.sh and final zip file through GitHub pages? Or Cloudflare pages? Then the repo stays private, but the built application can be accessed publicly, and behind a hard-to-guess url.
This is a great question! For some added context, this is the tool that bootstraps the developer environment. It installs Nix and Homebrew, configures the PostgreSQL service, and so on. I also work on developer UX full time, so I had hoped code signing would be a way to make the developer experience smooth and seamless (we already pay Apple for the signing certs, after all!).
> in an internal, trusted environment, why not post a "curl ... | sh" command?
It's internal code, so we can't use a public (e.g.) GitHub URL, and developers running this tool may not yet have SSH keys or the VPN set up yet.
> point them at the master/main.zip of the repo
Basically the same issue -- developers would have to install a Rust toolchain and compile the code themselves.
> Alternatively, since you're already using nixpkgs, that contains some wrappers and examples for signing Darwin binaries.
It would! But this tool is what developers use to install and configure Nix in the first place, so it's not a great fit here. (Although we do have it packaged in Nix for users that already have Nix installed.)
> Basically the same issue -- developers would have to install a Rust toolchain and compile the code themselves.
Why? The current script installs Rust for them, so the downloaded repo snapshot can do the same thing. Whatever is distributed in the signed package can be in that zip file instead.
I've been through the same hell and there's basically just one way to publish software for macOS with no hassle for the user: Completely lock yourself in their ecosystem. That means working on a Mac, building in Xcode using Swift and publishing in the official app store. If you want to do anything that resembles the free software approaches of Linux, stay far away from Apple. If you really need to publish on Mac for whatever reason, it is so much better to dump the hassle on the user. Even if the core targets are not developers. And who knows, if you ruin the experience for enough users, Apple might actually bother to fix their system.
Of course it's possible. Noone ever said anything on the contrary. The issue is how many hoops you have to jump through. And things like Tauri create at least as many problems as they solve once you build more complex things that are not minimal PoCs.
And why not respect the platform you want to support? Just like you get various Linux/BSD distros and they all have a different take on packaging and distributions... you also have the way macOS does it. There is no inherent expectation that "what I do on Linux" will work on macOS... or every Linux even.
That's exactly what's happening. But while most software devs still steer around Apple for these kinda of things, Windows and Linux have figured out a way to work nicely together. If you want to ship from one of them to the other, it is trivial - which is by no means expected or how it always was. And it did make everyone's lives easier.
It is "optional" in the sense that it removes hassle from the user. This is what the original article and my comment are all about. And don't get me started on building anything remotely more complex stuff for Mac from Windows or Linux. With the other too it's trivial to build and ship across by comparison.
I wouldn't call WSL hassle free for the user. Quite the contrary. And compiling stuff on Windows is a nightmare compared to Linux and macOS. It's become marginally better since you don't have to install the entire Visual Studio monstrosity anymore just to get a compiler that plays nice with the system. But it wasn't always like that. On macOS you could always install the necessary parts from Xcode without App store and activate it via command line.
>But while most software devs still steer around Apple for these kinda of things
Do they? Between Homebrew, MacPorts, the hundreds of thousands of Mac App Store apps, indie dev apps distributed outside MAS, not to mention all major proprietary apps like Abobe Creative Suite, MS Office, Pro Tools, and so on, probably more software devs steer around Linux than Apple..
To be exact though, Windows didn't figure out how to work together, they transplanted "a thing" (WSL) and I'm not sure that's a tool one can expect users who are not developers or IT folks to install or use. All "native" windows ways also require windowsy tooling (e.g. MSI, powershell etc...)
I did, and I'll highlight the relevant sections of your comment for you.
> I've been through the same hell and there's basically just one way to publish software for macOS with no hassle for the user: Completely lock yourself in their ecosystem. That means working on a Mac, building in Xcode using Swift and publishing in the official app store.
And yet all kinds of non-XCode major proprietary software exists, all kinds of third party indie dev software, all kinds of FOSS projects for Mac-only tools, as well as MacPorts, Homebrew, Nix, and everything in between.
Seems like part of the problem is that the author wants to both bootstrap homebrew, without users knowing the command line?
In which case, they should do the same thing other platforms do: user scripts.
Much like bat files on windows, macOS allows writing AppleScript which would sidestep their issues completely, or command files for shell scripts.
Otherwise they don’t need to sign command line tools. macOS doesn’t block tools run in the terminal to the best of my knowledge.
If they want to make an app, that also doubles as a command line tool, it’s pretty simple as well. You can check for command line options when running the main executable. I’ve done this several times. Apps aren’t magic.
They can self sign that app, and their end users can bypass the unknown developer warning, or just launch the command line part.
I believe you just need to remove the quarantine bit. I’ve definitely been distributing unsigned utility CLI tools to people on macOS 14. They’re built via CMake and so there’s no implicit signing.
xattr -d com.apple.quarantine Foo
The author of the post even mentions this BUT they’re operating in a very weird space where they want to make a command line tool but don’t want people to use a terminal to access it.
To me it’s a nonsensical issue. I suspect the author got flustered with a set of ideals and didn’t take the necessary step back to reevaluate their situation
Hm, this doesn't match my experience at all, I just use rcodesign (the third party rust tool), some plist and xar magic (that I honestly forgot) and call it a day
it's sad that we need third party tool to be able to codesign and notarize effectively (and from other OSes and CIs), but it works now
but maybe I miss something that the author needed and I didn't. The hardest part was cross-compiling for macOS but we did it too with these images from elasticsearch
It seems strange to try and make a sandboxed tool to perform work which is not sandboxed at all...
There is a class of macOS applications which can perform such functions and they're called Packages or Distributions (it's the wizard you see sometimes when an app needs to make changes to the system before you can use it). That's the way to deliver such applications from a downloadable source. Packages can be signed and notarised and none of the crazy workarounds listed in the post will be necessary.
It’s a weird time as a developer when it’s now easier to install WSL2 on an enterprise machine and use that for development than it is to use an actually Unix based OS
Last time I ran a Linux desktop, I had to reinstall my graphics drivers in a tty console every time I wanted to do a kernel upgrade, because the distro ones were ancient and that was the only way to get the latest from NVidia.
Personally, it always run fine at the beginning until it isnt and then you have to spend the rest of the day trying to figure out what is wrong. For desktop that is it, servers can just run 24/7 without problems (mine just run containers).
Plasma looks pretty good but is not nowhere as good as window's.
I heard that they fixed power management but haven't tried myself.
My last point is that hyper-v got soo good with GPU partitioning and can be fully managed using type-safe Powershell.
Until you realise that WSL2 is nothing but a Linux VM with some shoddily added sugar that completely breaks down as soon as real work demands handling encodings properly, opening windows at the end of a pipe, and all those other things that I regularly want do with a Unix like system. All of this works seamlessly on Linux or macOS, but Windows is still a pathetic shitshow, WSL2 or not.
> We also need to notarize the .app and the .dmg. Interestingly, you can only notarize .pkg, .dmg, and .app files (in .zips) — command-line tools can only be notarized if they’re embedded in one of the listed containers.
I don't think this is true, actually. I've been notarizing command-line builds of unxip by shoving it in a zip and uploading it to notarytool for a while: https://github.com/saagarjha/unxip/blob/main/release.sh. (You will note, amusingly, that I am suffering from the same -parse-as-library bug as the author.) After this is done I just chuck the zip file and upload the binary directly to GitHub. If you download the file from your browser and chmod +x it, you can double click on it and it runs. Obviously you can skip the chmod +x step by sending people the zip (which preserves permissions inside of itself).
Also, FWIW, Apple's tooling sucks but if you check Console while it evaluates the code sometimes it will tell you why it doesn't like it. Most of the evaluation code is open source, too, so you can look up log messages and errors there: https://github.com/apple-oss-distributions/Security
Yes, you can notarize command-line tools. The only downside is that you can't staple the notarization ticket to the tool, so it has to be downloaded by syspolicyd on first run.
In most cases that isn't an issue, though it won't work if your internet connection is down or if you're blocking syspolicyd to keep if from phoning home to Cupertino. ;-)
Incidentally, the other day I discovered that the Arc web browser neglected to staple the notarization ticket to their app.
There's a hidden assumption in the headline here: shipping a command-line tool to non-developers who don't use the command line is hard. If you want to make stuff double-clickable in the Finder you need to jump through a lot of hoops, yes.
Shipping a command-line tool for developers is easy. Put it on Homebrew or NPM or something, or just give them a URL for curl.
macOS is Unix, it's just that the Unix bits are carefully hidden away from most users.
Or MacPorts, which was created by Apple employee and inventor of the FreeBSD ports system Jordan Hubbard, has more packages than Homebrew, and has a much more correct design. Homebrew is a bit of a joke in comparison, not sure why it gets all the attention.
I don't know either! I've tried MacPorts in the past but for some reason I always ended up on Homebrew again -- I can't recall why. I should probably give it another try.
One possible minor advantage of Homebrew is that it works on both macOS and Linux. I recently found this very handy for setting up a project with developers on Macs and CI on Linux. Almost no OS-specific build steps were needed.
A couple more points, from a quick scan of the respective docs:
"Homebrew won’t install files outside its prefix and you can place a Homebrew installation wherever you like." The MacPorts docs are much less clear, I can't tell if it can do that.
Homebrew gives "brew install --cask firefox" as an example of installing a GUI app. MacPorts doesn't seem to have Firefox.
I'm curious what makes MacPorts "much more correct", if anyone wants to clarify that.
As this targets developers, aren't developers supposed to be familiar with the terminal. So doing `chmod +x` is trivial. Especially since they probably use a lot of other tooling to build, package, deploy, etc.
Seriously. You want to ask the CTO to make artifacts public because you don't want devs to set or clear an attribute bit? Talk about entirely the wrong priorities.
Where did this coddled sense of entitlement come from? Are they creating their own SSH keys? Do they know the basics of the git cli? Or are these "magic commands" too? Do they know how to do their job?
I sympathise that Apple's tooling could be better, but the conclusion and tone is ridiculous given what could charitably be described as 'unique' requirements.
Our internal CLI tools have all largely been hassle free regardless of devs OS. We do use GoReleaser which abstracts away the multi arch builds, signing, and publishing to GitHub and Homebrew, which can be criticised sometimes as a bit of a crutch. But it's also been zero hassle for all of our Mac devs (the most junior of which are still expected to be able to download and execute a binary from a terminal.) and lets us get on with our day without prioritising 'requirements' that aren't even worth spending any time on.
Why would you even ship a command-line tool if your apparent need is to avoid the command line?
Apple’s tools really could emit better messages when things go wrong. It’s sad that they hit a dead-end with this after everything they went through.
Windows isn’t a cakewalk either (you now have to use a hardware security module to store your certs, which means USB stick based signing or using a cloud service that supports HSMs, plus certs themselves are a racket in terms of pricing).
No, Windows 11 presents two scary-looking popups for unsigned apps:
- “Windows protected your PC” with some tiny blurb no-one seems to read and a single button labelled “Don’t run”.
- If you do read the tiny blurb and spot the “More info” link, you can click it to get a summary about Windows Defender SmartScreen having saved the day by protecting you from this app by an “unknown publisher”, and then a second button becomes available that says, “Run anyway”.
It’s pretty hostile and it effectively forces devs to throw 300-600usd/year at a cert provider plus spend time to configure signing via CI.
There are also two levels of certificate, and the better one is only available if you’ve been in business for several years with an office address and public phone number (cert providers need to call you at your public number to validate your business ID).
There are some ways around it (I think games distributed via Steam’s installer still don’t need to be signed) but those aren’t an option for many.
No, you have to do a bit of work to proceed with the installation. Windows with throw some scary warnings at you if you download something unsigned and try to run it. And if you're very unlucky and anger the SmartScreen, you may need to use power user level workarounds.
My company ships a simple CLI, it's written in Node.js so it can be installed on any platform using npm. A a Docker image is also provided. Most of the target audience has either of these installed.
I don't quite understand what the problem is with asking the developers to download the tool using curl. The developers could add their username and password to the command line...
I’ve not used macOS in forever but I never had issues with cli binaries installed with Brew or Nix or just manually added to some path. Are they forcing all binaries to be signed now?
For some reason they want to distribute this command line script as a double-click installer which doesn't need to be signed either. Users will simply need to override the "this is untrusted and you shouldn't open this" dialog.
Agreed. It doesn’t seem like there is anything you’d want to do in a binary that couldn’t be done by creating an installer .pkg and just including all the logic in the post install script.
The user gets a double-click “installer”, and like you said, you don’t even need to sign it.
This is really specific, but for this point in the article:
> There’s a long guide on Embedding a Command-Line Tool in a Sandboxed App, so I followed that, and then slowly, painfully, factored Xcode out of it, so that I wouldn’t have to figure out how to get a 10GB Xcode install onto the CI machine (remember, you need to be signed in to an Apple ID to download Xcode, and there’s no way to do it from the command-line).
You could actually solve this with Rust and no Xcode whatsoever. cacao [1] and cargo-bundle [2] will produce an app bundle you could sign/notarize/distribute without needing to ever open Xcode.
You wouldn't need to tangle with Swift, would only need enough Rust to hoist whatever script you're trying to run.
In general though I find the article a bit odd - there are also quite a few examples of signing/notarizing/etc for GitHub Actions out there already, I went through this exercise a week ago and it wasn't that bad.
Edit: Hell, you could also potentially just bundle your scripts up into a `.pkg` installer
At a past job the team had a custom CLI tool to manage all sorts of dev tooling. Worked great, highly recommended.
The distribution mechanism was to build from source[]. Once compiled, all other dev tasks were handled by the tool.
If you compile it yourself you don't run into any of these issues. This approach is only viable for developer tools of course, but that's what the post is about.
[]: This was a Go project, so building from source was relatively straight forward.
On success you can then call the private repo and execute the cli installer. If the bootstrap is piped then it doesn't ever have to be installed, and you don't have it junking up your system.
Extra points to pull down existing registed public keys and check if they are already there.
it's not clear to me what you want to happen. It sounds like you have a command line tool that takes no arguments. you want a Mac user to be able to download a file, click that file, have it run once? and then delete itself? Not sure about this last part
in any case I think there are plenty of apps that do this. Some are distributed as packages instead of apps.
I'm not a Mac dev expert at all but I have used enough apps to know it's possible. Many of those apps are open source maybe look into them if you want to solve this issue.
Electron apps have tools for automatically signing on GitHub actions so looking though what they do might help?
sorry I don't have concrete suggestions. I'm just confident there are solutions.
I'm totally sympathetic to the fact they it's a pain in the ass and Apple, probably intentionally, doesn't document the process well or at all
> Electron apps have tools for automatically signing on GitHub actions so looking though what they do might help?
These only sign/notarise the dmg or app bundles (or the binaries inside them), so I don’t think that will help the OP unless they’re willing to wrap their CLI app in an Electron app, unfortunately.
Apparently it's about a command line tool that's not downloaded and started on the command line, but instead downloaded in a browser and started by double clicking in the Finder (apparently for non-technical users?). TBF, this also wouldn't work on Windows without an expensive EV code signing certificate (which isn't exactly trivial to use with GH Actions either: https://melatonin.dev/blog/how-to-code-sign-windows-installe...)
...from what I'm seeing on macOS, downloading and running a cmdline tool (built with the Xcode cmdline toolchain on GH Actions) inside the terminal still works without all the hassle described in the article - of course I wouldn't be surprised if that stopped working too one day :/
PS: I kinda wish that operating systems had builtin WASI runners for this kind of bootstrapping work.
One idea, if the customers tend to be NixOS enjoyers, would be to put the tool in Nix Packages and then use nix-shell for them to access it.
Perhaps the nix packages configuration could reference a script with everything properly debugged and then the snake can eat its own tail.
It seems like a provisioner to install and configure stuff, in which case one obvious method to install is copy pasting bash code. Have we really reached an era where folks expect to use CLI tools without using a CLI?
Can nixpkgs install from brew? Maybe that’s the answer, use the annoying stuff and wrap it with something to make it less annoying. Or, you could wrap your CLI with Tauri and install with Cargo. Might be more robust but not in the comfort zone of a nix shop.
You'll get a kick out of knowing the tool I was attempting to codesign is the tool that performs the initial install and configuration of Nix (and Homebrew) for our engineers :)
> Do not attempt to “create software” for macOS. They don’t want you to. If you want to run your own programs, install Linux and suffer like you’re supposed to.
Yeah, I'm starting to understand that more and more. I'm not a big fan of the PWA SPA-approach, but given what's going on I'm going to have to change my tune.
Selecting the "Add to Dock..." option in Safari for YouTube Music has been kind of eye-opening. It works really well.
I'm not sure what they're thinking here. Are native desktop apps done and over? It kind of seems like that's what they want based on behavior.
> [workaround 2] Download the file with curl. Unix command-line tools like curl and tar don’t add the quarantine bit to the files they create on macOS, so it’s pretty easy to run software downloaded with them.
Wait I'm confused by this part (never used MacOS).
Which part of the chain mechanism mentioned above can be bypassed by just downloading the file via CURL?
I didn't seem to see the author mentioned (not explicitly at least) which steps added these "quarantine bits".
Do untagged files (like downloaded via curl) not require signing at all, or it's just lenient enough to not trigger the bugs/issues mentioned in the article (since their app is actually signed, after all)?
Why not just ship it as a Homebrew package instead? It's jankier than any of the usual Linux packaging systems, but should get the work done. (Of course, it doesn't solve the “running from Finder” problem, but then again you don't usually run your command line tools by double clicking them in file manager, do you?)
Do you understand the reasoning though? The explanation says the tool is responsible for "installing homebrew", but also that the developers this thing is for "refuse to install homebrew". Seems paradoxical.
But if this blog post gets Apple to improve their code signing process, then it does have value :)
The reason is that this is meant to be a script to bootstrap their startups systems to the way the poster wants.
But I think they have a really odd set of requirements in that they don’t want their devs to run a CLI tool from the terminal, but want to run it from Finder?
1. New team member gets new fresh Mac
2. New team member downloads the tool and runs it
3. New fresh Mac is now set up with a development environment suitable for the company, including a working Homebrew installation (or Nix or whatever)
If this is aimed for developers, going through all of that to provide an installable and clickable thing feels a bit unnecessary...
Alternatively, since you're already using nixpkgs, that contains some wrappers and examples for signing Darwin binaries. May be easier than doing it from scratch.