Alan's Lair

Application authors: please don't force users into your language or packaging details

Alan Franzoni -

This story has been boiling in my head since long; today I chose to (finally) publish it.

Long story short: in order to use a certain application, I should not need to understand how to use the language or its packaging ecosystem. Delivery and distribution is a relevant part of your app.

This does not apply to libraries, frameworks, or tools that are highly contextual for a certain language/environment ecosystem, and would be used only by a developer, in any case.

What do I mean? I'll start with an example. It's chronological, it's just the latest thing I came into; here's b2, the command line tool to get access to Backblaze B2 backup repositories. And those are its installation instructions:

  • What is pip?
  • Is such command safe to use?
  • Will it work in all situations?
    answers:
  • Python's package manager. It makes the b2 library available, and such library has the so-called b2 script, which exposes the b2 CLI executable.
  • possibly safe, but will install b2 within your global/per-user python dependencies, and may alter or install additional packages as dependencies, and such deps may be picked by other, unrelated software in the system;
  • it may require root to be run correctly.

Should every user of Backblaze B2 CLI know how to code in Python and understand how the Python environment works? I think that should not be necessary.

b2 is just an example, other famous tools - awscli and fpm come to my mind - just follow suit.

Most probably, if the tool you're downloading is written in C or C++, you'll probably expect to find a compiled binary, at least for some OSes and architectures; The same applies for Go, which makes binary creation for multiple platforms exceptionally easy. You should strive to provide some kind of equivalent piece of software for your users. .

But, even for tools written in Python or JavaScript (they look to be the ones that suffer most from the problem I'm describing), in most situations, the "right thing" to do is to provide a standalone binary, regardless of how it's written. You can provide standalone binaries for some archs (this is what borg does - it's a great backup utility written in Python), or you can rely on native or add-on packaging (additional repositories for Linux, choco for Windows, homebrew for Mac), like httpie. And, when you create such binary or package, you should make sure it's totally standalone (i.e. it doesn't alter global system state).

Why? Because leveraging packaging tools that were originally meant for developers of a language is brittle and risky, and can be hard to understand. You may end up compromising the system integrity if the wrong dependency slips into globally installed packages - this is especially true for Python, which is often used by many, many system tools. And, if all third-party apps did it that way, we could have continuous breakages. You can't assume yours is the only application installed in a certain system!

And, if you're writing an app, you can have full control of your execution environment and dependencies. You can choose a single Python or NodeJS version, and all of its dependencies. You don't need to support tons of variations!

In some situations, a CLI evolved out of a library. This is the case for pygments, a very nice syntax highlighting library. It's meant to be used as a library from inside python projects, but it also exposes a widely used pygmentize binary that is employed by many other tools. I'd love to have that CLI tool available as an independent, standalone package!

Final considerations:

  • As a software author, think whether you're building a library, a framework, or an application. If you're writing an application, think how you can deliver it to your final users, without the need for them to understand how you've written it.
  • If you're a maintainer for a package, think about what you're maintaining. If something is both a library and a command line tool, consider creating multiple (possibly independent) packages.

In the meantime:
If you find a Python CLI tool that you'd like to use, and it uses the pip way and you don't to mess with system dependencies, I recommend pipsi. I don't know if something similar exists for Node or other environments.

FAQ:
Q: But I'm a solo open source developer! Packaging this way would steal too much of my time!
A: That's totally fine. Just make sure you don't suggest dangerous things to people. If you're using a developer-only approach, just state that as your target; but the tools I've written about above (b2 and awscli) certainly are of a different class.

Q: Any suggestion for easy packaging?
A: I find homebrew very nice for Mac packaging and distribution. For Linux, I use fpm with docker - I've actually created a specific integration project fpm-within-docker. For Windows... no idea! I know about chocolatey and that's about it.

Photo by Malcolm Lightbody on Unsplash