Rust without crates.io

•

Rust is a lovely programming language but I’ve never quite come to terms with crates.io, or any other of these language-specific repositories where everyone uploads and downloads code willy-nilly. I have several objections:

  • If crates.io goes down or access is otherwise disrupted then the Rust community will stop work. It is profoundly unresilient to have a single point of failure like this. Certainly some people will have vendored their deps and others will have a panamax mirror handy, but for most, Rust as we know it stops if this one particular web service goes down.
  • There is no mediation of any kind between when a new library/version is published and when it is consumed. You need only one author in your maybe-hundreds-of-dependencies tree to be hacked, coerced or in a malicious mood for you to have a really bad day.
  • Any tampering with crates.io itself (espionage, disgruntlement, national security) could have an incredibly wide blast radius, or a incredibly wide set of targets from which to choose.
  • Since crates.io is the source for crates, it is normal for both developers and CI machines to be hitting this web service all the time. Opportunities for mischief are exacerbated when clients are phoning home so frequently.

So what’s the alternative? I think we all need to take a step back from the altar of developer velocity and take a deep breath. I don’t want dependencies hot off the press. Ideally I want someone independent of the authors playing a curatorial role.

Now, actually getting some human review of dependency updates is quite a hard thing to do. cargo-crev has been trying for years to make this happen. I would love if it was the solution but it isn’t yet, and I think it’s a little ambitious. Yes we would like to have super-experienced software developers reviewing all our libraries with cryptographic stamps of approval, but if they’re not available we could be the target of remote shell in a build.rs. Surely there’s a middle ground here?

What’s interesting is that this problem is largely solved for C and C++: Linux distributions like Debian package such a wide range of libraries that for many things that you want to develop or install, you don’t need any third-party libraries at all. It’s just a matter of finding the right apt-get incantations and off you go. Even if you can get 95% of your libraries from a common trusted source then your risk is decreased considerably.

Rust libraries don’t work quite the same as C/C++ ones. Normal Rust code can’t be dynamically linked—a binary will have all of its dependencies statically linked at build time, so you won’t typically see .so files for Rust libraries that are going to be consumed by other Rust code. Since there is no .so file, Debian has no package that installs the library. However if they want to ship a binary that was written in Rust, their builders can’t just be downloading stuff from crates.io. They need a way to package all of the software that represents that Debian release. To solve that problem they’ve taken all these little dependencies and put their full Rust source code in packages with names like librust-cratename-dev.

Hmm, how many such packages? Running on trixie (testing)…

$ aptitude search librust- | grep -vE "^v " | wc -l
2336

This is starting to look like a serious curation of the most important Rust crates, available from any Debian mirror. There are some double-ups to be sure, since in some cases multiple incompatible versions of the same crate had to be packaged. Still. Maybe there is enough Rust in Debian now that it’s viable to write interesting Rust software independently of crates.io? That would solve basically all my concerns and the situation is only going to improve as more Rust software gets packaged.

To be clear, I don’t expect that Debian Developers are auditing these packages in the manner of cargo-crev. The good thing is that they don’t actually need to for it to be a major improvement.

  • A DD isn’t going to upload a new patch release just ‘cause. It’s going to be because it has an important fix or because some other program has depended on it. On crates.io a maintainer is free to create new releases for any reason and cargo update is not going to evaluate how good that reason is.
  • A simple time delay will allow egregious malware like malicious build.rs scripts to be caught, whether that’s the super-long Debian stable cycle or even the several days required to migrate from unstable to testing. I assume that an urgent security issue would be distributed the same as any other Debian update.
  • They might decide to give the diff at least a cursory look, which is better than nothing.

How do we do this? It’s actually quite easy because the big-brained Debian developers have arranged all the Rust dependencies to follow the format of a cargo Directory Source. That is, all of the packages are installed in their own directories under /usr/share/cargo/registry, including implementing a cheeky workaround for the required .cargo-checksum.json files.

You can then add some brief incantations to your .cargo/config.toml, whether on a project- or user-wide basis:

[net]
offline = true

[source]

[source.apt]
directory = "/usr/share/cargo/registry"

[source.crates-io]
replace-with = "apt"

This overrides the default crates.io source and ensures dependencies can only be fulfilled locally by installing the relevant packages. This happily doesn’t require any changes to your projects themselves—you just have to be careful to use versions in your Cargo.toml (and Cargo.lock) that are resolvable on Debian, since it is a subset of those available on the wider crates.io.

I am quite certain that Debian wouldn’t have enough coverage yet for the monorepo at work, but I gave this a go on my one of my little CLI projects that has half a dozen dependencies. Apart from having to downgrade copypasta from 0.8.2 to 0.8.1 in the Cargo.lock, this builds and runs just fine. What a treat.

This little investigation has given me much more confidence in using Rust generally into the future. I feared that the “grab any dependency version you like” approach facilitated by crates.io would render Rust impervious to any sort of curation effort, such that anyone who was serious about my earlier concerns would have to stick to a language used to the old ways like C++. Fortunately, Debian is here to prove me wrong. A+ work by their Rust packaging team.

All power to those who like to live on the edge; I’ll be over here trying to minimise different types of dependencies.