This post extends the one about NixOS containers in ChromeOS to build Baguette images. I updated the nixos-crostini repository with experimental Baguette support. Give it a try and let me know how it works for you!

The ChromiumOS team is experimenting with a way (codename Baguette 🥖) for Chromebooks to run VM images directly instead of going through LXC containers.

The nixos-crostini repository provides a NixOS module to build LXC container images for Crostini (the established way to run arbitrary Linux guests in ChromeOS). When someone asked if it would be possible to support Baguette as well, I started taking a look at what it would take. This post describes the results.

# ChromeOS VMs

Under the hood, ChromeOS runs VMs through crosvm, a hardened virtual machine monitor. We already met it when investigating FIDO2 support in Linux ChromeOS guests.

Crostini uses crosvm to run a stripped-down VM called termina that boots quickly and runs the user’s containers. It also does a few more things:

  1. It mounts crosvm-tools, made available by the host through crosvm, into a guest directory (and later into containers as well).
  2. It runs vshd, allowing the host to get a shell on the guest.
  3. It handles the lifecycle of the VM and of its processes through maitred.

The default Baguette image is based on Debian and replicates all this. In addition, it also configures the VM to run garcon and sommelier directly (while in Crostini they run within the container). Together, they provide URI handling, file browsing, and X/Wayland forwarding: all the things that make Crostini container very pleasant to use.

# Baguette NixOS images

Our NixOS Baguette image will need to replicate the Debian image setup.

Typically, NixOS cannot run non-Nix executables due to the lack of FHS and of a global library path. Luckily, we won’t have to worry about that: crosvm-tools include their own libraries and dynamic linker, so they run without issues in NixOS.

Back when preparing NixOS LXC images for Crostini, we already figured out how to run garcon and sommelier when the user logs in, so that part is solved. Mounting crosvm-tools and adding systemd units for vshd and maitred was straightforward as well.

Next, I had to figure out how to correctly build an image of the format Baguette expects from the NixOS configuration: a BTRFS image built from a RootFS tarball. To build the tarball, I took a page from the lxc-container NixOS module. Then, to package it:

  • I initially re-used the Debian Python script, which depends on libguestfs-appliance and is not available for aarch64-linux in Nix1.
  • I later switched to a QEMU-based approach that works with Nix and supports ARM2.

After transferring the image to the Chromebook “Downloads” directory I figured out how to run it from crosh:

vmc start --vm-type BAGUETTE \
  --rootfs /home/chronos/user/MyFiles/Downloads/baguette_rootfs_raw.img \
  --writable-rootfs \
  baguette

Why such a weird CLI invocation? Because vmc create doesn’t recognize yet (as of ChromeOS 140) the option --vm-type, described in the baguette_image README. That flag was added to the source in August and it will probably need a bit more time before making it to production.

You might have heard about the #crostini-containerless flag. If you are trying this at home, you can run vmc start --vm-type BAGUETTE even without setting it. It only affects what happens when you use the ChromeOS UI to launch a Linux guest and, this way, you can try Baguette without losing your Crostini containers.

In order to correctly land to a shell, I symlinked the usermod executable under /usr/bin and made sure that a few Unix groups existed, in addition to the user chronos (the default in termina). Within the VM, I configured the DNS to rely on the host and a few environment variables required by the crosvm-tools.

The last piece of the puzzle was how to enable X/Wayland forwarding, which was failing. By diving into the source code I discovered that the /dev/wl0 device was missing read/write permissions for non-root users. A quick udev rule fixed that.

With that fixed as well, we are ready to shine! The baguette.nix file includes all the configuration in details, if you are curious. Here is the final result: a baguette-nixos VM correctly forwarding a Wayland session to ChromeOS.

A screenshot showing the `baguette-nixos` VM running Featherpad Wayland forwarding working in a Baguette VM.

# What’s next?

The nixos-crostini repository now includes experimental Baguette support. If you give it a try, let me know how it goes, I want to hear from you! You can reach me through any of the links in the footer.

I have marked it as experimental because of a few small issues. On ChromeOS 140, vmc cannot yet correctly import Baguette images. For this, we need to make the rootfs in the Chromebook’s Download folder as writable. In addition, vsh (used by vmc start) will always spawn a shell using the chronos user (which is then required to exist), even when instructed to do otherwise. I tried a few configurations (CLI flags for vmc or for vshd), but nothing seems to make it.

None of this should be a deal-breaker, especially if LXD was not giving you enough flexibility, performance, or control over the underlying (virtualized) hardware. Meanwhile, I will keep hacking to iron out these last details, integrate NixOS Baguette into the UI, and allow ourselves to pick the usernames we like the best.

Thanks for reading, and ‘til next time! 👋

# Footnotes

  1. I was running all experiments on the ARM-based Chromebook that I use for couch-computing. ↩

  2. Because I wanted the build scripts to run within the default penguin image, I also overrode the derivation so that, when /dev/kvm is missing, it will fallback to emulation. ↩