Skip to content

add disk script - #469

Merged
ryantm merged 1 commit into
mainfrom
rtm0128adddiskscript
Jan 28, 2026
Merged

ryantm merged 1 commit into
mainfrom
rtm0128adddiskscript

Conversation

@ryantm

@ryantm ryantm commented Jan 28, 2026 •

Copy link
Copy Markdown
Collaborator

Why

  • The nix modules disk builds too slowly.
  • Moving the squashfs and tarball steps out of nix should make it faster
  • We can also speed up things and make the disk faster with different compression choices

What changed

  • move squashfs and tarball steps into one script called disk-script that you can run with nix run -L .#disk-script or nix run -L .#disk-script-dev
  • run xargs cp in parallel
  • use lz4 compression for squashfs and 1M block size
  • use --best --recursive for pigz compression on tarball
  • add output for the time each stage takes

Test plan

  • ran it locally
xargs copy took 306 seconds
mksquashfs took 148 seconds
tar took 140 seconds

~10 minutes (note this was with the .#bundle already being built and cached on my local system.)

$ du -h /tmp/tmp.Lj42PAdmKN/disk.sqsh
21G     /tmp/tmp.Lj42PAdmKN/disk.sqsh

$ du -h /tmp/tmp.Lj42PAdmKN/disk.sqsh.tar.gz
17G     /tmp/tmp.Lj42PAdmKN/disk.sqsh.tar.gz

@ryantm
ryantm requested a review from a team as a code owner January 28, 2026 21:53
@ryantm
ryantm requested review from airportyh and vlinkz and removed request for a team and airportyh January 28, 2026 21:53
Why
===
* The nix modules disk builds too slowly.
* Moving the squashfs and tarball steps out of nix should make it
faster
* We can also speed up things and make the disk smaller with different
compression choices

What changed
===
* move squashfs and tarball steps into one script called disk-script
that you can run with `nix run -L .#disk-script` or `nix run -L .#disk-script-dev`
* run xargs cp in parallel
* use lz4 compression for squashfs and 1M block size
* use --best --recursive for pigz compression on tarball
* add output for the time each stage takes

Test plan
===
* ran it locally

```
xargs copy took 306 seconds
mksquashfs took 148 seconds
tar took 140 seconds
```
~10 minutes

```
$ du -h /tmp/tmp.Lj42PAdmKN/disk.sqsh
21G     /tmp/tmp.Lj42PAdmKN/disk.sqsh

$ du -h /tmp/tmp.Lj42PAdmKN/disk.sqsh.tar.gz
17G     /tmp/tmp.Lj42PAdmKN/disk.sqsh.tar.gz
```
@ryantm
ryantm force-pushed the rtm0128adddiskscript branch from bc590ec to bf47ba8 Compare January 28, 2026 21:57
@ryantm
ryantm enabled auto-merge January 28, 2026 22:00
@ryantm
ryantm merged commit a1dd68b into main Jan 28, 2026
3 checks passed
@ryantm
ryantm deleted the rtm0128adddiskscript branch January 28, 2026 22:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants