#+title: Workflow #+author: cashmere * Overview This repository is a custom FreeBSD ports overlay. Ports that live here are built and tested in an isolated poudriere jail; everything they depend on from the official tree is fetched as an official binary package. Only overlay ports are ever compiled from source. Tool choice, for the record: - *poudriere* :: builds ports in clean jails, supports overlays, bulk builds and single-port testing (=testport=). The standard tool for this job. - *synth* :: similar goals, but no overlay support. - *portmaster* / plain =make install= :: no build isolation. One privilege rule applies everywhere: poudriere always runs via =doas= (it manages jails and mounts; =doas.conf= already permits it without a password), and host-side =make= in a port directory never runs as root. * Layout #+begin_example ~/ports ├── devel/ # custom development ports │ ├── bash-language-server/ │ ├── just-lsp/ │ ├── nixd/ │ ├── py-rassumfrassum/ │ ├── pyrefly/ │ └── yaml-language-server/ ├── misc/kimi-code/ ├── security/pinentry-emacs/ ├── distfiles/ # shared distfiles cache ($DISTDIR) ├── work/ # host-side build directories ($WRKDIRPREFIX) ├── pkglist # all overlay ports; input for poudriere bulk └── WORKFLOW.org # this file #+end_example Category directories (=devel/=, =misc/=, =security/=) mirror the top-level layout of the official Ports Collection. The poudriere configuration is not stored here — its source of truth is =dotfiles.org= (see step 2 below). * Setup (once) ** 1. Install poudriere #+begin_src sh doas pkg install -y poudriere #+end_src ** 2. Configure poudriere The authoritative =poudriere.conf= lives in =dotfiles.org= under =Tasks → Ports → poudriere.conf=. Open =dotfiles.org= in Emacs and tangle it (=M-x org-babel-tangle=, or just save — auto-tangle is enabled). The block is tagged =:tangle /doas::/usr/local/etc/poudriere.conf=, so tramp writes the file as root via doas and prompts for the login password. What the config does, in short: - =PACKAGE_FETCH_BRANCH=quarterly= + =PACKAGE_FETCH_URL=pkg+https://pkg.FreeBSD.org/FreeBSD:15:amd64= :: fetch official binary packages for dependencies instead of compiling them. - =ZPOOL=zroot=, =ZROOTFS=/poudriere=, =BASEFS=/usr/local/poudriere= :: jails and trees live on the existing ZFS pool. - =DISTFILES_CACHE=/home/cashmere/ports/distfiles= :: poudriere reuses the distfiles already collected in this repo. - =USE_TMPFS=yes= :: build in memory where possible. ** 3. Export the ports environment Host-side =make= in a port directory must work as the unprivileged user, without flags. Three variables take care of that; they live in the =.env= block of =dotfiles.org= (tangle, then re-login or source =~/.env=): #+begin_src sh DISTDIR=$HOME/ports/distfiles WRKDIRPREFIX=$HOME/ports/work PORTSDIR=/usr/local/poudriere/ports/base export DISTDIR WRKDIRPREFIX PORTSDIR #+end_src =DISTDIR= keeps distfiles in this repo instead of a root-owned tree; =WRKDIRPREFIX= keeps build directories out of the port directories (a =work= symlink in the port directory still points at the real build directory); =PORTSDIR= points host-side =make= at the poudriere-managed base tree (step 5), so the framework files come from the same tree poudriere builds against and no =/usr/ports= checkout is needed at all. With these set, plain =make fetch= / =makesum= / =stage= / =makeplist= work in any port directory — never with =doas=. ** 4. Create the build jail One jail, named after the OS version and architecture (15.1-RELEASE amd64 at the time of writing): #+begin_src sh doas poudriere jail -c -j 151amd64 -v 15.1-RELEASE -a amd64 #+end_src ** 5. Create the ports trees Two trees. =base= is a git clone managed by poudriere itself, created pinned to the quarterly branch that matches =PACKAGE_FETCH_BRANCH= (=2026Q3= as of mid-2026). =custom= null-mounts this repository, so edits here are visible to poudriere immediately; poudriere never writes to it. #+begin_src sh doas poudriere ports -c -p base -m git+https -B 2026Q3 doas poudriere ports -c -p custom -m null -M /home/cashmere/ports #+end_src A =/usr/ports= checkout is not needed at all — =PORTSDIR= (step 3) points host-side =make= at the poudriere tree. If one exists, delete it; on this system it is its own ZFS dataset, so plain =rm= fails with =Device busy= — destroy the dataset instead: #+begin_src sh doas zfs destroy zroot/usr/ports #+end_src ** 6. Update the base tree The tree and the binary package repository (=PACKAGE_FETCH_BRANCH=quarterly=) must stay on the same quarterly branch, or version mismatches force poudriere to rebuild official-tree dependencies from source. Routine updates are a =git pull --rebase= of the pinned branch — poudriere never switches branches: #+begin_src sh doas poudriere ports -u -p base #+end_src When the quarter rolls over (e.g. =2026Q3= → =2026Q4=), switch the tree to the new branch, then update: #+begin_src sh doas git -C /usr/local/poudriere/ports/base checkout 2026Q4 doas poudriere ports -u -p base #+end_src * Building and testing ports ** A single port: =testport= #+begin_src sh doas poudriere testport -j 151amd64 -p base -O custom -o devel/py-rassumfrassum #+end_src =testport= builds the port in the jail and then runs the full QA cycle: stage, =check-plist=, install, deinstall, reinstall. Dependencies from the official tree are fetched as binary packages; only overlay ports compile. When a build fails, =testport -i= drops into an interactive shell inside the jail after the attempt (cleaned up on exit), and =-w= preserves the build directory for inspection: #+begin_src sh doas poudriere testport -i -j 151amd64 -p base -O custom -o devel/foo #+end_src Good smoke tests for a fresh setup: - =devel/py-rassumfrassum= :: pure Python, tiny dependency tree. - =security/pinentry-emacs= :: a shell script; depends only on =shells/bash= and =security/pinentry-curses=. ** Several ports: =bulk= The tracked =pkglist= file at the repository root lists every port in this overlay, one origin per line. Build them all with: #+begin_src sh doas poudriere bulk -j 151amd64 -p base -O custom -f /home/cashmere/ports/pkglist #+end_src =bulk= produces packages for everything on the list. It does not run the interactive install/deinstall tests =testport= runs, so use =testport= when validating a port and =bulk= when you just want the packages. ** Stopping a build #+begin_src sh doas poudriere jail -k -j 151amd64 #+end_src ** Clearing stale state After changing the poudriere configuration or switching the base tree's branch, remove the stale package cache metadata or poudriere may refuse to fetch packages: #+begin_src sh doas rm -rf /usr/local/poudriere/data/packages/151amd64-base/.building #+end_src ** Pruning old distfiles =poudriere distclean= cannot be used with this setup: with a shared =DISTFILES_CACHE= it only sees the trees it is passed — =-p base= alone would delete the overlay ports' distfiles (nothing in =base= references them), and =-p custom= fails because the overlay has no =Mk/= directory of its own. Prune by hand instead: after a version upgrade, delete the previous version's distfile from =distfiles/= — the current, referenced file names are listed in the port's =distinfo=. * Creating a new port A port is a directory inside a category (=devel/foo=, =misc/bar=, ...) containing at minimum: - =Makefile= :: port metadata and build logic (see the Porter's Handbook). - =pkg-descr= :: a paragraph describing the software. - =distinfo= :: checksums of the distfiles, generated by =makesum=. - =pkg-plist= :: the packing list. Not needed when the port uses =USE_PYTHON=autoplist= or sets =PLIST_FILES=. - =files/= :: patches and extra data files, if any. Existing ports to copy from: =devel/py-rassumfrassum= for a minimal Python port, =security/pinentry-emacs= for a no-build script port, =misc/kimi-code= for an npm-based port. Procedure, from the new port's directory. All =make= invocations run as the normal user; the environment from setup step 3 handles the paths. 1. Write the =Makefile= and =pkg-descr=. 2. Fetch the distfiles and generate =distinfo=: #+begin_src sh make fetch make makesum #+end_src 3. Extract and look around =work/= to verify the build/install assumptions: #+begin_src sh make extract #+end_src 4. Stage and generate the packing list. The first line of =makeplist= output is a reminder (=/you/have/to/check/what/makeplist/gives/you=), not a plist entry — =tail -n +2= strips it. Review the result: #+begin_src sh make stage make makeplist | tail -n +2 > pkg-plist #+end_src 5. Lint the port (optional but recommended): =portlint -AC= checks against porting conventions, =portclippy= checks variable ordering, =portfmt= rewrites the =Makefile= in canonical format: #+begin_src sh portlint -AC portclippy Makefile portfmt Makefile #+end_src 6. Test in the jail until =check-plist= and the install/deinstall cycle are clean: #+begin_src sh doas poudriere testport -j 151amd64 -p base -O custom -o devel/foo #+end_src * Upgrading a port ** Standard ports For ports that fetch a normal upstream distfile: 1. Bump =DISTVERSION= (or =PORTVERSION=) in the =Makefile=. 2. Refresh =distinfo=: #+begin_src sh make clean make fetch make makesum #+end_src 3. If the installed file list changed, regenerate =pkg-plist= as in step 4 of "Creating a new port". 4. Test: #+begin_src sh doas poudriere testport -j 151amd64 -p base -O custom -o devel/foo #+end_src 5. Prune the previous version's distfile (see "Pruning old distfiles"). ** npm-based ports (e.g. =misc/kimi-code=) These ports build their distfile by running =npm ci=, which requires a =files/package-lock.json= pinning the package to exactly =DISTVERSION=. After a version bump the checked-in lockfile still pins the old version, and =npm ci= aborts with =lock file's X@old does not satisfy X@new=. The lockfile therefore has to be regenerated before anything else. Procedure, from the port's directory: 1. Bump =DISTVERSION= in the =Makefile=. 2. Move the stale lockfile aside (in the port directory, not in =/tmp=): #+begin_src sh mv files/package-lock.json files/package-lock.json.old #+end_src 3. Regenerate the lockfile, the distfile and =distinfo=. The first =fetch= finds no lockfile and writes a fresh one to =work/package-lock.json.generated=; the second rebuilds the distfile reproducibly from that lockfile — the same file poudriere later consumes via =DISTFILES_CACHE=: #+begin_src sh make clean make fetch cp work/package-lock.json.generated files/package-lock.json make clean make fetch make makesum #+end_src 4. Extract and verify the upstream layout assumptions used by =do-build= / =do-install= still hold (for kimi-code: =dist/main.mjs=, =node_modules/node-pty=, =native/=): #+begin_src sh make extract #+end_src 5. Regenerate =pkg-plist= as in step 4 of "Creating a new port". 6. Test (this also runs =check-plist=): #+begin_src sh doas poudriere testport -j 151amd64 -p base -O custom -o misc/kimi-code #+end_src 7. Clean up: delete =files/package-lock.json.old= and prune the previous version's distfile (see "Pruning old distfiles"). * Troubleshooting - Everything compiles instead of being fetched as packages :: check, in order: (1) =/usr/local/etc/poudriere.conf= has =PACKAGE_FETCH_BRANCH=quarterly= (no typos) and a literal =FreeBSD:15:amd64= URL — an unescaped =${ABI}= resolves to an empty string; (2) the =base= tree is on the matching quarterly branch (see setup step 6); (3) the stale =.building= cache was removed (see "Clearing stale state"). - =npm ci= aborts with =lock file's X@old does not satisfy X@new= :: the lockfile was not regenerated after a version bump; see "npm-based ports". - =tar: Failed to open '/usr/ports/distfiles/...'= on the host :: the ports environment from setup step 3 is not active in this shell; re-source =~/.env= or re-login. - =check-plist= complains about =/you/have/to/check/what/makeplist/gives/you= :: the reminder line from =make makeplist= ended up in =pkg-plist=; strip the first line. - A build hangs or must be aborted :: =doas poudriere jail -k -j 151amd64=. - =USES=cargo= port seems to build all of Rust :: it does not; =lang/rust= is installed as a binary package. Only the port itself compiles.