diff options
| author | cashmere <cashmere@autistici.org> | 2026-09-04 09:05:24 +0200 |
|---|---|---|
| committer | cashmere <cashmere@autistici.org> | 2026-09-04 09:05:24 +0200 |
| commit | 4f49b16079b27d7ce633619e4cddfdf2bffbc952 (patch) | |
| tree | c296c1e7f3c3e2f6859c6d3a1b1b208779b89679 /www/server/debian | |
| download | relic-4f49b16079b27d7ce633619e4cddfdf2bffbc952.tar.gz | |
Diffstat (limited to 'www/server/debian')
| -rw-r--r-- | www/server/debian/index.md | 6 | ||||
| -rw-r--r-- | www/server/debian/nginx.md | 395 | ||||
| -rw-r--r-- | www/server/debian/service.md | 259 |
3 files changed, 660 insertions, 0 deletions
diff --git a/www/server/debian/index.md b/www/server/debian/index.md new file mode 100644 index 0000000..220d177 --- /dev/null +++ b/www/server/debian/index.md @@ -0,0 +1,6 @@ +# Debian/Ubuntu Specific Fossil Service Options + +- [SCGI under nginx](./nginx.md) +- [`systemd`](./service.md) + +*[Return to the top-level Fossil server article.](../)* diff --git a/www/server/debian/nginx.md b/www/server/debian/nginx.md new file mode 100644 index 0000000..0140b37 --- /dev/null +++ b/www/server/debian/nginx.md @@ -0,0 +1,395 @@ +# Serving via nginx on Debian and Ubuntu + +This document is an extension of [the platform-independent SCGI +instructions][scgii], which may suffice for your purposes if your needs +are simple. + +Here, we add more detailed information on nginx itself, plus details +about running it on Debian type OSes. This document was originally +written for and tested on Debian 10 (Buster) and Ubuntu 20.04, which +were common Tier 1 OS offerings for [virtual private servers][vps] +at the time. The same configuration appears to run on Ubuntu 22.04 +LTS without change. This material may not work for older OSes. It is +known in particular to not work as given for Debian 9 and older! + +We also cover [adding TLS](#tls) to the basic configuration, because several +details depend on the host OS and web stack details. Besides, TLS is +widely considered part of the baseline configuration these days. + +[scgii]: ../any/scgi.md +[vps]: https://en.wikipedia.org/wiki/Virtual_private_server + + +## <a id="benefits"></a>Benefits + +This scheme is considerably more complicated than the [standalone HTTP +server](../any/none.md) and [CGI options](../any/cgi.md). Even with the +benefit of this guide and pre-built binary packages, it requires quite a +bit of work to set it up. Why should you put up with this complexity? +Because it gives many benefits that are difficult or impossible to get +with the less complicated options: + +* **Power** — nginx is one of the most powerful web servers in the + world. The chance that you will run into a web serving wall that you + can’t scale with nginx is very low. + + To give you some idea of the sort of thing you can readily + accomplish with nginx, your author runs a single public web server + that provides transparent name-based virtual hosting for four + separate domains: + + * <p>One is entirely static, not involving any dynamic content or + Fossil integration at all.</p> + + * <p>Another is served almost entirely by Fossil, with a few select + static content exceptions punched past Fossil, which are handled + entirely via nginx.</p> + + * <p>The other two domains are aliases for one another — e.g. + `example.com` and `example.net` — with most of the content being + static. This pair of domains has several unrelated Fossil repo + proxies attached to various sections of the URI hierarchy.</p> + + By using nginx, I was able to do all of the above with minimal + repetition between the site configurations. + +* **Integration** — Because nginx is so popular, it integrates with +many different technologies, and many other systems integrate with it in +turn. This makes it great middleware, sitting between the outer web +world and interior site services like Fossil. It allows Fossil to +participate seamlessly as part of a larger web stack. + +* **Availability** — nginx is already in most operating system binary +package repositories, so you don’t need to go out of your way to get it. + + +## <a id="modes"></a>Fossil Service Modes + +Fossil provides four major ways to access a repository it’s serving +remotely, three of which are straightforward to use with nginx: + +* **HTTP** — Fossil has [a built-in HTTP server](../any/none.md). + While this method is efficient and it’s + possible to use nginx to proxy access to another HTTP server, we + don’t see any particularly good reason to make nginx reinterpret + Fossil’s own implementation of HTTP when we have a better option. + (But see [below](#http).) + +* **SSH** — This method exists primarily to avoid the need for HTTPS, + but we *want* HTTPS. (We’ll get to that [below](#tls).) + There is probably a way to get nginx to proxy Fossil to HTTPS via + SSH, but it would be pointlessly complicated. + +* **CGI** — This method is simple but inefficient, because it launches + a separate Fossil instance on every HTTP hit. + Since Fossil is a relatively small self-contained program, and it’s + designed to start up quickly, this method can work well in a + surprisingly large number of cases. + Nevertheless, we will avoid this option in this document because + we’re already buying into a certain amount of complexity here in + order to gain power. There’s no sense in throwing away any of that + hard-won performance on CGI overhead. + +* **SCGI** — The [SCGI protocol][scgip] provides the simplicity of CGI + without its performance problems. + +SCGI it is, then. + +[scgip]: https://en.wikipedia.org/wiki/Simple_Common_Gateway_Interface + + +## <a id="deps"></a>Installing the Dependencies + +The first step is to install some non-default packages we’ll need. SSH into +your server, then say: + + $ sudo apt install fossil nginx + +You can leave “`fossil`” out of that if you’re building Fossil from +source to get a more up-to-date version than is shipped with the host +OS. + + +## <a id="scgi"></a>Running Fossil in SCGI Mode + +For the following nginx configuration to work, it needs to contact a +background Fossil instance speaking the SCGI protocol. There are +[many ways](../) to set that up, such as [with `systemd`](./service.md) +on mainstream Linux distros. Another way is to [containerize][ctz] your +repository servers, then use the [`fslsrv` wrapper for Podman][fspm] to +generate `systemd` units for use by the front-end proxy. + +However you do it, you need to match up the TCP port numbers between it +and those in the nginx configuration below. + +[ctz]: ../../containers.md +[fspm]: https://tangentsoft.com/fossil/dir/bin + + +## <a id="config"></a>Configuration + +On Debian and Ubuntu systems the primary user-level configuration file +for nginx is `/etc/nginx/sites-enabled/default`. I recommend that this +file contain only a list of include statements, one for each site that +server hosts: + + include local/example.com + include local/foo.net + +Those files then each define one domain’s configuration. Here, +`/etc/nginx/local/example.com` contains the configuration for +`*.example.com` and its alias `*.example.net`; and `local/foo.net` +contains the configuration for `*.foo.net`. + +The configuration for our `example.com` web site, stored in +`/etc/nginx/sites-enabled/local/example.com` is: + +---- + + server { + server_name .example.com .example.net ""; + include local/generic; + include local/code; + + access_log /var/log/nginx/example.com-https-access.log; + error_log /var/log/nginx/example.com-https-error.log; + + # Bypass Fossil for the static documentation generated from + # our source code by Doxygen, so it merges into the embedded + # doc URL hierarchy at Fossil’s $ROOT/doc without requiring that + # these generated files actually be stored in the repo. This + # also lets us set aggressive caching on these docs, since + # they rarely change. + location /code/doc/html { + root /var/www/example.com/code/doc/html; + + location ~* \.(html|ico|css|js|gif|jpg|png)$ { + add_header Vary Accept-Encoding; + access_log off; + expires 7d; + } + } + + # Redirect everything under /code to the Fossil instance + location /code { + include local/code; + + # Extended caching for URLs that include unique IDs + location ~ "/(artifact|doc|file|raw)/[0-9a-f]{40,64}" { + add_header Cache-Control "public, max-age=31536000, immutable"; + include local/code; + access_log off; + } + + # Lesser caching for URLs likely to be quasi-static + location ~* \.(css|gif|ico|js|jpg|png)$ { + add_header Vary Accept-Encoding; + include local/code; + access_log off; + expires 7d; + } + } + } + +---- + +As you can see, this is a pure extension of [the basic nginx service +configuration for SCGI][scgii], showing off a few ideas you might want to +try on your own site, such as static asset proxying. + +You also need a `local/code` file containing: + + include scgi_params; + scgi_pass 127.0.0.1:12345; + scgi_param SCRIPT_NAME "/code"; + +We separate that out because nginx refuses to inherit certain settings +between nested location blocks, so rather than repeat them, we extract +them to this separate file and include it from both locations where it’s +needed. You see this above where we set far-future expiration dates on +files served by Fossil via URLs that contain hashes that change when the +content changes. It tells your browser that the content of these URLs +can never change without the URL itself changing, which makes your +Fossil-based site considerably faster. + +Similarly, the `local/generic` file referenced above helps us reduce unnecessary +repetition among the multiple sites this configuration hosts: + + root /var/www/$host; + + listen 80; + listen [::]:80; + + charset utf-8; + +There are some configuration directives that nginx refuses to substitute +variables into, citing performance considerations, so there is a limit +to how much repetition you can squeeze out this way. One such example +are the `access_log` and `error_log` directives, which follow an obvious +pattern from one host to the next. Sadly, you must tolerate some +repetition across `server { }` blocks when setting up multiple domains +on a single server. + +The configuration for `foo.net` is similar. + +See [the nginx docs](https://nginx.org/en/docs/) for more ideas. + + +## <a id="http"></a>Proxying HTTP Anyway + +[Above](#modes), we argued that proxying SCGI is a better option than +making nginx reinterpret Fossil’s own implementation of HTTP. If you +want Fossil to speak HTTP, just [set Fossil up as a standalone +server](../any/none.md). And if you want nginx to [provide TLS +encryption for Fossil](#tls), proxying HTTP instead of SCGI provides no +benefit. + +However, it is still worth showing the proper method of proxying +Fossil’s HTTP server through nginx if only to make reading nginx +documentation on other sites easier: + + location /code { + rewrite ^/code(/.*) $1 break; + proxy_pass http://127.0.0.1:12345; + } + +The most common thing people get wrong when hand-rolling a configuration +like this is to get the slashes wrong. Fossil is sensitive to this. For +instance, Fossil will not collapse double slashes down to a single +slash, as some other HTTP servers will. + + +## <a id="large-uv"></a> Allowing Large Unversioned Files + +By default, nginx only accepts HTTP messages [up to a +meg](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) +in size. Fossil chunks its sync protocol such that this is not normally +a problem, but when sending [unversioned content][uv], it uses a single +message for the entire file. Therefore, if you will be storing files +larger than this limit as unversioned content, you need to raise the +limit. Within the `location` block: + + # Allow large unversioned file uploads, such as PDFs + client_max_body_size 20M; + +[uv]: ../../unvers.wiki + + +## <a id="fail2ban"></a> Integrating `fail2ban` + +One of the nice things that falls out of proxying Fossil behind nginx is +that it makes it easier to configure `fail2ban` to recognize attacks on +Fossil and automatically block them. Fossil logs the sorts of errors we +want to detect, but it does so in places like the repository’s admin +log, a SQL table, which `fail2ban` doesn’t know how to query. By putting +Fossil behind an nginx proxy, we convert these failures to log file +form, which `fail2ban` is designed to handle. + +First, install `fail2ban`, if you haven’t already: + + sudo apt install fail2ban + +We’d like `fail2ban` to react to Fossil `/login` failures. The stock +configuration of `fail2ban` only detects a few common sorts of SSH +attacks by default, and its included (but disabled) nginx attack +detectors don’t include one that knows how to detect an attack on +Fossil. We have to teach it by putting the following into +`/etc/fail2ban/filter.d/nginx-fossil-login.conf`: + + [Definition] + failregex = ^<HOST> - .*POST .*/login HTTP/..." 401 + +That teaches `fail2ban` how to recognize the errors logged by Fossil. + +Then in `/etc/fail2ban/jail.local`, add this section: + + [nginx-fossil-login] + enabled = true + logpath = /var/log/nginx/*-https-access.log + +The last line is the key: it tells `fail2ban` where we’ve put all of our +per-repo access logs in the nginx config above. + +There’s a [lot more you can do][dof2b], but that gets us out of scope of +this guide. + + +[dof2b]: https://www.digitalocean.com/community/tutorials/how-to-protect-an-nginx-server-with-fail2ban-on-ubuntu-14-04 + + +## <a id="tls"></a> Adding TLS (HTTPS) Support + +One of the [many ways](../../ssl.wiki) to provide TLS-encrypted HTTP +access (a.k.a. HTTPS) to Fossil is to run it behind a web proxy that +supports TLS. Because one such option is nginx, it’s best to delegate +TLS to it if you were already using nginx for some other reason, such as +static content serving, with only part of the site being served by +Fossil. + +The simplest way by far to do this is to use [Let’s Encrypt][LE]’s +[Certbot][CB], which can configure nginx for you and keep its +certificates up to date. You need but follow their [nginx on Ubuntu 20 +guide][CBU]. We had trouble with this in the past, but either Certbot +has gotten smarter or our nginx configurations have gotten simpler, so +we have removed the manual instructions we used to have here. + +You may wish to include something like this from each `server { }` +block in your configuration to enable TLS in a common, secure way: + +``` +# Tell nginx to accept TLS-encrypted HTTPS on the standard TCP port. +listen 443 ssl; +listen [::]:443 ssl; + +# Reference the TLS cert files produced by Certbot. +ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; +ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; + +# Load the Let's Encrypt Diffie-Hellman parameters generated for +# this server. Without this, the server is vulnerable to Logjam. +ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; + +# Tighten things down further, per Qualys’ and Certbot’s advice. +ssl_session_cache shared:le_nginx_SSL:1m; +ssl_protocols TLSv1.2 TLSv1.3; +ssl_prefer_server_ciphers on; +ssl_session_timeout 1440m; + +# Offer OCSP certificate stapling. +ssl_stapling on; +ssl_stapling_verify on; + +# Enable HSTS. +include local/enable-hsts; +``` + +The [HSTS] step is optional and should be applied only after due +consideration, since it has the potential to lock users out of your +site if you later change your mind on the TLS configuration. +The `local/enable-hsts` file it references is simply: + + add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always; + +It’s a separate file because nginx requires that headers like this be +applied separately for each `location { }` block. We’ve therefore +factored this out so you can `include` it everywhere you need it. + +The [OCSP] step is optional, but recommended. + +You may find [Qualys’ SSL Server Test][QSLT] helpful in verifying that +you have set all this up correctly, and that the configuration is +strong. We’ve found their [best practices doc][QSLC] to be helpful. As +of this writing, the above configuration yields an A+ rating when run on +Ubuntu 22.04.01 LTS. + +[CB]: https://certbot.eff.org/ +[CBU]: https://certbot.eff.org/instructions?ws=nginx&os=ubuntufocal +[LE]: https://letsencrypt.org/ +[HSTS]: https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/ +[OCSP]: https://en.wikipedia.org/wiki/OCSP_stapling +[QSLC]: https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices +[QSLT]: https://www.ssllabs.com/ssltest/ + +<div style="height:50em" id="this-space-intentionally-left-blank"></div> + +*[Return to the top-level Fossil server article.](../)* diff --git a/www/server/debian/service.md b/www/server/debian/service.md new file mode 100644 index 0000000..f056ff1 --- /dev/null +++ b/www/server/debian/service.md @@ -0,0 +1,259 @@ +# Serving via systemd on Debian and Ubuntu + +[`systemd`][sdhome] is the service management framework in all major +in-support versions of Linux. There are multiple ways to run Fossil +under `systemd`. + +[sdhome]: https://www.freedesktop.org/wiki/Software/systemd/ +[wpa]: https://en.wikipedia.org/wiki/Systemd#Adoption + + +## Containerized Service + +Two of the methods for running [containerized Fossil][cntdoc] integrate +with `systemd`, potentially obviating the more direct methods below: + +* If you take [the Podman method][podman] of running containerized + Fossil, it opens the `podman generate systemd` option for you, as + exemplified in [the `fslsrv` script][fslsrv] used on this author’s + public Fossil-based web site. That script pulls its container images + from [my Docker Hub repo][dhrepo] to avoid the need for my public + Fossil server to have build tools and a copy of the Fossil source + tree. You’re welcome to use my images as-is, or you may use these + tools to bounce custom builds up through a separate container image + repo you manage. + +* If you’re willing to give up [a lot of features][nsweak] relative to + Podman, and you’re willing to tolerate a lot more manual + administrivia, [the nspawn method][nspawn] has a lot less overhead, + being a direct feature of `systemd` itself. + +Both of these options provide [better security][cntsec] than running +Fossil directly under `systemd`, among [other benefits][cntdoc]. + +[cntdoc]: ../../containers.md +[cntsec]: ../../containers.md#security +[dhrepo]: https://hub.docker.com/r/tangentsoft/fossil +[fslsrv]: https://tangentsoft.com/fossil/dir?name=bin +[nspawn]: ../../containers.md#nspawn +[nsweak]: ../../containers.md#nspawn-weaknesses +[podman]: ../../containers.md#podman + + +## User Service + +A fun thing you can easily do with `systemd` that you can’t directly do +with older technologies like `inetd` and `xinetd` is to set a server up +as a “user” service. + +You can’t listen on TCP port 80 with this method due to security +restrictions on TCP ports in every OS where `systemd` runs, but you can +create a listener socket on a high-numbered (≥ 1024) TCP port, +suitable for sharing a Fossil repo to a workgroup on a private LAN. + +To do this, write the following in +`~/.local/share/systemd/user/fossil.service`: + +> ```dosini +[Unit] +Description=Fossil user server +After=network-online.target + +[Service] +WorkingDirectory=/home/fossil/museum +ExecStart=/home/fossil/bin/fossil server --port 9000 repo.fossil +Restart=always +RestartSec=3 + +[Install] +WantedBy=multi-user.target +``` + +Unlike with `inetd` and `xinetd`, we don’t need to tell `systemd` which +user and group to run this service as, because we’ve installed it +under the account we’re logged into, which `systemd` will use as the +service’s owner. + +The result is essentially [the standalone server method](../any/none.md) +coupled with an intelligent service manager that will start it +automatically in the background on system boot, perform automatic +service restarts with back-off logic, and more, making this much more +robust than the by-hand launches of `fossil` in the platform-independent +Fossil server instructions. The service will stay up until we +explicitly tell it to shut down. + +This scheme couples well with [the generic SCGI instructions][scgi] as +it requires a way to run the underlying repository server in the +background. Given that its service port is then proxied by SCGI, it +follows that it doesn’t need to run as a system service. A user service +works perfectly well for this. + +Because we’ve set this up as a user service, the commands you give to +manipulate the service vary somewhat from the sort you’re more likely to +find online: + + $ systemctl --user daemon-reload + $ systemctl --user enable fossil + $ systemctl --user start fossil + $ systemctl --user status fossil -l + $ systemctl --user stop fossil + +That is, we don’t need to talk to `systemd` with `sudo` privileges, but +we do need to tell it to look at the user configuration rather than the +system-level configuration. + +This scheme isolates the permissions needed by the Fossil server, which +reduces the amount of damage it can do if there is ever a +remotely-triggerable security flaw found in Fossil. + +On some `systemd` based OSes, user services only run while that user is +logged in interactively. This is common on systems aiming to provide +desktop environments, where this is the behavior you often want. To +allow background services to continue to run after logout, say: + + $ sudo loginctl enable-linger $USER + +You can paste the command just like that into your terminal, since +`$USER` will expand to your login name. + +[scgi]: ../any/scgi.md + + + +### System Service Alternative + +There are some common reasons that you’d have good cause to install +Fossil as a system-level service rather than the prior user-level one: + +* You’re using [the new `fossil server --cert` feature][sslsrv] to get + TLS service and want it to listen directly on port 443, rather than be + proxied, as one had to do before Fossil got the ability to act as a + TLS server itself. That requires root privileges, so you can’t run + it as a user-level service. + +* You’re proxying Fossil with [nginx](./nginx.md) or similar, allowing + it to bind to high-numbered ports, but because it starts as a system + service, you can’t get Fossil into the same dependency chain to + ensure things start up and shut down in the proper order unless it + *also* runs as a system service. + +* You want to make use of Fossil’s [chroot jail feature][cjail], which + requires the server to start as root. + +There are just a small set of changes required: + +1. Install the unit file to one of the persistent system-level unit + file directories. Typically, these are: + + /etc/systemd/system + /lib/systemd/system + +2. Add `User` and `Group` directives to the `[Service]` section so + Fossil runs as a normal user, preferably one with access only to + the Fossil repo files, rather than running as `root`. + +[sslsrv]: ../../ssl-server.md +[cjail]: ../../chroot.md + + +## Socket Activation + +Another useful method to serve a Fossil repo via `systemd` is via a +socket listener, which `systemd` calls “[socket activation][sa],” +roughly equivalent to [the ancient `inetd` method](../any/inetd.md). +It’s more complicated, but it has some nice properties. + +We first need to define the privileged socket listener by writing +`/etc/systemd/system/fossil.socket`: + +> ```dosini +[Unit] +Description=Fossil socket + +[Socket] +Accept=yes +ListenStream=80 +NoDelay=true + +[Install] +WantedBy=sockets.target +``` + +Note the change of configuration directory from the `~/.local` directory +to the system level. We need to start this socket listener at the root +level because of the low-numbered TCP port restriction we brought up +above. + +This configuration says more or less the same thing as the socket part +of an `inetd` entry [exemplified elsewhere in this +documentation](../any/inetd.md). + +Next, create the service definition file in that same directory as +`fossil@.service`: + +> ```dosini +[Unit] +Description=Fossil socket server +After=network-online.target + +[Service] +WorkingDirectory=/home/fossil/museum +ExecStart=/home/fossil/bin/fossil http repo.fossil +StandardInput=socket + +[Install] +WantedBy=multi-user.target +``` + +Notice that we haven’t told `systemd` which user and group to run Fossil +under. Since this is a system-level service definition, that means it +will run as root, which then causes Fossil to [automatically drop into a +`chroot(2)` jail](../../chroot.md) rooted at the `WorkingDirectory` +we’ve configured above, shortly after each `fossil http` call starts. + +The `Restart*` directives we had in the user service configuration above +are unnecessary for this method, since Fossil isn’t supposed to remain +running under it. Each HTTP hit starts one Fossil instance, which +handles that single client’s request and then immediately shuts down. + +Next, you need to tell `systemd` to reload its system-level +configuration files and enable the listening socket: + + $ sudo systemctl daemon-reload + $ sudo systemctl enable fossil.socket + +And now you can manipulate the socket listener: + + $ sudo systemctl start fossil.socket + $ sudo systemctl status -l fossil.socket + $ sudo systemctl stop fossil.socket + +Notice that we’re working with the *socket*, not the *service*. The fact +that we’ve given them the same base name and marked the service as an +instantiated service with the “`@`” notation allows `systemd` to +automatically start an instance of the service each time a hit comes in +on the socket that `systemd` is monitoring on Fossil’s behalf. To see +this service instantiation at work, visit a long-running Fossil page +(e.g. `/tarball`) and then give a command like this: + + $ sudo systemctl --full | grep fossil + +This will show information about the `fossil` socket and service +instances, which should show your `/tarball` hit handler, if it’s still +running: + + fossil@20-127.0.0.1:80-127.0.0.1:38304.service + +You can feed that service instance description to a `systemctl kill` +command to stop that single instance without restarting the whole +`fossil` service, for example. + +In all of this, realize that we’re able to manipulate a single socket +listener or single service instance at a time, rather than reload the +whole externally-facing network configuration as with the far more +primitive `inetd` service. + +[sa]: http://0pointer.de/blog/projects/socket-activation.html + + +*[Return to the top-level Fossil server article.](../)* |
