From 4f49b16079b27d7ce633619e4cddfdf2bffbc952 Mon Sep 17 00:00:00 2001 From: cashmere Date: Fri, 4 Sep 2026 09:05:24 +0200 Subject: initial commit --- www/build.wiki | 543 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 543 insertions(+) create mode 100644 www/build.wiki (limited to 'www/build.wiki') diff --git a/www/build.wiki b/www/build.wiki new file mode 100644 index 0000000..7787461 --- /dev/null +++ b/www/build.wiki @@ -0,0 +1,543 @@ +Compiling and Installing Fossil + +

0.0 Using A Pre-compiled Binary

+ +[/uv/download.html|Pre-compiled binaries] are available for recent +releases. Just download +the appropriate executable for your platform +and put it on your $PATH. +To uninstall, simply delete the executable. +To upgrade from an older release, just overwrite the older binary with +the newer one. + +For details about how those binaries are built, see +[/wiki?name=Release+Build+How-To | the Release Build How-To wiki page]. + + +

0.1 Executive Summary

+ +Building and installing is very simple. Three steps: + +
    +
  1. Download and unpack a source tarball or ZIP. +
  2. ./configure; make +
  3. Move the resulting "fossil" or "fossil.exe" executable to someplace on +your $PATH. +
+ +
+ +

1.0 Obtaining The Source Code

+ +Fossil is self-hosting, so you can obtain a ZIP archive or tarball +containing a snapshot of the latest version directly from +Fossil's own fossil repository. Additionally, source archives of +released versions of +fossil are available from the [/uv/download.html|downloads page]. +To obtain a development version of fossil, follow these steps: + +
    +
  1. Point your web browser to [https://fossil-scm.org/]
    2. + +
  2. Click on the [/timeline|Timeline] +link at the top of the page.
    3. + +
  3. Select a version of Fossil you want to download. The latest +version on the trunk branch is usually a good choice. Click on its +link.
    4. + +
  4. Finally, click on one of the +"Zip Archive" or "Tarball" links, according to your preference. +These links will build a ZIP archive or a gzip-compressed tarball of the +complete source code and download it to your computer.
    5. +
+ +

Aside: Is it really safe to use an unreleased development version of +the Fossil source code?

+ +Yes! Any check-in on the +[/timeline?t=trunk | trunk branch] of the Fossil +[https://fossil-scm.org/ | self-hosting repository] +will work fine. (Dodgy code is always on a branch.) In the unlikely +event that you pick a version with a serious bug, it still won't +clobber your files. Fossil uses several +[./selfcheck.wiki | self-checks] prior to committing any +repository change that prevent loss-of-work due to bugs. + +The Fossil [./selfhost.wiki | self-hosting repositories], especially +the one at [https://fossil-scm.org/home], usually run a version +of trunk that is less than a week or two old. Look at the bottom +left-hand corner of this screen (to the right of "This page was +generated in...") to see exactly which version of Fossil is +rendering this page. It is always safe to use whatever version +of the Fossil code you find running on the main Fossil website. + +

2.0 Compiling

+ +
    +
  1. +Unpack the ZIP or tarball you downloaded then +cd into the directory created.
    2. + +
  2. (Optional, Debian-compatible Linux only) +Make sure you have all the necessary tools and libraries at hand by running: +sudo apt install tcl-dev tk libssl-dev zlib1g-dev. + +
  3. (Optional, Unix only) +Run ./configure to construct a makefile. + +
      +
    1. +The build system for Fossil on Unix-like systems assumes that the +OpenSSL development and runtime files are available on your system, +because unprotected repositories are trivial to attack otherwise. +Indeed, some public Fossil repositories — including Fossil's own — today +run in an HTTPS-only mode, so that you can't even do an anonymous clone +from them without using the TLS features added to Fossil by OpenSSL. To +weaken that stance could allow a +[https://en.wikipedia.org/wiki/Man-in-the-middle_attack|man in the +middle attack], such as one that substitutes malicious code into your +Fossil repository clone. + +You can force the Fossil build system to avoid searching for, building +against, and linking to the OpenSSL library by passing +--with-openssl=none to the configure script. + +If you do not have the OpenSSL development libraries on your system, +we recommend that you install them, typically via your OS's package +manager. The Fossil build system goes to a lot of effort to seek these +out wherever they may be found, so that is typically all you need to +do. + +For more advanced use cases, see the [./ssl.wiki#openssl-bin|OpenSSL +discussion in the "TLS and Fossil" document]. +
      2. + +
    2. +To build a statically linked binary, you can try adding +the --static option, but +[https://stackoverflow.com/questions/3430400/linux-static-linking-is-dead +| it may well not work]. If your platform of choice is affected by this, +the simplest workaround we're aware of is to build a Fossil container, +then [./containers.md#static | extract the static executable from it]. +
      3. + +
    3. +To enable the native [./th1.md#tclEval | Tcl integration feature] feature, +add the --with-tcl=1 and --with-tcl-private-stubs=1 options. +
      4. + +
    4. +Other configuration options can be seen by running +./configure --help +
      5. +
    + +
  4. Run "make" to build the "fossil" or "fossil.exe" executable. +The details depend on your platform and compiler.
    5. + +
      +
    1. Unix → the configure-generated Makefile should work on +all Unix and Unix-like systems. Simply type "make".
      2. + +
    2. Unix without running "configure" → if you prefer to avoid +running configure, you can also use: make -f Makefile.classic. You may +want to make minor edits to Makefile.classic to configure the build for your +system.
      3. + +
    3. MinGW / MinGW-w64 → The best-supported path is to build +via the MinGW specific Makefile under a POSIX build of GNU make: +"make -f win/Makefile.mingw".
      4. + +To enable the native [./th1.md#tclEval | Tcl integration feature], use a +command line like the following (all on one line): + +
      make -f win/Makefile.mingw FOSSIL_ENABLE_TCL=1 FOSSIL_ENABLE_TCL_STUBS=1 FOSSIL_ENABLE_TCL_PRIVATE_STUBS=1
      + +
    4. MSYS2 / Cygwin → This is something of a hybrid between +options "a" and "c" above: it configures and builds +fossil.exe much as on Linux, but you get a native Windows +executable out at the end. The primary downside is that this type of +executable can become confused when attempting to interoperate with +fully native Windows EXEs by making assumptions that only hold true when +all elements are running under the Cygwin/MSYS environment. The MSVC and +MinGW options do not have this limitation. + +Even so, there is value to Linux/Unix natives in having this hybrid +while off in Windows-land. The simpler of the two paths is MSYS2, since +it lets you install the necessary prerequisites in a single command +after installing the base environment: + +
      pacman -sS gcc make openssl-devel zlib-devel
      + +The equivalent in Cygwin's setup.exe requires stepping +through the GUI package chooser, and then if you miss one of the +prereqs, going all the way back through it again until you get it right. + +
    5. MSVC → Use the MSVC makefile.
      6. + +NB: Run the following nmake commands from a "x64 Native +Tools Command Prompt"; buildmsvc.bat is able to automatically load +the build tools (x64 by default, pass "x86" as the first argument to use the +x86 tools), so it can be called from a normal command prompt. + +First, change to the "win/" subdirectory ("cd win"), then run +"nmake /f Makefile.msc".

      Alternatively, the batch +file "win\buildmsvc.bat" may be used and it will attempt to +detect and use the latest installed version of MSVC.

      To enable +the optional OpenSSL support, +first download the official +source code for OpenSSL and extract it to an appropriately named +"openssl" subdirectory within the local +[/tree?ci=trunk&name=compat | compat] directory then make sure that some recent +Perl binaries are installed locally, +and finally run one of the following commands: +
      +nmake /f Makefile.msc FOSSIL_ENABLE_SSL=1 FOSSIL_BUILD_SSL=1 PERLDIR=C:\full\path\to\Perl\bin
+
      +
      +buildmsvc.bat FOSSIL_ENABLE_SSL=1 FOSSIL_BUILD_SSL=1 PERLDIR=C:\full\path\to\Perl\bin
+
      +To enable the optional native [./th1.md#tclEval | Tcl integration feature], +run one of the following commands or add the "FOSSIL_ENABLE_TCL=1" +argument to one of the other NMAKE command lines: +
      +nmake /f Makefile.msc FOSSIL_ENABLE_TCL=1
+
      +
      +buildmsvc.bat FOSSIL_ENABLE_TCL=1
+
      + +
    6. Cygwin → The same as other Unix-like systems. It is +recommended to configure using: "configure --disable-internal-sqlite", +making sure you have the "libsqlite3-devel" , "zlib-devel" and +"openssl-devel" packages installed first.
      7. +
    +
+ +

3.0 Installing

+ +
    +
  1. +The finished binary is named "fossil" (or "fossil.exe" on Windows). +Put this binary in a +directory that is somewhere on your PATH environment variable. +It does not matter where. +
    2. + +
  2. +(Optional:) +To uninstall, just delete the binary. +
    3. +
+ +

4.0 Additional Considerations

+ + + + +

5.0 Building a Docker Container

+ +The information on building Fossil inside an +[https://opencontainers.org/ | OCI container] is now in +[./containers.md | a separate document]. + +This includes the instructions on using the OCI container as an +expedient intermediary for building a statically-linked Fossil binary on +modern Linux platforms, which otherwise make this difficult. + + +

6.0 Building on/for Android

+ +

6.1 Cross-compiling from Linux

+ +The following instructions for building Fossil for Android via Linux, +without requiring a rooted OS, are adapted from +[forum:/forumpost/e0e9de4a7e | a forum post]. + +On the development machine, from the fossil source tree: + +
export CC=$NDK_PATH/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi21-clang
+./configure --with-openssl=none
+make
+
+ + +On the Android device, enable the USB debugging option from +Developer menu in Device Options. Connect the device to the development +system with USB. If it's configured and connected properly, +the device should show up in the output of adb devices: + +
sudo adb devices
+
+ +Copy the resulting fossil binary onto the device... + +
sudo adb push fossil /data/local/tmp
+
+ +And run it from an adb shell: + +
sudo adb shell
+> cd /data/local/tmp
+# Fossil requires a HOME directory to work with:
+> export HOME=$PWD
+> export PATH=$PWD:$PATH
+> fossil version
+This is fossil version 2.11 [e5653a4ceb] 2020-03-26 18:54:02 UTC
+
+ +The output might, or might not, include warnings such as: + +
WARNING: linker: ./fossil: unused DT entry: type 0x6ffffef5 arg 0x1464
+WARNING: linker: ./fossil: unused DT entry: type 0x6ffffffe arg 0x1ba8
+WARNING: linker: ./fossil: unused DT entry: type 0x6fffffff arg 0x2
+
+ +The source of such warnings is not 100% certain. +Some information about these (reportedly harmless) warnings can +be found +[https://stackoverflow.com/a/41900551 | on this StackOverflow post]. + + + +

7.0 Building for Fuzz Testing

+ +This feature is primarily intended for fossil's developers and may +change at any time. It is only known to work on Linux systems and has +been seen to work on x86/64 and ARM. + +Fossil has builtin support for processing specific features using +libfuzzer. The features which can be tested this way are +found in the help text for the [/help/test-fuzz|test-fuzz +command]. + +Fuzzing requires: + + * The clang C compiler. + * libfuzzer. On Ubuntu-derived systems, it can be installed with + apt install libfuzzer-XYZ, where XYZ is a version number + (several versions may be available on any given system) + + +Compile as follows: + +
make clean
+make TCCFLAGS='-DFOSSIL_FUZZ -fsanitize=fuzzer,address,undefined -O0 -g' CC=clang
+
+ +The resulting fossil binary differs from the standard +one primarily in that it runs the test-fuzz command by +default. It needs to be told what to fuzz and needs to be given a +directory of input files to seed the fuzzer with: + + +
$ mkdir cases
+  # Copy input files into ./cases. e.g. when fuzzing the markdown
+  # processor, copy any to-be-tested .md files into that directory.
+  # Then start the fuzzer:
+$ ./fossil-fuzz --fuzztype markdown cases
+
+ +As it works, it writes its mutated test files into the "cases" +directory, each one named in the form of a hash. When it finds a +problem it will produce a stack trace for the offending code, will +output the name of the file which triggered the crash (named +cases/SOME_HASH) and may, depending on the nature of the +problem, produce a file named crash-SOMETHING. In theory the +crash file can be fed directly back into the fuzzer to reproduce the +problem: + +
$ ./fossil-fuzz --fuzztype markdown crash-SOMETHING
+
+ +But whether or not it will genuinely crash may depend on static +app-level state which might not trigger the crash when running an +individual test. + +For a detailed information about the fuzzer's flags and features, see: + + * [https://llvm.org/docs/LibFuzzer.html] + * [https://github.com/google/fuzzing/blob/master/tutorial/libFuzzerTutorial.md] + +Flags for the fuzzer can be passed directly to fossil, +e.g. -jobs=4 to start four fuzzer jobs in parallel, but doing +so may cause the fuzzer to strip the --fuzztype flag, leading +to it testing the wrong thing. When passing on fuzzer-specific flags +along with --fuzztype, be sure to check your system's process +list to ensure that your --fuzztype flag is there. + + + +

8.0 Building WebAssembly Components

+ +Fossil uses one component built as +[https://developer.mozilla.org/en-US/docs/WebAssembly | WebAssembly] +a.k.a. WASM. Because compiling WASM code requires non-trivial +client-side tooling, the repository includes compiled copies of these +pieces. Most Fossil hackers should never need to concern themselves +with the WASM parts, but this section describes how to for those who +want or need to do so. + +The bits described in this section are necessary when updating +extsrc/pikchr.c from the upstream source, or the +fossil binary will use a different version of pikchr than +[/pikchrshow] does (as the latter runs the WASM build of pikchr). + +These instructions have only ever been tested on Linux systems. They +"should" work on any Unix-like system supported by Emscripten. The +fossil makefiles for Windows builds do not include any of the +WASM-related components (patches to add that would be welcomed, of +course). + +The first step is to configure the tree with support for +[https://emscripten.org/|Emscripten]. This requires that the system +has the Emscripten SDK (a.k.a. emsdk) installed, as documented at: + +[https://emscripten.org/docs/getting_started/downloads.html] + +For instructions on keeping the SDK up to date, see: + +[https://emscripten.org/docs/tools_reference/emsdk.html] + +
Getting Emscripten up and running is trivial and +painless, at least on Linux systems, but the installer downloads +many hundreds of megabytes of tools and dependencies, all of which +will be installed under the single SDK directory (as opposed to +being installed at the system level). It does, however, require +that python3 be installed at the system level and it can +optionally make use of a system-level cmake for certain tasks +unrelated to how fossil uses the SDK.
+ +After installing the SDK, configure the fossil tree with emsdk +support: + +
$ ./configure --with-emsdk=/path/to/emsdk \
+  --and-other-options...
+
+ +If the --with-emsdk flag is not provided, the configure +script will check for the environment variable EMSDK, which +is one of the standard variables the SDK environment uses. If that +variable is found, its value will implicitly be used in place of the +missing --with-emsdk flag. Thus, if the emsdk_env.sh +script is sourced into the shell before running the configure script, +the SDK will be detected even without the config flag. + +The configure script installs some makefile variables which tell the +build where to find the SDK and it generates a script named +tools/emcc.sh (from the template file +[/file/tools/emcc.sh.in|/tools/emcc.sh.in]), which is a +wrapper around the Emscripten C compiler (emcc). The wrapper +script uses the configure-time state to attempt to set up the various +environment variables which are required by emcc and will +fail if it cannot do so. Once it's set up the environment, it passes +on all of its arguments to emcc. + +The WASM-related build parts are set up such that none of them should +ever trigger implicitly (e.g. via dependencies resolution) in a normal +build cycle. They are instead explicitly built as described below. + +From the top of the source tree, all WASM-related components can be +built with: + +
$ make wasm
+ +
The file +[/file/extsrc/pikcher-worker.js|extsrc/pikcher-worker.js] +is hand-coded and intended to be loaded as a "Worker" in +JavaScript. That file loads the main module and provides an +interface via which a main JavaScript thread can communicate with +pikchr running in a Worker thread. The file +[/file/src/fossil.page.pikchrshowasm.js|src/fossil.page.pikchrshowasm.js] +implements the [/pikchrshow] app and demonstrates how +pikchr-worker.js is used.
+ +As of this writing, those parts include: + + * extsrc/pikchr.wasm is a WASM-compiled form of + extsrc/pikchr.c. + * extsrc/pikchr.js is JS/WASM glue code generated by Emscripten + to give JS code access to the API exported by the WASM file. + +When a new version of extsrc/pikchr.c is installed, the +files pikchr.{js,wasm} will need to be recompiled to account +for that. Running make wasm will, if the build is set up for +the emsdk, recompile those: + +
$ rm extsrc/pikchr.{js,wasm}
+# ^^^^ that rm has proven necessary in order to ensure rebuilds
+$ make wasm
+./tools/emcc.sh -o extsrc/pikchr.js ...
+$ ls -la extsrc/pikchr.{js,wasm}
+-rw-rw-r-- 1 stephan stephan 17263 Jun  8 03:59 extsrc/pikchr.js
+-rw-rw-r-- 1 stephan stephan 97578 Jun  8 03:59 extsrc/pikchr.wasm
+
+ +
If that fails with a message along the lines of +“setting `EXPORTED_RUNTIME_METHODS` expects `` but +got ``” then the emcc being invoked is too old: emcc +changed the format of list-type arguments at some point. The required +minimum version is unknown, but any SDK version from May 2022 or later +"should" (as of this writing) suffice. Any older version may or may not +work.
+ +After that succeeds, we need to run the normal build so that those +generated files can be compiled in to the fossil binary, accessible +via the [/help/www/builtin|/builtin page]: + +
$ make
+ +Before checking in those newly-built files, they need to be tested by +running the [/pikchrshow] page. If that page loads, the compilation +process fundamentally worked (a load failure will be made obvious to +the viewer). If it fails to load then the browser's dev tools console +likely provides at least a small hint (and sometimes a useful +hint) about the nature of the problem. Don't check those files in +until [/pikchrshow] runs, though! + +Should pikchr's C interface ever change, pikchr-worker.js +will need to be updated to accommodate it, but such modification is +typically trivial. + +

8.1 Solutions other than Emscripten?

+ +Emscripten is not the only option for building C as WASM, but it +provides a complete toolchain which eliminates many other steps which +must otherwise be accounted for on a per-project basis. Despite its +convenience, it behooves us to explore other build options for the +sake of portability and avoiding what amounts to vendor lock-in. + +For later refererence, here are articles specifically covering +building WASM projects without using Emscripten: + + * [https://surma.dev/things/c-to-webassembly/] + * [https://schellcode.github.io/webassembly-without-emscripten] -- cgit v1.3.1