initial commitHEADmaster

1 files changed, 1120 insertions, 0 deletions

diff --git a/www/sync.wiki b/www/sync.wiki
new file mode 100644
index 0000000..1518321
--- /dev/null
+++ b/www/sync.wiki
@@ -0,0 +1,1120 @@
+<title>The Fossil Sync Protocol</title>
+
+This document describes the wire protocol used to synchronize
+content between two Fossil repositories.
+
+<h2>1.0 Overview</h2>
+
+The global state of a fossil repository consists of an unordered
+[./fileformat.wiki|collection of artifacts].  Each artifact is 
+identified by a cryptographic
+hash of its content, expressed as a lower-case hexadecimal string.
+Synchronization is the process of sharing artifacts between
+repositories so that all repositories have copies of all artifacts.  Because
+artifacts are unordered, the order in which artifacts are received
+is unimportant.  It is assumed that the hash names
+of artifacts are unique - that every artifact has a different hash.
+To a first approximation, synchronization proceeds by sharing lists
+of hashes for available artifacts, then sharing the content of artifacts
+whose names are missing from one side or the other of the connection.
+In practice, a repository might contain millions of artifacts.  The list of
+hash names for this many artifacts can be large.  So optimizations are
+employed that usually reduce the number of hashes that need to be
+shared to a few dozen.
+
+Each repository also has local state.  The local state determines
+the web-page formatting preferences, authorized users, ticket formats,
+and similar information that varies from one repository to another.
+The local state is not usually transferred during a sync.  Except,
+some local state is transferred during a [/help/clone|clone]
+in order to initialize the local state of the new repository.  Also,
+an administrator can sync local state using
+the [/help/configuration|config push] and
+[/help/configuration|config pull]
+commands.
+
+<h3 id="crdt">1.1 Conflict-Free Replicated Datatypes</h3>
+
+The "bag of artifacts" data model used by Fossil is apparently an
+implementation of a particular
+[https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type|Conflict-Free
+Replicated Datatype (CRDT)] called a "G-Set" or "Grow-only Set".  The
+academic literature on CRDTs only began to appear in about 2011, and
+Fossil predates that research by at least 4 years.  But it is nice to
+know that theorists have now proven that the underlying data model of
+Fossil can provide strongly-consistent replicas using only
+peer-to-peer communication and without any kind of central
+authority.
+
+If you are already familiar with CRDTs and were wondering if Fossil
+used them, the answer is "yes".  We just don't call them by that name.
+
+
+<h2 id="transport">2.0 Transport</h2>
+
+All communication between client and server is via HTTP requests.
+The server is listening for incoming HTTP requests.  The client
+issues one or more HTTP requests and receives replies for each
+request.
+
+The server might be running as an independent server
+using the [/help/server|"fossil server" command], or it
+might be launched from inetd or xinetd using the
+[/help/http|"fossil http" command].  Or the server might
+be [./server/any/cgi.md|launched from CGI] or from
+[./server/any/scgi.md|SCGI].
+(See "[./server/|How To Configure A Fossil Server]" for details.)
+The specifics of how the server listens
+for incoming HTTP requests is immaterial to this protocol.
+The important point is that the server is listening for requests and
+the client is the issuer of the requests.
+
+A single [/help/push|push],
+[/help/pull|pull], or [/help?cmd=sync|sync]
+might involve multiple HTTP requests.
+The client maintains state between all requests.  But on the server
+side, each request is independent.  The server does not preserve
+any information about the client from one request to the next.
+
+Note: Throughout this article, we use the terms "server" and "client"
+to represent the listener and initiator of the interaction, respectively.
+Nothing in this protocol requires that the server actually be a back-room
+processor housed in a datacenter, nor does the client need to be a desktop
+or handheld device.  For the purposes of this article "client" simply means
+the repository that initiates the conversation and "server" is the repository
+that responds.  Nothing more.
+
+<h4 id="https">2.0.1 HTTPS Transport</h4>
+
+HTTPS differs from HTTP only in that the HTTPS protocol is
+encrypted as it travels over the wire.  The underlying protocol
+is the same.  This document describes only the underlying, unencrypted
+messages that go client to server and back to client.
+Whether or not those messages are encrypted does not come into play
+in this document.
+
+Fossil includes built-in 
+[./ssl-server.md|support for HTTPS encryption] in both client and server.
+
+<h4 id="ssh">2.0.2 SSH Transport</h4>
+
+When doing a sync using an "<code>ssh:…</code>" URL, the same HTTP transport protocol
+is used.  Fossil simply uses [https://en.wikipedia.org/wiki/Secure_Shell|ssh]
+to start an instance of the [/help?cmd=test-http|fossil test-http] command
+running on the remote machine.  It then sends HTTP requests and gets back HTTP
+replies over the SSH connection, rather than sending and receiving over an
+internet socket.  To see the specific "ssh" command that the Fossil client
+runs in order to set up a connection, add either of the the "--httptrace" or
+"--sshtrace" options to the "fossil sync" command line.
+
+This method is dependent on the remote <var>PATH</var> set by the SSH
+daemon, which may not be the same as your interactive shell's
+<var>PATH</var> on that same server. It is common to find
+<var>$HOME/bin</var> in the latter but not the former, for instance,
+leading to failures to sync over <code>ssh:…</code> URLs when you
+install the <code>fossil</code> binary in a nonstandard location, as
+with
+
+<verbatim>./configure --prefix=$HOME && make install</verbatim>
+
+The simpler of the two solutions to this problem is to install Fossil
+where sshd expects to find it, but when that isn't an option, you can
+instead give a URL like this:
+
+<verbatim>fossil clone ssh://myserver.example.com/path/to/repo.fossil?fossil=/home/me/bin/fossil</verbatim>
+
+That gives the local Fossil instance the absolute path to the binary on
+the remote machine for use when calling that Fossil instance through the
+SSH tunnel.
+
+<h4 id="file">2.0.3 FILE Transport</h4>
+
+When doing a sync using a "file:..." URL, the same HTTP protocol is
+still used.  But instead of sending each HTTP request over a socket or
+via SSH, the HTTP request is written into a temporary file.  The client
+then invokes the [/help?cmd=http|fossil http] command in a subprocess
+to process the request and and generate a reply.  The client then reads
+the HTTP reply out of a temporary file on disk, and deletes the two
+temporary files.  To see the specific "fossil http" command that is run
+in order to implement the "file:" transport, add the "--httptrace"
+option to the "fossil sync" command.
+
+<h3 id="srv-id">2.1 Server Identification</h3>
+
+The server is identified by a URL argument that accompanies the
+push, pull, or sync command on the client.  (As a convenience to
+users, the URL can be omitted on the client command and the same URL
+from the most recent push, pull, or sync will be reused.  This saves
+typing in the common case where the client does multiple syncs to
+the same server.)
+
+The client modifies the URL by appending the method name "<b>/xfer</b>"
+to the end.  For example, if the URL specified on the client command
+line is
+
+<pre>https://fossil-scm.org/home</pre>
+
+Then the URL that is really used to do the synchronization will
+be:
+
+<pre>https://fossil-scm.org/home/xfer</pre>
+
+<h3 id="req-format">2.2 HTTP Request Format</h3>
+
+The client always sends a POST request to the server.  The
+general format of the POST request is as follows:
+
+<pre>
+POST /home/xfer HTTP/1.0
+Host: fossil-scm.hwaci.com:80
+Content-Type: application/x-fossil
+Content-Length: 4216
+</pre>
+
+<i><pre>content...</pre></i>
+
+In the example above, the pathname given after the POST keyword
+on the first line is a copy of the URL pathname.  The Host: parameter
+is also taken from the URL.  The content type is always either
+"application/x-fossil" or "application/x-fossil-debug".  The "x-fossil"
+content type is the default.  The only difference is that "x-fossil"
+content is compressed using zlib whereas "x-fossil-debug" is sent
+uncompressed.
+
+A typical reply from the server might look something like this:
+
+<pre>
+HTTP/1.0 200 OK
+Date: Mon, 10 Sep 2007 12:21:01 GMT
+Connection: close
+Cache-control: private
+Content-Type: application/x-fossil; charset=US-ASCII
+Content-Length: 265
+</pre>
+
+<i><pre>content...</pre></i>
+
+The content type of the reply is always the same as the content type
+of the request.
+
+<h2 id="content">3.0 Fossil Synchronization Content</h2>
+
+A synchronization request between a client and server consists of
+one or more HTTP requests as described in the previous section.  This
+section details the "x-fossil" content type.
+
+<h3 id="lines">3.1 Line-oriented Format</h3>
+
+The x-fossil content type consists of zero or more "cards".  Cards
+are separated by the newline character ("\n").  Leading and trailing
+whitespace on a card is ignored.  Blank cards are ignored.
+
+Each card is divided into zero or more space separated tokens.
+The first token on each card is the operator.  Subsequent tokens
+are arguments.  The set of operators understood by servers is slightly
+different from the operators understood by clients, though the two
+are very similar.
+
+<h3 id="login">3.2 Login Cards</h3>
+
+Every message from client to server begins with one or more login
+cards.  Each login card has the following format:
+
+<pre><b>login</b>  <i>userid  nonce  signature</i></pre>
+
+The userid is the name of the user that is requesting service from the
+server, encoded in "fossilized" form (exactly as described for <a
+href="#error">the error card</a>).  The nonce is the SHA1 hash of the
+remainder of the message - all text that follows the newline character
+that terminates the login card.  The signature is the SHA1 hash of the
+concatenation of the nonce and the users password.
+
+When receving a login card, the server looks up the user and verifies
+that the nonce matches the SHA1 hash of the remainder of the message.
+It then checks the signature hash to make sure the signature matches.
+If everything checks out, then the client is granted all privileges of
+the specified user.
+
+Only one login card is permitted. A second login card will trigger
+a sync error. (Prior to 2025-07-21, the protocol permitted multiple
+logins, treating the login as the union of all privileges from all
+login cards. That capability was never used and has been removed.)
+
+As of version 2.27, Fossil supports transfering of the login card
+externally to the request payload via a Cookie HTTP header:
+
+<verbatim>
+  Cookie: x-f-l-c=...
+</verbatim>
+
+Where "..." is the URL-encoded login cookie. <code>x-f-l-c</code> is
+short for X-Fossil-Login-Cookie.
+
+
+<h3 id="file">3.3 File Cards</h3>
+
+Artifacts are transferred using either "file" cards, or "cfile"
+or "uvfile" cards.
+The name "file" card comes from the fact that most artifacts correspond to
+files that are under version control.
+The "cfile" name is an abbreviation for "compressed file".
+The "uvfile" name is an abbreviation for "unversioned file".
+
+<h4 id="ordinary-fc">3.3.1 Ordinary File Cards</h4>
+
+For sync protocols, artifacts are transferred using "file"
+cards.  File cards come in two different formats depending
+on whether the artifact is sent directly or as a
+[./delta_format.wiki|delta] from some
+other artifact.
+
+<pre>
+<b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i>
+<b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i>
+</pre>
+
+File cards are followed by in-line "payload" data.
+The content of the artifact
+or the artifact delta is the first <i>size</i> bytes of the
+x-fossil content that immediately follow the newline that
+terminates the file card.
+
+
+The first argument of a file card is the ID of the artifact that
+is being transferred.  The artifact ID is the lower-case hexadecimal
+representation of the name hash for the artifact.
+The last argument of the file card is the number of bytes of
+payload that immediately follow the file card.  If the file
+card has only two arguments, that means the payload is the
+complete content of the artifact.  If the file card has three
+arguments, then the payload is a
+[./delta_format.wiki|delta] and the second argument is
+the ID of another artifact that is the source of the delta.
+
+File cards are sent in both directions: client to server and
+server to client.  A delta might be sent before the source of
+the delta, so both client and server should remember deltas
+and be able to apply them when their source arrives.
+
+<h4 id="compressed-fc">3.3.2 Compressed File Cards</h4>
+
+A client that sends a clone protocol version "3" or greater will
+receive artifacts as "cfile" cards while cloning.  This card was
+introduced to improve the speed of the transfer of content by sending
+the compressed artifact directly from the server database to the
+client.  In this case, the containing response body is <em>not</em>
+compressed separately because the vast majority of the response is
+already compressed in cfile cards. In practice, version "3" is
+significantly faster than version "2".
+
+Compressed File cards are similar to File cards, sharing the same
+in-line "payload" data characteristics and also the same treatment of
+direct content or delta content.  Cfile cards come in two different formats
+depending on whether the artifact is sent directly or as a delta from
+some other artifact.
+
+<pre>
+<b>cfile</b> <i>artifact-id usize csize</i> <b>\n</b> <i>content</i>
+<b>cfile</b> <i>artifact-id delta-artifact-id usize csize</i> <b>\n</b> <i>content</i>
+</pre>
+
+The first argument of the cfile card is the ID of the artifact that
+is being transferred.  The artifact ID is the lower-case hexadecimal
+representation of the name hash for the artifact.  The second argument of
+the cfile card is the original size in bytes of the artifact.  The last
+argument of the cfile card is the number of compressed bytes of payload
+that immediately follow the cfile card.  If the cfile card has only
+three arguments, that means the payload is the complete content of the
+artifact.  If the cfile card has four arguments, then the payload is a
+delta and the second argument is the ID of another artifact that is the
+source of the delta and the third argument is the original size of the
+delta artifact.
+
+Unlike file cards, cfile cards are only sent in one direction during a
+clone from server to client for clone protocol version "3" or greater.
+
+<h4 id="private">3.3.3 Private artifacts</h4>
+
+"Private" content consist of artifacts that are not normally synced.
+However, private content will be synced when the
+the [/help?cmd=sync|fossil sync] command includes the "--private" option.
+
+Private content is marked by a "private" card:
+
+<pre><b>private</b></pre>
+
+The private card has no arguments and must directly precede a
+file card that contains the private content.
+
+<h4 id="uv-fc">3.3.4 Unversioned File Cards</h4>
+
+Unversioned content is sent in both directions (client to server and
+server to client) using "uvfile" cards in the following format:
+
+<pre><b>uvfile</b> <i>name mtime hash size flags</i> <b>\n</b> <i>content</i></pre>
+
+The <i>name</i> field is the name of the unversioned file.  The
+<i>mtime</i> is the last modification time of the file in seconds
+since 1970.  The <i>hash</i> field is the hash of the content
+for the unversioned file, or "<b>-</b>" for deleted content.
+The <i>size</i> field is the (uncompressed) size of the content
+in bytes.  The <i>flags</i> field is an integer which is interpreted
+as an array of bits.  The 0x0004 bit of <i>flags</i> indicates that
+the <i>content</i> is to be omitted.  The content might be omitted if
+it is too large to transmit, or if the sender merely wants to update the
+modification time of the file without changing the files content.
+The <i>content</i> is the (uncompressed) content of the file.
+
+The receiver should only accept the uvfile card if the hash and
+size match the content and if the mtime is newer than any existing
+instance of the same file held by the receiver.  The sender will not
+normally transmit a uvfile card unless all these constraints are true,
+but the receiver should double-check.
+
+A server will only accept uvfile cards if the login user has
+the "y" write-unversioned permission.
+
+Servers send uvfile cards in response to uvgimme cards received from
+the client.  Clients send uvfile cards when they determine that the server
+needs the content based on uvigot cards previously received from the server.
+
+<h3 id="push" name="pull">3.4 Push and Pull Cards</h3>
+
+Among the first cards in a client-to-server message are
+the push and pull cards.  The push card tells the server that
+the client is pushing content.  The pull card tells the server
+that the client wants to pull content.  In the event of a sync,
+both cards are sent.  The format is as follows:
+
+<pre>
+<b>push</b> <i>servercode projectcode</i>
+<b>pull</b> <i>servercode projectcode</i>
+</pre>
+
+The <i>servercode</i> argument is the repository ID for the
+client.  The <i>projectcode</i> is the identifier
+of the software project that the client repository contains.
+The projectcode for the client and server must match in order
+for the transaction to proceed.
+
+The server will also send a push card back to the client
+during a clone.  This is how the client determines what project
+code to put in the new repository it is constructing.
+
+The <i>servercode</i> argument is currently unused.
+
+<h3 id="clones">3.5 Clone Cards</h3>
+
+A clone card works like a pull card in that it is sent from
+client to server in order to tell the server that the client
+wants to pull content.  The clone card comes in two formats.  Older
+clients use the no-argument format and newer clients use the
+two-argument format.
+
+<pre>
+<b>clone</b>
+<b>clone</b> <i>protocol-version sequence-number</i>
+</pre>
+
+<h4>3.5.1 Protocol 3</h4>
+
+The latest clients send a two-argument clone message with a
+protocol version of "3".   (Future versions of Fossil might use larger
+protocol version numbers.)  Version "3" of the protocol enhanced version
+"2" by introducing the "cfile" card which is intended to speed up clone
+operations.  Instead of sending "file" cards, the server will send "cfile"
+cards
+
+<h4>3.5.2 Protocol 2</h4>
+
+The sequence-number sent is the number
+of artifacts received so far.  For the first clone message, the
+sequence number is 1.  The server will respond by sending file
+cards for some number of artifacts up to the maximum message size.
+
+The server will also send a single "clone_seqno" card to the client
+so that the client can know where the server left off.
+
+<pre>
+<b>clone_seqno</b>  <i>sequence-number</i>
+</pre>
+
+The clone message in subsequent HTTP requests for the same clone
+operation will use the sequence-number from the
+clone_seqno of the previous reply.
+
+In response to an initial clone message, the server also sends the client
+a push message so that the client can discover the projectcode for
+this project.
+
+<h4>3.5.3 Legacy Protocol</h4>
+
+Older clients send a clone card with no argument.  The server responds
+to a blank clone card by sending an "igot" card for every artifact in the
+repository.  The client will then issue "gimme" cards to pull down all the
+content it needs.
+
+The legacy protocol works well for smaller repositories (50MB with 50,000
+artifacts) but is too slow and unwieldy for larger repositories.
+The version 2 protocol is an effort to improve performance.  Further
+performance improvements with higher-numbered clone protocols are
+possible in future versions of Fossil.
+
+<h3 id="igot">3.6 Igot Cards</h3>
+
+An igot card can be sent from either client to server or from
+server to client in order to indicate that the sender holds a copy
+of a particular artifact.  The format is:
+
+<pre>
+<b>igot</b> <i>artifact-id</i> ?<i>flag</i>?
+</pre>
+
+The first argument of the igot card is the ID of the artifact that
+the sender possesses.
+The receiver of an igot card will typically check to see if
+it also holds the same artifact and if not it will request the artifact
+using a gimme card in either the reply or in the next message.
+
+If the second argument exists and is "1", then the artifact
+identified by the first argument is private on the sender and should
+be ignored unless a "--private" [/help?cmd=sync|sync] is occurring.
+
+The name "igot" comes from the English slang expression "I got" meaning
+"I have".
+
+<h4>3.6.1 Unversioned Igot Cards</h4>
+
+Zero or more "uvigot" cards are sent from server to client when
+synchronizing unversioned content.  The format of a uvigot card is
+as follows:
+
+<pre>
+<b>uvigot</b> <i>name mtime hash size</i>
+</pre>
+
+The <i>name</i> argument is the name of an unversioned file.
+The <i>mtime</i> is the last modification time of the unversioned file
+in seconds since 1970.
+The <i>hash</i> is the SHA1 or SHA3-256 hash of the unversioned file
+content, or "<b>-</b>" if the file has been deleted.
+The <i>size</i> is the uncompressed size of the file in bytes.
+
+When the server sees a "pragma uv-hash" card for which the hash
+does not match, it sends uvigot cards for every unversioned file that it
+holds.  The client will use this information to figure out which
+unversioned files need to be synchronized.
+The server might also send a uvigot card when it receives a uvgimme card
+but its reply message size is already oversized and hence unable to hold
+the usual uvfile reply.
+
+When a client receives a "uvigot" card, it checks to see if the
+file needs to be transferred from client to server or from server to client.
+If a client-to-server transmission is needed, the client schedules that
+transfer to occur on a subsequent HTTP request.  If a server-to-client
+transfer is needed, then the client sends a "uvgimme" card back to the
+server to request the file content.
+
+<h3 id="gimme">3.7 Gimme Cards</h3>
+
+A gimme card is sent from either client to server or from server
+to client.  The gimme card asks the receiver to send a particular
+artifact back to the sender.  The format of a gimme card is this:
+
+<pre>
+<b>gimme</b> <i>artifact-id</i>
+</pre>
+
+The argument to the gimme card is the ID of the artifact that
+the sender wants.  The receiver will typically respond to a
+gimme card by sending a file card in its reply or in the next
+message.
+
+The "gimme" name means "give me".  The imperative "give me" is
+pronounced as if it were a single word "gimme" in some dialects of 
+English (including the dialect spoken by the original author of Fossil).
+
+<h4>3.7.1 Unversioned Gimme Cards</h4>
+
+When synchronizing unversioned content, the client may send "uvgimme"
+cards to the server.  A uvgimme card requests that the server send
+unversioned content to the client.  The format of a uvgimme card is
+as follows:
+
+<pre>
+<b>uvgimme</b> <i>name</i>
+</pre>
+
+The <i>name</i> is the name of the unversioned file found on the
+server that the client would like to have.  When a server sees a
+uvgimme card, it normally responses with a uvfile card, though it might
+also send another uvigot card if the HTTP reply is already oversized.
+
+<h3 id="cookie">3.8 Cookie Cards</h3>
+
+A cookie card can be used by a server to record a small amount
+of state information on a client.  The server sends a cookie to the
+client.  The client sends the same cookie back to the server on
+its next request.  The cookie card has a single argument which
+is its payload.
+
+<pre>
+<b>cookie</b> <i>payload</i>
+</pre>
+
+The client is not required to return the cookie to the server on
+its next request.  Or the client might send a cookie from a different
+server on the next request.  So the server must not depend on the
+cookie and the server must structure the cookie payload in such
+a way that it can tell if the cookie it sees is its own cookie or
+a cookie from another server.  (Typically the server will embed
+its servercode as part of the cookie.)
+
+<h3 id="reqconfig">3.9 Request-Configuration Cards</h3>
+
+A request-configuration or "reqconfig" card is sent from client to
+server in order to request that the server send back "configuration"
+data.  "Configuration" data is information about users or website
+appearance or other administrative details which are not part of the
+persistent and versioned state of the project.  For example, the "name"
+of the project, the default Cascading Style Sheet (CSS) for the web-interface,
+and the project logo displayed on the web-interface are all configuration
+data elements.
+
+The reqconfig card is normally sent in response to the
+"fossil configuration pull" command.  The format is as follows:
+
+<pre>
+<b>reqconfig</b> <i>configuration-name</i>
+</pre>
+
+As of 2024-10-22, the configuration-name must be one of the
+following values:
+
+<table border=0 align="center">
+<tr><td valign="top">
+<ul>
+<li> css
+<li> header
+<li> mainmenu
+<li> footer
+<li> details
+<li> js
+<li> default-skin
+<li> logo-mimetype
+<li> logo-image
+<li> background-mimetype
+<li> background-image
+<li> icon-mimetype
+<li> icon-image
+<li> timeline-block-markup
+<li> timeline-date-format
+<li> timeline-default-style
+<ul></td><td valign="top"><ul>
+<li> timeline-dwelltime
+<li> timeline-closetime
+<li> timeline-hard-newlines
+<li> timeline-max-comment
+<li> timeline-plaintext
+<li> timeline-truncate-at-blank
+<li> timeline-tslink-info
+<li> timeline-utc
+<li> adunit
+<li> adunit-omit-if-admin
+<li> adunit-omit-if-user
+<li> default-csp
+<li> sitemap-extra
+<li> safe-html
+<li> th1-hooks
+<li> th1-uri-regexp
+<li> project-name
+<ul></td><td valign="top"><ul>
+<li> short-project-name
+<li> project-description
+<li> index-page
+<li> manifest
+<li> binary-glob
+<li> clean-glob
+<li> ignore-glob
+<li> keep-glob
+<li> crlf-glob
+<li> crnl-glob
+<li> encoding-glob
+<li> empty-dirs
+<li> <s title="removed 2020-08, version 2.12.1">allow-symlinks</s>
+<li> dotfiles
+<li> parent-project-code
+<li> parent-project-name
+<ul></td><td valign="top"><ul>
+<li> hash-policy
+<li> comment-format
+<li> mimetypes
+<li> forbid-delta-manifests
+<li> mv-rm-files
+<li> ticket-table
+<li> ticket-common
+<li> ticket-change
+<li> ticket-newpage
+<li> ticket-viewpage
+<li> ticket-editpage
+<li> ticket-reportlist
+<li> ticket-report-template
+<li> ticket-key-template
+<li> ticket-title-expr
+<li> ticket-closed-expr
+<ul></td><td valign="top"><ul>
+<li> user-color-map
+<li> xfer-common-script
+<li> xfer-push-script
+<li> xfer-commit-script
+<li> xfer-ticket-script
+<li> @reportfmt
+<li> @user
+<li> @concealed
+<li> @shun
+<li> @alias
+<li> @subscriber
+<li> @interwiki
+</ul></td></tr>
+</table>
+
+New configuration-names are likely to be added in future releases of
+Fossil.  If the server receives a configuration-name that it does not
+understand, the entire reqconfig card is silently ignored.  The reqconfig
+card might also be ignored if the user lacks sufficient privilege to
+access the requested information.
+
+The configuration-names that begin with an alphabetic character refer
+to values in the "config" table of the server database.  For example,
+the "logo-image" configuration item refers to the project logo image
+that is configured on the Admin page of the [./webui.wiki | web-interface].
+The value of the configuration item is returned to the client using a
+"config" card.
+
+If the configuration-name begins with "@", that refers to a class of
+values instead of a single value.  The content of these configuration items
+is returned in a "config" card that contains pure SQL text that is
+intended to be evaluated by the client.
+
+The @user and @concealed configuration items contain sensitive information
+and are ignored for clients without sufficient privilege.
+
+<h3 id="config">3.10 Configuration Cards</h3>
+
+A "config" card is used to send configuration information from client
+to server (in response to a "fossil configuration push" command) or
+from server to client (in response to a "fossil configuration pull" or
+"fossil clone" command).  The format is as follows:
+
+<pre>
+<b>config</b> <i>configuration-name size</i> <b>\n</b> <i>content</i>
+</pre>
+
+The server will only accept a config card if the user has
+"Admin" privilege.  A client will only accept a config card if
+it had sent a corresponding reqconfig card in its request.
+
+The content of the configuration item is used to overwrite the
+corresponding configuration data in the receiver.
+
+<h3 id="pragma">3.11 Pragma Cards</h3>
+
+The client may try to influence the behavior of the server by
+issuing a pragma card:
+
+<pre>
+<b>pragma</i> <i>name value...</i>
+</pre>
+
+The "pragma" card has at least one argument which is the pragma name.
+The pragma name defines what the pragma does.
+A pragma might have zero or more "value" arguments
+depending on the pragma name.
+
+New pragma names may be added to the protocol from time to time
+in order to enhance the capabilities of Fossil.
+Unknown pragmas are silently ignored, for backwards compatibility.
+
+The following are the known pragma names as of 2019-06-30:
+
+<ol>
+<li><b>send-private</b> The send-private pragma instructs the server to send all of its
+private artifacts to the client.  The server will only obey this
+request if the user has the "x" or "Private" privilege.
+
+<li><b>send-catalog</b> The send-catalog pragma instructs the server to transmit igot
+cards for every known artifact.  This can help the client and server
+to get back in synchronization after a prior protocol error.  The
+"--verily" option to the [/help?cmd=sync|fossil sync] command causes
+the send-catalog pragma to be transmitted.
+
+<li><b>uv-hash</b> <i>HASH</i> The uv-hash pragma is sent from client to server to provoke a
+synchronization of unversioned content.  The <i>HASH</i> is a SHA1
+hash of the names, modification times, and individual hashes of all
+unversioned files on the client.  If the unversioned content hash
+from the client does not match the unversioned content hash on the
+server, then the server will reply with either a "pragma uv-push-ok"
+or "pragma uv-pull-only" card followed by one "uvigot" card for
+each unversioned file currently held on the server.  The collection
+of "uvigot" cards sent in response to a "uv-hash" pragma is called
+the "unversioned catalog".  The client will used the unversioned
+catalog to figure out which files (if any) need to be synchronized
+between client and server and send appropriate "uvfile" or "uvgimme"
+cards on the next HTTP request.
+
+If a client sends a uv-hash pragma and does not receive back
+either a uv-pull-only or uv-push-ok pragma, that means that the
+content on the server exactly matches the content on the client and
+no further synchronization is required.
+
+<li><b>uv-pull-only</b></i> A server sends the uv-pull-only pragma to the client in response
+to a uv-hash pragma with a mismatched content hash argument.  This
+pragma indicates that there are differences in unversioned content
+between the client and server but that content can only be transferred
+from server to client.  The server is unwilling to accept content from
+the client because the client login lacks the "write-unversioned"
+permission.
+
+<li><b>uv-push-ok</b></i> A server sends the uv-push-ok pragma to the client in response
+to a uv-hash pragma with a mismatched content hash argument.  This
+pragma indicates that there are differences in unversioned content
+between the client and server and that content can be transferred
+in either direction.  The server is willing to accept content from
+the client because the client login has the "write-unversioned"
+permission.
+
+<li><b>ci-lock</b> <i>CHECKIN-HASH CLIENT-ID</i> A client sends the "ci-lock" pragma to the server to indicate
+that it is about to add a new check-in as a child of the
+CHECKIN-HASH check-in and on the same branch as CHECKIN-HASH.
+If some other client has already indicated that it was also
+trying to commit against CHECKIN-HASH, that indicates that a
+fork is about to occur, and the server will reply with
+a "ci-lock-fail" pragma (see below).  Check-in locks
+automatically expire when the check-in actually occurs, or
+after a timeout (currently one minute but subject to change).
+
+<li><b>ci-lock-fail</b> <i>LOGIN MTIME</i> When a server receives two or more "ci-lock" pragma messages
+for the same check-in but from different clients, the second a
+subsequent ci-lock will provoke a ci-lock-fail pragma in the
+reply to let the client know that it if continues with the
+check-in it will likely generate a fork.  The LOGIN and MTIME
+arguments are intended to provide information to the client to
+help it generate a more useful error message.
+
+<li><b>ci-unlock</b> <i>CLIENT-ID</i> A client sends the "ci-unlock" pragma to the server after
+a successful commit.  This instructs the server to release
+any lock on any check-in previously held by that client.
+The ci-unlock pragma helps to avoid false-positive lock warnings
+that might arise if a check-in is aborted and then restarted
+on a branch.
+
+<li><b>req-clusters</b> A client sends the "req-clusters" pragma
+to the server to ask the server to reply with "igot" cards for
+every [./fileformat.wiki#cluster|cluster artifact] that it holds.  The
+client typically does this when it thinks that it might be attempting
+to pull a long chain of cluster artifacts.  Sending the artifacts
+all at once can dramatically reduce the number of round trip
+messages needed to complete the synchronization.
+</ol>
+
+<h3 id="comment">3.12 Comment Cards</h3>
+
+Any card that begins with "#" (ASCII 0x23) is a comment card and
+is silently ignored.
+
+<h3 id="error">3.13 Message and Error Cards</h3>
+
+If the server discovers anything wrong with a request, it generates
+an error card in its reply.  When the client sees the error card,
+it displays an error message to the user and aborts the sync
+operation.  An error card looks like this:
+
+<pre>
+<b>error</b> <i>error-message</i>
+</pre>
+
+The error message is English text that is encoded in order to
+be a single token.
+A space (ASCII 0x20) is represented as "\s" (ASCII 0x5C, 0x73).  A
+newline (ASCII 0x0a) is "\n" (ASCII 0x6C, x6E).  A backslash
+(ASCII 0x5C) is represented as two backslashes "\\".  Apart from
+space and newline, no other whitespace characters nor any
+unprintable characters are allowed in
+the error message.
+
+The server can also send a message card that also prints a
+message on the client console, but which is not an error:
+
+<pre>
+<b>message</b> <i>message-text</i>
+</pre>
+
+The message-text uses the same format as an error message.
+
+<h3 id="unknown">3.14 Unknown Cards</h3>
+
+If either the client or the server sees a card that is not
+described above, then it generates an error and aborts.
+
+<h2 id="phantoms" name="clusters">4.0 Phantoms And Clusters</h2>
+
+When a repository knows that an artifact exists and knows the ID of
+that artifact, but it does not know the artifact content, then it stores that
+artifact as a "phantom".  A repository will typically create a phantom when
+it receives an igot card for an artifact that it does not hold or when it
+receives a file card that references a delta source that it does not
+hold.  When a server is generating its reply or when a client is
+generating a new request, it will usually send gimme cards for every
+phantom that it holds.
+
+A cluster is a special artifact that tells of the existence of other
+artifacts.  Any artifact in the repository that follows the syntactic rules
+of a cluster is considered a cluster.
+
+A cluster is line oriented.  Each line of a cluster
+is a card.  The cards are separated by the newline ("\n") character.
+Each card consists of a single character card type, a space, and a
+single argument.  No extra whitespace and no trailing or leading
+whitespace is allowed.  All cards in the cluster must occur in
+strict lexicographical order.
+
+A cluster consists of one or more "M" cards followed by a single
+"Z" card.  Each M card holds an argument which is an artifact ID for an
+artifact in the repository.  The Z card has a single argument which is the
+lower-case hexadecimal representation of the MD5 checksum of all
+preceding M cards up to and included the newline character that
+occurred just before the Z that starts the Z card.
+
+Any artifact that does not match the specifications of a cluster
+exactly is not a cluster.  There must be no extra whitespace in
+the artifact.  There must be one or more M cards.  There must be a
+single Z card with a correct MD5 checksum.  And all cards must
+be in strict lexicographical order.
+
+<h3 id="unclustered">4.1 The Unclustered Table</h3>
+
+Every repository maintains a table named "<b>unclustered</b>"
+which records the identity of every artifact and phantom it holds that is not
+mentioned in a cluster.  The entries in the unclustered table can
+be thought of as leaves on a tree of artifacts.  Some of the unclustered
+artifacts will be other clusters.  Those clusters may contain other clusters,
+which might contain still more clusters, and so forth.  Beginning
+with the artifacts in the unclustered table, one can follow the chain
+of clusters to find every artifact in the repository.
+
+<h2 id="strategies">5.0 Synchronization Strategies</h2>
+
+<h3 id="pull-strategy">5.1 Pull</h3>
+
+A typical pull operation proceeds as shown below.  Details
+of the actual implementation may vary slightly but the gist of
+a pull is captured in the following steps:
+
+<ol>
+<li>The client sends login and pull cards.
+<li>The client sends a cookie card if it has previously received a cookie.
+<li>The client sends gimme cards for every phantom that it holds.
+<hr>
+<li>The server checks the login password and rejects the session if
+the user does not have permission to pull.
+<li>If the number of entries in the unclustered table on the server is
+greater than 100, then the server constructs a new cluster artifact to
+cover all those unclustered entries.
+<li>The server sends file cards for every gimme card it received
+from the client.
+<li>The server sends igot cards for every artifact in its unclustered
+table that is not a phantom.
+<hr>
+<li>The client adds the content of file cards to its repository.
+<li>The client creates a phantom for every igot card in the server reply
+that mentions an artifact that the client does not possess.
+<li>The client creates a phantom for the delta source of file cards when
+the delta source is an artifact that the client does not possess.
+</ol>
+
+These ten steps represent a single HTTP round-trip request.
+The first three steps are the processing that occurs on the client
+to generate the request.  The middle four steps are processing
+that occurs on the server to interpret the request and generate a
+reply.  And the last three steps are the processing that the
+client does to interpret the reply.
+
+During a pull, the client will keep sending HTTP requests
+until it holds all artifacts that exist on the server.
+
+Note that the server tries
+to limit the size of its reply message to something reasonable
+(usually about 1MB) so that it might stop sending file cards as
+described in step (6) if the reply becomes too large.
+
+Step (5) is the only way in which new clusters can be created.
+By only creating clusters on the server, we hope to minimize the
+amount of overlap between clusters in the common configuration where
+there is a single server and many clients.  The same synchronization
+protocol will continue to work even if there are multiple servers
+or if servers and clients sometimes change roles.  The only negative
+effects of these unusual arrangements is that more than the minimum
+number of clusters might be generated.
+
+<h3 id="push-stragegy">5.2 Push</h3>
+
+A typical push operation proceeds roughly as shown below.  As
+with a pull, the actual implementation may vary slightly.
+
+<ol>
+<li>The client sends login and push cards.
+<li>The client sends file cards for any artifacts that it holds that have
+never before been pushed - artifacts that come from local check-ins.
+<li>If this is the second or later cycle in a push, then the
+client sends file cards for any gimme cards that the server sent
+in the previous cycle.
+<li>The client sends igot cards for every artifact in its unclustered table
+that is not a phantom.
+<hr>
+<li>The server checks the login and push cards and issues an error if
+anything is amiss.
+<li>The server accepts file cards from the client and adds those artifacts
+to its repository.
+<li>The server creates phantoms for igot cards that mention artifacts it
+does not possess or for file cards that mention delta source artifacts that
+it does not possess.
+<li>The server issues gimme cards for all phantoms.
+<hr>
+<li>The client remembers the gimme cards from the server so that it
+can generate file cards in reply on the next cycle.
+</ol>
+
+As with a pull, the steps of a push operation repeat until the
+server knows all artifacts that exist on the client.  Also, as with
+pull, the client attempts to keep the size of the request from
+growing too large by suppressing file cards once the
+size of the request reaches 1MB.
+
+<h3 id="sync-strategy">5.3 Sync</h3>
+
+A sync is just a pull and a push that happen at the same time.
+The first three steps of a pull are combined with the first five steps
+of a push.  Steps (4) through (7) of a pull are combined with steps
+(5) through (8) of a push.  And steps (8) through (10) of a pull
+are combined with step (9) of a push.
+
+<h3 id="uv-strategy">5.4 Unversioned File Sync</h3>
+
+"Unversioned files" are files held in the repository
+where only the most recent version of the file is kept rather than
+the entire change history.  Unversioned files are intended to be
+used to store ephemeral content, such as compiled binaries of the
+most recent release.
+
+Unversioned files are identified by name and timestamp (mtime).
+Only the most recent version of each file (the version with
+the largest mtime value) is retained.
+
+Unversioned files are synchronized using the
+[/help?cmd=unversioned|fossil unversioned sync] command.
+
+A schematic of an unversioned file synchronization is as follows:
+
+<ol>
+<li>The client sends a "pragma uv-hash" card to the server.  The argument
+    to the uv-hash pragma is a hash of all filesnames, mtimes, and
+    content hashes for the unversioned files held by the client.
+    <hr>
+<li>If the unversioned content hash from the client matches the unversioned
+    content hash on the server, then nothing needs to be done and the
+    server no-ops.  But if the hashes are different, then the server
+    replies with either a uv-pull-only or a uv-push-ok pragma followed by
+    uvigot cards for all unversioned files held on the server.
+    <hr>
+<li>The client examines the uvigot cards received from the server and
+    determines which unversioned files need to be exchanged in order
+    to bring the client and server into synchronization.  The client
+    then sends appropriate "uvgimme" or "uvfile" cards back to the
+    server.
+    <hr>
+<li>The server updates its unversioned file store with received "uvfile"
+    cards and answers "uvgimme" cards with "uvfile" cards in its reply.
+</ol>
+
+The last two steps might be repeated multiple
+times if there is more unversioned content to be transferred than will
+fit comfortably in a single HTTP request.
+
+<h2 id="summary">6.0 Summary</h2>
+
+Here are the key points of the synchronization protocol:
+
+<ol>
+<li>The client sends one or more PUSH HTTP requests to the server.
+    The request and reply content type is "application/x-fossil".
+<li>HTTP request content is compressed using zlib.
+<li>The content of request and reply consists of cards with one
+    card per line.
+<li>Card formats are:
+    <ul>
+    <li> <b>login</b> <i>userid nonce signature</i>
+    <li> <b>push</b> <i>servercode projectcode</i>
+    <li> <b>pull</b> <i>servercode projectcode</i>
+    <li> <b>clone</b>
+    <li> <b>clone</b> <i>protocol-version sequence-number</i>
+    <li> <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i>
+    <li> <b>file</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i>
+    <li> <b>cfile</b> <i>artifact-id size</i> <b>\n</b> <i>content</i>
+    <li> <b>cfile</b> <i>artifact-id delta-artifact-id size</i> <b>\n</b> <i>content</i>
+    <li> <b>uvfile</b> <i>name mtime hash size flags</i> <b>\n</b> <i>content</i>
+    <li> <b>private</b>
+    <li> <b>igot</b> <i>artifact-id</i> ?<i>flag</i>?
+    <li> <b>uvigot</b> <i>name mtime hash size</i>
+    <li> <b>gimme</b> <i>artifact-id</i>
+    <li> <b>uvgimme</b> <i>name</i>
+    <li> <b>cookie</b>  <i>cookie-text</i>
+    <li> <b>reqconfig</b> <i>parameter-name</i>
+    <li> <b>config</b> <i>parameter-name size</i> <b>\n</b> <i>content</i>
+    <li> <b>pragma</b> <i>name</i> <i>value...</i>
+    <li> <b>error</b> <i>error-message</i>
+    <li> <b>message</b> <i>text-messate</i>
+    <li> <b>#</b> <i>arbitrary-text...</i>
+    </ul>
+<li>Phantoms are artifacts that a repository knows exist but does not possess.
+<li>Clusters are artifacts that contain IDs of other artifacts.
+<li>Clusters are created automatically on the server during a pull.
+<li>Repositories keep track of all artifacts that are not named in any
+cluster and send igot messages for those artifacts.
+<li>Repositories keep track of all the phantoms they hold and send
+gimme messages for those artifacts.
+</ol>
+
+<h2 id="troubleshooting">7.0 Troubleshooting And Debugging Hints</h2>
+
+If you run the [/help?cmd=sync|fossil sync] command
+(or [/help?cmd=pull|pull] or [/help?cmd=push|push] or
+[/help?cmd=clone|clone]) with the --httptrace option, Fossil
+will keep a copy of each HTTP request and reply in files
+named:
+<ul>
+<li> <tt>http-request-</tt><i>N</i><tt>.txt</tt>
+<li> <tt>http-reply-</tt><i>N</i><tt>.txt</tt>
+</ul>
+
+In the above, <i>N</i> is an integer that increments with each
+round-trip.  If you are having trouble on the server side,
+you can run the "[/help?cmd=test-http|fossil test-http]" command in a
+debugger using one the "http-request-N.txt" files as input and
+single step through the processing performed by the server.
+
+The "--transport-command CMD" option on [/help?cmd=sync|fossil sync]
+(and similar) causes the external program "CMD" to be used to move
+the sync message to the server and retrieve the sync reply.  The
+CMD is given three arguments:
+<ol>
+<li> The URL of the server
+<li> The name of a temporary file that contains the output-bound sync
+     protocol text, with the HTTP headers
+<li> The name of a temporary file into which the CMD should write the
+     reply sync protocol text, again without any HTTP headers
+</ol>
+
+In a complex debugging situation, you can run the command
+"fossil sync --transport-command ./debugging_script" where
+"debugging_script" is some script of your own that invokes
+the anomalous behavior you are trying to debug.