From 4f49b16079b27d7ce633619e4cddfdf2bffbc952 Mon Sep 17 00:00:00 2001 From: cashmere Date: Fri, 4 Sep 2026 09:05:24 +0200 Subject: initial commit --- www/json-api/_template.md | 22 ++ www/json-api/api-artifact.md | 155 +++++++++++ www/json-api/api-auth.md | 271 ++++++++++++++++++ www/json-api/api-branch.md | 86 ++++++ www/json-api/api-checkout.md | 54 ++++ www/json-api/api-config.md | 36 +++ www/json-api/api-diff.md | 44 +++ www/json-api/api-dir.md | 53 ++++ www/json-api/api-finfo.md | 53 ++++ www/json-api/api-misc.md | 92 ++++++ www/json-api/api-query.md | 85 ++++++ www/json-api/api-settings.md | 127 +++++++++ www/json-api/api-stat.md | 59 ++++ www/json-api/api-tag.md | 229 +++++++++++++++ www/json-api/api-ticket.md | 136 +++++++++ www/json-api/api-timeline.md | 214 ++++++++++++++ www/json-api/api-user.md | 142 ++++++++++ www/json-api/api-version.md | 32 +++ www/json-api/api-wiki.md | 277 ++++++++++++++++++ www/json-api/conventions.md | 649 +++++++++++++++++++++++++++++++++++++++++++ www/json-api/hacking.md | 353 +++++++++++++++++++++++ www/json-api/index.md | 40 +++ www/json-api/intro.md | 172 ++++++++++++ www/json-api/tips.md | 102 +++++++ 24 files changed, 3483 insertions(+) create mode 100644 www/json-api/_template.md create mode 100644 www/json-api/api-artifact.md create mode 100644 www/json-api/api-auth.md create mode 100644 www/json-api/api-branch.md create mode 100644 www/json-api/api-checkout.md create mode 100644 www/json-api/api-config.md create mode 100644 www/json-api/api-diff.md create mode 100644 www/json-api/api-dir.md create mode 100644 www/json-api/api-finfo.md create mode 100644 www/json-api/api-misc.md create mode 100644 www/json-api/api-query.md create mode 100644 www/json-api/api-settings.md create mode 100644 www/json-api/api-stat.md create mode 100644 www/json-api/api-tag.md create mode 100644 www/json-api/api-ticket.md create mode 100644 www/json-api/api-timeline.md create mode 100644 www/json-api/api-user.md create mode 100644 www/json-api/api-version.md create mode 100644 www/json-api/api-wiki.md create mode 100644 www/json-api/conventions.md create mode 100644 www/json-api/hacking.md create mode 100644 www/json-api/index.md create mode 100644 www/json-api/intro.md create mode 100644 www/json-api/tips.md (limited to 'www/json-api') diff --git a/www/json-api/_template.md b/www/json-api/_template.md new file mode 100644 index 0000000..92b9074 --- /dev/null +++ b/www/json-api/_template.md @@ -0,0 +1,22 @@ +# JSON API: X +([⬑JSON API Index](index.md)) + +Jump to: + +* [Foo](#foo) +* [Bar](#bar) +* [Baz](#baz) + +--- + + +# Foo + + +# Bar + + +# Baz + + +# Footnotes diff --git a/www/json-api/api-artifact.md b/www/json-api/api-artifact.md new file mode 100644 index 0000000..c0e9a54 --- /dev/null +++ b/www/json-api/api-artifact.md @@ -0,0 +1,155 @@ +# JSON API: /artifact +([⬑JSON API Index](index.md)) + +Jump to: + +* [Checkin Artifacts (Commits)](#checkin) +* [File Artifacts](#file) +* [Wiki Artifacts](#wiki) + +--- + + +# Checkin Artifacts (Commits) + +Returns information about checkin artifacts (commits). + +**Status:** implemented 201110xx + +**Request:** `/json/artifact/COMMIT_HASH` + +**Required permissions:** "o" (was "h" prior to 20120408) + +**Response payload example: (CHANGED SIGNIFICANTLY ON 20120713)** + +```json +{ +"type":"checkin", +"name":"18dd383e5e7684e", // as given on CLI +"uuid":"18dd383e5e7684ecee327d3de7d3ff846069d1b2", +"isLeaf":false, +"user":"drh", +"comment":"Merge wideAnnotateUser and jsonWarnings into trunk.", +"timestamp":1330090810, +"parents":[ + // 1st entry is primary parent hash: + "3a44f95f40a193739aaafc2409f155df43e74a6f", + // Remaining entries are merged-in branch hashes: + "86f6e675eb3f8761d70d8b82b052ce2b297fffd2",\ + "dbf4ecf414881c9aad6f4f125dab9762589ef3d7"\ +], +"tags":["trunk"], +"files":[{ + "name":"src/diff.c", + // BLOB hashes, NOT commit hashes: + "uuid":"78c74c3b37e266f8f7e570d5cf476854b7af9d76", + "parent":"b1fa7e636cf4e7b6ed20bba2d2680397f80c096a", + "state":"modified", + "downloadPath":"/raw/src/diff.c?name=78c74c3b37e266f8f7e570d5cf476854b7af9d76" + }, + ...] +} +``` + +The "parents" property lists the parent hashes of the checkin. The +"parent" property of file entries refers to the parent hash of that +file. In the case of a merge there may be essentially an arbitrary +number. The first entry in the list is the "primary" parent. The primary +parent is the parent which was not pulled in via a merge operation. The +ordering of remaining entries is unspecified and may even change between +calls. For example: if, from branch C, we merge in A and B and then +commit, then in the artifact response for that commit the hash of branch +C will be in the first (primary) position, with the hashes for branches A +and B in the following entries (in an unspecified, and possibly +unstable, order). + +Note that the "uuid" and "parent" properties of the "files" entries +refer to raw blob hashes, not commit (a.k.a. check-in) hashes. See also +[the UUID vs. Hash discussion][uvh]. + + +# File Artifacts + +Fetches information about file artifacts. + +**FIXME:** the content type guessing is currently very primitive, and +may (but I haven't seen this) mis-diagnose some non-binary files as +binary. Fossil doesn't yet have a mechanism for mime-type mappings. + +**Status:** implemented 20111020 + +**Required permissions:** "o" + +**Request:** `/json/artifact/FILE_HASH` + +**Request options:** + +- `format=(raw|html|none)` (default=none). If set, the contents of the + artifact are included if they are text, else they are not (JSON does + not do binary). The "html" flag runs it through the wiki parser. The + results of doing so are unspecified for non-embedded-doc files. The + "raw" format means to return the content as-is. "none" is the same + as not specifying this flag, and elides the content from the + response. +- DEPRECATED (use format instead): `includeContent=bool` (=false) (CLI: + `--content|-c`). If true, the full content of the artifact is returned + for text-only artifacts (but not for non-text artifacts). As of + 20120713 this option is only inspected if "format" is not specified. + +**Response payload example: (CHANGED SIGNIFICANTLY ON 20120713)** + +```json +{ +"type":"file", +"name":"same name specified as FILE_HASH argument", +"size": 12345, // in bytes, independent of format=... +"parent": "hash of parent file blob. Not set for first generation.", +"checkins":[{ + "name":"src/json_detail.h", + "timestamp":1319058803, + "comment":"...", + "user":"stephan", + "checkin":"d2c1ae23a90b24f6ca1d7637193a59d5ecf3e680", + "branch":"json", + "state":"added|modified|removed" + }, + ...], +/* The following "content" properties are only set if format=raw|html */ +"content": "file contents", +"contentSize": "size of content field, in bytes. Affected by the format option!", +"contentType": "text/plain", /* currently always text/plain */ +"contentFormat": "html|raw" +} +``` + +The "checkins" array lists all checkins which include this file, and a +file might have different names across different branches. The size and +hash, however, are the same across all checkins for a given blob. + + +# Wiki Artifacts + +Returns information about wiki artifacts. + +**Status:** implemented 20111020, fixed to return the requested version +(instead of the latest) on 20120302. + +**Request:** `/json/artifact/WIKI_HASH` + +**Required permissions:** "j" + +**Options:** + +- DEPRECATED (use format instead): `bool includeContent` (=false). If + true then the raw content is returned with the wiki page, else no + content is returned.\ + CLI: `--includeContent|-c` +- The `--format` option works as for + [`/json/wiki/get`](api-wiki.md#get), and if set then it + implies the `includeContent` option. + +**Response payload example:** + +Currently the same as [`/json/wiki/get`](api-wiki.md#get). + +[uvh]: ../hashes.md#uvh diff --git a/www/json-api/api-auth.md b/www/json-api/api-auth.md new file mode 100644 index 0000000..96854da --- /dev/null +++ b/www/json-api/api-auth.md @@ -0,0 +1,271 @@ +# JSON API: X +([⬑JSON API Index](index.md)) + +Jump to: + +* [Introduction](#intro) +* [Capabilities (Access Rights)](#capabilities) +* [Login](#login) + * [Anonymous User Logins](#login-anonymous) +* [Logout](#logout) +* [Whoami](#whoami) + +--- + + +# Introduction + +**FIXME:** Ross found a bug: +[](/info/479aadb1d2645601) +(sending the authToken via `POST.envelope` isn't working. It should be.) + +The authentication-related operations are described in this section +(ordered alphabetically by operation name). + +The JSON API ties in to/piggybacks on fossil's existing cookie-based +login mechanism, but we must also keep in mind that not all JSON-using +clients support cookies. Cookie-capable clients can rely on cookies for +authentication, whereas non-cookie-aware clients will need to keep track +of the so-called "auth token" which is created via a login request, and +pass it in with each request (either as a GET parameter or a property of +the top-level POST request envelope) to prove to fossil that they are +authorized to access the requested resources. For most intents and +purposes, the "auth token" and the "login cookie" are the same thing (or +serve the same purpose), and the auth token is in fact just the value +part of the login cookie (which has a project-specific key). + +Note that fossil has two conventional user names which can show up in +various responses but do not refer to specific people: nobody and +anonymous. The nobody user is anyone who is not logged in. The anonymous +user is logged in but has no persistent user data (no associated user +name, email address, or similar). Normally the guest (nobody) user has +more access restrictions. The distinction between the two is largely +historical - it is a mechanism to keep bots from following the +multitudes of links generated by the HTML interface (especially the ZIP +files), while allowing interested users to do so by logging in as the +anonymous user (which bots have (or *had*, at the time) a harder time +doing because the password is randomly generated and protected from +brute-force attacks by hashing). In the JSON API, the distinction +between anonymous and nobody is not expected to be as prominent as it is +in the HTML interface because the reasons for the distinction don't +apply in *quite* the same ways to the JSON interface. It is possible, +however, that a given repo locks out guest access to, e.g. the wiki or +tickets, while still allowing anonymous (logged in) access. + + +# Capabilities (Access Rights) + +The "cap" request returns information about the so-called "capabilities" +(access rights) of the currently logged in user. This command is +basically the same as the "whoami" command, but returns capabilities in +a more exanded form (same data, different representation) and does not +include the authToken in the response. + +TODO: consider combining this and [whoami](#whoami) into a single +whoami response. We don't need both. We also don't really need the +permissionFlags member - the same info is already \[in a more cryptic form\] +in the capabilities string. + +**Status:** implemented 201109xx. + +**Required privileges:** none + +**Request:** `/json/cap` + +In CLI mode, permissions are not used/honored, and this command will +report that the caller has all permissions (which he effectively does). + +**Response payload example:** + +```json +{ +"userName":"json-demo", +"capabilities":"hgjorxz", /* raw fossil permissions list */ +"permissionFlags":{ /* Same info in a somewhat friendlier form */ + "setup": false, + "admin": false, + "delete": false, + "password": false, + "query": false, + "checkin": false, + "checkout": true, + "history": true, + "clone": false, + "readWiki": true, + "createWiki": false, + "appendWiki": false, + "editWiki": false, + "readTicket": true, + "createTicket": false, + "appendTicket": false, + "editTicket": false, + "attachFile": false, + "createTicketReport": false, + "readPrivate": false, + "zip": true, + "xferPrivate": false + } +} +``` + +**FIXME:** several new permissions have been added to fossil since +this API was implemented. + + + +# Login + +**Status:** implemented 20110915 (anonymous login 20110918) + +**Required privileges:** none + +**Request:** (see below for Anonymous user special case) + +- `/json/login?name=...&password=...` +- `/json/login?n=...&p=...` (for symmetry with existing login mechanism) + +Or `POST.payload`: `{ "name": ..., "password": ...}` + +(POST is highly preferred because it keeps the password out of web +server logs!) + +**Response payload example:** (structure changed 20111001) + +```json +{ +"authToken":"...", +"name":"json-demo", +"capabilities":"hgjorxz", +"loginCookieName": "fossil-XXXXX" /*project-specific cookie name*/ +/* TODO: add authTokenExpiry timestamp (cookie expiry time) */ +} +``` + +We "should" be able to inherit fossil's `REMOTE_USER` handling without +any special support, but that is untested so far. (If you happen to test +this, please update this doc with (or otherwise report) your results!) + +The response *also* sets the conventional fossil login cookie (for +clients which can make use of cookies), using fossil's existing +mechanism for this. + +Further requests which require authentication must include the +`authToken` (from the returned payload value) in the request (or it must +be available via fossil's standard cookie) or access may (depending on +the request) be denied. The `authToken` may optionally be set in the +request envelope or as a GET parameter, and it *must* be given if the +request requires restricted access to a resource. e.g. if reading +tickets is disabled for the guest user then all non-guest users must +send authentication info in their requests in order to be able to fetch +ticket info. + +Cookie-aware clients should send the login-generated cookie with each +request, in which case they do not need to explicitly include the +`authToken` in the JSON envelope/GET arguments. If submitted, the +`authToken` is used, otherwise the cookie, if set, is used. Note that +fossil uses a project-dependent cookie name in order to help thwart +attacks, so there is no simple mapping of cookie *name* to auth +token. That said, the cookie's *value* is also the auth token's value. + +> Special case: when accessing fossil over a local server instance +which was started with the `--localauth` flag, the `authToken` is ignored +(neither validated nor used for any form of authentication). + + + +## Anonymous User Logins + +The Anonymous user requires special handling because he has a random +password. + +First fetch the password and the so-called "captcha seed" via this +request: + +`/json/anonymousPassword` + +It will return a payload in the form: + +```json +{ +"seed": a_32_bit_unsigned_integer, +"password": "1234abcd" /*hexadecimal STRING*/ +} +``` + +The "seed" and "password" values of the response payload must be set as +the "anonymousSeed" and "password" fields (respectively) of the +subsequent login request. The login request is identical to +non-anonymous login except that extra "anonymousSeed" property is +required. + +The password value *may* be time-limited, and *may* eventually become +invalidated due to old age. This is unspecified. + +***Potential***** (low-probability) bug regarding the seed value:** from +what I hear, some unusual JSON platforms don't support full 32-bit +precision. If absolutely necessary we could chop off a bit or two from +the seed value (*if* it ever becomes a problem and if DRH blesses it). +Or we could just make it a double. + + + +# Logout + +**Status:** implemented 20110916 + +**Required privileges:** none, but must be logged in + +**Request:** + + - `/json/logout?authToken=...token fetched by /login` + +Or: set the `authToken` property of the POST envelope (as opposed to the +`POST.payload`) + +Or: fossil's normal cookie mechanism is the fallback for the auth token. + +**Response payload:** The same as the "whoami" response, containing the +info for the "nobody" user. + +This request requires the authentication token, and subsequent logouts +without an intervening login will fail with the "auth token not +provided" error. In effect this request removes the login entry from the +user account, making the token invalid for future requests. In HTTP +mode, on success fossil's login cookie is unset by this call. + + + +# Whoami + +This request fetches the current user's login name, capabilities, and +auth token. This can be used to check whether a login is active when the +client has not explicitly logged in (e.g. was logged in automatically +via a pre-existing cookie). + +**Status:** implemented 20110922 + +**Required privileges:** none + +**Request:** `/json/whoami` + +**Response payload example:** + +```json +{ +"name": "nobody", +"capabilities": "o", +"authToken": "fossil auth token (only for logged-in users)" +} +``` + +The reason authToken is included in the response is because it gives +client-side JavaScript code a way of fetching/checking for the auth +token at app startup. The token is normally sent as a cookie but parsing +the cookies in the browser is tedious, and fossil has a +project-dependent cookie name (which complicates parsing a bit). If +client code digs the cookie out of the browser, the app still wouldn't +know if the token is still valid, whereas whoami won't (or shouldn't!) +return an expired auth token. If the request does not include +authentication info (via the cookie, GET param, or request envelope) +then the response will not contain the authToken property and the user's +name will be "nobody". diff --git a/www/json-api/api-branch.md b/www/json-api/api-branch.md new file mode 100644 index 0000000..a2a002b --- /dev/null +++ b/www/json-api/api-branch.md @@ -0,0 +1,86 @@ +# JSON API: /branch +([⬑JSON API Index](index.md)) + +Jump to: + +* [Branch List](#list) +* [Create Branch](#create) + +--- + + +# Branch List + +**Status:** implemented, at least in draft form, on 20110921. + +**Required privileges:** "o" + +**Request:** `/json/branch/list` + +**Response payload example:** + +```json +{ +"range":"closed", +"current":"json", /* only when there is a local opened checkout */ +"branches":[ + "artifact_description", + "bch", + "ben-changes-report", + "ben-safe-make", + "ben-security", + "ben-testing", + … +] +} +``` + +*Potential* TODO: add "tip" property which names the most recently +modified branch? (How to get this?) + +HTTP GET/`POST.payload` options: + +- `range`: a string in the set ("open", "closed", "all"), + case-sensitive, but only the first letter is actually evaluated. + Default="open". Only branches with this state are returned. + +CLI mode options (same semantics as HTTP equivalents), must come last on +the CLI: + +- `-r|--range all|closed|open` +- `-a` (equivalent to `-r all`) +- `-c` (equivalent to `-r closed`). Only one of `-a`/`-c` may be specified, + and if both are specified then which one takes precedence is + unspecified. + + + +# Create Branch + +**Status:** implemented 20111002 + +**Required privileges:** "w" + +**Request:** `/json/branch/create` + +**Request options:** + +- `name=string` (REQUIRED) Name of new branch +- `basis=string` (default=trunk) Name of parent branch to branch from. +- `bgColor=string` (default=something ugly) In `#RRGGBB` form. (FIXME: + change the default to use fossil's random bgcolor technique.) +- `private=bool` (default=false) Determines whether the branch is + private or not. + +**Response payload example:** + +```json +{ +"name":"my-branch", +"basis":"my-other-branch", +"uuid":"de8115db4ce388ed8d0af666ae7d90e1410be4ca", +"isPrivate":true, +"bgColor":"#ff0000" +} +``` + diff --git a/www/json-api/api-checkout.md b/www/json-api/api-checkout.md new file mode 100644 index 0000000..0fd2437 --- /dev/null +++ b/www/json-api/api-checkout.md @@ -0,0 +1,54 @@ +# JSON API: /status +([⬑JSON API Index](index.md)) + +# Status of Local Checkout + +**Status:** implemented 20130223 + +**Required permissions:** n/a (local access only) + +**Request:** `/json/status` + +This command requires a local checkout and is the analog to the "fossil +status" command. + +**Request Options:** currently none. + +Payload example: + +```json +{ +"repository":"/home/stephan/cvs/fossil/fossil.fsl", +"localRoot":"/home/stephan/cvs/fossil/fossil/", +"checkout":{ + "uuid":"b38bb4f9bd27d0347b62ecfac63c4b8f57b5c93b", + "tags":["trunk"], + "datetime":"2013-02-22 17:34:19 UTC", + "timestamp":1361554459 + }, +/* "parent" info is currently missing. */ +"files":[ + {"name":"src/checkin.c", "status":"edited"} + ...], +"errorCount":0 /* see notes below */ +} +``` + +Notes: + +- The `checkout.tags` property follows the framework-wide convention + that the first tag in the list is the primary branch and any others + are secondary. +- `errorCount` is +1 for each missing file. Conflicts are not treated as + errors because the CLI 'status' command does not treat them as such. +- The `"status"` entry for each of the `"files"` entries will be either a + single string containing a single description of the status change, or + an array of strings if more than one change was made, e.g. `"edited"` + and `"renamed"`. The status strings include:\ + `deleted`, `new`, `notAFile`, `missing`, `updatedByMerge`, + `updatedByIntegrate`, `addedBymerge`, `addedByIntegrate`, + `conflict`, `edited`, `renamed` +- File objects with a `"renamed"` status will contain a `"priorName"` + property in addition to the `"name"` property reported in all cases. +- TODO: Info for the parent version is currently missing. +- TODO: "merged with" info for the checkout is currently missing. diff --git a/www/json-api/api-config.md b/www/json-api/api-config.md new file mode 100644 index 0000000..e89b26f --- /dev/null +++ b/www/json-api/api-config.md @@ -0,0 +1,36 @@ +# JSON API: /config +([⬑JSON API Index](index.md)) + +Jump to: + +* [Get Config](#get) +* [Set Config](#set) + +--- + + +# Fetch Config + +**Status:** Implemented 20120217 + +**Required permissions:** "s" + +**Request:** `/json/config/get/Area[/Area2/...AreaN]` + +Where "Area" can be any combination of: *skin*, *ticket*, *project*, +*all*, or *skin-backup* (which is not included in "all" by default). + +**Response payload example:** + +```json +{ +"ignore-glob":"*~", +"project-description":"For testing Fossil's JSON API.", +"project-name":"fossil-json-tests" +} +``` + + +# Set/Modify Config + +Not implemented. diff --git a/www/json-api/api-diff.md b/www/json-api/api-diff.md new file mode 100644 index 0000000..d001314 --- /dev/null +++ b/www/json-api/api-diff.md @@ -0,0 +1,44 @@ +# JSON API: /diff +([⬑JSON API Index](index.md)) + +# Diffs + +**Status:** implemented 20111007 + +**Required permissions:** "o" + +**Request:** `/json/diff[/version1[/version2]]` + +**Request options:** + +- `v1=string` Is the "from" version. It may optionally be the first + positional parameter/path element after the command name. +- `v2=string` Is the "to" version. It may optionally be the first + positional parameter/path element after the v1 part. +- `context=integer` (default=unspecified) Defines the number of context + lines to display in the diff.\ + CLI: `--context|-c` +- `sbs=bool` (default=false) Generates "side-by-side" diffs, but their + utility in JSON mode is questionable. +- `html=bool` (default=false) causes the output to be marked up with + HTML in the same manner as it is in the HTML interface. + +**Response payload example:** + +```json +{ +"from":"7a83a5cbd0424cefa2cdc001de60153aede530f5", +"to":"96920e7c04746c55ceed6e24fc82879886cb8197", +"diff":"@@ -1,7 +1,7 @@\\n-C factored\\\\sout..." +} +``` + +TODOs: + +- Unlike the standard diff command, which apparently requires a commit + hash, this one diffs individual file versions. If a commit hash is + provided, a diff of the manifests is returned. (That should be + considered a bug - we should return a combined diff in that case.) +- If hashes from two different types of artifacts are given, results + are unspecified. Garbage in, garbage out, and all that. +- For file diffs, add the file name(s) to the response payload. diff --git a/www/json-api/api-dir.md b/www/json-api/api-dir.md new file mode 100644 index 0000000..0e69394 --- /dev/null +++ b/www/json-api/api-dir.md @@ -0,0 +1,53 @@ +# JSON API: /dir +([⬑JSON API Index](index.md)) + +# Directory Listing + +**Status:** implemented 20120316 + +**Required privileges:** "o". Was "h" prior to 20120720, but the HTML +version of /dir changed permissions and this API was modified to match +it. + +**Request:** `/json/dir` + +Options: + +- `checkin=commit` (use "tip" for the latest). If checkin is not set + then all files from all versions of the tree are returned (but only + once per file - not with complete version info for each file in all + branches).\ + CLI: `--checkin|-ci CHECKIN` +- `name=subdirectory` name. To fetch the root directory, don't pass this + option, or use an empty value or "/".\ + CLI: use `--name|-n NAME` or pass it as the first argument after + the `dir` subcommand. + +**Response payload example:** + +```json +{ + "name":"/", /* dir name */ + "uuid":"ac6366218035ed62254c8d458f30801273e5d4fc", + "checkin":"tip", + "entries":[{ + "name": "ajax", /* file name/dir name fragment */ + "isDir": true, /* only set for directories */ + /* The following properties are ONLY set if + the 'checkin' parameter is provided. + */ + "uuid": "..." /*only for files, not dirs*/, + "size": number, + "timestamp": number + },...] +} +``` + +The checkin property is only set if the request includes it. The +entry-specific uuid and size properties (e.g. `entries[0].uuid`) are +only set if the checkin request property is set and they refer to the +latest version of that file for the given checkin. The `isDir` property is +only set on directory entries. + +This command does not recurse into subdirectories, though it "should be" +simple enough to add the option to do so. diff --git a/www/json-api/api-finfo.md b/www/json-api/api-finfo.md new file mode 100644 index 0000000..e6e757f --- /dev/null +++ b/www/json-api/api-finfo.md @@ -0,0 +1,53 @@ +# JSON API: /finfo +([⬑JSON API Index](index.md)) + +# File Information + +**Status:** implemented 2012-something, but output structure is likely +to change. + +**Required privileges:** "o" + +**Request:** `/json/finfo?name=path/to/file` + +Options: + +- `name=string`. Required. Use the absolute name of the file, including + any directory parts, and without a leading slash. e.g. + `"path/to/my.c"`.\ + CLI mode: `--name` or positional argument. +- `checkin=string`. Only return info related to this specific checkin, + as opposed to listing all checkins. If set, neither "before" nor + "after" have any effect.\ + CLI mode: `--checkin|-ci` +- `before=DATETIME` only lists checkins from on or before this time.\ + CLI mode: `--before|-b` +- `after=DATETIME` only lists checkins from on or after this time. + Using this option swaps the sort order, to provide reasonable + behaviour in conjunction with the limit option.\ + Only one of "before" and "after" may be specified, and if both are + specified then which one takes precedence is unspecified.\ + CLI mode: `--after|-a` +- `limit=integer` limits the output to (at most) the given number of + associated checkins.\ + CLI mode: `--limit|-n` + +**Response payload example (very likely to change!):** + +```json +{ +"name":"ajax/i-test/rhino-shell.js", +"checkins":[{ + "checkin":"6b7ddfefbfb871f793378d8f276fe829106ca49b", + "uuid":"2b735676d55e3d06d670ffbc643e5d3f748b95ea", + "timestamp":1329514170, + "user":"viriketo", + "comment":"<...snip...>", + "size":6293, + "state":"added|modified|removed" + },…] +} +``` + +**FIXME:** there is a semantic discrepancy between `/json/artifact`'s +`payload.checkins[N].uuid` and this command's. diff --git a/www/json-api/api-misc.md b/www/json-api/api-misc.md new file mode 100644 index 0000000..6bc788a --- /dev/null +++ b/www/json-api/api-misc.md @@ -0,0 +1,92 @@ +# JSON API: Misc. APIs +([⬑JSON API Index](index.md)) + +Some operations simply don't fit into a specific category (well, none +except "misc")... + +Jump to: + +* [Global State ("g")](#g) +* [Rebuild Repository](#rebuild) +* [Result Code Descriptions](#result-codes) + +--- + + +# Global State ("g") + +Fossil's internal state is maintained in a global object called "g". And +thus this command is named "g"... + +**Status:** implemented 20111009 + +**Required permissions:** "a" or "s" + +**Request:** `/json/g` + +**Response payload example:** this is a debugging-only request, and has +no stable response payload structure. The result object contains all +kinds of details gleaned from the fossil environment. + +Easter egg: this output can be added to ANY response by passing the +`debugFossilG` boolean in the POST envelope or GET parameters, or as the +`--json-debug-g` flag in CLI mode. This requires admin or setup +privileges, though, and it is silently ignored for other users. + + + +# Rebuild Repository + +This request does the same as the "fossil rebuild" command, rebuilding +the repo-internal structure. This is often required after upgrading the +fossil binary on a system. There "are very probably" cases where calling +this over HTTP will not work (e.g. if the user table has changed enough +that the access rights cannot be validated without a rebuild, i.e. a +chicken/egg scenario). Another consideration is that this request can +take a long time to run - rebuilding the fossil repo on my laptop takes +about 21 seconds, which is likely longer than the timeout set on an XHR +request, meaning that the rebuild transaction will fail. It can safely +be run in CLI mode, where timeouts are not normally a concern. As a +preliminary benchmark: rebuilding the fossil repo (as of late 2011) +takes just over 21 seconds on my 32-bit 1.6GHz netbook. That said, most +repos are much smaller and rebuild in under a few seconds. + +**Status:** implemented 20110929 + +**Required privileges:** "a" + +**Request:** `/json/rebuild` + +Requires admin access rights. + +**Response payload:** none (response envelope `resultCode` reports failure). +Potential TODO: return the amount of time it took to rebuild. + + + +# Result Code Descriptions + +This request returns the full list of result codes documented for +Fossil's JSON API. Each result code is returned as an object containing +metadata about the result code. + +**Status:** implemented 20111006 + +**Required permissions:** none + +**Request:** `/json/resultCodes` + +**Response payload example:** + +```json +[{ + "resultCode":"FOSSIL-1000", + "cSymbol":"FSL_JSON_E_GENERIC", + "number":1000, + "description":"Generic error" + }, + … many more objects with the same structure … +] +``` + + diff --git a/www/json-api/api-query.md b/www/json-api/api-query.md new file mode 100644 index 0000000..e246625 --- /dev/null +++ b/www/json-api/api-query.md @@ -0,0 +1,85 @@ +# JSON API: /query +([⬑JSON API Index](index.md)) + +# SQL Query + +**Status:** implemented 20111008 + +**Required privileges:** "a" or "s" + +**Request:** `/json/query` + +Potential FIXME: restrict this to queries which return results, as opposed +to those which may modify data. + +Options: + +- `sql=string` The SQL code to run. It is expected that it be a SELECT + statement, but that is not enforced. This parameter may be set as a + POST.payload property, as the POST.payload itself, GET, or as a + positional parameter coming after the command name (CLI and HTTP + modes, though the escaping would be unsightly in HTTP mode). +- `format=string` (default="o"). "o" specifies that each result row + should be in the form of key/value pairs (o=object). "a" means each + row should be an array of values. + +**Example request:** + +POST to: `/json/query` + +```json +{ +"authToken": "...", +"payload": { + "sql": "SELECT * FROM reportfmt", + "format": "o" + } +} +``` + +**Response payload example:** (assuming the above example) + + +``` +{ +"columns":[ + "rn", + "owner", + "title", + "mtime", + "cols", + "sqlcode" + ], + "rows":[ + { + "rn":1, + "owner":"drh", + "title":"All Tickets", + "mtime":1303870798, + }, + … + ] +} +``` + +The column names are provided in a separate field because their order +is guaranteed to match the order of the query columns, whereas object +key/value pairs might get reordered (typically sorted by key) when +travelling through different JSON implementations. In this manner, +clients can e.g. be sure to render the columns in the proper +(query-specified) order. + +When in "array" mode the "rows" results will be an array of arrays. For +example, the above "rows" property would instead look like: + +`[ [1, "drh", "All Tickets", 1303870798, … ], … ]` + +Note the column *names* are never *guaranteed* to be exactly as they +appear in the SQL *unless* they are qualified with an AS, e.g. `SELECT +foo AS foo...`. When generating reports which need fixed column names, it +is highly recommended to use an AS qualifier for every column, even if +they use the same name as the column. This is the only way to guarantee +that the result column names will be stable. (FYI: that behaviour comes +from sqlite3, not the JSON bits, and this behaviour *has* been known to +change between sqlite3 versions (so this is not just an idle threat of +*potential* future incompatibility).) diff --git a/www/json-api/api-settings.md b/www/json-api/api-settings.md new file mode 100644 index 0000000..5acde97 --- /dev/null +++ b/www/json-api/api-settings.md @@ -0,0 +1,127 @@ +# JSON API: /settings +([⬑JSON API Index](index.md)) + +Jump to: + +* [Fetch Settings](#get) +* [Set Settings](#set) + +--- + + +# Fetch Settings + +**Status:** Implemented 20230120 + +**Required permissions:** "o" + +**Request:** `/json/settings/get[?version=X]` + +**Response payload example:** + +```json +{ + "access-log":{ + "versionable":false, + "sensitive":false, + "defaultValue":"off", + "valueSource":null, + "value":null + }, +... + "binary-glob":{ + "versionable":true, + "sensitive":false, + "defaultValue":null, + "valueSource":"versioned", + "value":"*.gif\n*.ico\n*.jpg\n*.odp\n*.dia\n*.pdf\n*.png\n*.wav..." + }, +... + "web-browser":{ + "versionable":false, + "sensitive":true, + "defaultValue":null, + "valueSource":"repo", + "value":"firefox" + } +} +``` + +Each key in the payload is the name of a fossil-recognized setting, +modeled as an object. The keys of that are: + + +- `defaultValue`: the setting's default value, or `null` if no default + is defined. +- `value`: The current value of that setting. +- `valueSource`: one of (`"repo"`, `"checkout"`, `"versioned"`, or + `null`), specifying the data source where the setting was found. The + settings sources are checked in this order and the first one found + is the result: + - If `version=X` is provided, check-in `X` is searched for a + versionable-settings file. If found, its value is used and + `valueSource` will be `"versioned"`. If `X` is not a checkin, an + error response is produced with code `FOSSIL-3006`. + - If a versionable setting is found in the current checkout, its + value is used and `valueSource` will be `"versioned"` + - If the setting is found in checkout database's `vvar` table, its + value is used and `valueSource` will be `"checkout"`. + - If the setting is found in repository's `config` table, its + value is used and `valueSource` will be `"repo"`. + - If no value is found, `null` is used for both the `value` and + `valueSource` results. Note that _global settings are never + checked_ because they can leak information which have nothing + specifically to do with the given repository. +- `sensitive`: a value which fossil has flagged as sensitive can only + be fetched by a Setup user. For other users, they will always have + a `value` and `valueSource` of `null`. +- `versionable`: `true` if the setting is tagged as versionable, else + `false`. + +Note that settings are internally stored as strings, even if they're +semantically treated as numbers. The way settings are stored and +handled does not give us enough information to recognize their exact +data type here so they are passed on as-is. + + + +# Set Settings + +**Status:** Implemented 20230120 + +**Required permissions:** "s" + +**Request:** `/json/settings/set` + +This call requires that the input payload be an object containing a +mapping of fossil-known configuration keys (case-sensitive) to +values. For example: + +```json +{ + "editor": "emacs", + "admin-log": true, + "auto-captcha": false +} +``` + +It iterates through each property, which must have a data type of +`null`, boolean, number, or string. A value of `null` _unsets_ +(deletes) the setting. Boolean values are stored as integer 0 +or 1. All other types are stored as-is. It only updates the +`repository.config` database and never updates a checkout or global +config database, nor is it capable of updating versioned settings +(^Updating versioned settings requires creating a full check-in.). + +It has no result payload but this may be changed in the future it +practice shows that it should return something specific. + +Error responses include: + +- `FOSSIL-2002`: called without "setup" permissions. +- `FOSSIL-3002`: called without a payload object. +- `FOSSIL-3001`: passed an unknown config option. +- `FOSSIL-3000`: a value has an unsupported data type. + +If an error is triggered, any settings made by this call up until that +point are discarded. diff --git a/www/json-api/api-stat.md b/www/json-api/api-stat.md new file mode 100644 index 0000000..e4cd902 --- /dev/null +++ b/www/json-api/api-stat.md @@ -0,0 +1,59 @@ +# JSON API: /stat +([⬑JSON API Index](index.md)) + +# Repository Stats + +**Status:** implemented + +**Required privileges:** "o" + +**Request:** `/json/stat` + +**Response payload example:** (fields marked with `*` are omitted in +"brief" mode) + +```json +{ +"projectName":"Fossil", +"projectDescription":"Fossil SCM", /* added 20120217 */ +"repositorySize":24464384, +* "blobCount":13612, +* "deltaCount":9348, +* "uncompressedArtifactSize":589205834, +* "averageArtifactSize":43292, +* "maxArtifactSize":4620758, +* "compressionRatio":"24:1", +* "checkinCount":3150, +* "fileCount":456, +* "wikiPageCount":23, +* "ticketCount":940, +"ageDays":1512, +"ageYears":4.139744, +"projectCode":"25d3a4b83202c0d616a5ed17334f180dac4434dc", +"compiler":"gcc-4.1.2 20080704 (Red Hat 4.1.2-50)", +"sqlite":{ + "version":"2011-09-04 01:27:00 [6b657ae750] (3.7.8)", + "pageCount":23891, + "pageSize":1024, + "freeList":2705, + "encoding":"UTF-8", + "journalMode":"delete" +} +} +``` + +**Options:** + +- "Full detail" mode:\ + **HTTP/payload parameter:** full=bool\ + **CLI MODE:** -f|--full with no value.\ + If true then all properties are included, else certain properties + are omitted from the payload (because they take a relatively long + time to calculate).\ + **TODO:** rename this to verbose, for consistency.\ + **Default=false**. *This is in contrast to the HTML interface*, + which defaults to full detail mode. Testing shows stat to have a + relatively high per-call cost/run time, so it defaults + to "brief" mode by default. Full-detail mode can, on slow hardware, + take half a minute to respond, whereas non-full mode takes well + under one second. diff --git a/www/json-api/api-tag.md b/www/json-api/api-tag.md new file mode 100644 index 0000000..a9faede --- /dev/null +++ b/www/json-api/api-tag.md @@ -0,0 +1,229 @@ +# JSON API: /tag +([⬑JSON API Index](index.md)) + +Jump to: + +* [Add](#add) +* [Cancel](#cancel) +* [Find](#find) +* [List](#list) + +--- + + +# Add Tag + +**Status:** implemented 20111006 + +**Required permissions:** "i" + +**Request:** `/json/tag/add[/name[/checkin[/value]]]` + +**Request options:** + +- `name=string` The tag name. +- `checkin=string` The checkin to tag. May be a symbolic branch name. +- raw=bool (=false) If true, then the name is set as it is provided by + the client, else it gets "sym-" prefixed to it. Do not use this + unless you really know what you're doing. +- `value=string` (=null) An optional value. While tags *may* have values + in fossil, it is unusual for them to have a value. (This probably + has some interesting uses in custom UIs.) +- `propagate=bool` (=false) Sets the tag to propagate to all descendants + of the given checkin. + +In CLI modes, the name, checkin, and value parameters may optionally be +supplied as positional parameters (in that order, after the command +name). In HTTP mode they may optionally be the 4th-6th path elements or +specified via GET/`POST.envelope` parameters. + +**Response payload example:** + +```json +{ +"name":"my-tag", +"value":"abc", +"propagate":true, +"raw":false, +"appliedTo":"626ab2f3743543122cc11bc082a0603d2b5b2b1b" +} +``` + +The `appliedTo` property is the hash of the check-in to which the tag was +applied. This is the "resolved" version of the check-in name provided by +the client. + + +# Cancel Tag + +**Status:** implemented 20111006 + +**Required permissions:** "i" + +**Request:** `/json/tag/cancel[/name[/checkin]]` + +**Request options:** + +- `name=string` The tag name. May optionally be provided as the 4th path + element. +- `checkin=string` The checkin to untag. May be a symbolic branch name. + May optionally be provided as the 5th path element. + +In CLI modes, the name and checkin parameters may optionally be supplied +as positional parameters (in that order, after the command name) or +using the `-name NAME` and `-checkin NAME` options. In HTTP mode they may +optionally be the 4th and 5th path elements. + +**Response payload:** none (resultCode indicates failure) + + + +# Find Tag + +Fetches information about artifacts having a particular tag. + +Achtung: the output of this response is based on the HTML-mode +implementation, but it's not yet certain that it's exactly what we want +for JSON mode. The request options and response format may change. + +**Status:** implemented 20111006 + +**Required permissions:** "o" + +**Request:** `/json/tag/find[/tagName]` + +The response format differs somewhat depending on the options: + +- `name=string` The tag name to search for. Can optionally be the 3rd + path element. +- `limit=int` (defalt=0) Limits the number of results (0=no limit). + Since they are ordered from oldest to newest, the newest N results + will be returned. +- `type=string` (default=`*`) Searches only for the given type of + artifact (using fossil's conventional type naming: ci, e, t, w.) +- `raw=bool` (=false) If enabled, the response is an array of hashes + of the requested artifact type; otherwise, + it is an array of higher-level objects. If this is + true, the "name" property is interpreted as-is. If it is false, the + name is automatically prepended with "sym-" (meaning a branch). + (FIXME: the current semantics are confusing and hard to remember. + Re-do them.) + +**Response payload example, in RAW mode: (expect this format to change +at some point!)** + +```json +{ +"name":"sym-trunk" +"raw":true, +"type":"*", +"limit":2, +"artifacts":[ + "a28c83647dfa805f05f3204a7e146eb1f0d90505", + "dbda8d6ce9ddf01a16f6c81db2883f546298efb7" + ] +} +``` + +Very likely todo: return more information with that (at least the +artifact type and timestamp). Once the `/json/artifact` family of bits is +finished we could use that to return artifact-type-dependent values +here. + +**Response payload example, in non-raw mode:** + +``` +{ +"name":"trunk", +"raw":false, +"type":"*", +"limit":1, +"artifacts":[{ + "uuid":"4b0f813b8c59ac8f7fbbe33c0a369acc65407841", + "timestamp":1317833899, + "comment":"fixed [fc825dcf52]", + "user":"ron", + "eventType":"checkin" + }] +} +``` + + +# List Tags + +**Status:** implemented 20111006 + +**Required permissions:** "o" + +**Request:** `/json/tag/list[/checkinName]` + +Potential fixme: we probably have too many different response formats +here. We should probably break this into multiple subcommands. + +The response format differs somewhat depending on the options: + +- `checkin=string` Lists the tags only for the given CHECKIN (can be a + branch name). If set, includeTickets is ignored (meaningless in this + combination). This option can be set as either a GET/`POST.payload` + option, as the last element of the request path, e.g. + `/json/tag/list/MYBRANCH` *or* with `POST.payload` set to a string + value. +- `raw=bool` (default=false) uses "raw" tag names +- `includeTickets=bool` (default=false) Determines whether `tkt-` tags + are included. There is one of these for each ticket, so there can be + many of them (over 900 in the main fossil repo as of this writing). + +**Response format when raw=false and no checkin is specified:** + +```json +{ +"raw":false, +"includeTickets":false, +"tags":[ + "bgcolor", + "branch", + "closed", + …all tag names... + ] +} +``` + +Enabling the `raw` option will leave the internal `sym-` prefix on tags +which have them but will not change the structure. If `includeTickets` is +true then `tkt-` entries (possibly very many!) will be included in the +output, else they are elided. + +**General notes:** + +The `tags` property will be null if there are no tags, every non-empty +repo has at least one tag (for the trunk branch). + +**Response format when raw=false and checkin is specified:** + +```json +{ +"raw":false, +"tags":{ + "json":null, + "json-multitag-test":null + } +} +``` + +The `null`s there are the tag values (most tags don't have values). + +If `raw=true` then the tags property changes slightly: + +```json +{ +"raw":true, +"tags":{ + "branch":"json", + "sym-json":null, + "sym-json-multitag-test":null + } +} +``` + +TODO?: change the tag values to objects in the form +`{value:..., tipUuid:string, propagating:bool}`. diff --git a/www/json-api/api-ticket.md b/www/json-api/api-ticket.md new file mode 100644 index 0000000..1d02153 --- /dev/null +++ b/www/json-api/api-ticket.md @@ -0,0 +1,136 @@ +# JSON API: Tickets +([⬑JSON API Index](index.md)) + +Jump to: + +* [Ticket Reports](#reports) + * [Fetch a Report](#report-get) + * [List Reports](#report-list) + * [Run a Report](#report-run) + + +--- + +# Tickets + +This API is incomplete. It is missing at least the following features: + +- Content for a given ticket ID +- History for a given ticket ID? +- An option to enable/disable the generation of hyperlinks, as links + won't be useful in most non-browser clients. + + + +# Ticket Reports + + +## Fetch a Report + +**Status:** implemented 20111008 + +**Required privileges:** "t" (the thinking being that only those +permitted to create reports should be able to read their SQL code) + +**Request:** `/json/report/get[/REPORT_NUMBER]` + +**Request options:** + +- `report=number` The report number to fetch.\ + CLI: `-report|-r` \ + (Design note: `--number/-n` was not used because that parameter has a + different meaning (limit response count) in several commands.) May + optionally be defined via the 4th GET path element or CLI arg. + +**Response payload example:** + +```json +{ +"report":1, +"owner":"drh", +"title":"All Tickets", +"timestamp":"112443570187200", +"columns":"#ffffff Key:\r\n#f2dcdc Active\r\n...", +"sqlCode":"..." +} +``` + + +## List Reports + +**Status:** implemented 20111008 + +**Required privileges:** "r" or "n" + +**Request:** `/json/report/list` + +**Response payload example:** + +```json +[ + { + "report":1, + "title":"All Tickets", + "owner":"drh" + }, + … +] +``` + + +## Run a Report + +**Status:** implemented 20111008 + +**Required privileges:** "r" or "n" + +**Request:** `/json/report/run[/REPORT_NUMBER]` + +Request options: + +- `report|-r=int` Specifies which report to run. May optionally be + supplied as the 4th CLI arg or URL path element. +- `format|-f=string` (default="o") Specifies "array" or "object" output + format. + +**Response payload example:** + +```json +{ + "report":1, + "title":"All Tickets", + "sqlcode": "only set if requester has 't' privileges.", + "columnNames":[ … list of column names … ], + "tickets":[ + { + "bgcolor":"#cfe8bd", + "#":"fc825dcf52", + "timestamp":"112443570187200", + "type":"Code_Defect", + "status":"Fixed", + "subsystem":null, + "title":"\"config pull all\" asks to approve ssl cert" + }, + … + ] +} +``` + +Note that the column names of ticket reports are determined by the +reports themselves, and not C code. That means that we cannot return a +standard set of column names here. Fossil requires certain column naming +conventions for purposes of styling the HTML interface, e.g. the "\#" +column is always the uuid of the record and "bgcolor" (note the +different casing than bgColor used throughout the rest of this API!) has +a specific meaning to the HTML report browser. Fossil also allows the +tickets to be extended with client-specified fields, so we cannot +generically make these results fit into the API-wide naming scheme. Full +details are here: + +[](/doc/trunk/www/custom_ticket.wiki) + +and: + +[](/rptsql?rn=1) + +(That one may require non-default permission.) diff --git a/www/json-api/api-timeline.md b/www/json-api/api-timeline.md new file mode 100644 index 0000000..b0ff196 --- /dev/null +++ b/www/json-api/api-timeline.md @@ -0,0 +1,214 @@ +# JSON API: /timeline +([⬑JSON API Index](index.md)) + +Jump to: + +* [Introduction](#intro) +* [Branch Timeline](#branch) +* [Technote (formerly Event) Timeline](#technote) +* [Ticket Timeline](#ticket) +* [Wiki Timeline](#wiki) + +--- + + +# Introduction + +These requests return overview-level information about various types of +changes. The response payload differs for each artifact type, and the +current structures are almost certainly not "final" (e.g. we are still +undecided on how/whether to handle artifact links within commit messages +and whatnot). + +By default the entries are returned in chronological order from newest +to oldest, but some options might change that. + +FIXME (20120623): we have some inconsistent `type` vs. `eventType` in +the result sets. `type` is the current preferred choice (and it seems +unlikely that `eventType` is actually used in any client code). We +don't actually need either one (but a use for `type` is easily +envisioned), and we may get rid of both. + +**Common request options (via CLI, GET or POST.payload):** + +- `limit=integer` limits the number of entries in the response. Default + is unspecified (but is "quite possibly 20 or so"). A limit value of + 0 disables any limit, fetching all entries (which can take a really + long time and might overload clients which have very limited + memory).\ + CLI mode: `--limit|-n #` +- `after="YYYY-MM-DD[ HH:mm:ss]"` limits the search to items on or + after the given date string. Reverses the normal timeline sort + order. Alias: "a". Only one of "after" or "before" can be used, and + if both are specified then which one takes precedence is + unspecified.\ + CLI mode: `--after|-a "DATE[ TIME]"` +- `before="YYYY-MM-DD[ HH:mm:ss]"` limits the search to items on or + before the given date string.\ + CLI mode: `--before|-b "DATE[ TIME]" +- TODOs, still to be ported from the HTML-mode timeline: + - circa=DATETIME + - tag=string + - related=tag name + - string=search string + + +# Branch Timeline + +**Status:** partially implemented but undocumented because the utility +of the current impl is under question. It also doesn't understand most +of the common timeline options. + + +# Checkin Timeline + +**Status:** implemented 201109xx + +**Required privileges:** "o" + +**Request:** `/json/timeline/checkin` + +**Response payload example:** + +```json +{ +"limit": number, /* if not set, all records are returned */ +"timeline":[{ + "uuid":"be700e84336941ef1bcd08d676310b75b9070f43", + "timestamp":1317094090, + "comment":"Added /json/timeline/ci showFiles to ajax test page.", + "user":"stephan", + "isLeaf":true, + "bgColor":null, /* not quite sure why this is null? */ + "type":"ci", + "parents": ["primary parent hash", "...other parent hashes"], + "tags":["json"], + "files":[{ + "name":"ajax/index.html", + "uuid":"9f00773a94cea6191dc3289aa24c0811b6d0d8fe", + "parent":"50e337c33c27529e08a7037a8679fb84b976ad0b", + "state":"modified" + }] + },...] +} +``` + +(Achtung: the `parents` property was called `prevUuid` prior to 20120316.) + +The `parents` property lists the checkins which were parents of this +commit. The first entry in the array is the "primary parent" - the one +which was not involved in a merge with the child. + +**Request options:** + +- `files=bool` toggles the addition of a "files" array property which + contains objects describing the files changed by the commit, + including their hash, previous hash, and state change type + (modified, added, or removed). ([“uuid” here means hash][uvh])\ + CLI mode: `--show-files|-f` +- `tag|branch=string` selects only entries with the given tag or "close + to" the given branch. Only one of these may be specified and if both + are specified, which one takes precedence is unspecified. If the + given tag/branch does not exist, an error response is generated. The + difference between the two is subtle - tag filters only on the given + tag (analog to the HTML interface's "r" option) whereas branch can + also return entries from other branches which were merged into the + requested branch (analog to the HTML interface's "b" option). If one + of these is specified, the response payload will contain a "tag" + *or* "branch" property with the tag/branch name given by the client. + + +# Technote (formerly Event) Timeline + +**Status:** implemented 20180803 + +**Required privileges:** "j" + +**Request:** + +- `/json/timeline/technote` +- DEPRECATED: `/json/timeline/event` (technotes were formerly called `events`) + +**Response payload example:** + +```json +{ +"limit": number, /* if not set, all records are returned */ +"timeline":[{ + "name":"8d18bf27e9f9ff8b9017edd55afc35701407d418", + "uuid":"b23962c88c123924a77fd663e91af094780d920a", + "timestamp":1478376113, + "comment":"Style update due to [8d880f0bb4]", + "user":"andygoth", + "eventType":"e" + },...] +} +``` + +The `uuid` of each entry can be passed to `/json/artifact` to fetch the raw +event content. + + +# Ticket Timeline + +**Status:** implemented 201109xx + +**Required privileges:** "r" or "o" + +**Request:** `/json/timeline/ticket` + +**Response payload example:** + +```json +{ + "limit": number, /* if not set, all records are returned */ + "timeline":[{ + "uuid":"5065a5da060e181da49a618f8ae5dc245215e95b", + "timestamp":1316511322, + "user":"stephan", + "eventType":"t", + "comment":"Ticket [b64435dba9] <i>How to...</i>", + "briefComment":"Ticket [b64435dba9]: 2 changes", + "ticketUuid":"b64435dba9cceb709bd54fbc5883884d73f93491" + },...] +} +``` + +**Notice that there are two [hashes][uvh] for tickets** - `uuid` is the change +hash and `ticketUuid` is the actual ticket’s hash. This is an unfortunate +discrepancy vis-a-vis the other timeline entries, which only have one +hash. We may want to swap `uuid` to mean the ticket hash and change `uuid` +to `commitHash`. + + +# Wiki Timeline + +**Status:** implemented 201109xx + +**Required privileges:** "j" or "o" + +**Requests:** + +- `/json/timeline/wiki` +- `/json/wiki/timeline` (alias) + +**Response payload example:** + +```json +{ +"limit": number, /* if not set, all records are returned */ +"timeline":[{ + "uuid":"4b2026f06eb48eaf187209fcb05ba5438c3b0ef0", + "timestamp":1331351121, + "comment":"Changes to wiki page [Page3]", + "user":"stephan", + "eventType":"w" + },...] +} +``` + +The `uuid` of each entry can be passed to `/json/artifact` or +`/json/wiki/get?uuid=...` to fetch the raw page and the hash of the +parent version. + +[uvh]: ../hashes.md#uvh diff --git a/www/json-api/api-user.md b/www/json-api/api-user.md new file mode 100644 index 0000000..c4f055f --- /dev/null +++ b/www/json-api/api-user.md @@ -0,0 +1,142 @@ +# JSON API: /user +([⬑JSON API Index](index.md)) + +Jump to: + +* [Get User Info](#get) +* [List Users](#list) +* [Save User](#save) + +--- + + +# Get User Info + +**Status:** implemented 20110927. + +**Required privileges:** "a" or "s" + +**Request:** + +- POST `/json/user/get`\ + with `POST.payload.name=USERNAME` +- `/json/user/get?name=USERNAME` + +**Response payload example:** + +```json +{ + "uid":1, + "name":"stephan", + "capabilities":"abcdefhgijkmnopqrstuvwxz", + "info":"https://wanderinghorse.net/home/stephan/", + "timestamp":1316122562 +} +``` + +(What does that timestamp field represent, anyway?) + + +# List Users + +**Status:** implemented 20110927. + +**Required privileges:** "a" or "s" + +**Request:** `/json/user/list` + +**Response payload example:** + +```json +[ + { + "uid":1, + "name":"stephan", + "capabilities":"abcdefhgijkmnoprstuvwxz", + "info":"", + "timestamp":1316122562 + }, + ... more users... +] +``` + + + +# Save User + +Only admin/setup users may modify accounts other than their own. + +**Status:** implemented 20111021 *but* it is missing "login group" +support, so changes do not yet propagate to other repos within a group. + +**Required privileges:** 'p' or 'a' or 's', depending on the context. + +**Request:** `/json/user/save` + +All request options must come from the `POST.payload` and/or GET/CLI +parameters (exception: "name" must come from POST.payload or CLI). +GET/CLI parameters take precedence over those in `POST.payload`, the +intention being to use an input file as a template and overriding the +template's defaults via the CLI. The options include: + +- `name=string` Specifies the user name to change. When changing a + user's name, the current uid and the new name must be specified.\ + **Achtung:** due to fossil-internal ambiguity in the handling of the + "name" parameter, this parameter must come from the `POST.payload` + data or it will not be recognized. In CLI mode it may be specified + with the `--name` flag. +- `uid=int` Specifies the uid to change. At least one of uid or name are + required. A uid of -1 means to create a new user, in which case the + name must be provided. +- `password=string` Optionally changes the user's password. When + renaming existing or creating new users, be sure to always provide a + new password because any old password hash is invalidated by the + name change. +- `info=string` Optionally changes the user's info field. +- `capabilities=string` Optionally changes the user's capabilities + field. +- `forceLogout=bool` (=false, or true when renaming) Optionally clears + any current login info for the current user, which will invalidate + any active session. Requires 'a' or 's' privileges. Intended to be + used when disabling a user account, to ensure that any open session + is invalidated. When a user is renamed this option is implied (and + cannot be disabled) because renaming invalidates any currently + stored auth token (because the old name is part of the hash + equation). + +Fields which are not provided in the request will not be modified. +Non-admin/setup users cannot edit other users and may only change their +own data if they have the 'p' (password) privilege. + +As of 20120217, users who do not have the setup privilege may neither +change the setup privilege for any user nor edit another user who has +that privilege. That is, only users with setup access may propagate or +remove setup status and accounts with the setup privilege may only be +edited by themselves and other setup users. + +**Response payload:** Same as user/get, using the new/saved state of the +modified user. + +Example usage from the command line: + +```console +$ fossil json user save --name drh --password sqlite3 \ + --capabilities "as" --info "DRH" +$ fossil json user save --uid 1 --name richard \ + --password fossil \ + --info "Previously known as drh" +``` + +**Warnings:** + +- When creating a new user or renaming a user, if no (new) password is + specified in the save request then the user will not be able to log + in because the previous password (for existing users) is hashed + against the old name. +- Renaming a user invalidates any active login token because his old + name is a part of the hash. i.e. the user must log back in with the + new name after being renamed. + +**TODOs:** + +- Login group support. diff --git a/www/json-api/api-version.md b/www/json-api/api-version.md new file mode 100644 index 0000000..ab321b6 --- /dev/null +++ b/www/json-api/api-version.md @@ -0,0 +1,32 @@ +# JSON API: /version +([⬑JSON API Index](index.md)) + +# Version (a.k.a. HAI) + +**Status:** implemented + +**Required privileges:** none + +**Requests:** + +- `/json/version` +- `/json/HAI` (alias borrowed from LOLCATZ jargon) + +**Response payload example:** + +```json +{ +"manifestUuid":"20ff808f9809541d2eca6c49a17d5cbd16e1b93f", +"manifestVersion":"[20ff808f98]", +"manifestDate":"2011-09-09 16:49:23", +"manifestYear":"2011", +"releaseVersion":"1.19", +"releaseVersionNumber":119, +"jsonApiVersion": "YYYYMMDD" // added 20120409 +} +``` + +Those particular payload fields were chosen only because they're defined +in `VERSION.h`. We may want to add other information, but nothing comes to +mind at this time. + diff --git a/www/json-api/api-wiki.md b/www/json-api/api-wiki.md new file mode 100644 index 0000000..7e1899f --- /dev/null +++ b/www/json-api/api-wiki.md @@ -0,0 +1,277 @@ +# JSON API: /wiki +([⬑JSON API Index](index.md)) + +Jump to: + +* [Page List](#list) +* [Fetch a Page](#get) +* [Create or Save Page](#create-save) +* [Wiki Diffs](#diffs) +* [Preview](#preview) +* [Notes and TODOs](#todo) + +--- + + +# Page List + +Returns a list of all pages, not including their content (which can be +arbitrarily large). + +**Status:** implemented 201109xx + +**Required privileges:** "j" or "o" + +**Request:** `/json/wiki/list` + +**Options:** + +- `bool verbose` (=false) Changes the results to return much more + information. Added 20120219. +- `glob=wildcard` string (default=`*`). If set, only page names + matching the given wildcard are returned. Added 20120325.\ + CLI: `--glob|-g STRING` +- `like=SQL LIKE string` (default=`%`). If set, only page names matching + the given SQL LIKE string are returned. Note that this match is + case-INsensitive. If both glob and like are given then only one will + work and which one takes precedence is unspecified. Added 20120325.\ + CLI: `--like|-l STRING` +- `invert=bool` (default=false). If set to a true value, the glob/like + filter has a reverse meaning (pages *not* matching the wildcard are + returned). Added 20120329.\ + CLI: `-i/--invert` + +**Response payload: format depends on "verbose" option** + +Non-verbose mode: + +```json +["PageName1",..."PageNameN"] +``` + +In verbose mode: + +```json +[{ +"name":"Apache On Windows XP", +"uuid":"a7e68df71b95d35321b9d9aeec3c8068f991926c", +"user":"jeffrimko", +"timestamp":1310227825, +"size":793 /* in bytes */ +},...] +``` + +The verbose-mode output is the same as the [`/json/wiki/get`](#get) output, but +without the content. The size property of each reflects the *byte* +length of the raw (non-HTMLized) page content. + +**Potential TODOs:** + +- Allow specifying (in the request) a list/array of names, as opposed + to listing all pages. The page count is rarely very high, though, so + an "overload" is very unlikely. (i have one wiki with about 47 pages + in it.) + + +# Fetch a Page + +Fetches a single wiki page, including content and significant metadata. + +**Status:** implemented 20110922, but response format may change. + +**Required privileges:** "j" or "o" + +**Request:** + +- GET: `/json/wiki/get?name=PageName` +- GET: `/json/wiki/get/PageName` +- POST: `/json/wiki/get` with page name as `POST.payload` or + `POST.payload.name`. + +**Response payload example:** + +```json +{ +"name": "Fossil", +"uuid": "...hex string...", +"parent": "uuid of parent (not set for first version of page)", +"user": "anonymous", +"timestamp": 1286143975, +"size": 1906, /* In bytes, not UTF8 characters! + Affected by format option! */ +"content": "..." +} +``` + +**FIXME:** it's missing the mimetype (that support was added to fossil +after this was implemented). + +If given no page to load, or if asked to get a page which does not +exist, an error response is generated (a usage- or resource-not-found +error, respectively). + +**Options (via CLI/GET/`POST.payload`):** + +- `name=string`. The page to fetch. The latest version is fetched +unless the uuid paramter is supplied (in which case name is ignored). \ +CLI: `--name|-n string`\ +Optionally, the name may be the 4th positional argument/request path element. +- `uuid=string`. Fetches a specific version. The name parameter is +ignored when this is specified.\ +CLI: `--uid|-u string` +- `format=string ("raw"|"html"|"none")` (default="raw"). Specifies the +format of the "content" payload value.\ +CLI: `--format|-f string` \ +The "none" format means to return no content. In that case the size +refers to the same size as the "raw" format. + +**TODOs:** + +- Support passing an array of names in the request (and change + response to return an array). + + +# Create or Save Page + +**Status:** implemented 20110922. + +**Required privileges:** "k" (save) or "f" (create) + +**Request:** + +- `/json/wiki/create` +- `/json/wiki/save` + +These work only in HTTP mode, not CLI mode. (FIXME: now that we can +simulate POST from a file, these could be used in CLI mode.) + +The semantic differences between save and create are: + +- Save will fail if the page doesn't already exist whereas create will + fail if it does. The createIfNotExists option (described below) can + be used to create new pages using the save operation. +- The content property is optional for the create request, whereas it + is required to be a string for save requests (but it *may* be an + empty string). This requirement for save is *almost* arbitrary, and + is intended to prevent accidental erasing of existing page content + via API misuse. + +**Response payload example:** + +The same as for [`/json/wiki/get`](#get) but the page content is not +included in the response (only the metadata). + +**Request options** (via GET or `POST.payload` object): + +- `name=string` specifies the page name. +- `content=string` is the body text.\ + Content is required for save (unless `createIfNotExists` is true *and* + the page does not exist), optional for create. It *may* be an empty + string. +- `mimetype=string` specifies the mimetype for the body, noting any any + unrecognized/unsupported mimetype is silently treated as + `text/x-fossil-wiki`. +- Save (not create) supports a `createIfNotExists` boolean option which + makes it a functional superset of the create/save features. i.e. it + will create if needed, else it will update. If createIfNotExists is + false (the default) then save will fail if given a page name which + does not refer to an existing page. +- **TODO:** add `commitMessage` string property. The fossil internals + don't have a way to do this at the moment (they can as of late 2019). + Since fossil wiki commits have always had the same default commit message, this is not a + high-priority addition. See:\ + [](/doc/trunk/www/fileformat.wiki#wikichng) +- **Potential TODO:** we *could* optionally also support + multi-page saving using an array of pages in the request payload:\ + `[… page objects … ]` + + + + +# Wiki Diffs + +**Status:** implemented 20120304 + +**Required privileges:** "h" + +**Request:** + +- `/json/wiki/diff[/version1_UUID/version2_UUID]` + +**Response payload example:** + +```json +{ + "v1":"e32ccdcda59e930c77c1e01cebace5d71253f621", + "v2":"e15992f475760cdf3a9564d8f88cacb659ab4b07", + "diff":"@@ -1,4 +1,9 @@......" +} +``` + +**Options:** + +- `v1=uuid` and `v2=uuid` specify the two versions to diff, and are + required parameters. They may optionally be specified as the two + URL/CLI parameters following the "diff" sub-command/path. + +This command does not verify that both UUIDs actually refer to the same +page name, but do verify that they refer to wiki content. + +Trivia: passing the same UUIDs to the `/json/diff` command will produce +very different results - that one diffs the manifests of the commits. + +**TODOs:** + +- Add options for changing the format of the diff, e.g. side-by-side + and enabling the HTML markup supported by the main fossil HTML GUI. +- Potentially do a name comparison to verify that the diff is against + the same page. That said, when "renaming" pages it might be useful + to diff two different pages. + + +# Preview + +**Status:** implemented 20120310 + +**Required privileges:** "k" (to limit its use to only those who can +edit wiki pages). This limitation is up for debate/reconsideration. + +**Request:** + +- `/json/wiki/preview` + +This command wiki-processes arbitrary text sent from the client. To help +curb potential abuse, its use is restricted to those with "k" access +rights. + +The `POST.payload` property must be either: + +1) A string containing Fossil wiki markup. + +2) An Object with a `body` property holding the text to render and a + `mimetype` property describing the wiki format: + `text/x-fossil-wiki` (the default), `text/x-markdown`, or + `text/plain`. Any unknown type is treated as `text/x-fossil-wiki`. + +The response payload is a string containing the rendered page. Whether +or not "all HTML" is allowed depends on site-level configuration +options, and that changes how the input is processed. + +Note that the links in the generated page are for the HTML interface, +and will not work as-is for arbitrary JSON clients. In order to +integrate the parsed content with JSON-based clients the HTML will +probably need to be post-processed, e.g. using jQuery to fish out the +links and re-map wiki page links to a JSON-capable page handler. + + + +# Notes and TODOs + +- When server-parsing the wiki content, the generated + intra-wiki/intra-site links will only be useful in the context of + the original fossil UI (or a work-alike), not arbitrary JSON + client apps. + +Potential TODOs: + +- `/wiki/history` analog to the [](/whistory) page. diff --git a/www/json-api/conventions.md b/www/json-api/conventions.md new file mode 100644 index 0000000..f3b470c --- /dev/null +++ b/www/json-api/conventions.md @@ -0,0 +1,649 @@ +# JSON API: General Conventions +([⬑JSON API Index](index.md)) + +Jump to: + +* [JSON Property Naming](#property-names) +* [HTTP GET Requests](#http-get) +* [HTTP POST Requests](#http-post) + * [POST Request Envelope](#request-envelope) +* [Request Parameter Data Types](#request-param-types) +* [Response Envelope](#response-envelope) +* [HTTP Response Headers](#http-response-header) +* [CLI vs. HTTP Mode](#cli-vs-http) +* [Simulating POSTed data](#simulating-post-data) +* [Indentation/Formatting of JSON Output](#json-indentation) +* [JSONP](#jsonp) +* [API Result Codes](#result-codes) + +--- + + +# JSON Property Naming + +Since most JSON usage conventionally happens in JavaScript +environments, this API has (without an explicit decision ever having +been made) adopted the ubiquitous JavaScript convention of +`camelCaseWithStartingLower` for naming properties in JSON objects. + + +# HTTP GET Requests + +Many (if not most) requests can be made via simple GET requests, e.g. we +*could* use any of the following patterns for a hypothetical JSON-format +timeline: + +- `https://..../timeline/json` +- `/timeline?format=json` +- `/timeline?json=1` +- `/timeline.json` +- `/json/timeline?...options...` + +The API settled on the `/json/...` convention, primarily because it +simplifies dispatching and argument-handling logic compared to the +`/[.../]foo.json` approach. Using `/json/...` allows us to unify that +logic for all JSON sub-commands, for both CLI and HTTP modes. + + +# HTTP Post Requests + +Certain requests, mainly things like editing checkin messages and +committing new files entirely, require POST data. This is fundamentally +very simple to do - clients post plain/unencoded JSON using a common +wrapper envelope which contains the request-specific data to submit as +well as some request-independent information (like authentication data). + + +## POST Request Envelope + +POST requests are sent to the same URL as their GET counterpart (if any, +else their own path), and are sent as plain-text/unencoded JSON wrapped +in a common request envelope with the following properties: + +- `requestId`: Optional arbitrary JSON value, not used by fossil, but + returned as-is in responses. +- `command`: Provides a secondary mechanism for specifying which JSON + command should be run. A request path of /json/foo/bar is equivalent + to a request with path=/json and command=foo/bar. Note that subpaths + do not work this way. e.g. path=/json/foo, command=bar will not + work, but path=/json, command=foo/bar will. This option is + particularly useful when generating JSON for piping in to CLI mode, + but it also has some response-dispatching uses on the client side. +- `authToken`: Authentication token. Created by a login + request. Determines what access rights the user has, and any given + request may require specific rights. In principle this is required + by any request which needs non-guest privileges, but cookie-aware + clients do not manually need to track this (it is managed as a + cookie by the agent/browser). + - Note that when accessing fossil over a local server instance + which was started with the `--localauth` flag, the `authToken` + will be ignored and need not be sent with any requests. The user + will automatically be given full privileges, as if they were + using CLI mode. +- `payload`: Command-specific parameters. Most can optionally come in + via GET parameters, but those taking complex structures expect them + to be placed here. +- `indent`: Optionally specifies indentation for the output. 0=no + indention. 1=a single TAB character for each level of + indentation. >1 means that many spaces per level. e.g. indent=7 + means to indent 7 spaces per object/array depth level. cson also + supports other flags for fine-tuning the output spacing, and adding + them to this interface might be interesting at some + point. e.g. whether or not to add a newline to the output. CLI mode + adds extra indentation by default, whereas CGI/server modes produce + unindented output by default. +- `jsonp`: Optional String (client function name). Requests which + include this will be returned with `Content-Type + application/javascript` and will be wrapped up in a function call + using the given name. e.g. if `jsonp=foo` then the result would look like:\ +`foo( {...the response envelope...} )` + +The API allows most of those (normally all but the payload) to come in +as either GET parameters or properties of the top-level POSTed request +JSON envelope, with GET taking priority over POST. (Reminder to self: we +could potentially also use values from cookies. Fossil currently only +uses 1 cookie (the login token), and I'd prefer to keep it that way.) + +POST requests without such an envelope will be rejected, generating a +Fossil/JSON error response (as opposed to an HTTP error response). GET +requests, by definition, never have an envelope. + +POSTed client requests *must* send a Content-Type header of either +`application/json` , `application/javascript`, or `text/plain`, or the +JSON-handling code will never see the POST data. The POST handler +optimistically assumes that type `text/plain` "might be JSON", since +`application/json` is a newer convention which many existing clients +do not use (as of the time these docs were written, back in 2011). + +## POST Envelope vs. `POST.payload` + + +When this document refers to POST data, it is referring to top-level +object in JSON-format POSTed input data. When we say `POST.payload` we +refer to the "payload" property of the POST data. While fossil's core +handles *form-urlencoded* POST data, if such data is sent in then +parsing it as JSON cannot succeed, which means that (at worst) the +JSON-mode bits will "not see" any POST data. Data POSTed to the JSON API +must be sent non-form-urlencoded (i.e. as plain text). + +Framework-level configuration options are always set via the top-level +POST envelope object or GET parameters. Request-specific options are set +either in `POST.payload` or GET parameters (though the former is required +in some cases). Here is an example which demonstrates the possibly +not-so-obvious difference between the two types of options (framework +vs. request-specific): + +```json +{ +"requestId":"my request", // standard envelope property (optional) +"command": "timeline/wiki", // also standard +"indent":2, // output indention is a framework-level option +"payload":{ // *anything* in the payload is request-specific + "limit":1 +} +} +``` + +When a given parameter is set in two places, e.g. GET and POST, or +POST-from-a-file and CLI parameters, which one takes precedence +depends on the concrete command handler (and may be unspecified). Most +will give precedence to CLI and GET parameters, but POSTed values are +technically preferred for non-string data because no additional "type +guessing" or string-to-whatever conversion has to be made (GET/CLI +parameters are *always* strings, even if they look like a number or +boolean). + + + +# Request Parameter Data Types + +When parameters are sent in the form of GET or CLI arguments, they are +inherently strings. When they come in from JSON they keep their full +type (boolean, number, etc.). All parameters in this API specify what +*type* (or types) they must (or may) be. For strings, there is no +internal conversion/interpretation needed for GET- or CLI-provided +parameters, but for other types we sometimes have to convert strings to +other atomic types. This section describes how those string-to-whatever +conversions behave. + +No higher-level constructs, e.g. JSON **arrays** or **objects**, are +accepted in string form. Such parameters must be set in the POST +envelope or payload, as specified by the specific API. + +This API does not currently use any **floating-point** parameters, but +does return floating-point results in a couple of places. + +For **integer** parameters we use a conventional string-to-int algorithm +and assume base 10 (analog to atoi(3)). The API may err on the side of +usability when given technically invalid values. e.g. "123abc" will +likely be interpreted as the integer 123. No APIs currently rely on +integer parameters with more than 32 bits (signedness is call-dependent +but few, if any, use negative values). + +**Boolean** parameters are a bit schizophrenic... + +In **CLI mode**, boolean flags do not have a value, per se, and thus +require no string-to-bool conversion. e.g. +`fossil foo -aBoolOpt -non-bool-opt value`. + +Those which arrive as strings via **GET parameters** treat any of the +following as true: a string starting with a character in the set +`[1-9tT]`. All other string values are considered to be false for this +purpose. + +Those which are part of the **POST data** are normally (but not always - +it depends on the exact context) evaluated as the equivalent of +JavaScript booleans. e.g. if we have `POST.envelope.foo="f"`, and evaluate +it as a JSON boolean (as opposed to a string-to-bool conversion), the +result will be true because the underlying JSON API follows JavaScript +semantics for any-type-to-bool conversions. As long as clients always +send "proper" booleans in their POST data, the difference between +GET/CLI-provided booleans should never concern them. + +TODO: consider changing the GET-value-to-bool semantics to match the JS +semantics, for consistency (within the JSON API at least, but that might +cause inconsistencies vis-a-vis the HTML interface). + + +# Response Envelope + +Every response comes in the form of an HTTP response or (in CLI mode) +JSON sent to stdout. The body of the response is a JSON object following +a common envelope format. The envelope has the following properties: + + +- `fossil`: Fossil server version string. This property is basically + "the official response envelope marker" - if it is set, clients can + "probably safely assume" that the object indeed came from one of the + Fossil/JSON APIs. This API never creates responses which do not + contain this property. +- `requestId`: Only set if the request contained it, and then it is + echoed back to the caller as-is. This can be used to determine + (client-side) which request a given response is coming in for + (assuming multiple asynchronous requests are pending). In practice + this generally isn’t needed because response handling tends to be + done by closures associated with the original request object (at + least in JavaScript code). In languages without closures it might + have some use. It may be any legal JSON value - it need not be + confined to a string or number. +- `resultCode`: Standardized result code string in the form + `FOSSIL-####`. Only error responses contain a `resultCode`. +- `resultText`: Possibly a descriptive string, possibly + empty. Supplements the resultCode, but can also be set on success + responses (but normally isn't). Clients must not rely on any + specific values being set here. +- `payload`: Request-specific response payload (data type/structure is + request-specific). The payload is never set for error responses, + only for success responses (and only those which actually have a + payload - not all do). +- `timestamp`: Response timestamp (GMT Unix Epoch). We use seconds + precision because I did not know at the time that Fossil actually + records millisecond precision. +- `payloadVersion`: Not initially needed, but reserved for future use + in maintaining version compatibility when the format of a given + response type's payload changes. If needed, the "first version" + value is assumed to be 0, for semantic [near-]compatibility with the + undefined value clients see when this property is not set. +- `command`: Normalized form of the command being run. It consists of + the "command" (non-argument) parts of the request path (or CLI + positional arguments), excluding the initial "/json/" part. e.g. the + "command" part of "/json/timeline/checkin?a=b" (CLI: json timeline + checkin...) is "timeline/checkin" (both in CLI and HTTP modes). +- `apiVersion`: Not yet used, but reserved for a numeric value which + represents the JSON API's version (which can be used to determine if + it has a given feature or not). This will not be implemented until + it's needed. +- `warnings`: Reserved for future use as a standard place to put + non-fatal warnings in responses. Will be an array but the warning + structure/type is not yet specified. Intended primarily as a + debugging tool, and will "probably not" become part of the public + client interface. +- `g`: Fossil administrators (those with the "a" or "s" permissions) + may set the `debugFossilG` boolean request parameter (CLI: + `--json-debug-g`) to enable this property for any given response. It + contains a good deal of the server-side internal state at the time + the response was generated, which is often useful in debuggering + problems. Trivia: it is called "g" because that's the name of + fossil's internal global state object. +- `procTimeMs`: For debugging only - generic clients must not rely on + this property. Contains the number of milliseconds the JSON command + processor needed to dispatch and process the command. TODO: move the + timer into the fossil core so that we can generically time its + responses and include the startup overhead in the time calculation. + + + + +# HTTP Response Headers + +The Content-Type HTTP header of a response will be either +application/json, application/javascript, or text/plain, depending on +whether or not we are in JSONP mode or (failing that) the contents of +the "Accept" header sent in the request. The response type will be +text/plain if it cannot figure out what to do. The response's +Content-Type header *may* contain additional metadata, e.g. it might +look like: application/json; charset=utf-8 + +Apropos UTF-8: note that JSON is, by definition, Unicode and recommends +UTF-8 encoding (which is what we use). That means if your console cannot +handle UTF-8 then using this API in CLI mode might (depending on the +content) render garbage on your screen. + + + +# CLI vs. HTTP Mode + +CLI (command-line interface) and HTTP modes (CGI and standalone server) +are consolidated in the same implementations and behave essentially +identically, with only minor exceptions. + +An HTTP path of `/json/foo` translates to the CLI command `fossil json +foo`. CLI mode takes options in the fossil-convention forms (e.g. `--foo 3` +or `-f 3`) whereas HTTP mode takes them via GET/POST data (e.g. `?foo=1`). +(Note that per long-standing fossil convention CLI parameters taking a +value do not use an equal sign before the value!) + +For example: + +- HTTP: `/json/timeline/wiki?after=2011-09-01&limit=3` +- CLI: `fossil json timeline wiki --after 2011-09-01 --limit 3` + +Some commands may only work in one mode or the other (for various +reasons). In CLI mode the user automatically has full setup/admin +access. + +In HTTP mode, request-specific options can also be specified in the +`POST.payload` data, and doing so actually has an advantage over +specifying them as URL parameters: posting JSON data retains the full +type information of the values, whereas GET-style parameters are always +strings and must be explicitly type-checked/converted (which may produce +unpredictable results when given invalid input). That said, oftentimes +it is more convenient to pass the options via URL parameters, rather +than generate the request envelope and payload required by POST +requests, and the JSON API makes some extra effort to treat GET-style +parameters type-equivalent to their POST counterparts. If a property +appears in both GET and `POST.payload`, GET-style parameters *typically* +take precedence over `POST.payload` by long-standing convention (=="PHP +does it this way by default"). + +(That is, however, subject to eventual reversal because of the +stronger type safety provided by POSTed JSON. Philosophically +speaking, though, GET *should* take precedence, in the same way that +CLI-provided options conventionally override app-configuration-level +options.) + +One notable functional difference between CLI and HTTP modes is that in +CLI mode error responses *might* be accompanied by a non-0 exit status +(they "should" always be, but there might be cases where that does not +yet happen) whereas in HTTP mode we always try to exit with code 0 to +avoid generating an HTTP 500 ("internal server error"), which could keep +the JSON response from being delivered. The JSON code only intentionally +allows an HTTP 500 when there is a serious internal error like +allocation or assertion failure. HTTP clients are expected to catch +errors by evaluating the response object, not the HTTP result code. + + +# Simulating POSTed data + +We have a mechanism to feed request data to CLI mode via +files (simulating POSTed data), as demonstrated in this example: + +```console +$ cat in.json +{ "command": "timeline/wiki", "indent":2, "payload":{"limit":1}} +$ fossil json --json-input in.json # use filename - for stdin +``` + +The above is equivalent to: + +```console +$ echo '{"indent":2, "payload":{"limit":1}}' \ + | fossil json timeline wiki --json-input - +``` + +Note that the "command" JSON parameter is only checked when no json +subcommand is provided on the CLI or via the HTTP request path. Thus we +cannot pass the CLI args "json timeline" in conjunction with a "command" +string of "wiki" this way. + +***HOWEVER...*** + +Much of the existing JSON code was written before the `--json-input` +option was possible. Because of this, there might be some +"misinteractions" when providing request-specific options via *both* +CLI options and simulated POST data. Those cases will eventually be +ironed out (with CLI options taking precedence). Until then, when +"POSTing" data in CLI mode, for consistent/predictible results always +provide any options via the JSON request data, not CLI arguments. That +said, there "should not" be any blatant incompatibilities, but some +routines will prefer `POST.payload` over CLI/GET arguments, so there +are some minor inconsistencies across various commands with regards to +which source (POST/GET/CLI) takes precedence for a given option. The +precedence "should always be the same," but currently cannot be due to +core fossil implementation details (the internal consolidation of +GET/CLI/POST vars into a single set). + + + +# Indentation/Formatting of JSON Output + +CLI mode accepts the `--indent|-I #` option to set the indention level +and HTTP mode accepts `indent=#` as a GET/POST parameter. The semantics +of the indention level are derived from the underlying JSON library and +have the following meanings: 0 (zero) or less disables all superfluous +indentation (this is the default in HTTP mode). A value of 1 uses 1 hard +TAB character (ASCII 0x09) per level of indention (the default in CLI +mode). Values greater than 1 use that many whitespaces (ASCII 32d) per +level of indention. e.g. a value of 7 uses 7 spaces per level of +indention. There is no way to specify one whitespace per level, but if +you *really* want one whitespace instead of one tab (same data size) you +can filter the output to globally replace ASCII 9dec (TAB) with ASCII +32dec (space). Because JSON string values *never* contain hard tabs +(they are represented by `\t`) there is no chance that such a global +replacement will corrupt JSON string contents - only the formatting will +be affected. + +Potential TODO: because extraneous indention "could potentially" be used +as a form of DoS, the option *might* be subject to later removal from HTTP +mode (in CLI it's fine). + +In HTTP mode no trailing newline is added to the output, whereas in CLI +mode one is normally appended (exception: in JSONP mode no newline is +appended, to (rather pedantically and arbitraily) allow the client to +add a semicolon at the end if he likes). There is currently no option to +control the newline behaviour, but the underlying JSON code supports +this option, so adding it to this API is just a matter of adding the +CLI/HTTP args for it. + +Pedantic note: internally the indention level is stored as a single +byte, so giving large indention values will cause harmless numeric +overflow (with only cosmetic effects), meaning, e.g., 257 will overflow +to the value 1. + +Potential TODO: consider changing cson's indention mechanism to use a +*signed* number, using negative values for tabs and positive for +whitespace count (or the other way around). This would require more +doc changes than code changes :/. + + + +# JSONP + +The API supports JSONP-style output. The caller specifies the callback +name and the JSON response will be wrapped in a function call to that +name. For HTTP mode pass the `jsonp=string` option (via GET or POST +envelope) and for CLI use `--jsonp string`. + +For example, if we pass the JSONP name `myCallback` then a response will +look like: + +```js +myCallback({...response...}) +``` + +Note that fossil does not evaluate the callback name itself, other than +to verify that it is-a string, so "garbage in, garbage out," and all +that. (Remember that CLI and GET parameters are *always* strings, even +if they *look* like numbers.) + + + +# API Result Codes + +Result codes are strings which tell the client whether or not a given +API call succeeded or failed, and if it failed *perhaps* some hint as to +why it failed. + +The result code is available via the resultCode property of every +*error* response envelope. Since having a result code value for success +responses is somewhat redundant, success responses contain no resultCode +property. In practice this simplifies error checking on the client side. + +The codes are strings in the form `FOSSIL-####`, where `####` is a +4-digit integral number, left-padded with zeros. The numbers follow +these conventions: + +- The number 0000 is reserved for the "not an error" (OK) case. Since + success responses do not contain a result code, clients won't see + this value (except in documentation). +- All numbers with a leading 0 are reserved for *potential* future use + in reporting non-fatal warnings. +- Despite *possibly* having leading zeros, the numbers are decimal, + not octal. Script code which uses eval() or similar to produce + integers from them may need to take that into account. +- The 1000ths and 100ths places of the number describe the general + category of the error, e.g. authentication- vs. database- vs. usage + errors. The 100ths place is more specific than the 1000ths place, + allowing two levels of sub-categorization (which "should be enough" + for this purpose). This separation allows the server administrator + to configure the level of granularity of error reporting. e.g. some + admins consider error messages to be security-relevant and like to + "dumb them down" on their way to the client, whereas developers + normally want to see very specific error codes when tracking down a + problem. We can offer a configuration option to "dumb down" error + codes to their generic category by simply doing a modulo 100 + (or 1000) against the native error code number. e.g. FOSSIL-1271 + could (via a simple modulo) be reduced to FOSSIL-1200 or + FOSSIL-1000, depending on the paranoia level of the sysadmin. I have + tried to order the result code numbers so that a dumb-down level of + 2 provides reasonably usable results without giving away too much + detail to malicious clients.\ + (**TODO:** `g.json.errorDetailParanoia` is used to set the + default dumb-down level, but it is currently set at compile-time. + It needs to be moved to a config option. We have a chicken/egg scenario + with error reporting and db access there (where the config is + stored).) +- Once a number is assigned to a given error condition (and actually + used somewhere), it may not be changed/redefined. JSON clients need + to be able to rely on stable result codes in order to provide + adequate error reporting to their clients, and possibly for some + error recovery logic as well (i.e. to decide whether to abort or + retry). + +The *tentative* list of result codes is shown in the following table. +These numbers/ranges are "nearly arbitrarily" chosen except for the +"special" value 0000. + +**Maintenance reminder:** these codes are defined in +[`src/json_detail.h`](/finfo/src/json_detail.h) (enum +`FossilJsonCodes`) and assigned default `resultText` values in +[`src/json.c:json_err_cstr()`](/finfo/src/json.c). Changes there need +to be reflected here (and vice versa). Also, we have assertions in +place to ensure that C-side codes are in the range 1000-9999, so do +not just go blindly change the numeric ranges used by the enum. + + +**`FOSSIL-0###`: Non-error Category** + +- `FOSSIL-0000`: Success/not an error. Succesful responses do not + contain a resultCode, so clients should never see this. +- `FOSSIL-0###`: Reserved for potential future use in reporting + non-fatal warnings. + + + +**`FOSSIL-1000`: Generic Errors Category** + +- `FOSSIL-1101`: Invalid request. Request envelope is invalid or + missing. +- `FOSSIL-1102`: Unknown JSON command. +- `FOSSIL-1103`: Unknown/unspecified error +- `FOSSIL-1104`: RE-USE +- `FOSSIL-1105`: A server-side timeout was reached. (i’m not sure we + can actually implement this one, though.) +- `FOSSIL-1106`: Assertion failed (or would have had we + continued). Note: if an `assert()` fails in CGI/server modes, the HTTP + response will be code 500 (Internal Server Error). We want to avoid + that and return a JSON response instead. All of that said - there seems + to be little reason to implement this, since assertions are "truly + serious" errors. +- `FOSSIL-1107`: Allocation/out of memory error. This cannot be reasonably + reported because fossil aborts if an allocation fails. +- `FOSSIL-1108`: Requested API is not yet implemented. +- `FOSSIL-1109`: Panic! Fossil's `fossil_panic()` or `cgi_panic()` was + called. In non-JSON HTML mode this produces an HTTP 500 + error. Clients "should" report this as a potential bug, as it + "possibly" indicates that the C code has incorrect argument- or + error handling somewhere. +- `FOSSIL-1110`: Reading of artifact manifest failed. Time to contact + your local fossil guru. +- `FOSSIL-1111`: Opening of file failed (e.g. POST data provided to + CLI mode). + + +**`FOSSIL-2000`: Authentication/Access Error Category** + +- `FOSSIL-2001`: Privileged request was missing authentication + token/cookie. +- `FOSSIL-2002`: Access to requested resource was denied. Oftentimes + the `resultText` property will contain a human-language description of + the access rights needed for the given command. +- `FOSSIL-2003`: Requested command is not available in the current + operating mode. Returned in CLI mode by commands which require HTTP + mode (e.g. login), and vice versa. FIXME: now that we can simulate + POST in CLI mode, we can get rid of this distinction for some of the + commands. +- `FOSSIL-2100`: Login Failed. +- `FOSSIL-2101`: Anonymous login attempt is missing the + "anonymousSeed" property (fetched via [the `/json/anonymousPassword` + request](api-auth.md#login-anonymous)). Note that this is more + specific form of `FOSSIL-3002`. + + +ONLY FOR TESTING purposes should the remaning 210X sub-codes be +enabled (they are potentially security-relevant, in that the client +knows which part of the request was valid/invalid): + +- `FOSSIL-2102`: Name not supplied in login request +- `FOSSIL-2103`: Password not supplied in login request +- `FOSSIL-2104`: No name/password match found + + +**`FOSSIL-3000`: Usage Error Category** + +- `FOSSIL-3001`: Invalid argument/parameter type(s) or value(s) in + request +- `FOSSIL-3002`: Required argument(s)/parameter(s) missing from + request +- `FOSSIL-3003`: Requested resource identifier is ambiguous (e.g. a + shortened hash that matches multiple artifacts, an abbreviated + date that matches multiple commits, etc.) +- `FOSSIL-3004`: Unresolved resource identifier. A branch/tag/uuid + provided by client code could not be resolved. This is a special + case of #3006. +- `FOSSIL-3005`: Resource already exists and overwriting/replacing is + not allowed. e.g. trying to create a wiki page or user which already + exists. FIXME? Consolidate this and resource-not-found into a + separate category for dumb-down purposes? +- `FOSSIL-3006`: Requested resource not found. e.g artifact ID, branch + name, etc. + + +**`FOSSIL-4000`: Database-related Error Category** + +- `FOSSIL-4001`: Statement preparation failed. +- `FOSSIL-4002`: Parameter binding failed. +- `FOSSIL-4003`: Statement execution failed. +- `FOSSIL-4004`: Database locked (this is not used anywhere, but + reserved for future use). + +Special-case DB-related errors... + +- `FOSSIL-4101`: Fossil Schema out of date (repo rebuild required). +- `FOSSIL-4102`: Fossil repo db could not be found. +- `FOSSIL-4103`: Repository db is not valid (possibly corrupt). +- `FOSSIL-4104`: Check-out not found. This is similar to FOSSIL-4102 + but indicates that a local checkout is required (but was not + found). Note that the 4102 gets triggered earlier than this one, and + so can appear in cases when a user might otherwise expect a 4104 + error. + + +Some of those error codes are of course "too detailed" for the client to +do anything with (e.g.. 4001-4004), but their intention is to make it +easier for Fossil developers to (A) track down problems and (B) support +clients who report problems. If a client reports, "I get a FOSSIL-4000, +how can I fix it?" then the developers/support personnel can't say much +unless they know if it's a 4001, 4002, 4003, 4004, or 4101 (in which +case they can probably zero in on the problem fairly quickly, since they +know which API call triggered it and they know (from the error code) the +general source of the problem). + +## Why Standard/Immutable Result Codes are Important + +- They are easily internationalized (i.e. associated with non-English + error text) +- Clients may be able to add automatic retry strategies for certain + problem types by examining the result code. e.g. if fossil returns a + locking or timeout error \[it currently does no special + timeout/locking handling, by the way\] the client could re-try, + whereas a usage error cannot be sensibly retried with the same + inputs. +- The "category" structure described above allows us some degree of + flexibility in how detailed the reported errors are reported. +- While the string prefix "FOSSIL-" on the error codes may seem + superfluous, it has one minor *potential* advantage on the client + side: when managing several unrelated data sources, these error + codes can be immediately identified (by higher-level code which may + be ignorant of the data source) as having come from the fossil API. + Think "ORA-111" vs. "111". diff --git a/www/json-api/hacking.md b/www/json-api/hacking.md new file mode 100644 index 0000000..1de360a --- /dev/null +++ b/www/json-api/hacking.md @@ -0,0 +1,353 @@ +# JSON API: Hacker's Guide +([⬑JSON API Index](index.md)) + +Jump to: + +* [Before Committing Changes](#before-committing) +* [JSON C API](#json-c-api) +* [Reporting Errors](#reporting-errors) +* [Getting Command Arguments](#command-args) +* [Creating JSON Data](#creating-json) + * [Creating JSON Values](#creating-json-values) + * [Converting SQL Query Results to JSON](#query-to-json) + +This section will only be of interest to those wanting to work on the +Fossil/JSON code. That said... + +If you happen to hack on the code and find something worth noting here +for others, please feel free to expand this section. It will only +improve via feedback from those working on the code. + +--- + + +# Before Committing Changes... + +Because this code lives in the trunk, there are certain +guidelines which must be followed before committing any changes: + +1. Read the [checkin preparation list](/doc/trunk/www/checkin.wiki). +2. Changes to the files `src/json_*.*`, and its related support code + (e.g. `ajax/*.*`), may be made freely without affecting mainline + users. Changes to other files, unless they are trivial or made for + purposes outside the JSON API (e.g. an unrelated bug fix), must be + reviewed carefully before committing. When in doubt, create a branch + and post a request for a review. +3. The Golden Rule is: *do not break the trunk build*. + + + +# JSON C API + +libcson, the underlying JSON API, is a separate project, included in +fossil in "amalgamation" form: see `extsrc/cson_amalgamation.[ch]`. It has +thorough API docs and a good deal of information is in its wiki: + +[](https://fossil.wanderinghorse.net/wikis/cson/) + +In particular: + +[](https://fossil.wanderinghorse.net/wikis/cson/?page=CsonArchitecture) + +gives an overview of its architecture. Occasionally new versions of it +are pulled into the Fossil tree, but other developers generally need not +concern themselves with that. + +(Trivia: the cson wiki's back-end is fossil using this very JSON API, +living on top of a custom JavaScript+HTML5 application.) + +Only a small handful of low-level fossil routines actually input or +output JSON text (only for reading in POST data and sending the +response). In the C code we work with the higher-level JSON value +abstractions provided by cson (conceptually similar to an XML DOM). All +of the JSON-defined data types are supported, and we can construct JSON +output of near arbitrary complexity with the caveat that *cyclic data +structures are strictly forbidden*, and *will* cause memory corruption, +crashes, double free()'s, or other undefined behaviour. Because JSON +cannot, without client-specific semantic extensions to JSON, represent +cyclic structures, it is not anticipated that this will be a +problem/limitation when generating output for fossil. + + + + +# Architecture of JSON Commands + +In order to consolidate CLI/HTTP modes for JSON handling, this code +foregoes fossil's conventional command/path dispatching mechanism. Only +the top-most "json" command/path is dispatched directly by fossil's +core. The disadvantages of this are that we lose fossil's conventional +help text mechanism (which is based on code comments in the +command/path's dispatcher impl) and the ability to write abbreviated +command names in CLI mode ("json" itself may be abbreviated, but not the +subcommands). The advantages are that we can handle CLI/HTTP modes +almost identically (there are a couple of minor differences) by unifying +them under the same callback functions much more easily. + +The top-level "json" command/path uses its own dispatching mechanism +which uses either the path (in HTTP mode) or CLI positional arguments to +dispatch commands (stopping at the first "flag option" (e.g. -foo) in +CLI mode). The command handlers are simply callback functions which +return a cson\_value pointer (the C representation of an arbitrary JSON +value), representing the "payload" of the response (or NULL - not all +responses need a payload). On error these callbacks set the internal +JSON error state (detailed in a subsection below) and return NULL. The +top-level dispatcher then creates a response envelope and returns the +"payload" from the command (if any) to the caller. If a callback sets +the error state, the top-level dispatcher takes care to set the error +information in the response envelope. In summary: + +- The top-level dispatchers (`json_page_top()` and `json_cmd_top()`) + are called by fossil's core when the "json" command/path is called. + They initialize the JSON-mode global state, dispatch the requested + command, and handle the creation of the response envelope. They + prepare all the basic things which the individual subcommands need + in order to function. +- The command handlers (most are named `json_page_something()`) + implement the `fossil_json_f()` callback interface (see + [`src/json_detail.h`](/finfo/src/json_detail.h)). They are + responsible for permissions checking, setting any error state, and + passing back a payload (if needed - not all commands return a + payload). It is strictly forbidden for these callbacks to produce + any output on stdout/stderr, and doing so effectively corrupts the + out-bound JSON and HTTP headers. + +There is a wrench in all of that, however: the vast majority of fossil's +commands "fail fast" - they will `exit()` if they encounter an error. To +handle that, the fossil core error reporting routines have been +refactored a small bit to operate differently when we are running in +JSON mode. Instead of the conventional output, they generate a JSON +error response. In HTTP mode they exit with code 0 to avoid causing an +HTTP 500 error, whereas in CLI mode they will exit with a non-0 code. +Those routines still `exit()`, as in the conventional CLI/HTTP modes, but +they will exit differently. Because of this, it is perfectly fine for a +command handler to exit via one of fossil's conventional mechanisms +(e.g. `db_prepare()` can be fatal, and callbacks may call `fossil_panic()` +if they really want to). One exception is `fossil_exit()`, which does +_not_ generate any extra output and will `exit()` the app. In the JSON +API, as a rule of thumb, `fossil_exit()` is only used when we *want* a +failed request to cause an HTTP 500 error, and it is reserved for +allocation errors and similar truly catastrophic failures. That said... +libcson has been hacked to use `fossil_alloc()` and friends for memory +management, and those routines exit on error, so alloc error handling in +the JSON command handler code can afford to be a little lax (the +majority of *potential* errors clients get from the cson API have +allocation failure as their root cause). + +As a side-note: the vast majority (if not all) of the cson API calls are +"NULL-safe", meaning that will return an error code (or be a no-op) if +passed NULL arguments. e.g. the following chain of calls will not crash +if the value we're looking for does not exist, is-not-a String (see +`cson_value_get_string()` for important details), or if `myObj` is NULL: + +```c +const char * str = + cson_string_cstr( // get the C-string form of a cson_string + cson_value_get_string( // get its cson_string form + cson_object_get(myObj,"foo") // search for key in an Object + ) + ); +``` + +If `"foo"` is not found in `myObj` (or if `myObj` is NULL) then v will be +NULL, as opposed to stepping on a NULL pointer somewhere in that call +chain. + +Note that all cson JSON values except Arrays and Objects are *immutable* +- you cannot change a string's or number's value, for example. They also +use reference counting to manage ownership, as documented and +demonstrated on this page: + +[](https://fossil.wanderinghorse.net/wikis/cson/?page=TipsAndTricks) + +In short, after creating a new value you must eventually *either* add it +to a container (Object or Array) to transfer ownership *or* call +`cson_value_free()` to clean it up (exception: the Fossil/JSON command +callbacks *return* a value to transfer ownership to the dispatcher). +Sometimes it's more complex than that, but not normally. Any given value +may legally be stored in any number of containers (or multiple times +within one container), as long as *no cycles* are introduced (cycles +*will* cause undefined behaviour). Ownership is shared using reference +counting and the value will eventually be freed up when its last +remaining reference is freed (e.g. when the last container holding it is +cleaned up). For many examples of using cson in the context of fossil, +see the existing `json_page_xxx()` functions in `json_*.c`. + + +# Reporting Errors + +To report an error from a command callback, one abstractly needs to: + +- Set g.json.resultCode to one of the `FSL_JSON_E_xxx` values + (defined in [`src/json_detail.h`](/finfo/src/json_detail.h)). +- *Optionally* set `g.zErrMsg` to contain the (dynamically-allocated!) + error string to be sent to the client. If no error string is set + then a standard/generic string is used for the given error code. +- Clean up any resources created so far by the handler. +- Return NULL. If it returns non-NULL, the dispatcher will destroy the + value and not include it in the error response. + +That normally looks something like this: + +``` +if(!g.perm.Read){ + json_set_err(FSL_JSON_E_DENIED, "Requires 'o' permissions."); + return NULL; +} +``` + +`json_set_err()` is a variadic printf-like function, and can use the +printf extensions supported by mprintf() and friends (e.g. `%Q` and `%q`) +(but they are normally not needed in the context of JSON). If the error +string is NULL or empty then `json_err_cstr(errorCode)` is used to fetch +the standard/generic error string for the given code. + +When control returns to the top-level dispatching function it will check +`g.json.resultCode` and, if it is not 0, create an error response using +the `g.json.resultCode` and `g.zErrMsg` to construct the response's +`resultCode` and `resultText` properties. + +If a function wants to output an error and exit by itself, as opposed +to returning to the dispatcher, then it must behave slightly +differently. See the docs for `json_err()` (in +[`src/json.c`](/finfo/src/json.c)) for details, and search that file +for various examples of its usage. It is also used by fossil's core +error-reporting APIs, e.g. `fossil_panic()` (defined in [`src/main.c`](/finfo/src/main.c)). +That said, it would be "highly unusual" for a callback to need to do +this - it is *far* simpler (and more consistent/reliable) to set the +error state and return to the dispatcher. + + +# Getting Command Arguments + +Positional parameters can be fetched usinig `json_command_arg(N)`, where +N is the argument position, with position 0 being the "json" +command/path. In CLI mode positional arguments have their obvious +meaning. In HTTP mode the request path (or the "command" request +property) is used to build up the "command path" instead. For example: + +CLI: `fossil json a b c` + +HTTP: `/json/a/b/c` + +HTTP POST or CLI with `--json-input`: /json with POSTed envelope +`{"command": "a/b/c" …}` + +Those will have identical "command paths," and `json_command_path(2)` +would return the "b" part. + +Caveat: a limitation of this support is that all CLI flags must come +*after* all *non-flag* positional arguments (e.g. file names or +subcommand names). Any argument starting with a dash ("-") is considered +by this code to be a potential "flag" argument, and all arguments after +it are ignored (because the generic handling cannot know if a flag +requires an argument, which changes how the rest of the arguments need +to be interpreted). + +To get named parameters, there are several approaches (plus some special +cases). Named parameters can normally come from any of the following +sources: + +- CLI arguments, e.g. `--foo bar` +- GET parameters: `/json/...?foo=bar` +- Properties of the POST envelope +- Properties of the `POST.payload` object (if any). + +To try to simplify the guessing process the API has a number of +functions which behave ever so slightly differently. A summary: + +- `json_getenv()` and `json_getenv_TYPE()` search the so-called "JSON + environment," which is a superset of the GET/POST/`POST.payload` (if + `POST.payload` is-a Object). +- `json_find_option_TYPE()`: searches the CLI args (only when in CLI + mode) and the JSON environment. +- The use of fossil's `P()` and `PD()` macros is discouraged in JSON + callbacks because they can only handle String data from the CLI or + GET parameters (not POST/`POST.payload`). (Note that `P()` and `PD()` + *normally* also handle POSTed keys, but they only "see" values + posted as form-urlencoded fields, and not JSON format.) +- `find_option()` (from `src/main.c`) "should" also be avoided in + JSON API handlers because it removes flag from the g.argv + arguments list. That said, the JSON API does use `find_option()` in + several of its option-finding convenience wrappers. + +For example code: the existing command callbacks demonstrate all kinds +of uses and the various styles of parameter/option inspection. Check out +any of the functions named `json_page_SOMETHING()`. + + +# Creating JSON Data + + +## Creating JSON Values + +cson has a fairly rich API for creating and manipulating the various +JSON-defined value types. For a detailed overview and demonstration I +recommend reading: + +[](https://fossil.wanderinghorse.net/wikis/cson/?page=HowTo) + +That said, the Fossil/JSON API has several convenience wrappers to save +a few bytes of typing: + +- `json_new_string("foo")` is easier to use than + `cson_value_new_string("foo", 3)`, and + `json_new_string_f("%s","foo")` is more flexible. +- `json_new_int()` is easier to type than `cson_value_new_integer()`. +- `cson_output_Blob()` and `cson_parse_Blob()` can write/read JSON + to/from fossil `Blob`-type objects. + +It also provides several lower-level JSON features which aren't of +general utility but provide necessary functionality for some of the +framework-level code (e.g. `cson_data_dest_cgi()`), which is only used +by the deepest of the JSON internals). + + + +## Converting SQL Query Results to JSON + +The `cson_sqlite3_xxx()` family of functions convert `sqlite3_stmt` rows +to Arrays or Objects, or convert single columns to a JSON-compatible +form. See `json_stmt_to_array_of_obj()`, +`json_stmt_to_array_of_array()` (both in `src/json.c`), and +`cson_sqlite3_column_to_value()` and friends (in +`extsrc/cson_amalgamation.h`). They work in an intuitive way for numeric +types, but they optimistically/naively *assume* that any fields of type +TEXT or BLOB are actually UTF8 data, and treat them as such. cson's +string class only handles UTF8 data and it is semantically illegal to +feed them anything but UTF8. Violating this will likely result in +down-stream errors (e.g. when emitting the JSON string output). **The +moral of this story is:** *do not use these APIs to fetch binary data*. +JSON doesn't do binary and the `cson_string` class does not +protect itself against clients feeding it non-UTF8 data. + +Here's a basic example of using these features: + +```c +Stmt q = empty_Stmt; +cson_value * rows = NULL; +db_prepare(&q, "SELECT a AS a, b AS b, c AS c FROM foo"); +rows = json_stmt_to_array_of_obj( &sql, NULL ); +db_finalize(&q); +// side note: if db_prepare()/finalize() fail (==they exit()) +// then a JSON-format error reponse will be generated. +``` + +On success (and if there were results), `rows` is now an Array value, +each entry of which contains an Object containing the fields (key/value +pairs) of each row. `json_stmt_to_array_of_array()` returns each row +as an Array containing the column values (with no column name +information). + +**Note the seemingly superfluous use of the "AS" clause in the above +SQL.** Having them is actually significant! If a query does *not* use AS +clauses, the row names returned by the db driver *might* be different +than they appear in the query (this is documented behaviour of sqlite3). +Because the JSON API needs to return stable field names, we need to use +AS clauses to be guaranteed that the db driver will return the column +names we want. Note that the AS clause is often used to translate column +names into something more JSON-conventional or user-friendly, e.g. +"SELECT cap AS capabilities...". Alternately, we can convert the +individual `sqlite3_stmt` column values to JSON using +`cson_sqlite3_column_to_value()`, without referring directly to the +db-reported column name. diff --git a/www/json-api/index.md b/www/json-api/index.md new file mode 100644 index 0000000..757babe --- /dev/null +++ b/www/json-api/index.md @@ -0,0 +1,40 @@ +# JSON API Index + +This is the client-side documentation of Fossil's JSON API. The JSON +API aims to provide access to many of the primary fossil features via +AJAX-style interfaces. + +* [Introduction](intro.md) +* [General API Conventions](conventions.md) +* [Tips & Tricks](tips.md) +* [Hacking Guide](hacking.md) + +General warnings regarding the APIs linked to in the following list: + +- **NOTE** that request/response examples shown in the individual API +pages do not show [the standard request/response envelope](conventions.md) +(for brevity and sanity). +- **Achtung:** just because a given feature is described as being +implemented does not mean that the implementation is "final" - it may be +changed at any time until we find/implement useful APIs. + +The APIs, alphabetically by category: + +* [Artifact Info](api-artifact.md) +* [Authentication](api-auth.md) +* [Branches](api-branch.md) +* [Checkout Status](api-checkout.md) +* [Config](api-config.md) +* [Diffs](api-diff.md) +* [Directory Listing](api-dir.md) +* [File Info](api-finfo.md) +* [The Obligatory Misc. Category](api-misc.md) +* [Repository Stats](api-stat.md) +* [Settings](api-settings.md) +* [SQL Query](api-query.md) +* [Tags](api-tag.md) +* [Tickets](api-ticket.md) +* [Timeline](api-timeline.md) +* [User Management](api-user.md) +* [Version](api-version.md) +* [Wiki](api-wiki.md) diff --git a/www/json-api/intro.md b/www/json-api/intro.md new file mode 100644 index 0000000..d8a9f14 --- /dev/null +++ b/www/json-api/intro.md @@ -0,0 +1,172 @@ +# JSON API Introduction +([⬑JSON API Index](index.md)) + +Jump to: + +* [Why?](#why) +* [Building JSON Support](#builing) +* [Goals & Non-goals](#goals) +* [Potential Client-side Uses](#potential-uses) +* [Technical Problems and Considerations](#considerations) + +--- + + +# Why? + +In September, 2011, Fossil contributor Stephan Beal had the great +pleasure of meeting D. Richard Hipp, Fossil's author, for lunch in +Munich, Germany. During the conversation Richard asked, "what does +Fossil need next?" Stephan's first answer was, "refactoring into a +library/client, as opposed to a monolithic app." We very quickly +agreed that the effort required would be "herculean," and second +choice was voiced, "a JSON API." They briefly discussed the idea and +Richard gave his blessing. That night work began. + +Why a JSON API? Because it is the next best thing to the +"librification" of Fossil, in that it makes Fossil's features +available to near-arbitrary applications using a simple, globally +available data format. + + +# Building JSON Support + +In environments supported by fossil's `configure` script, +simply pass `--enable-json` to it: + +``` +$ ./configure --prefix=$HOME --enable-json ... +``` + +When built without that option, JSON support is disabled. **When +reconfiguring the source tree**, ***always be sure to do a "make +clean"*** (or equivalent for your platform) between builds (preferably +*before* reconfiguring), to ensure that everything is rebuilt properly. +If you fail to do that after enabling JSON on a tree which has already +been built, most of the sources will not be rebuilt properly. The reason +is that the JSON files are actually unconditionally compiled, but when +built without `--enable-json` they compile to empty object files. Thus +after a reconfigure the (empty) object files are still up-to-date +vis-a-vis the sources, and won't be rebuilt. + +To build Fossil with JSON support on Windows using the Microsoft C +compiler: + +``` +cd win +nmake -f Makefile.msc FOSSIL_ENABLE_JSON=1 +``` + +It has been seen to compile in VC versions 6 and higher. + + +# Goals & Non-goals + +The API described here is most certainly not +[*REST*](http://en.wikipedia.org/wiki/Representational_state_transfer)-conformant, +but is instead JSON over HTTP. The error reporting techniques of the +REST conventions (using HTTP error codes) "does not mesh" with my ideas +of separation of transport- vs. app-side errors. Additionally, REST +requires HTTP methods which are not specified by CGI (namely PUT and +DELETE), which means we can't possibly implement a REST-compatible +interface on top of fossil (which uses CGI mode even for its built-in +server). + +The **overall goals** of this effort include: + +- A JSON-based API off of which clients can build customized Fossil + UIs and special-purpose applications. e.g. a desktop notification + applet which polls for new timeline data. +- Arbitrary JSON-using clients should be able to use it. Though JSON + originates from JavaScript, it is truly a cross-platform data format + with a very high adoption rate. (There’s even a JSON implementation + for Oracle PL/SQL.) +- Fossil’s CGI and Server modes are the main targets and should be + supported equally. CLI JSON mode is of secondary concern (but is in + practice easier to test, so it’s generally implemented first). + +The ***non-goals*** include: + +- We won’t be able to implement *every* feature of Fossil via a JSON + interface, and we won’t try to. +- Binary data (e.g. commits of binary files or downloading ZIP files) + is not an initial goal, but "might be interesting" once the overall + infrastructure is in place and working well. See below for more + details about binary data in JSON. +- A "pure REST" interface is seemingly not possible due to REST + relying on HTTP methods not specified in the CGI standard (PUT and + DELETE). Additionally, REST-style error reporting cannot be used by + non-HTTP clients (which this code supports). + +Adding JSON support also gives us a framework off of which to +build/enhance other features. Some examples include: + +- **Internationalization**. Errors are reported via standard codes and + the raw artifact data is language-independent. +- The ability to author **special-case clients**, e.g. a ticket + poller. +- Use **arbitrary HTTP-capable languages** to implement such tools. + Programming languages which can execute programs and intercept their + stdout output can use the JSON API via a local fossil binary. +- **Automatable tests.** Many of fossil's test results currently have + to be "visually reviewed" for correctness after changes (e.g. + changes in the HTML interface). JSON structures can be + programmatically checked for correctness. Artifacts are immutable, + which allows us to be very specific in what data to expect as output + (for artifact-specific requests the payload data will often (but not + always) be the same across all requests and all time). + + +# Potential Client-side Uses + +Some of the potential client-side uses of this API include... + +- Custom apps/applets to fetch timeline/ticket/etc. information from + arbitrary repositories. There are many possibilities here, including + "dashboard" sites which monitor several repositories. +- Custom post-commit triggers, by polling for changes and reacting to + them (e.g. sending mails). +- A custom wiki front-end which uses fossil as the back-end storage, + inheriting its versioning and user access support while providing a + completely custom wiki-centric UI. Such a wiki need not have, on the + surface, anything to do with fossil or source control, as fossil + would just become a glorified wiki back-end. This approach also + allows clients to serve wiki pages in a format of their choice - + since all rendering would be done client-side, they could use + whatever format they like. + + + +# Technical Problems and Considerations + +A random list of considerations which need to be made and potential +problem areas... + +- **Binary data:** JSON is a text serialization method, and it takes + up the “payload” area of each HTTP request, so there is no + reasonable way to include binary data in the JSON message without + some sort of codec like Base64, for which there is no provision in + the current JSON API. You will therefore find no JSON API for + committing changes to a file in the repository, for example. Other + Fossil APIs such as [`/raw`](/help/www/raw) or + [`/fileedit`](../fileedit-page.md) may serve you better. +- **64-bit integers:** The JSON standard does not specify integer precision, + because it targets many different platforms, and not all of + them can support more than 32 bits. JavaScript (from which JSON + derives) supports 53 bits of integer precision, which may affect how + a given client-side JSON implementation sends large integers to Fossil’s JSON + API. Our JSON parser can cope with integers larger than 32 bits on input, and it + can emit them, but it requires platform support. If you’re running + Fossil on a 64-bit host, you should not run into problems in + this area, but if you’re on a legacy 32-bit only or a mixed 32/64-bit + system, it’s possible that some integers in the API could be + clipped. Realize however that this is a rare case: Fossil currently + cannot store files large enough to exceed a 32-bit `size_t` value, + and `time_t` won’t roll past 32-bit integers until 2038. We’re aware + of no other uses of integers in this API that could even in + principle exceed the range of a 32-bit integer. +- **Timestamps:** For portability, this API uses UTC Unix epoch + timestamps. (`time_t`) They are the most portable time representation out + there, easily usable in most programming environments. (In + hindsight, we might better have used a higher-precision time format, + but changing that now would break API compatibility.) diff --git a/www/json-api/tips.md b/www/json-api/tips.md new file mode 100644 index 0000000..885bdf8 --- /dev/null +++ b/www/json-api/tips.md @@ -0,0 +1,102 @@ +# JSON API: Tips and Tricks +([⬑JSON API Index](index.md)) + +Jump to: + +* [Beware of Content-Type and Encoding...](#content-type) + * [Using `curl` and `wget`](#curl-wget) +* [Example JavaScript](#javascript) +* [Demo Apps](#demo-apps) + +--- + + +# Beware of Content-Type and Encoding... + +When posting data to fossil, make sure that the request sends: + +- **Content-Type** of `application/json`. Fossil also (currently) + accepts `application/javascript` and `text/plain` as JSON input, + but `application/json` is preferred. The client may optionally + send `;charset=utf-8` with the Content-Type, but any other + encoding produces undefined results. Behaviour without the charset + or with `;charset=utf-8` suffix is identical. +- **POST data must be an non-form-encoded JSON string** + (ASCII or UTF-8). jQuery, by default, form-urlencodes it, which the + fossil json bits cannot read. e.g. post the result of + `JSON.stringify(requestObject)`, without any additional encoding on + top of it. +- **When POSTing via jQuery**, set these AJAX options: + - `contentType:'application/json'` + - `dataType:'text'` + - `data:JSON.stringify(requestObject)` +- **When POSTing via XMLHttpRequest** (XHR), be sure to: + - `xhr.open( … )` + - `xhr.setRequestHeader("Content-Type", "application/json")` + - `xhr.send( JSON.stringify( requestObject ) )` + +The response will be (except in the case of an HTTP 500 error or +similar) a JSON or JSONP string, ready to be parsed by your favourite +`JSON.parse()` implementation or `eval()`'d directly. + + +## Using `curl` and `wget` + +Both [curl](https://curl.haxx.se/) and +[wget](https://www.gnu.org/software/wget/) can be used to post data to +this API from the command line or scripts, but both require an extra +parameter to set the request encoding. + +Example: + +```console +$ cat x.json +{ +"payload": { + "sql": "SELECT * FROM reportfmt limit 1", + "format": "o" + } +} + +# Fossil has been started locally with: +# fossil server --localauth +# which allows the following requests to work without extra +# authenticaion: + +$ wget -q -O- \ + --post-file=x.json \ + --header="Content-Type: application/json" \ + 'http://localhost:8080/json/query' + +$ curl \ + --data-binary @x.json \ + --header 'Content-Type: application/json' \ + 'http://localhost:8080/json/query' +``` + +The relevant parts for encoding are the `--header` flag for `wget` and +`curl`, noting that they have different syntaxes for each +(`--header=X` vs `--header X`). + + +# Example JavaScript (Browser and Shell) + +In the fossil source tree, [in the ajax directory](/dir/ajax), is test/demo code +implemented in HTML+JavaScript. While it is still quite experimental, it +demonstrates one approach to creating client-side wrapper APIs for +remote Fossil/JSON repositories. + +There is some additional JS test code, which uses the Rhino JS engine +(i.e. from the console, not the browser), under +[`ajax/i-test`](/dir/ajax/-itest). That adds a Rhino-based connection +back-end to the AJAJ API and uses it for running integration-style +tests against an arbitrary JSON-capable repository. + + + +# Demo Apps + +Known in-the-wild apps using this API: + +- The wiki browsers/editors at [](https://fossil.wanderinghorse.net/wikis/) + -- cgit v1.3.1