diff options
Diffstat (limited to 'www/hashpolicy.wiki')
| -rw-r--r-- | www/hashpolicy.wiki | 199 |
1 files changed, 199 insertions, 0 deletions
diff --git a/www/hashpolicy.wiki b/www/hashpolicy.wiki new file mode 100644 index 0000000..58a1396 --- /dev/null +++ b/www/hashpolicy.wiki @@ -0,0 +1,199 @@ +<title>Hash Policy</title> + +<h2>Executive Summary</h2> + +<b>Or: How To Avoid Reading This Article</b> + +There was much angst over the [http://www.shattered.io|SHAttered attack] +against SHA1 when it was announced in early 2017. If you are concerned +about this and its implications for Fossil, simply +[./quickstart.wiki#install|upgrade to Fossil 2.1 or later], and the +problem will go away. Everything will continue to work as before. + + * Legacy repositories will continue working just as + they always have, without any conversions or upgrades. + * Historical check-ins will keep their same historical + SHA1 names. + * New check-ins will get more secure SHA3-256 hash names. + * Everything will continue as if nothing happened. + * Your workflow will be unchanged. + +But if you are curious and want a deeper understanding of what is +going on, read on... + + +<h2>Introduction</h2> + +The first snapshot-based distributed version control system +was [http://www.monotone.ca|Monotone]. Many of the ideas behind the design +of Fossil were copied from Monotone, including the use of a SHA1 hash to +assign names to artifacts. Git and Mercurial did the same thing. + +The SHA1 hash algorithm is used only to create names for artifacts in Fossil +(and in Git, Mercurial, and Monotone). It is not used for security. +Nevertheless, when the [http://www.shattered.io|SHAttered attack] found +two different PDF files with the same SHA1 hash, many users learned that +"SHA1 is broken". They see that Fossil (and Git, Mercurial, and Monotone) +use SHA1 and they therefore conclude that "Fossil is broken". This is +not true, but it is a public relations problem. So the decision +was made to migrate Fossil away from SHA1. + +This article describes how that migration is occurring. + +<h2>Use Of Hardened SHA1</h2> + +In Fossil version 2.0 ([/timeline?c=version-2.0|2017-03-03]), +the internal SHA1 implementation was changed from a generic +FIPS PUB 180-4 SHA1 implementation to a "Hardened SHA1" +[[https://github.com/cr-marcstevens/sha1collisiondetection|1]] +[[https://marc-stevens.nl/research/papers/C13-S.pdf|2]]. + +The Hardened SHA1 algorithm automatically detects when the artifact +being hashed is specifically designed to exploit the known weaknesses +in the SHA1 algorithm, and when it detects such an attack it changes +the hash algorithm (by increasing the number of rounds in the compression +function) to make the algorithm secure again. If the attack detection +gets a false-positive, that means that Hardened SHA1 will get a different +answer than the standard FIPS PUB 180-4 SHA1, but the creators of +Hardened SHA1 (see the second paper +[[https://marc-stevens.nl/research/papers/C13-S.pdf|2]]) +report that the probability of a false-positive is vanishingly small - +less than 1 false positive out of 10<sup><font size=1>27</font></sup> +hashes. + +Hardened SHA1 is slower (and a lot bigger) but Fossil does not do that +much hashing, so performance is not really an issue. + +All versions of Fossil moving forward will use Hardened SHA1. So if +someone says "SHA1 is broken, and Fossil uses SHA1, therefore Fossil is +broken," you can rebut the argument by pointing out that Fossil uses +<em>Hardened SHA1</em>, not generic SHA1, and Hardened SHA1 is <em>not</em> +broken. + +<h2>Support For SHA3-256</h2> + +Prior to Fossil version 2.0 ([/timeline?c=version-2.0|2017-03-03]), +all artifacts in all Fossil repositories were named +by only a SHA1 hash. +Version 2.0 extended the [./fileformat.wiki|Fossil file format] +to allow artifacts to be named by either SHA1 or SHA3-256 hashes. +(SHA3-256 is the only variant of SHA3 that +Fossil uses for artifact naming, so for the remainder of this article +it will be called simply "SHA3". Similarly, "Hardened SHA1" will +be shortened to "SHA1" in the remaining text.) + +To be clear: Fossil (version 2.0 and later) +allows the SHA1 and SHA3 hashes to be mixed within +the same repository. Older check-ins, created years ago, +continue to be named using their legacy SHA1 hashes while +newer check-ins are named using modern SHA3 hashes. There +is no need to "convert" a repository from SHA1 over to SHA3. +You can see this in Fossil itself. The recent +[9d9ef82234f63758] check-in uses a SHA3 hash whereas the older +[1669115ab9d05c18] check-in uses a SHA1 hash. + +Other than permitting the use of SHA3 in addition to SHA1, there +were no file format changes in Fossil version 2.0 relative +to the previous version 1.37. Both Fossil 2.0 and Fossil 1.37 read +and write all the same repositories and sync with one another, as long +as none of the repositories contain artifacts named using SHA3. If +a repository does contain artifacts named using SHA3, Fossil 1.37 will +not know how to interpret those artifacts and will generate various warnings +and errors. + +<h2>How Fossil Decides Which Hash Algorithm To Use For New Artifacts</h2> + +If newer versions of Fossil are able to use either SHA1 or SHA3 to +name artifacts, which hash algorithm is actually used? That question +is answered by the "hash policy". These are the supported hash policies: + +<table cellpadding=10> +<tr> +<td valign='top'>sha1</td> +<td>Name all new artifacts using the (Hardened) SHA1 hash algorithm.</td> +</tr> +<tr> +<td valign='top'>auto</td> +<td>Name new artifacts using the SHA1 hash algorithm, but if any +artifacts are encountered which are already named using SHA3, then +automatically switch the hash policy to "sha3"</td> +</tr> +<tr> +<td valign='top'>sha3</td> +<td>Name new artifacts using the SHA3 hash algorithm if the artifact +does not already have a SHA1 name. If the artifact already has a SHA1 +name, then continue to use the older SHA1 name. Use SHA3 for new +artifacts that have never before been encountered.</td> +</tr> +<tr> +<td valign='top'>sha3-only</td> +<td>Name new artifacts using the SHA3 hash algorithm even if the artifact +already has a SHA1 name. In other words, force the use of SHA3. This can +cause some artifacts to be added to the repository twice, once under their +SHA1 name and again under their SHA3 name, but delta compression will +prevent that from causing repository size problems.</td> +</tr> +<tr> +<td valign='top'>shun-sha1</td> +<td>Like "sha3-only" but at this level do not accept a push of SHA1-named +artifacts. If another Fossil instance tries to push a SHA1-named artifact, +that artifact is discarded and ignored. +</tr> +</table> + +For Fossil 2.0, and obviously also for Fossil 1.37 and before, the +only hash policy supported was the one now called "sha1", meaning that +all new artifacts were named +using a SHA1 hash. +Even though Fossil 2.0 added the capability of understanding SHA3 hashes, it +never actually generates any SHA3 hashes. + +From Fossil 2.1 through 2.9, the default hash policy for legacy repositories +changed to "auto", meaning that +Fossil continued to generate only SHA1 hashes until it +encountered one artifact with a SHA3 hash. Once those older versions of +Fossil saw a single SHA3 hash, they +automatically switched to "sha3" mode and thereafter generated +only SHA3 hashes. + +When a new repository is created by cloning, the hash policy is copied +from the parent. + +For new repositories created using the +[/help/new|fossil new] command the default hash policy is "sha3". +That means new repositories +will normally hold nothing except SHA3 hashes. The hash policy for new +repositories can be overridden using the "--sha1" option to the +"fossil new" command. + +If you are still on Fossil 2.1 through 2.9 but you want Fossil to go +ahead and start using SHA3 hashes, change the hash policy to +"sha3" using a command like this: + +<verbatim> +fossil hash-policy sha3 +</verbatim> + +The next check-in will use a SHA3 hash, so that when that check-in is pushed +to colleagues, their clones will include the new SHA3-named artifact, so +their local Fossil instances will automatically convert their clones to +"sha3" mode as well. + +Of course, if some members of your team stubbornly refuse to upgrade past +Fossil 1.37, you should avoid changing the hash policy and creating +artifacts with SHA3 names, because once you do that your recalcitrant +coworkers will no longer be able to collaborate. + +<h2>A Pure SHA3 Future</h2> + +Fossil 2.10 changed the default hash policy to "sha3" mode even for +legacy repositories, so if you +upgrade to the latest version of Fossil, all of your new artifacts will +use a SHA3 hash. Legacy SHA1 artifacts continue to use their original +names, but new artifacts will use SHA3 names. You might not even notice +this automatic change over to stronger hashes. + +We decided to make the change to pure SHA3 since the last known distributor +of Fossil 1.x binaries — Debian 9 — was finally replaced in June 2019 +by Debian 10, which included Fossil 2.8. All other known sources of +Fossil 1.x binaries upgraded well before that point. |