diff options
Diffstat (limited to 'www/rebaseharm.md')
| -rw-r--r-- | www/rebaseharm.md | 522 |
1 files changed, 522 insertions, 0 deletions
diff --git a/www/rebaseharm.md b/www/rebaseharm.md new file mode 100644 index 0000000..9b72d38 --- /dev/null +++ b/www/rebaseharm.md @@ -0,0 +1,522 @@ +# Rebase Considered Harmful + +Fossil deliberately omits a "rebase" command because the original +designer of Fossil (and [original author][vhist] of this article) considers rebase to be +an anti-pattern to be avoided. This article attempts to +explain that point of view. + +[vhist]: /finfo?name=www/rebaseharm.md&ubg + +## 1.0 Rebasing is dangerous + +Most people, even strident advocates of rebase, agree that rebase can +cause problems when misused. The Git rebase documentation talks about the +[golden rule of rebasing][golden]: never rebase on a public +branch. Horror stories of misused rebase abound, and the rebase +documentation devotes considerable space toward explaining how to +recover from rebase errors and/or misuse. + +## <a id="cap-loss"></a>2.0 Rebase provides no new capabilities + +Sometimes sharp and dangerous tools are justified, +because they accomplish things that cannot be +done otherwise, or at least cannot be done easily. +Rebase does not fall into that category, +because it provides no new capabilities. + +### <a id="orphaning"></a>2.1 A rebase is just a merge with historical references omitted + +A rebase is really nothing more than a merge (or a series of merges) +that deliberately forgets one of the parents of each merge step. +To help illustrate this fact, +consider the first rebase example from the +[Git documentation][gitrebase]. The merge looks like this: + +~~~ pikchr toggle center +scale = 0.8 +circle "C0" fit +arrow right 50% +circle same "C1" +arrow same +circle same "C2" +arrow same +circle same "C3" +arrow same +circle same "C5" +circle same "C4" at 1cm above C3 +arrow from C2 to C4 chop +arrow from C4 to C5 chop +~~~ + +And the rebase looks like this: + +~~~ pikchr toggle center +scale = 0.8 +circle "C0" fit +arrow right 50% +circle same "C1" +arrow same +circle same "C2" +arrow same +circle same "C3" +arrow same +circle same "C4'" +circle same "C4" at 1cm above C3 +arrow from C2 to C4 chop +~~~ + +As the [Git documentation][gitrebase] points out, check-ins C4\' and C5 +are identical. The only difference between C4\' and C5 is that C5 +records the fact that C4 is its merge parent but C4\' does not. + +Thus, a rebase is just a merge that forgets where it came from. + +The Git documentation acknowledges this fact (in so many words) and +justifies it by saying "rebasing makes for a cleaner history." I read +that sentence as a tacit admission that the Git history display +capabilities are weak and need active assistance from the user to +keep things manageable. +Surely a better approach is to record +the complete ancestry of every check-in but then fix the tool to show +a "clean" history in those instances where a simplified display is +desirable and edifying, but retain the option to show the real, +complete, messy history for cases where detail and accuracy are more +important. + +So, another way of thinking about rebase is that it is a kind of +merge that intentionally forgets some details in order to +not overwhelm the weak history display mechanisms available in Git. +Wouldn't it be better, less error-prone, and easier on users +to enhance the history display mechanisms in Git so that rebasing +for a clean, linear history became unnecessary? + +### <a id="clean-diffs"></a>2.2 Rebase does not actually provide better feature-branch diffs + +Another argument, often cited, is that rebasing a feature branch +allows one to see just the changes in the feature branch without +the concurrent changes in the main line of development. +Consider a hypothetical case: + +~~~ pikchr toggle center +scale = 0.8 +circle "C0" fit fill white +arrow right 50% +circle same "C1" +arrow same +circle same "C2" +arrow same +circle same "C4" +arrow same +circle same "C6" +circle same "C3" at last arrow.width + C0.rad*2 heading 30 from C2 +arrow right 50% +circle same "C5" +arrow from C2 to C3 chop +box ht C3.y-C2.y wid C6.e.x-C0.w.x+1.5*C1.rad at C2 behind C0 fill 0xc6e2ff color 0xaac5df +box ht previous.ht wid previous.wid*0.55 with .se at previous.ne \ + behind C0 fill 0x9accfc color 0xaac5df +text "feature" with .s at previous.n +text "main" with .n at first box.s +~~~ + +In the above, a feature branch consisting of check-ins C3 and C5 is +run concurrently with the main line in check-ins C4 and C6. Advocates +for rebase say that you should rebase the feature branch to the tip +of main in order to remove main-line development differences from +the feature branch's history: + +~~~ pikchr toggle center +# Duplicated below in section 5.0 +scale = 0.8 +circle "C0" fit fill white +arrow right 50% +circle same "C1" +arrow same +circle same "C2" +arrow same +circle same "C4" +arrow same +circle same "C6" +circle same "C3" at last arrow.width + C0.rad*2 heading 30 from C2 +arrow right 50% +circle same "C5" +arrow from C2 to C3 chop +C3P: circle same "C3'" at first arrow.width + C0.rad*2 heading 30 from C6 +arrow right 50% from C3P.e +C5P: circle same "C5'" +arrow from C6 to C3P chop + +box ht C3.y-C2.y wid C5P.e.x-C0.w.x+1.5*C1.rad with .w at 0.5*(first arrow.wid) west of C0.w \ + behind C0 fill 0xc6e2ff color 0xaac5df +box ht previous.ht wid previous.e.x - C2.w.x with .se at previous.ne \ + behind C0 fill 0x9accfc color 0xaac5df +~~~ + + +You could choose to collapse C3\' and C5\' into a single check-in +as part of this rebase, but that's a side issue we'll deal with +[separately](#collapsing). + +Because Fossil purposefully lacks rebase, the closest you can get to this same check-in +history is the following merge: + +~~~ pikchr toggle center +scale = 0.8 +circle "C0" fit fill white +arrow right 50% +circle same "C1" +arrow same +circle same "C2" +arrow same +circle same "C4" +arrow same +circle same "C6" +circle same "C3" at last arrow.width + C0.rad*2 heading 30 from C2 +arrow right 50% +circle same "C5" +arrow same +circle same "C7" +arrow from C2 to C3 chop +arrow from C6 to C7 chop + +box ht C3.y-C2.y wid C7.e.x-C0.w.x+1.5*C1.rad with .w at 0.5*(first arrow.wid) west of C0.w \ + behind C0 fill 0xc6e2ff color 0xaac5df +box ht previous.ht wid previous.e.x - C2.w.x with .se at previous.ne \ + behind C0 fill 0x9accfc color 0xaac5df +~~~ + + +Check-ins C5\' and C7 check-ins hold identical code. The only +difference is in their history. + +The argument from rebase advocates +is that with merge it is difficult to see only the changes associated +with the feature branch without the commingled mainline changes. +In other words, diff(C2,C7) shows changes from both the feature +branch and from the mainline, whereas in the rebase case +diff(C6,C5\') shows only the feature branch changes. + +But that argument is comparing apples to oranges, since the two diffs +do not have the same baseline. The correct way to see only the feature +branch changes in the merge case is not diff(C2,C7) but rather diff(C6,C7). + +<div align=center> + +| Rebase | Merge | What You See | +|---------------|-------------|----------------------------------------| +| diff(C2,C5\') | diff(C2,C7) | Commingled branch and mainline changes | +| diff(C6,C5\') | diff(C6,C7) | Branch changes only | + +</div> + +Remember: C7 and C5\' are bit-for-bit identical, so the output of the +diff is not determined by whether you select C7 or C5\' as the target +of the diff, but rather by your choice of the diff source, C2 or C6. + +So, to help with the problem of viewing changes associated with a feature +branch, perhaps what is needed is not rebase but rather better tools to +help users identify an appropriate baseline for their diffs. + +## <a id="siloing"></a>3.0 Rebase encourages siloed development + +The [golden rule of rebasing][golden] is that you should never do it +on public branches, so if you are using rebase as intended, that means +you are keeping private branches. Or, to put it another way, you are +doing siloed development. You are not sharing your intermediate work +with collaborators. This is not good for product quality. + +[Nagappan, et. al][nagappan] studied bugs in Windows Vista and found +that the best predictor of bugs is the distance on the org-chart between +the stake-holders. The bug rate is inversely related to the +amount of communication among the engineers. +Similar findings arise in other disciplines. Keeping +private branches does not prove that developers are communicating +insufficiently, but it is a key symptom of that problem. + +[Weinberg][weinberg] argues programming should be "egoless." That +is to say, programmers should avoid linking their code with their sense of +self, as that makes it more difficult for them to find and respond +to bugs, and hence makes them less productive. Many developers are +drawn to private branches out of sense of ego. "I want to get the +code right before I publish it." I sympathize with this sentiment, +and am frequently guilty of it myself. It is humbling to display +your stupid mistake to the whole world on an Internet that +never forgets. And yet, humble programmers generate better code. + +What is the fastest path to solid code? Is it to continue staring at +your private branch to seek out every last bug, or is it to publish it +as-is, whereupon the many eyeballs will immediately see that last stupid +error in the code? Testing and development are often done by separate +groups within a larger software development organization, because +developers get too close to their own code to see every problem in it. + +Given that, is it better for those many eyeballs to find your problems +while they're still isolated on a feature branch, or should that vetting +wait until you finally push a collapsed version of a private working +branch to the parent repo? Will the many eyeballs even see those errors +when they’re intermingled with code implementing some compelling new feature? + +## <a id="timestamps"></a>4.0 Rebase causes timestamp confusion + +Consider the earlier example of rebasing a feature branch: + +~~~ pikchr toggle center +# Copy of second diagram in section 2.2 above +scale = 0.8 +circle "C0" fit fill white +arrow right 50% +circle same "C1" +arrow same +circle same "C2" +arrow same +circle same "C4" +arrow same +circle same "C6" +circle same "C3" at last arrow.width + C0.rad*2 heading 30 from C2 +arrow right 50% +circle same "C5" +arrow from C2 to C3 chop +C3P: circle same "C3'" at first arrow.width + C0.rad*2 heading 30 from C6 +arrow right 50% from C3P.e +C5P: circle same "C5'" +arrow from C6 to C3P chop + +box ht C3.y-C2.y wid C5P.e.x-C0.w.x+1.5*C1.rad with .w at 0.5*(first arrow.wid) west of C0.w \ + behind C0 fill 0xc6e2ff color 0xaac5df +box ht previous.ht wid previous.e.x - C2.w.x with .se at previous.ne \ + behind C0 fill 0x9accfc color 0xaac5df +~~~ + +What timestamps go on the C3\' and C5\' check-ins? If you choose +the same timestamps as the original C3 and C5, then you have the +odd situation C3' is older than its parent C6. We call that a +"timewarp" in Fossil. Timewarps can also happen due to misconfigured +system clocks, so they are not unique to rebase, but they are very +confusing and so best avoided. The other option is to provide new +unique timestamps for C3' and C5' but then you lose the information +about when those check-ins were originally created, which can make +historical analysis of changes more difficult. It might also +complicate the legal defense of prior art claims. + +## <a id="lying"></a>5.0 Rebase misrepresents the project history + +By discarding parentage information, rebase attempts to deceive the +reader about how the code actually came together. + +Git’s rebase feature is more than just an +alternative to merging: it also provides mechanisms for changing the +project history in order to make editorial changes. Fossil shows that +you can get similar effects without modifying historical records, +allowing users to: + + 1. Edit check-in comments to fix typos or enhance clarity + 2. Attach supplemental notes to check-ins or whole branches + 3. Hide ill-conceived or now-unused branches from routine display + 4. Fix faulty check-in date/times resulting from misconfigured + system clocks + 5. Cross-reference check-ins with each other, or with + wiki, tickets, forum posts, and/or embedded documentation + +…and so forth. + +Fossil allows all of this not by removing or modifying existing +repository entries, but rather by adding new supplemental records. +Fossil keeps the original incorrect or unclear inputs and makes them +readily accessible, preserving the original historical record. Fossil +doesn’t make the user tell counter-factual “stories,” it only allows the +user to provide annotations to provide a more readable edited +presentation for routine display purposes. + +Git needs rebase because it lacks these annotation facilities. Rather +than consider rebase a desirable feature missing in Fossil, ask instead +why Git lacks support for making editorial changes to check-ins without +modifying history? Wouldn't it be better to fix the version control +tool rather than requiring users to fabricate a fictitious project +history? + +## <a id="collapsing"></a>6.0 Collapsing check-ins throws away valuable information + +One of the oft-cited advantages of rebasing in Git is that it lets you +collapse multiple check-ins down to a single check-in to make the +development history “clean.” The intent is that development appear as +though every feature were created in a single step: no multi-step +evolution, no back-tracking, no false starts, no mistakes. This ignores +actual developer psychology: ideas rarely spring forth from fingers to +files in faultless finished form. A wish for collapsed, finalized +check-ins is a wish for a counterfactual situation. + +The common counterargument is that collapsed check-ins represent a +better world, the ideal we're striving for. What that argument overlooks +is that we must throw away valuable information to get there. + +### <a id="empathy"></a>6.1 Individual check-ins support mutual understanding + +Ideally, future developers of our software can understand every feature +in it using only context available in the version of the code they start +work with. Prior to widespread version control, developers had no choice +but to work that way. Pre-existing codebases could only be understood +as-is or not at all. Developers in that world had an incentive to +develop software that was easy to understand retrospectively, even if +they were selfish people, because they knew they might end up being +those future developers! + +Yet, sometimes we come upon a piece of code that we simply cannot +understand. If you have never asked yourself, "What was this code's +developer thinking?" you haven't been developing software for very long. + +When a developer can go back to the individual check-ins leading up to +the current code, they can work out the answers to such questions using +only the level of personal brilliance necessary to be a good developer. To +understand such code using only the finished form, you are asking future +developers to make intuitive leaps that the original developer was +unable to make. In other words, you are asking your future maintenance +developers to be smarter than the original developers! That's a +beautiful wish, but there's a sharp limit to how far you can carry it. +Eventually you hit the limits of human brilliance. + +When the operation of some bit of code is not obvious, both Fossil and +Git let you run a [`blame`](/help/blame) on the code file to get +information about each line of code, and from that which check-in last +touched a given line of code. If you squash the check-ins on a branch +down to a single check-in, you throw away the information leading up to +that finished form. Fossil not only preserves the check-ins surrounding +the one that included the line of code you're trying to understand, its +[superior data model][sdm] lets you see the surrounding check-ins in +both directions; not only what lead up to it, but what came next. Git +can't do that short of crawling the block-chain backwards from the tip +of the branch to the check-in you’re looking at, an expensive operation. + +We believe it is easier to understand a line of code from the 10-line +check-in it was a part of — and then to understand the surrounding +check-ins as necessary — than it is to understand a 500-line check-in +that collapses a whole branch's worth of changes down to a single +finished feature. + +[sdm]: ./fossil-v-git.wiki#durable + +### <a id="bisecting"></a>6.2 Bisecting works better on small check-ins + +Git lets a developer write a feature in ten check-ins but collapse it +down to an eleventh check-in and then deliberately push only that final +collapsed check-in to the parent repo. Someone else may then do a bisect +that blames the merged check-in as the source of the problem they’re +chasing down; they then have to manually work out which of the 10 steps +the original developer took to create it to find the source of the +actual problem. + +An equivalent push in Fossil will send all 11 check-ins to the parent +repository so that a later investigator doing the same sort of bisect +sees the complete check-in history. That bisect will point the +investigator at the single original check-in that caused the problem. + +### <a id="comments"></a>6.3 Multiple check-ins require multiple check-in comments + +The more comments you have from a given developer on a given body of +code, the more concise documentation you have of that developer's +thought process. To resume the bisecting example, a developer trying to +work out what the original developer was thinking with a given change +will have more success given a check-in comment that explains what the +one check-in out of ten blamed by the "bisect" command was trying to +accomplish than if they must work that out from the eleventh check-in's +comment, which only explains the "clean" version of the collapsed +feature. + +### <a id="cherrypicking"></a>6.4 Cherry-picks work better with small check-ins + +While working on a new feature in one branch, you may come across a bug +in the pre-existing code that you need to fix in order for work on that +feature to proceed. You could choose to switch briefly back to the +parent branch, develop the fix there, check it in, then merge the parent +back up to the feature branch in order to continue work, but that's +distracting. If the fix isn't for a critical bug, fixing it on the +parent branch can wait, so it's better to maintain your mental working +state by fixing the problem in place on the feature branch, then check +the fix in on the feature branch, resume work on the feature, and later +merge that fix down into the parent branch along with the feature. + +But now what happens if another branch *also* needs that fix? Let us say +our code repository has a branch for the current stable release, a +development branch for the next major version, and feature branches off +of the development branch. If we rebase each feature branch down into +the development branch as a single check-in, pushing only the rebase +check-in up to the parent repo, only that fix's developer has the +information locally to perform the cherry-pick of the fix onto the +stable branch. + +Developers working on new features often do not care about old stable +versions, yet that stable version may have an end user community that +depends on that version, who either cannot wait for the next stable +version or who wish to put off upgrading to it for some time. Such users +want backported bug fixes, yet the developers creating those fixes have +poor incentives to provide those backports. Thus the existence of +maintenance and support organizations, who end up doing such work. +(There is [a famous company][rh] that built a multi-billion dollar +enterprise on such work.) + +This work is far easier when each cherry-pick transfers completely and +cleanly from one branch to another, and we increase the likelihood of +achieving that state by working from the smallest check-ins that remain +complete. If a support organization must manually disentangle a fix from +a feature check-in, they are likely to introduce new bugs on the stable +branch. Even if they manage to do their work without error, it takes +them more time to do the cherry-pick that way. + +[rh]: https://en.wikipedia.org/wiki/Red_Hat + +### <a id="backouts"></a>6.5 Back-outs also work better with small check-ins + +The inverse of the cherry-pick merge is the back-out merge. If you push +only a collapsed version of a private working branch up to the parent +repo, those working from that parent repo cannot automatically back out +any of the individual check-ins that went into that private branch. +Others must either manually disentangle the problematic part of your +merge check-in or back out the entire feature. + +## <a id="better-plan"></a>7.0 Cherry-pick merges work better than rebase + +Perhaps there are some cases where a rebase-like transformation +is actually helpful, but those cases are rare, and when they do +come up, running a series of cherry-pick merges achieves the same +topology with several advantages: + + 1. In Fossil, cherry-pick merges preserve an honest and clear record + of history. Fossil remembers where a cherry-pick came from, and + it shows this in its timeline, so other developers can understand + how a cherry-pick based commit came together. + + Git lacks the ability to remember the source of a cherry-pick as + part of the commit. This fact has no direct bearing on this + document’s thesis, but we can make a few observations. First, Git + forgets history in more cases than in rebasing. Second, if Git + remembered the source of cherry-picks in commits, Git users might + have a better argument for avoiding rebase, because they’d have an + alternative that *didn’t* lose history. + + 2. Fossil’s [test before commit philosophy][tbc] means you can test a + cherry-pick before committing it. Because Fossil allows multiple + cherry-picks in a single commit and it remembers them all, you can + do this for a complicated merge in step-wise fashion. + + Git commits cherry-picks straight to the repository, so that if it + results in a bad state, you have to do something drastic like + `git reset --hard` to repair the damage. + + 3. Cherry-picks keep both the original and the revised check-ins, + so both timestamps are preserved. + +[tbc]: ./fossil-v-git.wiki#testing + +## <a id="conclusion"></a>8.0 Summary and conclusion + +Rebasing is an anti-pattern. It is dishonest. It deliberately +omits historical information. It causes problems for collaboration. +And it has no offsetting benefits. + +For these reasons, rebase is intentionally and deliberately omitted +from the design of Fossil. + + +[golden]: https://www.atlassian.com/git/tutorials/merging-vs-rebasing#the-golden-rule-of-rebasing +[gitrebase]: https://git-scm.com/book/en/v2/Git-Branching-Rebasing +[nagappan]: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-2008-11.pdf +[weinberg]: https://books.google.com/books?id=76dIAAAAMAAJ |