The Setup
In this guide, I will talk about how conflicts happen, why they happen, and how BitKeeper helps you merge them when they can’t be merged automatically. In order to do this, I will use a real example from a real open source software product and I will walk through the process of discovering what others did to the code and how to merge two different branches. But first, a little backstory.
A while ago, we decided at BitKeeper to create a C-like language that runs on top of the Tcl runtime, we called it L. At the beginning of the project, we imported the Tcl CVS repository into BitKeeper and created our own branch of Tcl. For three years we worked on the project, almost completely ignoring all the development in the main Tcl branch, and synchronizing only
occasionally with them. Now it was time to synchronize the project with them again, and I was in charge of doing it.
How bad is it?
The first step was seeing how big the merge was going to be. I imported the latest Tcl repository into BitKeeper and called it Tcl-Baseline. Then I cloned my own copy of our own branch and called it Tcl-BitMover. I also made Tcl-Baseline the parent repository of Tcl-BitMover in order to save typing URLs in a bunch of commands that use the parent by default.
I went to the directory that holds our branch, and tried to gauge how much work had been done in the baseline.
dirac Tcl-BitMover $ bk repogca -d:REV:\ :D:\\n
1.3971.1.1127 2008/03/30
The bk repogca command prints the Greatest Common Ancestor [1] of the two repositories. This is the newest ChangeSet that is shared by the two repositories [2]. The -d option just formats the output: I asked for the revision number (:REV:) and the date (:D:).
- The proper term would be Greatest Lower Bound, but for historical reasons it is called GCA in BitKeeper.
- Technically, it can even be a list if the two repositories have been merged in what is called a criss-cross
merge.
I can see from the output of that command that the last time our branch was synchronized with the main Tcl branch was in 2008. How much work has been done in our branch and how much work has been done in the main Tcl branch since then? Let’s find out:
dirac Tcl-BitMover $ bk changes -qnd:REV:\ :D: -L | wc -l
358
dirac Tcl-BitMover $ bk changes -qnd:REV:\ :D: -R | wc -l
697
Let’s break down those two commands. The command bk changes lets us explore ChangeSets. The first argument is formatting: -q to avoid printing extra information, like the repository paths, -n to print a new line after each line, and -d:REV:\ :D: which tells bk changes to print the revision number and the date for each ChangeSet.
Notice that the last argument is different between the two invocations of bk changes. It is -L for the first one, asking for local changes. That is, ChangeSets that are in this repository (Tcl-BitMover), but not in its parent (Tcl-Baseline).
In the second invocation, the last argument is -R, which means remote. It’s the exact opposite: it asks for changes that are in the parent (Tcl-Baseline) but are not in the local repository (Tcl-BitMover).
Finally, wc -l is a standard Unix command that counts the number of lines. Since the -n argument to bk changes prints one new line per ChangeSet, the output tells us how many ChangeSets are in the remote repository that are not in the local repository and how many ChangeSets are in the local repository that are not in the remote.
Seems like the Tcl developers have been busy, they have created 697 ChangeSets since the last time we synchronized. We have created 358 ChangeSets in the same amount of time.
Pulling and merging
Let’s pull those ChangeSets and see how bad the merge is:
dirac Tcl-BitMover $ bk pull
Pull file:///Users/ob/example/Tcl-Baseline
-> file:///Users/ob/example/Tcl-BitMover
[pull] 100% |==============================|
Conflicts during automerge of generic/tcl.h
Conflicts during automerge of generic/tclBasic.c
Conflicts during automerge of generic/tclCmdIL.c
Conflicts during automerge of generic/tclExecute.c
Conflicts during automerge of generic/tclIOUtil.c
Conflicts during automerge of generic/tclInt.h
Conflicts during automerge of generic/tclInterp.c
Conflicts during automerge of generic/tclMain.c
Conflicts during automerge of generic/tclRegexp.h
Conflicts during automerge of tests/interp.test
Conflicts during automerge of unix/Makefile.in
Conflicts during automerge of unix/configure
Conflicts during automerge of win/Makefile.in
Conflicts during automerge of win/configure
resolve: 14 unresolved conflicts, starting manual resolve process for:
generic/tcl.h
generic/tclBasic.c
generic/tclCmdIL.c
generic/tclExecute.c
generic/tclIOUtil.c
generic/tclInt.h
generic/tclInterp.c
generic/tclMain.c
generic/tclRegexp.h
tests/interp.test
unix/Makefile.in
unix/configure
win/Makefile.in
win/configure
[resolve] 100% |==============================|
(content conflict) generic/tcl.h>>
The normal mode of operation for bk pull is to bring in the remote ChangeSets, merge them automatically, and update the local repository with them. Sometimes however, there are conflicts that can’t be merged automatically. In that case, BitKeeper will tell us which files have conflicts in them, and print a prompt indicating which kind of conflict it is and asking what to do about it.
The first command you need to know is simply ?, which prints what commands are available. Let’s type that at the prompt:
(content conflict) generic/tcl.h>> ?
---------------------------------------------------------------------------
File: generic/tcl.h
New work has been added locally and remotely and must be merged.
GCA: 1.210.1.44
Local: 1.230
Remote: 1.210.1.78
---------------------------------------------------------------------------
Commands are:
? - print this help
! - escape to an interactive shell
a - abort the patch, DISCARDING all merges
cl - clear the screen
C - commit the merged file
CC - commit the merged file with comments
d - diff the local file against the remote file
D - run side-by-side graphical difftool on local and remote
dl - diff the GCA vs. local file
dr - diff the GCA vs. remote file
dlm - automerge (if not yet merged) and diff the local file vs. merge file
drm - automerge (if not yet merged) and diff the remote file vs merge file
e - automerge (if not yet merged) and then edit the merged file
f - merge with graphical three-way filemerge
f2 - merge with graphical two-way filemerge
h - revision history of all changes
hl - revision history of the local changes
hr - revision history of the remote changes
H - show merge help in helptool
m - automerge the two files
p - graphical picture of the file history
q - immediately exit resolve
s - merge the two files using SCCS' algorithm
sd - side-by-side diff of the local file vs. the remote file
ul - use the local version of the file
ur - use the remote version of the file
v - view the merged file
vl - view the local file
vr - view the remote file
S - skip this file and resolve it later
x - explain the choices
Typical command sequence: 'e' 'C';
Difficult merges may be helped by 'p'.
(content conflict) generic/tcl.h>>
As you can see, ? also gives some suggestions on how to proceed. Let’s do what it suggests and use the p command to see a graphical picture of the file history:
BitKeeper’s Revtool GUI showing the GCA and local ChangeSets. The GCA ChangeSet is highlighted in white and the local tip is highlighted in green.
What you’re looking at is BitKeeper’s history presentation tool, bk revtool, operating in the file that has the conflict. In this view, we have the common ancestor for the file highlighted in white while the local tip of the file is highlighted in green. We can’t see the remote tip (the parent’s) because it is all the way to the right. If we scroll, it will come into view.
BitKeeper’s Revtool GUI showing the remote tip highlighted in red.
At this point it is important to note that one of the advantages of operating at the file level is that we are presented with more relevant information than if we operated at the repository level. For instance, it’s possible for the file conflict to be the result of only parallel changes to the file, while the repositories have hundreds of parallel ChangeSets.
Let’s merge the file by using the f command in the prompt:
(content conflict) generic/tcl.h>> f
The f command launches BitKeeper’s fm3tool GUI, which is a graphical merge tool. It looks like this:
BitKeeper’s file merge tool (fm3tool) showing the differences between the local and remote revisions and the GCA.
Let’s unpack what we’re looking at by labeling each of the four panes.
BitKeeper’s fm3tool with labels showing what each of the four panes are showing.
The top left pane (labeled LOCAL) contains the local history and local contents of the file. The top right pane (labeled REMOTE) contains the remote history and remote contents of the file. The bottom left pane (labeled MERGED) is where the result goes. And finally, the bottom right pane (labeled HELP) is just a reminder of what the colors mean and what the mouse actions and keyboard shortcuts do.
We are first going to focus on understanding the top two panes, namely LOCAL and REMOTE. These two panes consist each of the history of the changes on top, and the contents of the file at the bottom. Let’s look first at the pane labeled LOCAL.
Top left pane of BitKeeper’s fm3tool showing the history and contents of the local version of the file.
The contents of the file are in a special format. It shows the difference (diffs) between the GCA (the common revision to both local and remote) and the local version of the file. The turquoise color means unchanged, while the sky blue color means added.
View of the differences and history from GCA to LOCAL.
Now let’s take a look at the upper right pane, the REMOTE pane.
View of the differences and history between GCA and REMOTE.
This pane shows the history and contents of the remote file. A future version of BitKeeper will show the comments for the revision that deleted the lines, rather than the one that added the deleted lines, which is immensely more useful.
All the information necessary for understanding both the LOCAL and REMOTE changes is presented at the same time. Armed with these two views, we can understand what the conflict is about.
Looking at the LOCAL pane, we can see in the diffs that a new constant called TCL_REG_PCRE was added as part of the feature “PCRE support” mentioned in the comments.
On the REMOTE pane, we can see that the comments for the three defines TCL_REG_NLANCH, TCL_REG_NEWLINE, and TCL_REG_CANMATCH were fixed for spelling and grammar.
The conflict happened because those two changes are too close together for BitKeeper to merge automatically. Basically when two parallel branches change the same region of code, it is too dangerous to auto merge the changes.
We can see that the right answer in this case is to take the comment corrections from the REMOTE (you can see in the LOCAL pane that those lines were not changed in LOCAL, which is why they are highlighted in turquoise), and also take the addition of the TCL_REG_PCRE define from the LOCAL side. We accomplish that by clicking on each of the regions we want to take with the mouse (all of the clicks are left-clicks).
Note that since this was the only conflict, the information bar in the help pane changed color from red to light yellow, and it says now that there are zero unresolved conflicts. That means that we are done with this file. We can simply hit the key s (for save) and fm3tool will quit after saving the merged file.
We’ll go back to the prompt, where we can use the C (in uppercase) command to commit the merged file.
(content conflict) generic/tcl.h>> C
(content conflict) generic/tclBasic.c>>
The C command also moved us to the next file with conflicts: tclBasic.c. Let’s see what that one has in store for us. We can bring up fm3tool again by typing the f command.
(content conflict) generic/tclBasic.c>> f
This is what will appear:
BitKeeper’s fm3tool for tclBasic.h.
That first conflict doesn’t look too scary. It’s basically an update to copyright comments so we will resolve it in the same way as before, by taking both changes, first the REMOTE and then LOCAL.
We are not done with this file yet. We have solved the first conflict, but there is another one (it said “2 unresolved” in the red bar). In order to move to the next conflict, we can hit the } key or click on the forward double arrow button (it looks like a fast forward button). Let’s use the button:
This conflict looks more complicated. It looks like the REMOTE side changed a bunch of lines and the local side also changed a lot of lines in the same area. One thing to note tho is that all that turquoise in the left side (LOCAL) means that the changes on the local side are not as large as the changes in the remote. When conflicts are this large, sometimes it helps to remove the diffs from the GCA and only look at what both the LOCAL and REMOTE did
without the added noise of what was in the GCA. Let’s do that now by clicking on that “GCA” checkbox on the toolbar.
Without the added noise of removed lines in the right pane, the conflict no longer looks too scary:
Without the GCA, you can see the changed lines in Sky Blue on both sides.
On the left side (LOCAL) we can see that two new commands were added: “L”, and “pointer”. We still don’t know what all those changes are on the right hand side. Let’s explore what happened. First we are going to call out to bk revtool to see the history. We do that by selecting “FILE” and then “Run Revtool”.
Selecting “File|Run Revtool” launches bk revtool from fm3tool.
Once revtool is up, each of the three revisions of interest will be highlighted: GCA in white, LEFT in green and RIGHT in red. Since we are trying to figure out what happened in the REMOTE, we are going to click on the GCA (white) and control-click on the REMOTE (red). This will bring up the diffs between those two revisions in revtool.
Once we have revtool showing the diffs, we’ll click on the “Diff tool” button at the top of the toolbar to bring up bk difftool. It will look like this:
BitKeeper’s Difftool between GCA and REMOTE tip.
We are now seeing a diff of the GCA and the tip of the REMOTE file. That is, we are seeing everything that the remote side changed. Note that most of these diffs have been automatically merged by BitKeeper. We are only interested in understanding the differences that resulted in the conflict. We can do that by hitting the spacebar to move to the next conflict until we see the one we are interested in.
We have now found the conflict we care about. Take a look at the yellow regions, those are the portions of the line that were changed.
BitKeeper’s difftool showing the changes in GCA vs REMOTE that caused the conflict we are resolving.
It looks like they just added a new field to the structure doesn’t it? If you look carefully at these differences, you’ll see that most of the entries had just a NULL value added as the next to last field. Let’s follow the exact same procedure with the local side. We’ll go back to revtool, then click on GCA and control-click on LOCAL (green), then select “Diff tool” from the toolbar and hit “next” until we find the diff that added those conflicting lines:
BitKeeper’s Difftool between GCA and LOCAL for the conflict we are resolving.
It looks like when those two new entries were added, the structure only had four fields, but seeing the diffs in REMOTE we learned that a new field was added in the next-to-last position, and that it’s NULL for most entries. It is probably safe to assume we can merge this conflict by taking the changes made in REMOTE (after all, the declaration of the structure was probably updated to have an extra field), and then taking the changes done in LOCAL with one modification: the addition of an extra field in the next-to-last position with the value NULL.
Fortunately, fm3tool allows us not only to take blocks of code form each of the two sides, but also to modify them in the bottom left pane (the MERGE pane). The way we’ll accomplish this is as follows: we are going to go back to fm3tool, click on the REMOTE changes, then click on the LOCAL changes, and then click on the MERGE pane where we’ll manually add the extra field. Finally, we’ll hit the ESC key to get out of edit mode and hit the s key to save. Let’s do that now.
Now that we have finally merged the file, we can use the C command again in the prompt to keep merging the other files with conflicts.
(content conflict) generic/tclBasic.c>> C
(content conflict) generic/tclCmdIL.c>>
I’m not going to show the process of merging all the other files (after all, there were 14 of them!). But the process is very similar. Once we are done merging all the files, BitKeeper will pop up BitKeeper’s Check-in tool so that we can comment on the merge ChangeSet.
Conclusion
In this guide we saw how to use BitKeeper to merge conflicts using fm3tool, revtool, and difftool. For more information see the online help available by running bk help.