HOWTO: Building BitKeeper on Windows

Our current build environment on Windows uses an old version of MSYS that we have been maintaining.
To build with this setup:

  • install the binary release of BitKeeper
  • Open a Command Prompt
  • mkdir c:\build
  • cd \build
  • download our prebuilt build environment
  • bk tar zxf win32-buildenv.tgz
  • Make a directory for your BK repository and cd to it.
  • Clone BitKeeper source:
    bk clone -sALL bk://bkbits.net/u/bk/dev
  • Lauch the build environment:
    cd dev
    buildenv
    That last step runs buildenv.bat which will set up your build environment and leave you at an MSYS bash prompt in the src directory of this BK repository.
    The batch file will complain about not being able to resolve data.bitkeeper.com ā€“ ignore that.
    It will also ask if you wish to remove your old non-existent buildenv, just say no.
  • Build as follows:

./build p
./build image

* You will end up with two executables (install images) in the `utils` subdirectory.  One that opens a console during the installation and one that does not (should be obvious from the name).

**Warning**: The build may fail if your machine has a current MSYS from http://mingw.org  installed in `C:\MinGW`

Iā€™m unable to build bk on Windows. I have bk 7.3.2 installed from the binary download and am having trouble getting bk to work via ssh, so I was motivated to build from source so I could tweak it to show the output from the ssh session to help me debug the problem.

I understand from the post requesting testing of a 7.3 release candidate that bk cannot be built from the source tarball on Windows. My problem is I canā€™t clone the dev source repo:

C:\bk>bk clone -sALL bk://bkbits.net/u/bk/dev
Unable to connect to host 'bkbits.net'.

I have verified name resolution is giving me 216.218.207.91 for bkbits.net. Is bkd down on bkbits.net?

Yes, bkbits.net is down and I donā€™t think it is going to be restored. When I get the time and can find a Windows machine I will see if I can come up with new build instructions. No ETA at this point. :frowning:

I appreciate all the time youā€™ve volunteered to support the BitKeeper community when you have paying work to keep you busy. If thereā€™s anything you or any other former bk staff can do to get me the pieces that would have been fetched from bkbits.net to build on Windows, Iā€™d be extremely grateful. I suspect Iā€™d be able to get the ssh:// protocol working again if only I could hack away at bk on Windows.

I added win32tools to the last release on github.

That is normally unpacked in C:\build\win32tools and used by bk/src/update_buildenv to write C:\build\buildenv

You might need the buildenv to run the script so I also included buildenv.tgz which is a copy of it built and probably want was in the win32-buildenv download.

If you get in working then we should figure out how to package it. Probably a separate win32 repository on github.

Actually, I just noticed that the win32-buildenv.tgz download is on the bitkeeper.org website
https://www.bitkeeper.org/downloads/win32-buildenv.tgz

Yes, I have the buildenv from bitkeeper.org already, but when I tried to use it I was still hitting an attempt to fetch stuff from bkbits.net. Iā€™ll bang on it again soon and let you know how it goes. Thanks for the followup and pointers.

Iā€™ve spent a bit of time trying to get the build working on 64-bit Windows 10 and Windows 11 without succerss yet. In addition to trying with the 7.3.3 source tarball, Iā€™ve tried using a github clone as well as a bk repo Harlan Stenn had squirelled away apparently dating to a time when 7.3.3 sources were available from bkbits.net.

One very minor problem was that on a 64-bit Windows 10 system, the 8.3 path to the installed BitKeeper 7.3.2 was ā€œc:\progra~2\bitkee~1ā€ rather than progra~1.

If the bitkeeper.com Win7 build VM is still stored somewhere, it might be a bunch easier if the 7.3.2 working \build tree from that box were available.

I tried saving several build log by adding ā€œ| tee build_p.logā€ but sadly those logs I now see contain only ā€œsh mkconf.sh > conf.mk || { rm -f conf.mk; false; }ā€.

I do have these notes I kept on one attempt:

bk build steps on windows 10 x64

verified bk installed (/build/bk-7.3.2-wscott/src)
verified git installed (git version 2.39.1.windows.1)

from an elevated cmd:

mkdir c:\build
cd /d c:\build
git clone httpsā€¦//github.com/bitkeeper-scm/bitkeeper.git dev

from an elevated VS 2022 x64_x86 Cross Tools Command Prompt:

echo set path=%path% >p.bat
rem edited p.bat to remove the three git directories to ensure no mingw-like tools on path
p.bat
cd /d c:\build
rem re: build instructions said to use bk tar zxf ā€¦ but:
rem c:\build>bk tar zxf win32-buildenv.tgz
rem bk: ā€˜tarā€™ is not a BitKeeper command.
rem however win10 tar is fine:
tar zxf win32-buildenv.tgz
tar zxf win32tools.tgz
cd dev
buildenv

I answered no to removing buildenv.
Then from bash:

BUILD src $ ./build p | tee ā€¦/build_p.log

That reported errors and failed in case those were not significant I proceeded

BUILD src $ ./build image | tee ā€¦/build_image.log

but it errored out very similarly. Both failures seem to be because I wasnā€™t building in a bk repo.
However:

c:\build>dev\src\bk version
BitKeeper version is NOFILE for x86-win32
Built by: NOFILE in /build/dev/src
Built on: Mon Jan 30 2023 11:05:56 Eastern Standard Time (123 seconds ago)
Running on: x86-win32,Windows_10_Enterprise-64
Latest version: bk-7.3.3 (released 5 years ago)

butā€¦

c:\build>dir dev\src*.exe
Volume in drive C is OS
Volume Serial Number is B4FE-DB48

Directory of c:\build\dev\src

01/30/2023 11:05 AM 7,676,383 bk.exe
01/30/2023 11:03 AM 2,611,678 bkg.exe
01/30/2023 11:03 AM 28,160 svcmgr.exe
3 File(s) 10,316,221 bytes
0 Dir(s) 611,159,904,256 bytes free

c:\build>cd dev\src

c:\build\dev\src>bk helptool
Cannot find the graphical interpreter.

c:\build\dev\src>set path=c:\build\dev\src;%path%

c:\build\dev\src>bk helptool
Cannot find the graphical interpreter.

c:\build\dev\src>bk clone \ntp\stable\tracking \ntp\stable\tracking2
Clone file:///ntp/stable/tracking
ā†’ file:///ntp/stable/tracking2
clone 100% |==============================| OK

c:\build\dev\src>bk repocheck \ntp\stable\tracking2
Cannot find the L interpreter.

c:\build\dev\src>

Thanks for your heroic volunteer efforts keeping the bk community going. I will make some more attempts to get bk building on Windows so I can try to restore the SSH support that appears to have been intentionally disabled for Windows bk.

Iā€™ve had some further success thanks to finding a bk-bugfix repo used to build on a Debian system a few years ago that has most of the needed nested repos populated. I did need to manually add the 8.3 path to bk to my bash path since Iā€™m on 64-bit windows and itā€™s under \Program Files (x86)\BitKeeper::

BUILD src $ PATH=/c/PROGRA~2/BITKEE~1:$PATH; export PATH
BUILD src $ ./build p
[ā€¦]
here: searching bk://bkbits.net/bk/bugfixā€¦connect failure
here: searching bk://bkbits.net//repos/bk/devā€¦connect failure
here: No other sources for ./src/win32/dll/scc known
make[2]: *** [populate] Error 1
make[1]: *** [win32] Error 2
make: *** [p] Error 2
BUILD src $

If anyone has a bitkeeper repo with src/win32/dll/scc populated and can provide a tarball of that tree, that would be awfully helpful.

Along similar lines, even on Unix systems itā€™s no longer possible to bk clone a bk source repo due to lack of the BAM server at bkbits.net. I can provide details on which deltas are not populated if someone has access to a more-complete repo and is willing to help out.

1 Like

I found a virtual machine where I built Windows in the past. I donā€™t have time to mess with it much, but here is the copy of that bk/src/win32/dll directory with those submodules already populated.

I am pretty that directory just contains binaries. I am not sure where the source for those DLLs even live. Thankfully those DLLs are completely optional and you could just tell the makefiles to skip them.

Thanks, Wayne. The tree you provided allowed the build to complete. There was a ā€˜ps2pdf: not foundā€™ trying to create the refcard. Also the installer apparently invokes Windows find.exe when a GNU find is desired, causing the following chmod to fail, but it doesnā€™t seem to matter. clone, repocheck, and GUI tools all seem in order.

I donā€™t know if 7.3.3 is any better than 7.3.2 on Windows, but since there is no 7.3.3 installer on bitkeeper.org, I posted the one I built:

https://people.nwtime.org/hart/bk-7.3.3-x86-win32-setup.exe

Wayne, good news. I was able to get ssh:// URLs working again on Windows bk with minimal source changes. I disabled the --j/ā€“parallel support #ifdef WIN32 (leaving the code in, but eliminating the actually parallelization) and reverted the change to use a socketpair instead of a pipe for child process stdio introduced in the same 2014 changeset that added -j.

However I was wrong about the installer issue with the wrong ā€˜findā€™ being used. It does matter, as itā€™s a symptom of the entire gnu/bin tree being missing. Iā€™m surprised so much worked without it.

Once again, it seems like the lack of bkbits.net was the immediate stumbling block:

C:\build\bk-bugfix\src\gnu>bk here add MSYS
here: searching /net/nfs1/nfs/home/bk/bk-bugfix...repo gone
here: searching ssh://bk@nbkbits/dev-oss...ssh: Could not resolve hostname nbkbits: Name or service not known
connect failure
here: searching bk://bkbits.net/bk/bugfix...connect failure
here: searching bk://bkbits.net//repos/bk/dev...connect failure
here: No other sources for ./src/win32/msys known

C:\build\bk-bugfix\src\gnu>

If you can find time to zip up that tree from the WinXP build VM Iā€™d really appreciate it. Iā€™m sure I could hack around it by using the 7.3.2 binaries, but Iā€™m hoping to end up with a working bk source repo, not just a working bk source tree.

Harlan mentioned he has a rack in the same he.net datacenter as bkbits.net is/was. Possibly he could get it going or get a copy of the data.

Thanks again for your time.

Just a tickle to see if I can distract Wayne long enough to get some missing gnu bits of the bk source.