Nix package for BitKeeper


#1

I have been playing with getting BitKeeper supported on Nix, but
suddenly find that things are getting very busy. :wink:

If someone wants to mess with Nix here is a package I started:

{stdenv, fetchurl, perl, gperf, bison, groff,
 pkgconfig, libXft, fontconfig}:

stdenv.mkDerivation rec {
  name = "bitkeeper-7.2ce";

  src = fetchurl {
    url = "http://www.bitkeeper.org/downloads/7.2ce/bk-7.2ce.tar.gz";
    md5 = "ba76ad934d7d647a1714cd8b24e72ae3";
  };

  buildInputs = [ perl gperf bison groff libXft pkgconfig ];

  buildPhase = ''
    cd src
    make -j6 p
    make image
  '';

  installPhase = ''
    ./utils/bk-* $out/bitkeeper
    mkdir -p $out/bin
    $out/bitkeeper/bk links $out/bin
    chmod g-w $out
  '';

  meta = {
    description = "BitKeeper";
    longDescription = ''
      Long BitKeeper desc
    '';
    homepage = http://www.bitkeeper.org/;
    license = stdenv.lib.licenses.asl20;
    platforms = with stdenv.lib.platforms; all;
    maintainers = [ #Add your name here!
      stdenv.lib.maintainers.wscott
    ];
  };
}

This builds something that looks like this:

$ ls -l /nix/store/pqq0n30vychsy17b7py3d13z5gzyjj6f-bitkeeper-7.2ce
total 8K
dr-xr-xr-x 2 root root 4096 Dec 31  1969 bin
dr-xr-xr-x 8 root root 4096 Dec 31  1969 bitkeeper
$ ls -l /nix/store/pqq0n30vychsy17b7py3d13z5gzyjj6f-bitkeeper-7.2ce/bin
total 4K
lrwxrwxrwx 2 root root 72 Dec 31  1969 bk -> /nix/store/pqq0n30vychsy17b7py3d13z5gzyjj6f-bitkeeper-7.2ce/bitkeeper/bk
$ ls -l /nix/store/pqq0n30vychsy17b7py3d13z5gzyjj6f-bitkeeper-7.2ce/bitkeeper/
total 3852K
-r-xr-xr-x   6 root root      1489 Dec 31  1969 applypatch
-r-xr-xr-x   6 root root       696 Dec 31  1969 b64wrap
-r-xr-xr-x   2 root root   2556888 Dec 31  1969 bk
-r--r--r--   5 root root    921738 Dec 31  1969 bkhelp.txt
...

But it doesn’t actually work in practice because of how bk works. Normally /usr/bin/bk is a symlink to something like /opt/bitkeeper/bk where the package was installed. The code in src/port/platform.c tries to find the pathname to that directory for use internally by looking up ARGV[0] on the user’s PATH and then following the symlink. However in Nix that symlink is actually a series of links. And the readlink() in src/port/platforminit.c doesn’t handle that case.

We probably need to fix bk to handle that case. Or perhaps replace /nix/store/HASH-bitkeeper/bin/bk with a shell script that just calls ../bitkeeper/bk.


#2

I pushed a fix for that symlink issue to bk:// bkbits.net/u/bk/dev

So now the above form should work. But unfortunately, at the moment our bkweb code doesn’t support downloading a tarball of a certain REV directly so I can’t update the recipe to use the new code.

I hope to release bk-7.3 soon based on that tree and then this nixpkg should work.


#3

I’m a NixOS maintainer and developer, so I’m willing to upstream a working package expression, FWIW. It’ll need to build cleanly on NixOS as well - have you been testing on NixOS, or just on Nix-on-some-other-Linux? (NixOS has enhanced chroot security features that can cause builds to fail sometimes, if unaccounted for.)

Also, there should be no need to wait for 7.3 if the symlink fix is all that’s really preventing things from working. We run into similar upstream code a lot, and have a tool called wrapProgram that can just do exactly what you say: sit in bin, and do nothing but run exec /path/to/bk $@. This is very often needed for things like Java programs; here’s an example in a package I maintain:

Alternatively, if you have a unified diff, you can ship .patch files to be used by the build system, as a last resort, by specifying patches = [ ./fix-symlink-resolution.patch ]; inside the Nix file. This is how we backport needed fixes from upstreams.


#4

Here is the patch to fix symlinks. We just need that simple one-line change.

http://bkbits.net/u/bk/dev/?PAGE=patch&REV=57728090MNpxizRfc3Eo79B8cW87cw

I only run the nix package manager from Linux, but I have done builds in a chroot before to make sure they are clean. If you would be willing to maintain a valid bk package, that would be good for me.


#5

OK I think I have this one.

The bk-7.3ce released will be soon. That one will includes code to use the system’s zlib, lz4, pcre, tomcrypt & tommath libraries (if we can find them). (Thanks @calhariz for the initial push there.)

Here is the nix bk package I am currently working on:

{stdenv, fetchurl, perl, gperf, bison, groff,
 pkgconfig, libXft, fontconfig, pcre, libtomcrypt, libtommath, lz4, zlib}:

stdenv.mkDerivation rec {
  name = "bitkeeper-7.3ce";

  src = fetchurl {
    url = "http://work.bitkeeper.com/~wscott/bk-20160707164603.tar.gz";
    md5 = "379ab9e9092e4a3a42542c765677e484";
  };

  # need 'pcre' here so pcre-config is on PATH for Tcl build
  buildInputs = [ perl gperf bison groff libXft pkgconfig pcre ];

  # setup env vars for conf.mk below
  inherit pcre libtomcrypt libtommath lz4 zlib;

  buildPhase = ''
    cd src
    cat <<EOF > conf.mk
PCRE_SYSTEM=1
export PCRE_SYSTEM
PCRE_CFLAGS=-I$pcre/include
export PCRE_CFLAGS
PCRE_LDFLAGS=`$pcre/bin/pcre-config --libs`
export PCRE_LDFLAGS
TOMCRYPT_CFLAGS=-I$libtomcrypt/include -I$libtommath/include
export TOMCRYPT_CFLAGS
TOMCRYPT_LDFLAGS=-L$libtomcrypt/lib -ltomcrypt -L$libtommath/lib -ltommath
export TOMCRYPT_LDFLAGS
LZ4_SYSTEM=1
export LZ4_SYSTEM
LZ4_CFLAGS=-I$lz4/include
export LZ4_CFLAGS
LZ4_LDFLAGS=-L$lz4/lib -llz4
export LZ4_LDFLAGS
CC=gcc -DHAVE_GMTOFF
LD=gcc
XLIBS=
RANLIB=ranlib
EOF
    make -j6 V=1 p
    make image
  '';

  installPhase = ''
    ./utils/bk-* $out/bitkeeper
    mkdir -p $out/bin
    $out/bitkeeper/bk links $out/bin
    chmod g-w $out
  '';

  meta = {
    description = "BitKeeper";
    longDescription = ''
      Long BitKeeper desc
    '';
    homepage = http://www.bitkeeper.org/;
    license = stdenv.lib.licenses.asl20;
    platforms = with stdenv.lib.platforms; all;
    maintainers = [
      stdenv.lib.maintainers.wscott
    ];
  };
}

It builds a correct tree that seems to work fine. It also demonstrates how the new version will support the ability to override the locations of the libraries. That file I am overwriting is normally auto-generated, but it has very simplistic logic to search for system libraries. For something like Nix is was easier to just write the correct file.

Once we release bk-7.3 I will make a pull request to nixpkgs to include this.

$ ldd /nix/store/sfy1njd9r7ah7k066acxn0dwk75jgs92-bitkeeper-7.3ce/bin/bk 
    linux-vdso.so.1 =>  (0x00007fff4efbd000)
    libtomcrypt.so.0 => /nix/store/xww89c44kw8ff2576184w1ksky7ccmrn-libtomcrypt-1.17/lib/libtomcrypt.so.0 (0x00007f6a53f80000)
    libtommath.so.1 => /nix/store/hwsma8cxb3zk9vzclzdkay6zl114m86n-libtommath-1.0/lib/libtommath.so.1 (0x00007f6a53d66000)
    libpcre.so.1 => /nix/store/13cbz82vfdvy60vx6ghnjr7057wq1bdx-pcre-8.38/lib/libpcre.so.1 (0x00007f6a53af8000)
    liblz4.so.1 => /nix/store/k9jk5xljv8yxh4ybwlw2mfs9w6aqfy31-lz4-131/lib/liblz4.so.1 (0x00007f6a538e1000)
    libz.so.1 => /nix/store/h6a5qnn3alf2xq0k7d1hzqqbss9d028x-zlib-1.2.8/lib/libz.so.1 (0x00007f6a536c7000)
    libc.so.6 => /nix/store/763vw2zyc9434dj7rw6c2kczj0blvssy-glibc-2.23/lib/libc.so.6 (0x00007f6a53326000)
    libpthread.so.0 => /nix/store/763vw2zyc9434dj7rw6c2kczj0blvssy-glibc-2.23/lib/libpthread.so.0 (0x00007f6a53109000)
    /nix/store/763vw2zyc9434dj7rw6c2kczj0blvssy-glibc-2.23/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x000055637c08f000)

#6

With the bk-7.3 release that whole ugly block in the middle that writes conf.mk is gone and it still builds correctly. We look run pkg-config to see about library locations and nix is helpful enough to keep those updated.

However, the GUI tools from this build don’t work. When bk runs our local wish it hangs when trying to open the first X window. I will need to figure out what is happening there. Nevermind, user error. Works fine.


#7

I have a pull request for my Nix package.

Waiting on feedback.


#8

I’ve upstreamed this as three separate commits:

These were tested on my NixOS server. Hydra should build binaries soon, meaning you’ll get binary artifacts quickly enough (hopefully).

I also lightly modified the expression to be somewhat more aesthetically pleasing, and added myself to the maintainer group.