Unable to build bitkeeper on FreeBSD 10

I am trying to build bitkeeper on FreeBSD 10 and after installing security/libtomcrypt port, I get the error message:

sh mkconf.sh > conf.mk || { rm -f conf.mk; false; }
TOMCRYPT required to build bk
Makefile:46: conf.mk: No such file or directory
gmake[1]: *** [Makefile:662: conf.mk] Error 1

Running

pkg info libtomcrypt

come back with:

libtomcrypt-1.17_4
Name           : libtomcrypt
Version        : 1.17_4
Installed on   : Thu Jul 14 22:42:35 2016 CDT
Origin         : security/libtomcrypt
Architecture   : freebsd:10:x86:64
Prefix         : /usr/local
Categories     : security
Licenses       : PD
Maintainer     : ports@FreeBSD.org
WWW            : http://libtom.net/
Comment        : Comprehensive, modular, and portable cryptographic toolkit
Options        :
	DOCS           : on
	GMP            : off
	LIBTOMMATH     : on
	TOMSFASTMATH   : off
Shared Libs provided:
	libtomcrypt_prof.so.0
	libtomcrypt.so.0
Annotations    :
Flat size      : 2.89MiB
Description    :
LibTomCrypt is a portable ANSI C cryptographic library that supports symmetric
ciphers, one-way hashes, pseudo-random number generators, public key
cryptography (via RSA,DH or ECC/DH) and a plethora of support routines.

WWW: http://libtom.net/

OK. I probably need to update that error message.

I didn’t write a full autoconf replacement that searched every possible location. I just checked to see if it is in the default search path for the compiler and if pkg-config can find it.

If that search doesn’t work you need to configure it manually.

$ cp conf.mk.local.template conf.mk.local
$ vi conf.mk.local 
... now uncomment the TOMCRYPT lines and fix them up ...

BTW building from a bk repository would have worked because it would have noticed it didn’t have a tomcrypt and populated our tomcrypt component and used that.

Thanks @wscott. Something interesting is that running

pkg config libtomcrypt

or

pkg config tomcrypt

comes back with

pkg: No such configuration options:

However if I use the info command from pkg I can get the files that are installed with libtomcrypt:

pkg info -l libtomcrypt libtomcrypt-1.17_4: /usr/local/include/tomcrypt.h /usr/local/include/tomcrypt_argchk.h /usr/local/include/tomcrypt_cfg.h /usr/local/include/tomcrypt_cipher.h /usr/local/include/tomcrypt_custom.h /usr/local/include/tomcrypt_hash.h /usr/local/include/tomcrypt_mac.h /usr/local/include/tomcrypt_macros.h /usr/local/include/tomcrypt_math.h /usr/local/include/tomcrypt_misc.h /usr/local/include/tomcrypt_pk.h /usr/local/include/tomcrypt_pkcs.h /usr/local/include/tomcrypt_prng.h /usr/local/include/tomcrypt_test.h /usr/local/lib/libtomcrypt.a /usr/local/lib/libtomcrypt.so /usr/local/lib/libtomcrypt.so.0 /usr/local/lib/libtomcrypt.so.0.0.117 /usr/local/lib/libtomcrypt_prof.a /usr/local/lib/libtomcrypt_prof.so /usr/local/lib/libtomcrypt_prof.so.0 /usr/local/lib/libtomcrypt_prof.so.0.0.117 /usr/local/share/doc/libtomcrypt/pdf/crypt.pdf /usr/local/share/licenses/libtomcrypt-1.17_4/LICENSE /usr/local/share/licenses/libtomcrypt-1.17_4/PD /usr/local/share/licenses/libtomcrypt-1.17_4/catalog.mk

I believe Package Next Generation (pkgng) was first added in FreeBSD 9

That is ‘pkg-config’ it probably isn’t installed on FreeBSD.

Did you try what I suggested with creating a conf.mk.local file uses the conf.mk.local.template and setting the defaults for TOMCRYPT? I am pretty sure it will work.

Thanks @wscott I’ll give that a try this weekend. I’m looking at creating a FreeBSD port of bitkeeper so I might have more questions as I get further along

Ok… got a little further… the port pulls down the src and installs/configures dependencies and blows up with this output:

===> License APACHE20 accepted by the user ===> bitkeeper-7.3ce depends on file: /usr/local/sbin/pkg - found ===> Fetching all distfiles required by bitkeeper-7.3ce for building ===> Extracting for bitkeeper-7.3ce => SHA256 Checksum OK for bk-7.3ce.tar.gz. ===> Patching for bitkeeper-7.3ce ===> bitkeeper-7.3ce depends on executable: bash - found ===> bitkeeper-7.3ce depends on executable: gmake - found ===> bitkeeper-7.3ce depends on package: perl5>=5.20<5.21 - found ===> bitkeeper-7.3ce depends on shared library: libtommath.so - found (/usr/local/lib/libtommath.so) ===> bitkeeper-7.3ce depends on shared library: libpcre.so - found (/usr/local/lib/libpcre.so) ===> bitkeeper-7.3ce depends on shared library: liblz4.so - found (/usr/local/lib/liblz4.so) ===> bitkeeper-7.3ce depends on shared library: libtomcrypt.so - found (/usr/local/lib/libtomcrypt.so) ===> Configuring for bitkeeper-7.3ce ===> Building for bitkeeper-7.3ce gmake[1]: Entering directory '/usr/ports/devel/bitkeeper/work/bk-7.3ce' gmake -C src all gmake[2]: Entering directory '/usr/ports/devel/bitkeeper/work/bk-7.3ce/src' sh mkconf.sh > conf.mk || { rm -f conf.mk; false; } Building cmd.h gmake[3]: Entering directory '/usr/ports/devel/bitkeeper/work/bk-7.3ce/src/libc' perl fslayer/gen_fslayer.pl cd string && perl mk_str_cfg.pl Died at mk_str_cfg.pl line 40. gmake[3]: *** [string/Makefile:27: string/str.cfg] Error 2 gmake[3]: Leaving directory '/usr/ports/devel/bitkeeper/work/bk-7.3ce/src/libc' gmake[2]: *** [Makefile:438: libc/libc.a] Error 2 gmake[2]: *** Waiting for unfinished jobs.... gmake[2]: Leaving directory '/usr/ports/devel/bitkeeper/work/bk-7.3ce/src' gmake[1]: *** [Makefile:16: all] Error 2 gmake[1]: Leaving directory '/usr/ports/devel/bitkeeper/work/bk-7.3ce' ===> Compilation failed unexpectedly. Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer. *** Error code 1 Stop. make: stopped in /usr/ports/devel/bitkeeper

It looks like mk_str_cfg.pl is expecting the system to have gcc installed (clang is now the default compiler for FreeBSD). Where can I override the compiler setting that somewhere since bitkeeper builds with clang on OSX?

Well, to build the package, just edit that file and keep moving. When you get done we can look at the diffs you needed to use and figure out how to generalize the code.

You might be able to use just ‘cc’ which is kinda generic now. But that code is depending on a certain gcc warning.

Or install ‘gcc’ run it, and save str.cfg. Then just have the package pluck down that str.cfg before running. (It will probably be empty)

So an update… the basics of bitkeeper compiles but dies when building tommath:

/usr/ports/devel/bitkeeper/work/bk-7.3ce/src/tommath/bn_mp_grow.c:35:20: error: unexpected type name 'mp_digit': expected expression tmp = OPT_CAST(mp_digit) XREALLOC (a->dp, sizeof (mp_digit) * size);

During compilation there are errors stating that the product root cannot be found ( which is mostly a path issue I am guessing with the ports system).

The prelim patches are in the files directory with the patch-directory_directory_file pattern

Is there a place where I can upload my port and get your feedback on how to continue?

Have you modified anything? I’m assuming yes because you want to send us something. If so, I’d just send is a diff -Nur patch (send to dev@bitkeeper.com).

Also is there any place that has a freebsd 10 VM that I can just rent for a while and try and debug this?
Or do you know if the same problem would happen on freebsd8? We have that in our cluster.

Hi @mcvoy Digital Ocean now supports FreeBSD droplets. I have sent the patch to dev@bitkeeper.com

We’ve just put together a FreeBSD10 vm for ourselves and I have a dumb question…

What’s the relationship between pkg and ports?

We have verified that if you copy $BKSRC/src/conf.mk.local.template to $BKSRC/src/conf.mk.local and uncomment the lines for tomcrypt and tommath our build works without problem.

On a clean FreeBSD10 VM, I had to additionally pkg install:

  • libXft
  • gcc
  • gperf
  • bison
  • flex
  • perl5
  • gmake
  • pcre
  • liblz4
  • libtomcrypt
  • libtommath
  • pkgconf

My conf.mk.local looks like this:

# Copyright 2016 BitMover, Inc
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#-----------------------------------------------------------------------

# When bk cannot locate the needed libraries they can be configured
# manually by copying this file to conf.mk.local uncommenting/editing
# some of the options below


# PCRE_CPPFLAGS="-I/usr/local/include"
# PCRE_LDFLAGS="-L/usr/local/lib -lpcre"

TOMCRYPT_CPPFLAGS="-I/usr/local/include"
TOMCRYPT_LDFLAGS="-L/usr/local/lib -ltomcrypt"

TOMMATH_CPPFLAGS="-I/usr/local/include"
TOMMATH_LDFLAGS="-L/usr/local/lib -ltommath"

#LZ4_CPPFLAGS="-I/usr/local/include"
#LZ4_LDFLAGS="-L/usr/local/lib -llz4"

At this point, you can simply run gmake p in $BKSRC/src.

Not exactly sure what more is required to turn this into a port/pkg/whatever but shouldn’t be too hard from here.

@hasanihunter fyi, I am looking at the patch you sent in to see a way to make a port/pkg (I’m also attempting to make a macport)

Thanks George.

The pkg infrastructure is more like apt-get as it only deals with the compiled binaries and the ports way allows you to build from source. There are times where I might mix packages and ports (if I need to compile with specific options for example but don’t want to compile everything else)

The patch I emailed does create the conf.mk.local in $BKSRC. Since make then calls gmake, I think some assumptions about the user’s current working directory might not necessarily be true with building from ports rather than typing gmake at the command line

Right, found the appropriate info about pkg vs ports. Learning quickly.

One difference that I haven’t had a chance to dig into yet is that you’re using clang and I used gcc.

So those files you sent, would ultimately live in /usr/ports/devel/bitkeeper (or something like that)?

Yes in /usr/ports/devel/bitkeeper. I’m trying to use clang since it replaced gcc in the base system a few years ago

Another update, I was able to fully build with clang (fresh tree, gmake CC=clang p). Lunching and will start seriously digging into your port files next.

The recent work by @georgn was pushed to bk://bkbits.net/u/dev

This improves building on freebsd10:

  • building with cd src && ./build p will automatically use gmake instead of make if it is seen on the path
  • the build works without gcc installed

I’ll take a look this weekend… thanks