Help with BAM and nested repos?

While I’ve been using bk for a while, there are two features I have yet to use: nested repos, and BAM.

I now have a use-case that I think is good for these, and I’m not sure how to get it started. I’m looking for help/guidance.

We have a collection of packages that we build from source tarballs.

The bundle for a nameserver consists of scripts to build each of the tarballs:

  • bind
  • json-c
  • libidn2
  • openssl
  • ply
  • some related support scripts.

The bundle for pgsql consists of:

  • openssl
  • postgresql
  • some related support scripts.

The bundle for mariadb consists of:

  • cmake
  • jemalloc
  • mariadb
  • openssl
  • some related support scripts.

The list goes on.

The bundle for a full webserver installation has a couple hundred things in it.

For some app servers, we’ll install the webserver bundle, perhaps an SQL bundle, and then an applications bundle (eg, bugzilla).

You can see that several of these bundles will contain things like openssl, or perl, or python, or …

So it seems to me that we could create a repo called something like “app-bundles”, and that would be the collection of all of these things, as nested repos. Each bundle would be a separate top-level subdirectory of app-bundle/ .

And when we want to deploy these on a new box, we’d just check out the bundles we wanted.

I thought there might be a way to have a nested repo for openssl, etc, and then include the openssl nested repo (at a particular tag) in all of the other nested repos that needed an instance of openssl.

I’m not seeing how to do this now.

The tarballs would be where we’d use BAM.

So I’m seeing ‘bk populate’ and ‘bk partition’. I’m not understanding ‘bk portal’ yet.

I gather I’d be using ‘bk partition’ to do the initial pass at putting each bundle into its own nested repo. And I’m still going to want to see if I can have “compound nested repos”, where there’s a nested repo for openssl, and the other collections that need openssl would check out the appropriate version of openssl.

I’m also not seeing how I would spin up a new nested repo for a new bundle.

Do I need to check out the entire ‘app-bundle’ and see what I can do to make sure that checks out no default nested repo components, then add in the new stuff to the new subdirectory, and then use ‘bk partition’ to make it a new nested repo?

I hope this makes some sense…

I think I would suggest not trying it. The nested stuff is has a number of places where you can shoot yourself in the foot and our support is pretty lacking at this point. I would stick to the parts of bk that have been working well for you.