Conan recipe builder pattern

Conan recipe builder pattern

I work a lot with the conan package manager. They have a huge library of already created conan recipes in their conan-center index which have many builds for common compilers (thanks JFrog for the CI power). But sometimes you need a different compiler version, a different compiler or different options for your conan recipe build. Thanks to conan package tools and conan itself, this is really easy to do. Because I have done that for a lot of recipes, I’ve called this technic conan recipe builder pattern. The nice thing about that technic is that you can use normal CI jobs to generate your conan recipe builds without writing your own conanfile and just use an existing recipe. This technic can be used for any conan repo, not just the files from the conan-center index itself.

Building zlib

For example we could build “our” version of zlib. We just create a normal git repo like a normal conan recipe repo, but we just need a build.py file and no conanfile.py, because we will fetch the file from the conan-center index master branch. Install conan and the conan package tools via pip install conan conan-package-tools if not already done.

Here is the build.py file for the zlib builder:

import os
import shutil
from cpt.packager import ConanMultiPackager
from conans import tools

def create_clean_build_folder(folder):
    if os.path.exists(folder):
        shutil.rmtree(folder)
    os.makedirs(folder)

if __name__ == "__main__":
    url = "https://github.com/conan-io/conan-center-index/archive/master.zip"
    recipe_folder = "conan-center-index-master/recipes/zlib/1.2.8"
    build_folder = "_build"
    reference="zlib/1.2.8@"

    create_clean_build_folder(build_folder)

    # download the conan index as zip and extract to a sub directory
    tools.get(
        url,
        filename="conan-index.zip",
        destination=build_folder,
        pattern=os.path.join(recipe_folder, "*"), # extract just zlib/1.2.8
        retry=2,
        retry_wait=5
    )
    conanfile = os.path.join(build_folder, recipe_folder, "conanfile.py")

    # normal cpt build, but with the downloaded conanfile
    builder = ConanMultiPackager(conanfile=conanfile, reference=reference)
    builder.add_common_builds()
    builder.run()

It looks like a normal build.py file but we’ve download the conanfile.py from the index. We just create a local directory (_build) and download and unzip the file via tools.get (using the pattern variable to just export the zlib/1.2.8 subdirectory). This is a method from conan which normally will be used in a conanfile.py file, but conan is a normal python package, so we can use it outside of a conanfile.py.

At the end we just make a normal conan package tools build, but we define the conanfile parameter (pointing to the downloaded conanfile from the conan-center index). You can use any parameter as defined on the conan package tools repo and the conan recipe itself.

Conclusion

As you can see it is really easy to compile custom builds from existing conan recipes. Combine this technic with your CI tool of choice and you get automatic builds too.