Skip to content

Link C libraries into a module

A c or lua-c component compiles C and C++ into a native module of Tarantool, and links it with the libraries it needs. A library of the system the module finds on every machine it is installed on, as it finds the C library; any other one goes with the module, linked into it or copied beside it. This page builds two packages that do both: one with zlib, which pkg-config describes, and one with a C library the package builds itself.

The fields are those of the c and lua-c backends:

Field What it does
libraries, library_dirs the libraries the module links, -l<name> or a file, and where the linker looks for them
pkg_config packages whose flags of the compiler and of the linker pkg-config gives
link the components of the package whose libraries the module links
bundle the libraries whose shared libraries go beside the module
rpath where the loader looks for the libraries of the module, $ORIGIN its directory
static true: the module needs nothing of the machine, each library linked from its .a

The commands below ran on Ubuntu with gcc 15, zlib 1.3.1, CMake 4.2 and Tarantool 3.8; the paths of the project are shortened to ....

A library pkg-config knows

zcrc/app.manifest.toml, a package of one C module, zcrc.crc:

manifest_version = "0.1"

[package]
name = "zcrc"

[platform]
tarantool = ">=3.0.0"
tt = ">=3.0.0"

# The C module zcrc.crc: the CRC-32 of zlib.
[components.zcrc]
path = "zcrc"

[components.zcrc.build]
backend = "lua-c"
module = "crc"
sources = ["crc.c"]
pkg_config = ["zlib >= 1.2"]

zcrc/zcrc/crc.c:

#include <lua.h>
#include <lauxlib.h>
#include <zlib.h>

/* crc(s): the CRC-32 of the string s, as zlib computes it. */
static int
crc(lua_State *L)
{
    size_t len;
    const char *s = luaL_checklstring(L, 1, &len);
    lua_pushinteger(L, crc32(0, (const Bytef *)s, len));
    return 1;
}

/* version(): the version of the zlib the module runs with. */
static int
version(lua_State *L)
{
    lua_pushstring(L, zlibVersion());
    return 1;
}

int
luaopen_zcrc_crc(lua_State *L)
{
    lua_newtable(L);
    lua_pushcfunction(L, crc);
    lua_setfield(L, -2, "crc");
    lua_pushcfunction(L, version);
    lua_setfield(L, -2, "version");
    return 1;
}

pkg_config asks pkg-config for the flags of zlib, --cflags for every compile and --libs for the link, so the block names neither the headers nor the library:

$ cd zcrc
$ tt package build
$ tt run -e "local z = require('zcrc.crc') print(z.crc('hello'), z.version()) os.exit()"
907060870   1.3.1
$ readelf -d .rocks/lib/tarantool/zcrc/crc.so | grep -E 'NEEDED|RUNPATH'
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

The module needs libz.so.1 of the machine it runs on. A package other machines install, whose module is to load wherever Tarantool runs, says static = true, and its build links each library from its static library. On Debian and Ubuntu libz.a is not built with -fPIC, which a module, a shared object, needs, and the link says so:

$ echo 'static = true' >> app.manifest.toml
$ tt package build
   ⨯ building component "zcrc": linking: exit status 1
     cc -shared -Wl,-Bsymbolic-functions -o .../zcrc/_build/rock/default/lib/zcrc/crc.so .../zcrc/_build/component/zcrc/obj/crc.c.de186406.o /usr/lib/x86_64-linux-gnu/libz.a
     /usr/bin/x86_64-linux-gnu-ld.bfd: /usr/lib/x86_64-linux-gnu/libz.a(zutil.o): warning: relocation against `z_errmsg' in read-only section `.text'
     /usr/bin/x86_64-linux-gnu-ld.bfd: /usr/lib/x86_64-linux-gnu/libz.a(zutil.o): relocation R_X86_64_PC32 against symbol `z_errmsg' can not be used when making a shared object; recompile with -fPIC
     /usr/bin/x86_64-linux-gnu-ld.bfd: final link failed: bad value
     collect2: error: ld returned 1 exit status
     hint: the static library /usr/lib/x86_64-linux-gnu/libz.a was not built with -fPIC: a module is a shared object, which takes position-independent code only: build the library with -fPIC (CFLAGS=-fPIC, or -DCMAKE_POSITION_INDEPENDENT_CODE=ON of cmake), or link its shared library

The build took /usr/lib/x86_64-linux-gnu/libz.a for -lz: pkg-config was asked with --static, and the compiler named the static library, cc -print-file-name=libz.a. Where libz.a is built with -fPIC - a zlib the package builds itself, below - the module links it in and needs libc.so.6 alone. Here the shared library goes with the module instead.

Ship a shared library with the module

bundle copies the shared library of a library the module links beside it, under the name the module asks the loader for, and has the module look for it there, by an rpath of its own directory, $ORIGIN:

$ echo 'bundle = ["z"]' >> app.manifest.toml
$ tt package build
$ ls .rocks/lib/tarantool/zcrc
crc.so
libz.so.1
$ readelf -d .rocks/lib/tarantool/zcrc/crc.so | grep -E 'NEEDED|RUNPATH'
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN]

libz.so.1 is /usr/lib/x86_64-linux-gnu/libz.so.1.3.1, which libz.so of the system links to, copied under its SONAME. The build with static = true passes: the module needs libz.so.1, which it finds beside itself, and libz.so.1 needs libc.so.6 alone. A library the bundled one needs is not bundled with it: tt names it, a warning without static = true, an error with it, and bundle lists it too.

The copy is a module of the package, so it goes wherever the module goes. Packed and installed into another directory, the module loads, with no LD_LIBRARY_PATH:

$ echo 1.0.0 > VERSION
$ tt package pack --without-deps
.../zcrc/_build/pack/zcrc-1.0.0-linux-amd64.tt
$ mkdir ../app && cd ../app
$ tt package install ../zcrc/_build/pack/zcrc-1.0.0-linux-amd64.tt
installed zcrc 1.0.0 into project scope
$ find .rocks/lib -type f | sort
.rocks/lib/tarantool/zcrc/crc.so
.rocks/lib/tarantool/zcrc/libz.so.1
$ tarantool -e "print(require('zcrc.crc').crc('hello')) os.exit()"
907060870

bundle is of lua-c only. Tarantool loads a module of c, of box.schema.func, from a copy in a temporary directory, where nothing is beside it; such a module links its libraries in statically.

On macOS the copy gets the install name @rpath/<name>, with install_name_tool, and is signed again ad hoc, with codesign, before the module is linked against it with -Wl,-rpath,@loader_path.

A library the package builds

A C library the package carries in its sources is built by a component of its own, with the tool of its project, and the module links what that component made. hashes/app.manifest.toml:

manifest_version = "0.1"

[package]
name = "hashes"

[platform]
tarantool = ">=3.0.0"
tt = ">=3.0.0"

[products.default]
components = ["fnv", "hashes"]
default = true

# libfnv.a and fnv.h, for the module below to link; not in the package.
[components.fnv]
path = "fnv"

[components.fnv.build]
backend = "cmake"
output = [
    { path = "libfnv.a", to = "link" },
    { path = "fnv.h", to = "link" },
]

# The C module hashes.fnv, linked with libfnv.a.
[components.hashes]
path = "lua"

[components.hashes.build]
backend = "lua-c"
module = "fnv"
sources = ["fnv_lua.c"]
link = ["fnv"]

hashes/fnv/CMakeLists.txt, a static library of position-independent code, and its header in the build tree:

cmake_minimum_required(VERSION 3.15)
project(fnv C)
add_library(fnv STATIC fnv.c)
set_target_properties(fnv PROPERTIES POSITION_INDEPENDENT_CODE ON)
configure_file(fnv.h fnv.h COPYONLY)

hashes/fnv/fnv.h:

#include <stddef.h>
#include <stdint.h>

uint32_t fnv1a(const char *s, size_t len);

hashes/fnv/fnv.c:

#include "fnv.h"

uint32_t
fnv1a(const char *s, size_t len)
{
    uint32_t h = 2166136261u;
    for (size_t i = 0; i < len; i++)
        h = (h ^ (unsigned char)s[i]) * 16777619u;
    return h;
}

hashes/lua/fnv_lua.c, which includes the header of the library as a header of the system:

#include <lua.h>
#include <lauxlib.h>
#include <fnv.h>

static int
hash(lua_State *L)
{
    size_t len;
    const char *s = luaL_checklstring(L, 1, &len);
    lua_pushinteger(L, fnv1a(s, len));
    return 1;
}

int
luaopen_hashes_fnv(lua_State *L)
{
    lua_pushcfunction(L, hash);
    return 1;
}

to = "link" sends a file of output to _build/link/<product>/<component>/, a header to include/ of it: a library for the components built after it, which is neither installed nor packed. link = ["fnv"] gives the compiles of the module -I of that include/ and its link -L of the directory and -lfnv:

$ cd ../hashes
$ tt package build
$ find _build/link -type f | sort
_build/link/default/fnv/include/fnv.h
_build/link/default/fnv/libfnv.a
$ find .rocks/lib -type f | sort
.rocks/lib/tarantool/hashes/fnv.so
$ tt run -e "print(require('hashes.fnv')('hello')) os.exit()"
1335831723

The product builds fnv before hashes, in the order of its components; a product that builds it after is an error of the manifest:

$ sed -i 's/components = \["fnv", "hashes"\]/components = ["hashes", "fnv"]/' app.manifest.toml
$ tt package build
   ⨯ invalid manifest: products.default.components: component "hashes" links the libraries of component "fnv", and the product builds it after "hashes": list "fnv" before "hashes"

A manifest without [products] builds its components in the order of their names. A build of one component takes what the components of its link made in an earlier build, and fails when there is none yet:

$ sed -i 's/components = \["hashes", "fnv"\]/components = ["fnv", "hashes"]/' app.manifest.toml
$ rm -rf _build .rocks
$ tt package build hashes
   ⨯ building component "hashes": link[0]: component "fnv" has not been built: build it first, as a build of the whole product does before "hashes"
$ tt package build fnv
$ tt package build hashes

The same works with make and shell, whose output is relative to the working directory of the build. A component of link that makes a shared library, libfnv.so with to = "link", is linked with it, and bundle = ["fnv"] ships it beside the module, as libz.so.1 above.

Where the loader looks

library_dirs is where the linker looks for libraries. It is not where the loader looks when Tarantool loads the module: a directory of the build machine is not on the machine the package is installed on. rpath says where the loader looks, each entry -Wl,-rpath,<entry>: $ORIGIN is the directory of the module (@loader_path on macOS), so rpath = ["$ORIGIN/../vendor"] finds a library installed beside the directory of the module, and an absolute entry is a directory of the machine the module is installed on, which it is to have.