Skip to content

Rock servers

The dependencies of a package are rocks, which come from rock servers: an HTTP(S) LuaRocks server, or a local directory with a LuaRocks manifest. Either may publish a rock index beside the manifest or instead of it.

Which servers

The list of servers is taken from the first of these layers that names any:

  1. --registry, which can be repeated, of the command;
  2. TT_REGISTRIES, comma-separated;
  3. [platform].registries of app.manifest.toml;
  4. the built-in defaults: https://rocks.tarantool.org/ and https://luarocks.org/.

The layers replace one another rather than merge: resolution takes a rock from the first server that has it, so a server appended to the defaults would change where a rock comes from. A relative directory is taken from the current directory in TT_REGISTRIES and from the manifest's directory in the manifest.

tt registry list prints the list a command would use, and the layer of each server:

$ tt registry list
URL                           SOURCE
https://rocks.tarantool.org/  default
https://luarocks.org/         default

A dependency may name a server of its own: crud = { version = "1.5.2", registry = "https://my.mirror/" }. The rock is then resolved from that server and installed from it, whatever the list is; the rocks it depends on come from the list. A directory there is taken from the manifest's directory, as in [platform].registries, whichever directory tt runs in.

A rock index

A server may publish a tt-rocks-index.json where its LuaRocks manifest would be. The index names every file of a rock version by its URL, with its sha256: the rock of each platform, and the rockspec apart from them. The files may be elsewhere, on release pages for instance, while the index is a file in git:

{
  "format": 1,
  "repository": {
    "emmylua": {
      "0.30.0pre1-1": {
        "artifacts": [
          {
            "arch": "linux-x86_64",
            "sha256": "…",
            "size": 19777374,
            "url": "https://github.com/tarantool-contrib/rocks-repo/releases/download/emmylua-0.30.0pre1-1/emmylua-0.30.0pre1-1.linux-x86_64.rock"
          }
        ],
        "rockspec": {
          "sha256": "…",
          "url": "emmylua-0.30.0pre1-1.rockspec"
        }
      }
    }
  }
}

A url is absolute https, or relative to the directory of the index, and its last segment is the name of the file: <name>-<version>.<arch>.rock or <name>-<version>.rockspec. A field tt does not know is ignored; an index of a newer format is an error that asks to upgrade tt.

A version may be withdrawn with "yanked": true. Resolving afresh passes it over - tt package add, tt package update, a first build - while a lock that pins it keeps resolving to it and installs it: holding the versions a lock has is what every command but tt package update does. A constraint only a yanked version satisfies fails with a note to pin it with ==.

tt reads the index before any manifest, from an HTTP server and from a directory alike. When it is there, it is the whole catalogue of the server; when it is there and does not read, the server is in error rather than read through its manifests. From such a server:

  • resolving reads the rockspec of a version, checked against its sha256, and downloads no rock;
  • the lock pins the sha256 of every file of the version, under artifacts:
[[lock.products.default.dependencies]]
name = 'emmylua'
version = '0.30.0pre1-1'
source = 'registry'
artifacts = {linux-x86_64 = 'sha256:…', macosx-aarch64 = 'sha256:…', rockspec = 'sha256:…'}
  • every install of the rock - by tt package build, tt package fetch and tt test, and by tt package install of an archive packed --without-deps, from the lock of the archive - checks the file of its platform against those digests before it unpacks it, whichever server serves it. A file that does not match is not installed. A platform the lock pins no file for is an error too: tt package resolve pins the files the server serves now. A source rock (.src.rock) is the one exception: when the lock pins the rockspec of the version and no source rock, as it does for a version the server served as a bare rockspec, a source rock installs if the rockspec it bundles has the pinned sha256. The sources in it are checked against the sha256 the serving index lists, as the sources a bare rockspec names were against nothing.

Writing an index

tt writes the index of every directory it makes a rock server of, beside the LuaRocks manifests and listing the same rocks: tt package pack --out, tt package download --dir and tt wasm rock --out. A project that depends on a rock from such a directory pins the sha256 of its files in the lock. tt package index DIR writes it for a directory filled otherwise, with the rocks the jobs of a CI matrix packed, say:

$ tt package index dist
.../dist/tt-rocks-index.json
indexed 2 version(s) of .../dist

It lists every <name>-<version>.<arch>.rock and <name>-<version>.rockspec of DIR at a URL relative to the index, its name, with its sha256 and size, and writes the LuaRocks manifests too, so DIR serves LuaRocks as well. Since tt reads the index alone when there is one, a rock copied into such a directory by hand is served once the index is written again.

An index kept apart from the rocks, a file in a git branch of its own with the rocks on release pages, is named with --index, and the URL the files are downloaded from with --base-url:

$ gh release upload v1.0.0 dist/*.rock dist/*.rockspec
$ tt package index dist --index index/tt-rocks-index.json \
    --base-url https://github.com/example/mylib/releases/download/v1.0.0

Without --base-url the URLs are relative to the index, which DIR is then below: tt package index site/v1.0.0 --index site/tt-rocks-index.json names the files v1.0.0/<file>. With either flag, only the index is written.

An index that is there is updated, not replaced. The versions of DIR are listed anew from its files, and one marked "yanked": true stays so, which is how a version is yanked: by hand, in the index. A version the index lists with files of DIR that are gone is dropped; one it lists with files elsewhere, those of another release say, is kept. A version of DIR the index lists with files elsewhere is an error: a published version does not change, so a new build of it is a new revision (tt package pack --revision). The same index is written as the same bytes, its keys sorted, so a commit of it shows what changed; a field tt does not know is not kept. A rock index for release pages walks through both, an index beside the rocks and one apart from them, for a module and the project that depends on it.

Searching

tt package search TERM asks every server for rocks whose name contains TERM and lists every version each one offers. Unlike resolution, it does not stop at the first server.

A mirror for offline builds

tt package download --dir DIR downloads rocks into DIR and indexes them, with the LuaRocks manifests and a rock index, so that DIR is a rock server itself. Without arguments it mirrors the locked dependencies of the project at their exact versions, dev ones included; with names it downloads those.

$ tt package download --dir ./vendor-rocks
$ TT_REGISTRIES=./vendor-rocks tt package build

Running it again extends the mirror in place. A dependency that does not come from a rock server, a path one, has no rock to mirror and is skipped with a warning.

Each version is downloaded as the server offers it best: the rock of this platform or an .all.rock, else its source rock, else its bare rockspec. A rockspec names its sources by source.url, a git repository most often, which a build would fetch, so download packs them into the source rock luarocks pack <rockspec> makes, <name>-<version>.src.rock, and puts it beside the rockspec: the rockspec and the sources, a git repository cloned at the tag or branch the rockspec pins. The download fails when the sources cannot be fetched. A build from the mirror takes the source rock and fetches nothing:

$ tt package download --dir ../mirror
.../mirror/checks-3.4.1-1.rockspec
.../mirror/checks-3.4.1-1.src.rock
indexed 2 rock(s) in .../mirror
$ TT_REGISTRIES=$PWD/../mirror tt package build --locked

Packing the same sources again gives the same bytes, so a lock resolved against the mirror keeps matching it. A rock whose build downloads something itself, a CMake build that fetches a dependency, say, still needs the network: the mirror holds what the rockspec names, not what its build does.

The components of the project are another matter: what the build of a component downloads itself, download downloads too, when its backend can say what that is, into a directory of the backend in DIR. The crates the Cargo.lock of a cargo component pins go into DIR/cargo, and a build whose servers have DIR builds them offline, as Build a module in Rust shows. The components of the path dependencies of the lock are downloaded so too.

tt wasm rock packs a server of the same kind for the wasm rock of tarantool-wasm: the rock of every platform of a release, with its index; see Run WebAssembly components.

tt rocks

tt rocks runs LuaRocks itself, for the commands tt package has no equivalent of. It queries the servers of TT_REGISTRIES or of the manifest instead of its own defaults when either is set. A rock with a C part is built against the headers of the Tarantool that runs it: those of the project's .tarantool/include/tarantool, of TT_CLI_TARANTOOL_PREFIX, or the first found beside its executable, then under the install prefix it was built with (see Headers for rocks with a C part).

LuaRocks reads manifests only, not a rock index. A server that publishes nothing but the index is left out of those servers, and of TT_CLI_REPO_ROCKS, with a warning; one named with --server or --only-server is an error. A directory is such a server when it holds tt-rocks-index.json and no manifest; an HTTP(S) server when a HEAD of tt-rocks-index.json succeeds and one of each manifest answers 403, 404 or 410. A server that does not answer is kept, for LuaRocks to report on.

LuaRocks builds nothing of the source rock of a tt package, --format rock-src: its rockspec is that of the binary rocks, of build type none, so LuaRocks installs it with no module in it, only the rockspec and the license files in doc/; of a package with no [package].repository, it fails to unpack the rock instead. tt rocks fails a command that installed such a rock, install and build say, and removes the rock from the tree again, so that it satisfies no dependency on it; install the binary rock of the platform instead, or depend on the rock in app.manifest.toml and run tt package build, which builds the source rock with its manifest. LuaRocks has printed that the rock is installed by then, and the error of tt, which follows, says it is not. A tree the command made, which holds no rock once the source rock is gone, is removed whole, its manifest and lock file with it. The check covers the tree of the working directory or the one --tree names. Plain luarocks does the same install with no such check: it says the rock is installed, and require of its module fails. What LuaRocks installs of each rock tt package pack makes is in For LuaRocks users.