Migration from older versions¶
This page describes how to migrate existing projects to a newer version of tt
and lists breaking changes.
Contents¶
2.x -> 3.0.0¶
The main differences between tt 3.0 and tt 2.x are:
- No tt environment or
tt.yaml. An application is now a project. Binaries and installed applications live in installation scopes: project, user or system. - Commands are grouped:
tt instance,tt binary,tt package,tt dataand others. - Only Tarantool 3 is supported. Cartridge and the tarantoolctl layout are no longer supported.
Migration checklist:
- Convert the tt environment into projects, one per application.
- Replace tt 2.x commands in scripts: instance commands, targets, binaries, and building and packaging.
- If applications run under systemd, recreate their units.
- Check how scripts handle exit codes and confirmations.
The sections below start with changes that affect most users, followed by less common cases.
tt.yaml is not read¶
tt 3.0 reads no tt.yaml: neither the environment file nor the system
/etc/tarantool/tt.yaml. The following are also removed:
- The
-S(--system),-L(--local),-c(--cfg) and-s(--self) flags oftt. - The
TT_CLI_CFGandTT_SYSTEM_CONFIG_DIRenvironment variables. - The
-c(--config) flag oftt backup startandtt backup finalize. It selected thett.yamlused to resolve an<APP>:<INSTANCE>target. Use--scopeand--templateto find the application instead. - The
tt cfgandtt envcommands.
tt no longer delegates to the tt binary in bin_dir. To use the project's
tt, run source .env/activate.sh to add it to PATH.
If a command finds no project but finds tt.yaml, tt.yml or instances.yml
in the current or a parent directory, it reports a tt 2.x environment and points
to this page.
Convert an environment into projects¶
A project replaces an application in a tt environment. It is a directory
containing app.manifest.toml (created by tt new) or instance templates at
.templates/<name>/instances.yaml. tt searches upward from the current
directory for a project.
A tt environment with several applications in instances.enabled becomes one
project per application. For example, a tt 2.x environment might look like this:
<environment>/
├── tt.yaml
├── instances.available/
│ ├── app1/
│ │ ├── init.lua
│ │ └── instances.yml
│ └── app2/
└── instances.enabled/
├── app1 -> ../instances.available/app1
└── app2 -> ../instances.available/app2
Stop the application's instances with tt 2.x. Then, for each application:
- Use the real application directory, not the
instances.enabledsymlink, and runtt newthere. This directory becomes the project directory.
The application name now comes from [package].name in the manifest, not the
directory or symlink name. tt new uses the directory name only if it is a
valid package name: lowercase letters, digits and dashes. For a directory
named my_app, run tt new -n my-app. Targets and data directories will use
my-app.
-
Move the instance description and cluster configuration:
-
instances.ymlto.templates/default/instances.yaml; config.yamlto.templates/default/config.yaml.
Record names stay the same. Records with no fields, as written by tt 2.x, are
read unchanged. A Tarantool instance named in a record must be in the cluster
configuration's topology. Applications that call box.cfg themselves need
neither a topology nor config.yaml; see
box.cfg applications.
Set the restart policy in the record's restart field, for example
restart: on-failure, instead of env.restart_on_failure.
- Move the data of all instances at once, from
var/lib/<instance>to.tarantool/var/lib/<app>/<instance>. Logs, pid files and sockets now go in.tarantool/var/logand.tarantool/var/run.
Warning: While
var/libexists and.tarantool/var/libdoes not,tt instance startrefuses to start a project instance whose data would go in.tarantool/var/lib. This prevents it from starting with an empty database. Once.tarantool/var/libexists, this check is off: an instance whose data is still invar/libstarts with an empty database.
-
Install the project's binaries with
tt binary install. This replacesenv.bin_dirandenv.inc_dir: -
Versions are stored in
.tarantool/versions/<product>/<product>_<version>/. - Active versions are linked into
.tarantool/bin, and their headers into.tarantool/include/tarantool.
Without project binaries, the project uses the first tarantool on PATH.
- Remove
tt.yaml.
The resulting project:
/srv/apps/app1/
├── app.manifest.toml
├── init.lua
├── .templates/
│ └── default/
│ ├── config.yaml
│ └── instances.yaml
└── .tarantool/
├── bin/
└── var/
Replace the remaining tt.yaml settings¶
tt.yaml |
tt 3.0 |
|---|---|
env.instances_enabled |
None: one project per application, or applications installed into the user or system scope |
env.bin_dir, env.inc_dir |
bin/ and include/ in the scope selected by tt binary --scope |
env.restart_on_failure |
The instance record's restart field |
env.tarantoolctl_layout |
None: the tarantoolctl layout is not supported |
env.output_history_max |
The argument of \history in the tt connect console, such as \history 15; 10 without an argument |
app.run_dir, log_dir, wal_dir, memtx_dir, vinyl_dir |
<root>/var/{run,log,lib}/<app>/<instance>/ under the scope root; the cluster configuration can specify other paths |
modules.directory |
TT_CLI_MODULES_PATH; see External modules |
repo.rocks |
The rock servers shown by tt registry; see below |
repo.distfiles |
--local-repo DIR and --local-dir DIR of tt binary install and tt binary search |
ee.credential_path |
--credentials-file of tt binary |
templates |
Pass the path to your own template to tt create |
The behavior of repo.rocks has changed:
- In tt 2.x,
repo.rocksadded a rock server that LuaRocks queried before its default servers. - In tt 3.0,
TT_REGISTRIESor the manifest's[platform].registriesreplace the default servers for bothtt packageandtt rocks. TT_CLI_REPO_ROCKSstill adds a server thattt rocksqueries first.
See Rock servers.
The instance commands are under tt instance¶
Instance commands are grouped under tt instance. The old top-level commands
are removed:
tt 2.x |
tt 3.0 |
|---|---|
tt start, stop, restart, kill, status, logrotate, check, clean |
tt instance start, stop, ... |
tt log, tt log -f |
tt instance logs, tt instance logs --follow |
tt start -i |
tt instance start --foreground |
tt kill -f, tt clean -f |
tt instance kill --yes, tt instance clean --yes |
tt status -f json, tt status --pretty |
tt instance status -o json, tt instance status -o pretty-table |
tt connect |
tt connect or tt instance connect: the same command |
Other changes:
-fno longer means confirmation, follow or output format in these commands.- Commands prompt for confirmation only in a terminal. If a script piped
yintott stopor rantt --no-prompt stop, pass--yesinstead. Without it, the command exits with code 1. tt instance start --foregroundruns each instance under its own watchdog, as in background mode. It also writes instance output tott.logand does not restart instances. The command exits with:0if the instances stopped in time;137if an instance did not stop within itsstop_timeoutand was killed;- otherwise, the exit code of the first instance that ended on its own with another code.
- A command targeting several instances continues after an operation fails for one instance. It exits with code 3 if the command succeeded for only some instances.
A target names an instance¶
Instance commands accept these targets:
<app>:<instance>: one instance;<app>:*: all instances of an application;- a comma-separated list of these targets;
- no target: all instances of the current project.
A bare <app>, which meant all instances of the application in tt 2.x, is now
rejected. This prevents a missing instance name from accidentally stopping them
all:
tt 2.x |
tt 3.0 |
|---|---|
tt start app |
tt instance start app:*, or tt instance start in the project |
tt start app:storage-001 |
tt instance start app:storage-001 |
tt start: every application in instances.enabled |
Run tt instance start in each project, or tt instance start app1:*,app2:* for applications installed into the system scope |
tt connect app |
tt connect app:<instance> |
<app> is [package].name from the project manifest, or the project directory
name if there is no manifest. Outside a project of that name, <app> refers to
an application installed into the system scope. Use
--scope project|user|system to select the scope explicitly. See
Targets.
Removed support¶
- Tarantool 1.10 and 2.x.
tt binaryneither lists nor installs Tarantool, Tarantool Enterprise or the Enterprise SDK below version 3. - Tarantool Enterprise is now installed under its version, such as
3.8.1, rather than thegc64-andnogc64-names used in 2.x. - A previously installed Enterprise 2.x can still be listed, switched to and removed under its old name.
- The tarantoolctl layout and tarantoolctl applications.
- Cartridge:
tt cartridge,tt replicaset --cartridge, thecartridgetemplate oftt create, andtt pack --cartridge-compat.
Binaries are managed by tt binary¶
Binary management commands are grouped under tt binary:
tt 2.x |
tt 3.0 |
|---|---|
tt install tarantool 3.2.0 |
tt binary install tarantool 3.2.0 |
tt install tarantool-ee gc64-3.2.0-0-r40 |
tt binary install tarantool gc64-3.2.0-0-r40, or 3.2.0ee for the latest build of 3.2.0 |
tt install tarantool-dev ~/src/tarantool/build |
tt binary install tarantool --from-build ~/src/tarantool/build |
tt uninstall tarantool 3.2.0 |
tt binary uninstall tarantool 3.2.0 |
tt search tarantool |
tt binary search tarantool |
tt binaries list |
tt binary list |
tt binaries switch tarantool 3.2.0 |
tt binary switch tarantool 3.2.0 |
tt download <version> |
tt binary install tarantool-sdk <version>: installs the SDK rather than just downloading it |
tt install --local-repo tarantool |
tt binary install tarantool --local-repo DIR: Git sources in DIR/tarantool |
tt install --local-repo tarantool-ee |
tt binary install tarantool 3ee --local-dir DIR: SDK archives in DIR |
--local-repoused to be a switch that readrepo.distfilesfromtt.yaml. It now takes a directory path, as--local-dirdoes, relative to the current directory.- Each
tt binarycommand acts on one scope:--scope project(the default),userorsystem. It never falls back to another scope.
See Manage binaries.
tt pack, tt build and tt init are replaced by tt package¶
An application is built and packed as a package
described in app.manifest.toml. Every tt package command needs a manifest;
create one with tt new.
tt 2.x |
tt 3.0 |
|---|---|
tt init |
tt new. There is no converter for Cartridge or tarantoolctl applications |
tt build |
tt package build |
tt pack tgz, deb, rpm |
tt package pack: creates a .tt archive installed with tt package install |
tt pack --with-integrity-check KEY |
tt package pack --with-integrity-check KEY in a distribution built with integrity = "rsa"; see Sign packages |
tt pack --unit-params-file |
tt service systemd install; see systemd units |
tt pack --use-docker, --cartridge-compat |
None |
The archive includes .rocks/, .templates/, runtimes and the files listed in
[package].include.
Note: An
init.luain the project root is included only if[package].includelists it. If the cluster configuration loads it withapp.file, add it to that list or load a component's code withapp.moduleinstead.
tt run runs the Tarantool of the project that contains the current
directory. It previously read tt.yaml.
- The project root is the nearest directory above with
app.manifest.tomlor instance templates in.templates/<name>/. A package is not required. - Tarantool runs in the current directory, and paths are relative to it. For
example,
tt run ../script.luain a subdirectory runs the script from the parent directory. - Only
-hand--helpbefore the script name are options oftt run. A--before the script name ends these options and is dropped. - All arguments from the script name on pass to the script unchanged:
tt run script.lua a -- -hpassesa -- -hto the script. The tt 2.x help described--differently.
box.cfg applications are script records¶
In tt 2.x, an application that calls box.cfg itself ran from init.lua,
<instance>.init.lua or a single <name>.lua in instances.enabled. tt 3.0
does not recognize these layouts. To run a script, set the script field in an
instances.yaml record:
legacy-001:
script: init.lua # relative to the application directory
- These records are not part of the cluster configuration's topology. An
application made up only of script records needs no
config.yaml. TARANTOOL_CFG, which pointed toinstances.yml, is no longer set. Use the record'senvfield for environment variables the script needs.- For data directories and the binary port that the script does not set,
box.cfgusesTT_WAL_DIR,TT_MEMTX_DIR,TT_VINYL_DIRandTT_LISTENfrom the layout.
See Run a box.cfg application.
The system environment is the system scope¶
In tt 2.x, tt -S used the environment in /etc/tarantool/tt.yaml.
Applications lived in /etc/tarantool/instances.enabled, with data, logs and
pid files in the sys_env directories under /var/lib/tarantool,
/var/log/tarantool and /var/run/tarantool.
In tt 3.0, the system scope replaces this environment. Its root is
/opt/tarantool, or the path set by TT_SYSTEM_ROOT. It holds applications
installed from
packages:
# tt package install --scope system myapp-1.0.0-linux-amd64.tt
# tt instance start myapp:*
The layout is:
/opt/tarantool/apps/<app>: a link to the active application version;/opt/tarantool/var/{lib,log,run}/<app>/<instance>/: instance files;/opt/tarantool/etc/<app>/: this host'sconfig.yamlandinstances.yaml.
Before the first start, move each instance's data from
/var/lib/tarantool/sys_env/<app>/<instance> to
/opt/tarantool/var/lib/<app>/<instance>. Outside a project, tt does not check
the old data location.
The tt 3.0 deb and rpm packages no longer install /etc/tarantool/tt.yaml, the
sys_env directories or /etc/logrotate.d/tt.conf. A deb package upgrade
leaves the old files in place until the package is purged.
Warning: The old logrotate rule runs
tt -S logrotate, which now fails. Remove that rule. In your own rule, reopen logs withtt instance logrotate <app>:*. See Logs for an example.
systemd units come from tt service systemd¶
The units that tt 2.x tt pack added to packages ran
tt -L <environment> start as forking services. They do not work with tt 3.0.
For an application installed into the system scope,
tt service systemd install <app> creates:
- A template unit,
<app>@.service, that runs one instance withtt instance start --foreground. - A drop-in for each instance, using
restartandstop_timeoutfrom its record.
After installation, use systemctl to manage these instances.
tt instance start, stop, restart and kill refuse to manage them.
In tt 2.x, a multi-instance application also used <app>@.service, but in
/usr/lib/systemd/system. The tt 3.0 unit goes in /etc/systemd/system and
overrides the old one. Migrate in this order:
- Stop and disable the old units.
- Remove the package that installed them.
- Pack the application and install the archive into the system scope.
- Install the units with
tt service systemd install <app>.
The command supports only the system scope, not the project or user scope.
Replace --unit-params-file parameters as follows:
systemd-unit-params.yml |
tt 3.0 |
|---|---|
FdLimit |
The unit sets LimitNOFILE=65535. Set a different value in your own drop-in, <app>@.service.d/override.conf |
instance-env |
The env field of the instance record or the "*" record |
TT, ConfigPath, ExecArgs |
None: the unit runs the scope's tt on the installed application |
See Run instances under systemd.
tt cluster publish asks for a confirmation¶
tt cluster publishprompts for confirmation before writing configuration, but only in a terminal. In publishing scripts, pass--yes(-y); otherwise, the command exits with code 1 and writes nothing.--forceskips validation, not confirmation.tt cluster show --validateandtt cluster publishvalidate against the Tarantool cluster configuration JSON Schema. Validation is stricter than in tt 2.x: unknown keys at the top level and in configuration scopes are rejected. Errors use the format<path> [code] <message>.
Removed commands¶
tt 2.x |
tt 3.0 |
|---|---|
tt env, . <(tt env) |
source .env/activate.sh. The first tt binary install in a project creates this file |
tt cfg dump |
None. tt registry list shows the rock servers |
tt enable |
None: projects are found from the current directory, and tt package install installs applications |
tt instances |
tt instance status for instances, tt package list --scope system for installed applications |
tt cartridge |
None |
tt create¶
tt createneeds nott.yaml.- It creates an application in the current directory or under
--dst, without a symlink ininstances.enabled. - Specify your own template by its directory or
.tgzarchive path, for examplett create ./templates/basic. - The
rundirtemplate variable is removed. - Built-in templates place
instances.yamlandconfig.yamlin.templates/default/.single_instanceuses.yamlrather than.yml. Thecartridgetemplate is removed. - Templates that create
app.manifest.tomlreject names that are invalid for a package. For example,vshard_clusterconverts--name my_apptomy-appbut rejectsMy_App.
Exit codes¶
In tt 2.x, all errors except those from tt backup and tt restore returned
code 1. In tt 3.0, the codes are:
| Code | Meaning |
|---|---|
1 |
The request cannot be carried out as given: a usage error, a missing file or project, or a required confirmation without a terminal |
2 |
A failure of the system where the command ran: an unreachable or unresponsive server, a hostname that cannot be resolved, or a filesystem operation denied due to permissions or insufficient space. For example, tt connect to a port with no listener now exits with 2 rather than 1 |
3 |
A command targeting several resources succeeded for only some of them |
If a script checked for code 1, check for any nonzero code instead. tt backup
and tt restore keep their own codes from 3 upward. See tt help errors for
all codes.
tt backup-id is tt backup id¶
The command that prints an identifier for a new backup has moved under
tt backup. In backup scripts, replace id=$(tt backup-id) with
id=$(tt backup id). The identifier format is unchanged.
External modules are found by TT_CLI_MODULES_PATH¶
- External modules are found only in the colon-separated directories in
TT_CLI_MODULES_PATH. Relative paths are resolved from the working directory. modules.directoryfromtt.yamland the defaultmodulesdirectory next to it are no longer searched. A module left there will not be found, and the command it replaced will run as a built-in tt command again.- A module named after a command group, such as
cluster, replaces the whole group:tt cluster publish ...runs the module with argumentspublish .... tt modules,tt help,tt completionandtt versioncannot be replaced.-Iruns the built-in tt command even if a module with the same name exists.
See External modules.
tt daemon runs the commands of tt 3.0¶
A request to tt daemon must now name a tt 3.0 command. For example, a request
for tt start test_app changes as follows:
- tt 2.x:
{"command_name":"start", "params":["test_app"]}; - tt 3.0:
{"command_name":"instance", "params":["start", "test_app:*"]}.
Global tt 2.x flags such as -L are removed. Pass --yes for commands that
require confirmation: the daemon runs commands without a terminal. See
Working with tt daemon.
Building tt and importing its packages¶
TT_CLI_BUILD_SSL accepts default, openssl and openssl_static. The values
no, shared and static are rejected:
tt 2.x |
tt 3.0 |
|---|---|
Unset or no: no TLS |
Unset or default: TLS in pure Go, without cgo |
shared |
openssl |
static |
openssl_static |
For Go code that imports tt:
- The module path is
github.com/tarantool/tt/v3. lib/cluster,lib/connect,lib/dialandlib/integrityhave moved togithub.com/tarantool/tt/sdk/cluster,sdk/connect,sdk/dialandsdk/integrity. Each is a separate module with an alpha API.InjectedCmdsfromcli/cmdis removed. A distribution adds commands as tt modules and selects its integrity provider withcore.WithIntegrity. See Distributions and modules.
tt cat and tt play are tt data cat and tt data play¶
tt cat and tt play are removed without aliases. Use tt data cat and
tt data play instead. The new commands read files in Go, so no local Tarantool
is needed. Flag names are retained, but their behavior has changed:
tt 2.x |
tt 3.0 |
|---|---|
--format yaml\|json\|lua |
-o/--format. json is JSON Lines: one object per line |
--to N excludes N |
--to N includes N, as does --from N |
--from/--to compare one LSN with every replica's |
Bound one replica's LSNs: select it with --replica, or use the only replica in the files. For several replicas, use --from-vclock/--to-vclock |
| A bound may split a transaction | A transaction within the bounds is taken whole |
All .snap and .xlog files in a directory, snapshots first |
The newest snapshot within the bounds, then logs in vclock order, excluding data already in the snapshot |
--show-system filters data changes only |
Also hides RAFT, synchronous replication and NOP rows |
--timestamp: Unix seconds or RFC 3339 |
Also 2024-11-14 17:02:36.8181, 2024-11-14 17:02 or 2024-11-14, in local time unless a zone follows |
play: one request per row, no transactions |
One IPROTO stream, with --batch requests in flight; supports --txn-mode, --on-error and --dry-run |
Additional changes:
- If
--fromor--from-vclockis set, the snapshot is not read. Snapshot row numbers are not LSNs, so bounds apply only to write-ahead logs (WAL). catoutput retains each row'sHEADERandBODY. RAFT and synchronous replication row bodies are now decoded.- A snapshot contains no vinyl space data; that data lives in
.runfiles.tt data catreads them when explicitly named or found with-r.tt data playrejects them. tt data playexits with code 3 if the target rejected some transactions, or 2 if the connection failed.tt coredumphas not moved into the group.
1.3.1 -> 2.0.0¶
New format of tt config file¶
The tt 2.0.0 configuration file format is incompatible with previous versions:
- The root
ttsection is removed. - Common environment settings have moved to the
envsection. All relative paths in this section are resolved from the configuration file's directory. - Relative paths in the
appsection are resolved from the application directory.
New runtime artifacts layout¶
The runtime artifacts layout has changed:
- Relative paths are resolved from the application directory. For an instance
defined by a single script, a directory is created in
instances.enabled. - Since relative paths already contain the application name, only the instance name is appended. The default local environment layout in 2.0.0 is:
instances.enabled/app/
├── init.lua
├── instances.yml
└── var
├── lib
│ ├── inst1
│ └── inst2
├── log
│ ├── inst1
│ └── inst2
└── run
├── inst1
└── inst2
To move artifacts from version 1.*:
- Create the artifact directories in the application directory: var/lib, var/log and var/run.
- Copy the instance subdirectories from the 1.* environment into the application
directory. For example, to copy data:
cp -r <env_dir>/var/lib/app/* <instances_enabled>/app/var/lib/
These changes do not affect absolute paths: the application name is always appended to them.
Working directory is changed¶
The instance process now uses the application directory as its working directory, rather than tt's current directory. Move or copy files that the instance accesses by relative paths into the application directory.