Hacker's Handbook

Getting the hang of Rebar3

Posted: 2025-01-07

Rebar3 is the standard build tool and package manager for the Erlang programming language. While the official documentation is pretty good, it can be hard for a beginner to grasp what Rebar3 is really doing at times, and why; in particular how profiles and releases and dependencies work, where the build results end up, and so on.

This guide assumes you already have installed Erlang and Rebar and you understand the basics of Erlang/OTP applications and their directory structure.

Running

Like most typical build tools, basic usage is just rebar3 <command>, with commands for instantiating a new project, compiling, running tests, building a release package, etc.

Common commands are:

• rebar3 help
• rebar3 new
• rebar3 compile
• rebar3 shell
• rebar3 release
• rebar3 eunit
• rebar3 ct

For the beginner, note that simply running erl to launch Erlang will get you an interactive Erlang shell but will not add your application to the code path, unless you do so yourself with a flag like -pa <ebin-dir>.

Instead, use rebar3 shell and Rebar3 will launch an Erlang shell with the code path set up for you. This will also launch your Erlang applications in the background according to your configuration, so that you can start interacting with them (mainly for debugging). If you want to get a shell without launching anything, use rebar3 shell --start-clean.

For details on all the commands, see the official documentation.

Configuration

Rebar3 reads the rebar.config file to know what to do. This consists of one or more plain Erlang tuples: {...}, each terminated by a full stop and newline, for example:

{erl_opts, [debug_info]}.

These may contain numbers, double-quoted strings "...", symbols ("atoms") such as erl_opts, lists [...], and other nested tuples.

Apart from compilation options and other details, the deps section of this file lists dependencies (other Erlang apps which will be fetched automatically):

{deps, [{getopt, "1.0.2"},
        {cowboy, {git, "https://github.com/ninenines/cowboy.git",
                 {tag, "2.11.0"}}},
        ... ]}.

the relx section defines how a Release (the package that you ship) is put together:

{relx, [{release, { my_release, "1.0.2"},
         [my_app1, my_app2]},
        {include_erts, true},
        ... ]}.

and the profiles section specifies the different Rebar3 Profiles:

{profiles, [{prod, [... prod-specific-options ...
                   ]},
            {test, [... test-specific-options ...
                   ]},
           ]}.

In addition, if the file rebar.config.script (an Erlang script) exists, it will be executed by Rebar3 to perform dynamic configuration.

There are many other things that can be configured. For a complete list, see the official documentation.

Generated files

Rebar3 does not write into your source directories, and instead outputs all generated files under a separate directory which by default is named _build/. It's always safe to delete the whole build directory and recompile everything.

Source files

Rebar3 expects that applications follow the standard Erlang application structure. A Rebar3 project can be either a single application with a rebar.config file in the project root directory and a src/ subdirectory, or it can consist of a collection of applications in a subdirectory named apps/ (alternatively lib/), with the main rebar.config in the root directory and each app having its own src/. Such a collection is called an "umbrella project".

An umbrella application is usually published as a Release - a complete Erlang system to run on some target machine. Often, only a top level rebar.config file is needed, but individual apps (apps/app1/, apps/app2/, ...) may have their own rebar.config files in order to use individual build options, pre- or post-build hooks, etc.

A single application can be made into a Release but it can also be published as a standalone library (that others can use as a dependency), or turned in to an escript (a standalone executable).

Releases

A release is a package that can be installed and run on a target machine, where the operator doesn't necessarily know anything about the implementation. When Rebar3 builds a release, typically using a command like

rebar3 as prod release

or

rebar3 as prod tar

it puts the files under _build/$PROFILE/rel/$RELNAME, where $PROFILE in this case would be prod (see Profiles below) and $RELNAME is taken from the configuration. A typical release specification in your rebar.config looks something like this:

{relx, [{release, { my_release, DEFAULT_VERSION_STRING },
         [app1, app2, ...]},
        {sys_config, "./config/sys.config"},
        {vm_args, "./config/vm.args"},
        {overlay, [{copy, "LICENSE" , "LICENSE"},
                   {copy, "docs/README.md", "docs/REAME.md"}
                  ]}
       ]}.

A start script bin/$RELNAME will be generated automatically, providing standard CLI commands for your release, like bin/my_release start. The listed Erlang apps [app1, app2, ...] will be included in the release package and will be launched when the script runs, using the included sys_config and vm_args configuration files.

External Dependencies

Dependencies can be specified either just by name and version, as in {deps, [{gproc, "0.9.0"},...]}, in which case they are downloaded via the Hex package manager, or as a Git URL, as in {deps, [{cowboy, {git, "https://github.com/ninenines/cowboy.git", {tag,"2.11.0"}}},...]}, in which case they are checked out and built. See Profiles below for details about where the code ends up.

Note that listing an app as a dependency does not automatically include it in the final Release package - for that to happen, it must also be included in the relx specification (see above). For instance, libraries only used for building or testing may be listed as dependencies but should not be in the release spec.

The relx section does not however need to list every app that should be included in the Release. Each individual app should contain a *.app metadata file, which lists its specific startup dependencies {applications, ...}, so if an app a declares that it has a runtime dependency on app b, and Rebar3 has been told to include a, it will automatically also include b, and so on, transitively, so that the Release package will always contain all apps required for running.

Conversely, listing an app in the release spec or a *.app file does not tell Rebar3 how to find and download that app if it is not part of your source code - all external dependencies need to be declared in the deps section.

Dependency pinning

When new dependencies have been fetched, Rebar3 updates the rebar.lock file with more exact information about the version, such as the Git hash, not just the branch or tag name used in the deps declaration. This file should typically be kept under version control to ensure repeatable builds. See the Rebar3 documentation for details.

Checkout dependencies - locally sourced apps

You can also create a subdirectory or symbolic link named _checkouts, containing apps or links to apps that you have as local files, maybe not yet published or committed, such as a library that you're currently making changes to. Apps found under _checkouts take precedence over any other apps with the same names, even if they already exist under _build.

Testing

There are two main test frameworks in Erlang: EUnit for lightweight unit tests, and Common Test which is more complex and can do system level testing. To run all EUnit tests in your applications, say rebar3 eunit. To run all tests written with Common Test, say rebar3 ct.

Rebar3 will ensure that the Erlang code path is set up to find both your code and your test suites. (You should put Common Test files in a separate test/ subdirectory).

To run the Dialyzer type analysis tool, say rebar3 dialyzer.

You can define aliases in rebar.config to simplify common tasks like running tests; for example:

{alias, [{check, [dialyzer, eunit, ct]}]}.

letting you say simply rebar3 check to run all three.

Profiles

The default profile simply means the rebar.config without any specific profile applied. This will be used when you just say e.g. rebar3 compile. To apply a profile such as prod to a command, say rebar3 as prod compile. You can use any profile names you like, but some names have special meaning to Rebar3:

• The prod profile will automatically apply the prod mode (see below).
• When running the commands rebar3 eunit or rebar3 ct, the profile named test will be automatically applied.

For example, if your tests require the meck library to run, you can add it as a dependency to only the test profile, like this:

{profiles, [{test, [{deps, [meck]}]}]}.

Where do the files go?

When Rebar3 builds things, it puts the generated files under _build/$PROFILE/. For example, Erlang apps compiled with rebar3 compile end up under _build/default/lib, but when compiled with rebar3 as prod compile the files are placed under _build/prod/lib.

External dependencies, as specified in {deps, ...}, are treated specially:

• They are always built using their individual prod profiles, no matter what profile Rebar3 has been told to use currently.
• The files are placed under _build/default/lib (rather than _build/prod/lib), because they should be available under the default profile.
• When Rebar builds other profiles than the default, it does not rebuild the external dependencies. Instead it creates symbolic links from _build/$PROFILE/lib to the already built files under _build/default/lib.

The exception is dependencies specified as part of an individual profile, as in {profiles, [{test, [{deps, [meck]}]}]}, which get stored under that profile (in this case _build/test/lib) since they should not be available under the default profile.

These locations are typically not the final destination for the compiled files. Usually, they will later get copied into a Release package under _build/$PROFILE/rel for distribution as a tarball or similar.

Modes

Modes are shortcuts for some basic settings, for example {mode, prod} sets some typical options for production. The builtin modes are:

• prod: Include the Erlang Runtime System in the release package, don't include source code, and strip any debug information. Copy files into the release package instead of using symbolic links.
• minimal: Like prod but does not include the Erlang Runtime System.
• dev: The inverse of prod.

In particular, {mode, dev} implies the {dev_mode, true} option, which creates symbolic links instead of copying files when composing a release. This means that you don't need to rebuild the release when you make a small change during development; just recompiling is enough.

Conclusion

We have gone through the most important concepts in Rebar3 and shown how they interact and where the resulting files end up. We hope this has been useful to beginners and seasoned programmers alike.