Requirements ------------ - C++14 compatible compiler - [CMake](http://www.cmake.org) - [GMP (GNU multiprecision library)](http://gmplib.org/) Platform-Specific Setup ----------------------- - [Linux (Ubuntu)](ubuntu.md) - [Windows (msys2)](msys2.md) - [Windows (Visual Studio)](msvc.md) - [macOS (homebrew)](osx-10.9.md) - Linux/macOS/WSL via [Nix](https://nixos.org/nix/): Call `nix-shell` in the project root. That's it. - There is also an [**experimental** setup based purely on Nix](nix.md) that works fundamentally differently from the make/CMake setup described on this page. Generic Build Instructions -------------------------- Setting up a basic release build: ```bash git clone https://github.com/leanprover/lean4 cd lean4 mkdir -p build/release cd build/release cmake ../.. make ``` Note: that if you have a CPU with lots of cores you will get a faster build if you specify the number of parallel jobs using the `-j n` option on make. For example, on an AMD Ryzen 9 `make` takes 00:04:55, whereas `make -j 10` takes 00:01:38. Your results may vary depending on the speed of your hard drive. Setting up a basic debug build: ```bash git clone https://github.com/leanprover/lean4 cd lean4 mkdir -p build/debug cd build/debug cmake -D CMAKE_BUILD_TYPE=DEBUG ../.. make ``` This will compile the Lean library and binary into the `stage1` subfolder; see below for details. Add `-jN` for an appropriate `N` to `make` for a parallel build. To install the build see [Dev setup using elan](index.md#dev-setup-using-elan) below. Useful CMake Configuration Settings ----------------------------------- Pass these along with the `cmake ../..` command. * `-D CMAKE_BUILD_TYPE=`\ Select the build type. Valid values are `RELEASE` (default), `DEBUG`, `RELWITHDEBINFO`, and `MINSIZEREL`. * `-D CMAKE_C_COMPILER=`\ `-D CMAKE_CXX_COMPILER=`\ Select the C/C++ compilers to use. Official Lean releases currently use Clang; see also `.github/workflows/ci.yml` for the CI config. Lean will automatically use [CCache](https://ccache.dev/) if available to avoid redundant builds, especially after stage 0 has been updated (see below). Lean Build Pipeline ------------------- Since version 4, Lean is a partially bootstrapped program: most parts of the frontend and compiler are written in Lean itself and thus need to be built before building Lean itself - which is needed to again build those parts. This cycle is broken by using pre-built C files checked into the repository (which ultimately go back to a point where the Lean compiler was not written in Lean) in place of these Lean inputs and then compiling everything in multiple stages up to a fixed point. The build directory is organized in these stages: ```bash stage0/ # Bootstrap binary built from stage0/src/. # We do not use any other files from this directory in further stages. bin/lean stage1/ include/ config.h # config variables used to build `lean` such as use allocator runtime/lean.h # runtime headers, used by extracted C code, uses `config.h` share/lean/ Makefile # used by `leanmake` lib/ lean/**/*.olean # the Lean library (incl. the compiler) compiled by the previous stage's `lean` temp/**/*.{c,o} # the library extracted to C and compiled by `leanc` libInit.a libStd.a libLean.a # static libraries of the Lean library libleancpp.a # a static library of the C++ sources of Lean bin/ lean # the Lean compiler & server linked together from the above libraries leanc # a wrapper around a C compiler supplying search paths etc leanmake # a wrapper around `make` supplying the Makefile above stage2/... stage3/... ``` Stage 0 can be viewed as a blackbox since it does not depend on any local changes and is equivalent to downloading a bootstrapping binary as done in other compilers. The build for any other stage starts by building the runtime and standard library from `src/`, using the `lean` binary from the previous stage in the latter case, which are then assembled into a new `bin/lean` binary. Each stage can be built by calling `make stageN` in the root build folder. Running just `make` will default to stage 1, which is usually sufficient for testing changes on the test suite or other files outside of the stdlib. However, it might happen that the stage 1 compiler is not able to load its own stdlib, e.g. when changing the .olean format: the stage 1 stdlib will use the format generated by the stage 0 compiler, but the stage 1 compiler will expect the new format. In this case, we should continue with building and testing stage 2 instead, which will both build and expect the new format. Note that this is only possible because when building a stage's stdlib, we use the previous compiler but never load the previous stdlib (since everything is `prelude`). We can also use stage 2 to test changes in the compiler or other "meta" parts, i.e. changes that affect the produced (.olean or .c) code, on the stdlib and compiler itself. We are not aware of any "meta-meta" parts that influence more than two stages of compilation, so stage 3 should always be identical to stage 2 and only exists as a sanity check. In summary, doing a standard build via `make` involves these steps: 1. compile the `stage0/src` archived sources into `stage0/bin/lean` 1. use it to compile the current library (*including* your changes) into `stage1/lib` 1. link that and the current C++ code from `src/` into `stage1/bin/lean` You now have a Lean binary and library that include your changes, though their own compilation was not influenced by them, that you can use to test your changes on test programs whose compilation *will* be influenced by the changes. Finally, when we want to use new language features in the library, we need to update the stage 0 compiler, which can be done via `make -C stageN update-stage0`. `make update-stage0` without `-C` defaults to stage1. ### Further Bootstrapping Complications As written above, changes in meta code in the current stage usually will only affect later stages. This is an issue in two specific cases. * For *non-builtin* meta code such as `notation`s or `macro`s in `Notation.lean`, we expect changes to affect the current file and all later files of the same stage immediately, just like outside the stdlib. To ensure this, we need to build the stage using `-Dinterpreter.prefer_native=false` - otherwise, when executing a macro, the interpreter would notice that there is already a native symbol available for this function and run it instead of the new IR, but the symbol is from the previous stage! To make matters more complicated, while `false` is a reasonable default incurring only minor overhead (`ParserDescr`s and simple macros are cheap to interpret), there are situations where we *need* to set the option to `true`: when the interpreter is executed from the native code of the previous stage, the type of the value it computes must be identical to/ABI-compatible with the type in the previous stage. For example, if we add a new parameter to `Macro` or reorder constructors in `ParserDescr`, building the stage with the interpreter will likely fail. Thus we need to set `interpreter.prefer_native` to `true` in such cases to "freeze" meta code at their versions in the previous stage; no new meta code should be introduced in this stage. Any further stages (e.g. after an `update-stage0`) will then need to be compiled with the flag set to `false` again since they will expect the new signature. For an example, see https://github.com/leanprover/lean4/commit/da4c46370d85add64ef7ca5e7cc4638b62823fbb. * For the special case of *quotations*, it is desirable to have changes in built-in parsers affect them immediately: when the changes in the parser become active in the next stage, macros implemented via quotations should generate syntax trees compatible with the new parser, and quotation patterns in macro and elaborators should be able to match syntax created by the new parser and macros. Since quotations capture the syntax tree structure during execution of the current stage and turn it into code for the next stage, we need to run the current stage's built-in parsers in quotation via the interpreter for this to work. Caveats: * Since interpreting full parsers is not nearly as cheap and we rarely change built-in syntax, this needs to be opted in using `-Dinternal.parseQuotWithCurrentStage=true`. * The parser needs to be reachable via an `import` statement, otherwise the version of the previous stage will silently be used. * Only the parser code (`Parser.fn`) is affected; all metadata such as leading tokens is taken from the previous stage. For an example, see https://github.com/leanprover/lean4/commit/f9dcbbddc48ccab22c7674ba20c5f409823b4cc1#diff-371387aed38bb02bf7761084fd9460e4168ae16d1ffe5de041b47d3ad2d22422 (from before the flag defaulted to `false`). To modify either of these flags both for building and editing the stdlib, adjust the code in `stage0/src/stdlib_flags.h`. The flags will automatically be reset on the next `update-stage0` when the file is overwritten with the original version in `src/`. Development Setup ----------------- After building a stage, you can invoke `make -C stageN test` (or, even better, `make -C stageN test ARGS=-jN` to make `ctest` parallel) to run the Lean test suite. `make test` without `-C` defaults to stage1. While the Lean tests will automatically use that stage's corresponding Lean executables, for running tests or compiling Lean programs manually, you need to put them into your `PATH` yourself. A simple option for doing that is to use [`elan`](https://github.com/leanprover/elan), see the next section. You can use any of the [supported editors](../setup.md) for editing the Lean source code. If you set up `elan` as below, opening `src/` as a *workspace folder* should ensure that stage 0 will be used for file in that directory. You should also set the `LEAN_SRC_PATH` environment variable to the path of the `src/` directory to enable go-to-definition in the stdlib (automatically set when using `nix-shell`). Dev setup using elan -------------------- You can use [`elan`](https://github.com/leanprover/elan) to easily switch between stages and build configurations based on the current directory, both for the `lean/leanc/leanmake` binaries in your shell's PATH and inside your editor. If you haven't already installed elan, you can do so, without installing a default version of Lean, using ```bash curl https://raw.githubusercontent.com/leanprover/elan/master/elan-init.sh -sSf | sh -s -- --default-toolchain none ``` You can use `elan toolchain link` to give a specific stage build directory a reference name, then use `elan override set` to associate such a name to the current directory. We usually want to use `stage0` for editing files in `src` and `stage1` for everything else (e.g. tests). ``` # in the Lean rootdir elan toolchain link lean4 build/release/stage1 elan toolchain link lean4-stage0 build/release/stage0 # make `lean` etc. point to stage1 in the rootdir and subdirs elan override set lean4 cd src # make `lean` etc. point to stage0 anywhere inside `src` elan override set lean4-stage0 ``` You can also use the `+toolchain` shorthand (e.g. `lean +lean4-debug`) to switch toolchains on the spot. `lean4-mode` will automatically use the `lean` executable associated with the directory of the current file as long as `lean4-rootdir` is unset and `~/.elan/bin` is in your `exec-path`. Where Emacs sources the `exec-path` from can be a bit unclear depending on your configuration, so alternatively you can also set `lean4-rootdir` to `"~/.elan"` explicitly. You might find that debugging through elan, e.g. via `gdb lean`, disables some things like symbol autocompletion because at first only the elan proxy binary is loaded. You can instead pass the explicit path to `bin/lean` in your build folder to gdb, or use `gdb $(elan which lean)`. Troubleshooting --------------- * Call `make` with an additional `VERBOSE=1` argument to print executed commands.