diff --git a/doc/SUMMARY.md b/doc/SUMMARY.md index 9c6b306b92..f2bc424987 100644 --- a/doc/SUMMARY.md +++ b/doc/SUMMARY.md @@ -55,12 +55,16 @@ # Development -- [Commit Convention](./commit_convention.md) +- [Development Guide](./dev/index.md) +- [Commit Convention](./dev/commit_convention.md) - [Building Lean](./make/index.md) - [Ubuntu Setup](./make/ubuntu-16.04.md) - [macOS Setup](./make/osx-10.9.md) - - [Windows Setup](./make/msys2.md) + - [Windows MSYS2 Setup](./make/msys2.md) + - [Windows with WSL](./make/wsl2.md) - [Nix Setup (*Experimental*)](./make/nix.md) -- [Building This Manual](./mdbook.md) -- [Fixing Tests](./fixing_tests.md) -- [Debugging](./debugging.md) +- [Unit Testing](./dev/testing.md) +- [Building This Manual](./dev/mdbook.md) +- [Fixing Tests](./dev/fixing_tests.md) +- [Debugging](./dev/debugging.md) +- [C++ Coding Style](./dev/cpp_coding_style.md) \ No newline at end of file diff --git a/doc/coding_style.md b/doc/coding_style.md deleted file mode 100644 index 774afebd10..0000000000 --- a/doc/coding_style.md +++ /dev/null @@ -1,244 +0,0 @@ -Coding Style -============ - -[C++11](http://en.wikipedia.org/wiki/C%2B%2B11) features --------------------------------------------------------- - -We make extensive use of new features in the C++ 11 standard. -Developers must be familiar with the standard to be able to understand -the code. -Here are some of the features that are extensively used. - -- Type inference (aka `auto` keyword). -- Initializer lists. -- Lambda functions and expressions. -- `nullptr` constant. -- Strongly typed enumerations. -- Right angle bracket. -- Thread local storage. -- Threading facilities. -- Tuple types. - -Comments --------- - -The comments in the Lean codebase contain -[Doxygen](http://www.stack.nl/~dimitri/doxygen/) commands. -Doxygen is the de facto standard tool for generating documentation from -annotated C++ sources. - -Namespaces ----------- - -All code is in the `lean` namespace. Each frontend is stored in a -separate nested namespace. For example, the SMT 2.0 frontend is stored -in the `lean::smt` namespace. - -Exception: some debugging functions are stored outside of the `lean` -namespace. These functions are called `print` and are meant to be used -when debugging Lean using `gdb`. - -Smart pointers --------------- - -We only use `std::shared_ptr` template for class `C` only if we expect -to create only a few objects (< 1000) of class `C`. Otherwise, we -implement our own intrusive smart pointer. For example, the class -`expr` is an intrusive smart pointer to `expr_cell`. We may have -millions of `expr` objects. We say it is intrusive because the -reference counter is stored in `expr_cell`. - -We use `std::unique_ptr` to make sure unique resources will be freed -correctly. - -Template --------- -We organize template source code using the approach described at http://www.codeproject.com/Articles/3515/How-To-Organize-Template-Source-Code - -Idioms ------- - -We use some popular C++ idioms: - -- [Pimpl](http://c2.com/cgi/wiki?PimplIdiom) -- [RAII](http://en.wikipedia.org/wiki/Resource_Acquisition_Is_Initialization) Resource Acquisition Is Initialization - -Formatting ----------- - -* We use 4 spaces for indentation. - -* Class, method, and function names are lower case -We use `_` for composite names. Example: `type_checker`. - -* Class/struct fields should start with the prefix `m_`. - -Example: - - class point { - int m_x; - int m_y; - public: - ... - }; - -* We do **not** use the `#ifndef-#define-#endif` idiom for header files. -Instead we use `#pragma once`. - -* We write `type const & v` instead of `const type & v`. - -* We use `const` extensively. - -* `if-then-else` - -The following forms are acceptable: - - if (cond) { - ... - } else { - ... - } - -and - - if (cond) - statement1; - else - statement2; - -In *exceptional cases*, we also use - - if (cond) statement; - -and - - if (cond) statement1; else stament2; - -* `if-then-else-if-else` - -The following forms are acceptable: - - if (cond) { - ... - } else if (cond) { - ... - } else { - ... - } - -and - - if (cond) - statement1; - else if (cond) - statement2; - else - statement3; - -* We frequently format code using extra spaces - -For example, we write - - environment const & m_env; - cache m_cache; - normalizer m_normalizer; - volatile bool m_interrupted; - -instead of - - environment const & m_env; - cache m_cache; - normalizer m_normalizer; - volatile bool m_interrupted; - -* We use the macro `lean_assert` for assertions. -The macro `lean_assert` is extensively used when writing unit tests. - -* Spaces in expressions -We write `a == b` instead of `a==b`. -Similarly, we write `x < y + 1` instead of `x``. If you need to cast within a class - hierarchy, use ``static_cast<>`` to upcast. Google doesn't support - RTTI. - - "public:" should be preceded by a blank line - - Missing space before ``{`` - - Found C system header after C++ system header. Should be: - environment.h, c system, c++ system, other. - - Labels should always be indented at least one space. If this is - a member-initializer list in a constructor or the base class list in - a class definition, the colon should be on the following line. - - You don't need a ``;`` after a ``}`` - - No ``#ifndef`` header guard found - - Streams are highly discouraged. - - Extra space before ``(`` in function call - - Else clause should never be on same line as else - - Extra space before ``)`` - - Is this a non-const reference? If so, make const or use a pointer. - - All parameters should be named in a function - - Do not use anonymous constructor notation (e.g., ``{a1, ..., an}``) - -Modified Features: - - - Add ``#include `` for ``list<>`` - - => *Check* ``std::list`` instead of ``list`` because we do have our own ``lean::list`` type. - - - Add ``#include `` for copy - - => *Check* ``std::copy`` instead of ``copy`` because we do have our own ``lean::copy`` method. - - - Do not use namespace using-directives. Use using-declarations instead. - - => *Allow* this if filename contains "tests/" - - - Small and focused functions are preferred: foo() - has xxx non-comment lines (error triggered by exceeding 500 lines). - - => *Allow* this if filename contains "tests/" - - - Include the directory when naming .h files - - => *Allow* this if the included filename is "version.h" which is generated by cmake. - -[google-style]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml -[cpplint]: /src/cmake/Modules/cpplint.py - -Git pre-push hook ------------------ - -Since [git 1.8.2][git-pre-push-hook], git introduced *pre-push* hook -which is executed *before* actual push operation is performed. Using this, -we can *automatically* run the style checker over the changed files *before* -we push commits to repositories. This is useful because it prevents us -from accidentally pushing the commits which contain style problems. - -[git-pre-push-hook]: https://github.com/git/git/blob/master/Documentation/RelNotes/1.8.2.txt - -To activate the hook, execute - -```bash -ln -s ../../script/pre-push .git/hooks -``` - -in the project root directory. - -You can change the ``CHECKER`` variable in the script if you want to use other -checkers. diff --git a/doc/dev/bootstrap.md b/doc/dev/bootstrap.md new file mode 100644 index 0000000000..f4fcb24bc3 --- /dev/null +++ b/doc/dev/bootstrap.md @@ -0,0 +1,122 @@ + +# Lean Build Bootstrapping + +Since version 4, Lean is a partially bootstrapped program: most parts of the +frontend and compiler are written in Lean itself and thus need to be built before +building Lean itself - which is needed to again build those parts. This cycle is +broken by using pre-built C files checked into the repository (which ultimately +go back to a point where the Lean compiler was not written in Lean) in place of +these Lean inputs and then compiling everything in multiple stages up to a fixed +point. The build directory is organized in these stages: + +```bash +stage0/ + # Bootstrap binary built from stage0/src/. + # We do not use any other files from this directory in further stages. + bin/lean +stage1/ + include/ + config.h # config variables used to build `lean` such as use allocator + runtime/lean.h # runtime headers, used by extracted C code, uses `config.h` + share/lean/ + Makefile # used by `leanmake` + lib/ + lean/**/*.olean # the Lean library (incl. the compiler) compiled by the previous stage's `lean` + temp/**/*.{c,o} # the library extracted to C and compiled by `leanc` + libInit.a libStd.a libLean.a # static libraries of the Lean library + libleancpp.a # a static library of the C++ sources of Lean + bin/ + lean # the Lean compiler & server linked together from the above libraries + leanc # a wrapper around a C compiler supplying search paths etc + leanmake # a wrapper around `make` supplying the Makefile above +stage2/... +stage3/... +``` + +Stage 0 can be viewed as a blackbox since it does not depend on any local +changes and is equivalent to downloading a bootstrapping binary as done in other +compilers. The build for any other stage starts by building the runtime and +standard library from `src/`, using the `lean` binary from the previous stage in +the latter case, which are then assembled into a new `bin/lean` binary. + +Each stage can be built by calling `make stageN` in the root build folder. +Running just `make` will default to stage 1, which is usually sufficient for +testing changes on the test suite or other files outside of the stdlib. However, +it might happen that the stage 1 compiler is not able to load its own stdlib, +e.g. when changing the .olean format: the stage 1 stdlib will use the format +generated by the stage 0 compiler, but the stage 1 compiler will expect the new +format. In this case, we should continue with building and testing stage 2 +instead, which will both build and expect the new format. Note that this is only +possible because when building a stage's stdlib, we use the previous compiler +but never load the previous stdlib (since everything is `prelude`). We can also +use stage 2 to test changes in the compiler or other "meta" parts, i.e. changes +that affect the produced (.olean or .c) code, on the stdlib and compiler itself. +We are not aware of any "meta-meta" parts that influence more than two stages of +compilation, so stage 3 should always be identical to stage 2 and only exists as +a sanity check. + +In summary, doing a standard build via `make` involves these steps: + +1. compile the `stage0/src` archived sources into `stage0/bin/lean` +1. use it to compile the current library (*including* your changes) into `stage1/lib` +1. link that and the current C++ code from `src/` into `stage1/bin/lean` + +You now have a Lean binary and library that include your changes, though their +own compilation was not influenced by them, that you can use to test your +changes on test programs whose compilation *will* be influenced by the changes. + +Finally, when we want to use new language features in the library, we need to +update the stage 0 compiler, which can be done via `make -C stageN update-stage0`. +`make update-stage0` without `-C` defaults to stage1. + +## Further Bootstrapping Complications + +As written above, changes in meta code in the current stage usually will only +affect later stages. This is an issue in two specific cases. + +* For *non-builtin* meta code such as `notation`s or `macro`s in + `Notation.lean`, we expect changes to affect the current file and all later + files of the same stage immediately, just like outside the stdlib. To ensure + this, we need to build the stage using `-Dinterpreter.prefer_native=false` - + otherwise, when executing a macro, the interpreter would notice that there is + already a native symbol available for this function and run it instead of the + new IR, but the symbol is from the previous stage! + + To make matters more complicated, while `false` is a reasonable default + incurring only minor overhead (`ParserDescr`s and simple macros are cheap to + interpret), there are situations where we *need* to set the option to `true`: + when the interpreter is executed from the native code of the previous stage, + the type of the value it computes must be identical to/ABI-compatible with the + type in the previous stage. For example, if we add a new parameter to `Macro` + or reorder constructors in `ParserDescr`, building the stage with the + interpreter will likely fail. Thus we need to set `interpreter.prefer_native` + to `true` in such cases to "freeze" meta code at their versions in the + previous stage; no new meta code should be introduced in this stage. Any + further stages (e.g. after an `update-stage0`) will then need to be compiled + with the flag set to `false` again since they will expect the new signature. + + For an example, see https://github.com/leanprover/lean4/commit/da4c46370d85add64ef7ca5e7cc4638b62823fbb. + +* For the special case of *quotations*, it is desirable to have changes in + built-in parsers affect them immediately: when the changes in the parser + become active in the next stage, macros implemented via quotations should + generate syntax trees compatible with the new parser, and quotation patterns + in macro and elaborators should be able to match syntax created by the new + parser and macros. Since quotations capture the syntax tree structure during + execution of the current stage and turn it into code for the next stage, we + need to run the current stage's built-in parsers in quotation via the + interpreter for this to work. Caveats: + * Since interpreting full parsers is not nearly as cheap and we rarely change + built-in syntax, this needs to be opted in using `-Dinternal.parseQuotWithCurrentStage=true`. + * The parser needs to be reachable via an `import` statement, otherwise the + version of the previous stage will silently be used. + * Only the parser code (`Parser.fn`) is affected; all metadata such as leading + tokens is taken from the previous stage. + + For an example, see https://github.com/leanprover/lean4/commit/f9dcbbddc48ccab22c7674ba20c5f409823b4cc1#diff-371387aed38bb02bf7761084fd9460e4168ae16d1ffe5de041b47d3ad2d22422 + (from before the flag defaulted to `false`). + +To modify either of these flags both for building and editing the stdlib, adjust +the code in `stage0/src/stdlib_flags.h`. The flags will automatically be reset +on the next `update-stage0` when the file is overwritten with the original +version in `src/`. \ No newline at end of file diff --git a/doc/commit_convention.md b/doc/dev/commit_convention.md similarity index 100% rename from doc/commit_convention.md rename to doc/dev/commit_convention.md diff --git a/doc/dev/cpp_coding_style.md b/doc/dev/cpp_coding_style.md new file mode 100644 index 0000000000..6cf2aceb40 --- /dev/null +++ b/doc/dev/cpp_coding_style.md @@ -0,0 +1,155 @@ +[google-style]: https://google.github.io/styleguide/cppguide.html +[cpplint]: /src/cmake/Modules/cpplint.py + +# Coding Style + +The Lean project is moving away from using any C++ as more and more of +the compiler is being bootstrapped in Lean itself. But the remaining +C++ codebase is using modified version of [Google's C++ Style +Guide][google-style]. + +## [C++11](http://en.wikipedia.org/wiki/C%2B%2B11) features + +Lean makes extensive use of new features in the C++ 11 standard. +Developers must be familiar with the standard to be able to understand +the code. Here are some of the features that are extensively used. + +- Type inference (aka `auto` keyword). +- Initializer lists. +- Lambda functions and expressions. +- `nullptr` constant. +- Strongly typed enumerations. +- Right angle brackets with no space is now allowed in C++ 11. +- Thread local storage. +- Threading facilities. +- Tuple types. +- Smart pointers. +- When using ``std::list`` make sure to include the `std::` + qualifier so you do not accidentally use the ``lean::list`` type. +- When using ``std::copy`` make sure to include the `std::` + qualifier so you do not accidentally use the ``lean::copy`` type. +- Small and focused functions are preferred: foo(). Try not to + exceed 500 lines in a function, except in tests. +- Do **not** use the `#ifndef-#define-#endif` idiom for header files. + Instead use `#pragma once`. +- Write `type const & v` instead of `const type & v`. +- Use `const` extensively. +- Use the macro `lean_assert` for assertions. The macro `lean_assert` + is extensively used when writing unit tests. + +## Naming + +- Class, method, and function names are lower case +Use `_` for composite names. Example: `type_checker`. +- Class/struct fields should start with the prefix `m_`. + + Example: + ```c++ + class point { + int m_x; + int m_y; + public: + ... + }; + ``` + +## Namespaces + +All code is in the `lean` namespace. Each frontend is stored in a +separate nested namespace. For example, the SMT 2.0 frontend is stored +in the `lean::smt` namespace. + +Exception: some debugging functions are stored outside of the `lean` +namespace. These functions are called `print` and are meant to be used +when debugging Lean using `gdb`. + +Do not use `using namespace` in a header file. + +## Templates + +Organize template source code using the approach described at http://www.codeproject.com/Articles/3515/How-To-Organize-Template-Source-Code + +## Idioms + +Use some popular C++ idioms: + +- [Pimpl](http://c2.com/cgi/wiki?PimplIdiom) +- [RAII](http://en.wikipedia.org/wiki/Resource_Acquisition_Is_Initialization) Resource Acquisition Is Initialization + +## Formatting + +* Use 4 spaces for indentation. + +* `if-then-else` curly brackets not always required. The following + forms are acceptable: + + ```c++ + if (cond) { + ... + } else { + ... + } + ``` + and + + ```c++ + if (cond) + statement1; + else + statement2; + ``` + + In *exceptional cases*, we also use + + ```c++ + if (cond) statement; + ``` + + and + + ```c++ + if (cond) statement1; else stament2; + ``` + * `if-then-else-if-else` + + The following forms are acceptable: + + ```c++ + if (cond) { + ... + } else if (cond) { + ... + } else { + ... + } + ```c++ + and + + ```c++ + if (cond) + statement1; + else if (cond) + statement2; + else + statement3; + ``` + +* Format code using extra spaces to make code more readable. For example: + + ```c++ + environment const & m_env; + cache m_cache; + normalizer m_normalizer; + volatile bool m_interrupted; + ``` + instead of: + + ```c++ + environment const & m_env; + cache m_cache; + normalizer m_normalizer; + volatile bool m_interrupted; + ``` + +* Spaces in expressions. Write `a == b` instead of `a==b`. Similarly, + we write `x < y + 1` instead of `x