This PR fixes an inconsistency in the way Lake and Lean view the
transitivity of a `meta import`. Lake now works as Lean expects and
includes the meta segment of all transitive imports of a `meta import`
in its transitive trace.
This PR moves Lake's `tests/lake/examples/targets` test from `examples`
to `tests` (and thus disabling it by default).
It is being
[flaky](https://github.com/leanprover/lean4/actions/runs/20111185289/attempts/1)
for some unknown reason, so I am disabling until I have a better
opportunity to debug it.
This PR adds a workspace-index to the name of the package used by build
target. To clarify the distinction between the different uses of a
package's name, this PR also deprecates `Package.name` for more
use-specific variants (e.g., `Package.keyName`, `Package.prettyName`,
`Package.origName`).
More to come. (WIP)
This PR adds a module resolution procedure to Lake to disambiguate
modules that are defined in multiple packages.
On an `import`, Lake will now check if multiple packages within the
workspace define the module. If so, it will verify that modules have
sufficiently similar definitions (i.e., artifacts with the same content
hashes). If not, Lake will report an error.
This verification is currently only done for direct imports. Transitive
imports are not checked for consistency. An overhaul of transitive
imports will come later.
This PR fixes a breakage in Lake's TOML test caused by String API
changes. It also removes a JSON parser workaround that has since been
fixed, and it more generally polishes up the code.
This PR prevents symbol clashes between (non-`@[export]`) definitions
from different Lean packages.
Previously, if two modules define a function with the same name and were
transitively imported (even privately) by some downstream module,
linking would fail due to a symbol clash. Similarly, if a user defined a
symbol with the same name as one in the `Lean` library, Lean would use
the core symbol even if one did not import `Lean`.
This is solved by changing Lean's name mangling algorithm to include an
optional package identifier. This identifier is provided by Lake via
`--setup` when building a module. This information is weaved through the
elaborator, interpreter, and compiler via a persistent environment
extension that associates modules with their package identifier.
With a package identifier, standard symbols have the form
`lp_<pkg-id>_<mangled-def>`. Without one, the old scheme is used (i.e.,
`l_<mangled-def>`). Module initializers are also prefixed with package
identifier (if any). For example, the initializer for a module `Foo` in
a package `test` is now `initialize_test_Foo` (instead of
`initialize_Foo`). Lake's default for native library names has also been
adjusted accordingly, so that libraries can still, by default, be used
as plugins. Thus, the default library name of the `lean_lib Foo` in
`package test` will now be `libtest_Foo`.
When using Lake to build the Lean core (i.e., `bootstrap = true`), no
package identifier will be used. Thus, definitions in user packages can
never have symbol clashes with core.
Closes#222.
This PR changes all module build keys in Lake to be scoped by their
package. This enables building modules with the same name in different
packages (something previously only well-supported for executable
roots).
API-wise, the `BuildKey` definitions `module` and `moduleFacet` have
been deprecated and replaced with `packageModule` and
`packageModuleFacet`. The `moduleTargetIndicator` has also been removed
(with its purpose subsumed by `packageModule`).
This PR adds `Job.sync` as a standard way of declaring a synchronous
job.
It makes some non-behavior changes to related Job APIs to improve
compilation.
This PR enables Lake users to require Reservoir dependencies by a
semantic version range. On a `lake update`, Lake will fetch the
package's version information from Reservoir and select the newest
version of the package that satisfies the range.
### Using Version Ranges
Version ranges can be specified through the `version` field of a TOML
`require` or the `@` clause of a Lean `require`. They are only
meaningful on Reservoir dependencies.
**lakefile.lean**
```lean-4
require "Seasawher" / "mdgen" @ "2.*"
```
**lakefile.toml**
```toml
[[require]]
name = "mdgen"
scope = "Seasawher"
version = "2.*"
```
The syntax for these versions ranges is a mix of
[Rust's](https://doc.rust-lang.org/stable/cargo/reference/specifying-dependencies.html?highlight=caret#version-requirement-syntax)
and
[Node's](https://github.com/npm/node-semver/tree/v7.7.3?tab=readme-ov-file#ranges)
with some Lean-friendly deviations.
### Comparators
The basic unit of semantic version ranges are version comparators. They
take a base version and a comparison operator and match versions which
compare positively with their base. Lake supports the following
comparison operators.
* `<`, `<=` / `≤`, `>`, `>=` / `≥`, `=`, `!=` / `≠`
Unlike Rust and Node, Lake supports Unicode alternatives for the
operators. It also adds the not-equal operator (`!=` / `≠`) to make
excluding broken versions easier.
Comparators can be combined into clauses via conjunction or disjunction:
* **AND clauses**: Rust-style `≥1.2.3, <1.8.0` or Node-style `1.2.3
<1.8.0`
* **OR clauses**: Node-style `1.2.7 || >=1.2.9, <2.0.0`
When the base version of a comparator has a `-` suffix (e.g.,
`>1.2.3-alpha.3`) it will match versions of the same core (`1.2.3`) with
suffixes that lexicographically compare (e.g., `1.2.3-alpha.7` or
`1.2.3-beta.2`) but will not match suffixed versions of different cores
(e.g., `3.4.5-rc5`). An empty `-` suffix can be used to disable this
behavior. For example, `<2.0.0-` will match `1.2.3-beta.2` and
`2.0.0-alpha.1`.
### Range Macros
In addition to the basic comparators, Lake also supports standard
shorthand for specifying more complex ranges. Namely, it supports the
caret (`^`) and tilde (`~`) operator along with wildcard ranges.
**Caret Ranges**
* `^1` => `≥1.0.0, <2.0.0-`
* `^1.2` => `≥1.2.0, <2.0.0-`
* `^1.2.3` => `≥1.2.3, <2.0.0-`
* `^1.2.3-beta.2` => `≥1.2.3-beta.2, <2.0.0-`
* `^0.2` => `≥0.0.0, <0.3.0-`
* `^0.2.3` => `≥0.2.3, <0.3.0-`
* `^0.0.3` => `≥0.0.3, <0.0.4-`
* `^0` => `≥0.0.0, <1.0.0-`
* `^0.0` => `≥0.0.0, <0.1.0-`
**Tilde Ranges**
* `~1` => `≥1.0.0, <2.0.0-`
* `~1.2` => `≥1.2.0, <1.3.0-`
* `~1.2.3` => `≥1.2.3, <1.3.0-`
* `~1.2.3-beta.2` => `≥1.2.3-beta.2, <1.3.0-`
* `^0` => `≥0.0.0, <1.0.0-`
* `^0.2.3` => `≥0.2.3, <0.3.0-`
* `^0.0.3` => `≥0.0.3, <0.0.4-`
* `~0` => `≥0.0.0, <1.0.0-`
* `~0.0` => `≥0.0.0, <0.1.0-`
* `~0.0.0` => `≥0.0.0, <0.1.0-`
**Wildcard Ranges**
* `*` => `≥0.0.0`
* `1.x` => `≥1.0.0, <2.0.0-`
* `1.*.x` => `≥1.0.0, <2.0.0-`
* `1.2.*` => `≥1.2.0, <1.3.0-`
These ranges closely follow Rust's and Node's syntax. Like Node but
unlike Rust, wildcard ranges support `x` and `X` as alternative syntax
for wildcards.
This PR implements zero cost `BaseIO` by erasing the `IO.RealWorld`
parameter from argument lists and structures. This is a **major breaking
change for FFI**.
Concretely:
- `BaseIO` is defined in terms of `ST IO.RealWorld`
- `EIO` (and thus `IO`) is defined in terms of `EST IO.RealWorld`
- The opaque `Void` type is introduced and the trivial structure
optimization updated to account for it. Furthermore, arguments of type
`Void s` are removed from the argument lists of the C functions.
- `ST` is redefined as `Void s -> ST.Out s a` where `ST.Out` is a pair
of `Void s` and `a`
This together has the following major effects on our generated code:
- Functions that return `BaseIO`/`ST`/`EIO`/`IO`/`EST` now do not take
the dummy world parameter anymore. To account for this FFI code needs to
delete the dummy world parameter from the argument lists.
- Functions that return `BaseIO`/`ST` now return their wrapped value
directly. In particular `BaseIO UInt32` now returns a `uint32_t` instead
of a `lean_object*`. To account for this FFI code might have to change
the return type and does not need to call `lean_io_result_mk_ok` anymore
but can instead just `return` values right away (same with extracting
values from `BaseIO` computations.
- Functions that return `EIO`/`IO`/`EST` now only return the equivalent
of an `Except` node which reduces the allocation size. The
`lean_io_result_mk_ok`/`lean_io_result_mk_error` functions were updated
to account for this already so no change is required.
Besides improving performance by dropping allocation (sizes) we can now
also do fun new things such as:
```lean
@[extern "malloc"]
opaque malloc (size : USize) : BaseIO USize
```
This PR fixes a bug with Lake's cache where revisions were stored at the
incorrect path. Revisions were stored at `<rev>/<pkg>.jsonl` rather than
the correct `<pkg>/<rev>.jsonl`.
This PR fixes `input_dir` tracking to also recurse through
subdirectories. The `filter` of an `input_dir` will be applied to each
file in the directory tree (the path names of directories will not be
checked).
Closes#10827.
This PR fixes a bug where partially up-to-date files built with `--old`
could be stored in the cache as fully up-to-date. Such files are no
longer cached. In addition, builds without traces now only perform an
modification time check with `--old`. Otherwise, they are considered
out-of-date.
This PR changes the Lake's remote cache interface to scope cache outputs
by toolchain and/or platform were useful.
Packages that set `platformIndependent = true` will not be scoped by
platform and the core build (i.e., `bootstrap = true`) will not be
scoped by toolchain. Lake's detected platform and toolchain can be
overridden with the new `--platform` and `--toolchain` options to `cache
get` and `cache put`.
Lake no longer accepts the `--scope` option when using `cache get` with
Reservoir.. The `--repo` option must be used instead.
This PR adds a new `allowImportAll` configuration option for packages
and libraries. When enabled by an upstream package or library,
downstream packages will be able to `import all` modules of that package
or library. This enables package authors to selectively choose which
`private` elements, if any, downstream packages may have access to.
