This PR upstreams `unnecessarySeqFocus` linter from batteries to core
lean as a "clippy" linter - i.e. one that is disabled by default and
executed by `lake lint --clippy`.
---------
Co-authored-by: Mac Malone <tydeu@hatpress.net>
Co-authored-by: Thomas R. Murrills <68410468+thorimur@users.noreply.github.com>
This PR upstreams `dupNamespace` linter from batteries to work with new
core environment linting framework, as a "clippy" linter - i.e. one that
is not enabled by default.
Stacked on top of #13513.
---------
Co-authored-by: Mac Malone <tydeu@hatpress.net>
Co-authored-by: Thomas R. Murrills <68410468+thorimur@users.noreply.github.com>
This PR extends `lake lint --builtin-lint` to also support text linters
(i.e. those using `logLint`/`logLintIf`), in addition to the environment
linters added in #13431. Text-linter warnings emitted during the build
are persisted into each module's `.olean` via a new
`Lean.Linter.lintLogExt` environment extension; `lake lint` re-runs the
build for the target modules and reads the entries back, reporting them
alongside the environment linter output.
---------
Co-authored-by: Mac Malone <tydeu@hatpress.net>
Co-authored-by: Thomas R. Murrills <68410468+thorimur@users.noreply.github.com>
This PR removes the Makefile-based Lake test runner and replaces it with
`run_test.sh` and `run_clean.sh`. These are still not designed for use
with the `with_*.sh` runners, but this a step closer to that
eventuality.
It also adds a line to `CLAUDE.md` informing Claude how to build and run
a single Lake test.
This PR adds a check for empty `lake build` invocations (as an empty
build usually indicates a misconfiguration). Builds with no jobs will
now print "Nothing to build." and invocations of `lake build` with no
default targets configured will produce a warning. This will be promoted
to an error in the future. The warning (and future error) can be
suppressed with the new `--allow-empty` CLI option.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This PR adds builtin environment linting support to Lake, accessible via
`lake lint` flags. It also introduces two builtin linters upstreamed
from Mathlib (`defLemma` and `checkUnivs`) and a `builtinLint` package
configuration option.
Builtin linting is triggered via flags on `lake lint`:
- `--builtin-lint`: run default builtin linters (in addition to the lint
driver if configured)
- `--builtin-only`: run only builtin linters, skip the lint driver
- `--clippy`: run only non-default (clippy) linters
- `--lint-all`: run all builtin linters (default + clippy)
- `--lint-only <name>`: run a specific builtin linter by name
- Using `--clippy`, `--lint-all`, or `--lint-only` implicitly enables
builtin lint mode
The `builtinLint` package option is a tristate (`Option Bool`):
- `true`: always run builtin lints via `lake lint`; when a lint driver
is also configured, builtin lints run first, then the driver, and the
command fails if either reports errors.
- `false`: never run builtin lints automatically; `lake check-lint`
fails unless a lint driver is configured.
- `none` (default): currently equivalent to `false`; in a future
release, `none` will fall back to builtin lints when no lint driver is
configured.
The linter framework introduces a `LintScope` enum (`.default`,
`.clippy`, `.all`) replacing the previous boolean `clippy` parameter in
`getChecks` and `formatLinterResults`. A `@[builtin_nolint]` attribute
(available without imports) allows suppressing specific linters per
declaration.
---------
Co-authored-by: Mac Malone <tydeu@hatpress.net>
Co-authored-by: Thomas R. Murrills <68410468+thorimur@users.noreply.github.com>
This PR adds `JobAction.reuse` and `JobAction.unpack` which provide more
information captions for what a job is doing for the build monitor.
`reuse` is set when using an artifact from the Lake cache, `unpack` is
set when unpacking module `.ltar` archives and release (Reservoir or
GitHub) archives.
This PR adds a basic support for `lake builtin-lint` command that is
used to run environment linters and in the future will be extend to deal
with the core syntax linters.
This PR adds three new `lake cache` subcommands for staged cache
uploads: `stage`, `unstage`, and `put-staged`. These are designed to
function as parallels for the commands of the same name in Mathlib's
`lake exe cache`.
- `lake cache stage`: Copies the build outputs of a mappings file from
the Lake cache to a staging directory.
- `lake cache unstage`: Copies the build outputs from a staging
directory back into the Lake cache.
- `lake cache put-staged`: Uploads build outputs from a staging
directory to a remote cache service. Unlike `lake cache put`, this
command does not load the workspace configuration. As a result, platform
and toolchain settings must be supplied manually via `--platform` and
`--toolchain` if needed.
This PR also removes deprecation warnings when using environment
variables to configure the cache service for `lake cache put` (and `lake
cache put-staged`).
🤖 Prepared with Claude Code
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This PR changes Lake's materialization process to run remove untracked
files in tracked directories (via `git clean -xf`) when updating
dependency repositories. This ensures stale leftovers in the source tree
are removed.
In particular, if a `.hash` ends up in the source tree and the package
is updated, that `.hash` file will be stale but nonetheless trusted by a
Lake build. This will cause incorrect trace computation and break
builds. This happened with ProofWidgets in Mathlib (see [this Zulip
discussion](https://leanprover.zulipchat.com/#narrow/channel/113488-general/topic/ProofWidgets.20not.20up-to-date)).
This PR serves as alternative to #13130 (by @kim-em) and instead
provides a more generic solution to the problem. Nonetheless, thank them
for diagnosing this issue in the first place!
This PR adds a check that rejects Lake configurations where multiple
executables share the same root module name. Previously, Lake would
silently compile the root module once and link it into all executables,
producing identical binaries regardless of differing `srcDir` settings.
Lake (and Lean) rely on module names being unique within a package.
Rather than attempting to support duplicate module names, Lake now
produces a clear error at configuration load time, for both TOML and
Lean configuration files.
Closes#13013🤖 Prepared with Claude Code
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This PR changes the Lake `CacheMap` data structure to track the
platform-dependence of outputs. Platform-independent packages will no
longer include platform-dependent mappings in the output files produced
by `lake build -o`.
This PR adds the `fixedToolchain` Lake package configuration option.
Setting this to `true` informs Lake that the package is only expected to
function on a single toolchain (like Mathlib). This causes Lake's
toolchain update procedure to prioritize its toolchain and avoids the
need to separate input-to-output mappings for the package by toolchain
version in the Lake cache.
This PR changes `lake cache get` to download artifacts by default.
Artifacts can be downloaded on demand with the new `--mappings-only`
option (`--download-arts` is now obsolete).
In the future, the plan is to have Lake download mappings when cloning
dependencies. Then, `lake cache get` will primarily be used to download
artifacts eagerly. Thus, it makes sense to have that as the default.
This PR changes Lake to only emit `.nobuild` traces (introduced in
#12076) if the normal trace file already exists. This fixes an issue
where a `lake build --no-build` would create the build directory and
thereby prevent a cloud release fetch in a future build.
This PR enables Lake to download artifacts from a remote cache service
on demand as part of a `lake build`. It also refactors much of the cache
API to be more type safe.
The newly documented `lake cache add` command loads input-to-output
mappings from a file and stores them in the cache with optional
information about which cache service and what scope they come from.
With this information, Lake can now download artifacts on demand during
a `lake build`.
The `lake cache get` command has also changed its default behavior to
download just the input-to-outputs mapping and then lazily fetch
artifacts from Reservoir as part of a `lake build`. The original eager
behavior can be forced via the new `--download-arts` option.
This PR sets up the new integrated test/bench suite. It then migrates
all benchmarks and some related tests to the new suite. There's also
some documentation and some linting.
For now, a lot of the old tests are left alone so this PR doesn't become
even larger than it already is. Eventually, all tests should be migrated
to the new suite though so there isn't a confusing mix of two systems.
This PR changes all `lean-toolchain` to use relative toolchain paths
instead of `lean4` and `lean4-stage0` identifiers, which removes the
need for manually linking toolchains via Elan.
After this PR, at least Elan 4.2.0 and 0.0.224 of the Lean VS Code
extension will be needed to edit core.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This PR adds a system-wide Lake configuration file and uses it to
configure the remote cache services used by `lake cache`.
The system configuration is written in TOML. The exact location of the
file is system dependent and can be controlled via the `LAKE_CONFIG`
environment variable, but is usually located at `~/.lake/config.toml`.
As an example, one can configure a custom S3 cache service like so:
**~/.lake/config.toml**
```toml
cache.defaultService = "my-s3"
cache.defaultUploadService = "my-s3"
[[cache.service]]
name = "my-s3"
kind = "s3"
artifactEndpoint = "https://my-s3.com/a0"
revisionEndpoint = "https://my-s3.com/r0"
```
If no `cache.defaultService` is configured, Lake will use Reservoir for
downloads by default. A Reservoir mirror (or Reservoir-like service) can
be configured using `kind = "reservoir"` and setting an `apiEndpoint`. A
list of configured cache service (one name per line) can be obtained via
`lake cache services`.
This PR fixes a bug where Lake recached artifacts already present within
the cache. As a result, Lake would attempt to overwrite the read-only
artifacts, causing a permission denied error.
This PR fixes a bug with `cache clean` where it would fail if the cache
directory does not exist.
This introduces a `removeDirAllIfExists` utility which is also now used
in `lake clean`. While `lake clean` did previously check for a
nonexistent build directory, this version should be more robust to
racing runs of `lake clean` as well.
This PR adds support for manually re-releasing nightlies when a build
issue or critical fix requires it. When a `workflow_dispatch` triggers
the nightly release job and a `nightly-YYYY-MM-DD` tag already exists,
the CI now creates `nightly-YYYY-MM-DD-rev1` (then `-rev2`, etc.)
instead of silently skipping.
### Lake `ToolchainVer`
- Extend `ToolchainVer.nightly` with an optional `rev : Option Nat`
field
- Parse `-revK` suffixes from nightly tags in `ofString`
- Ordering: `nightly-YYYY-MM-DD` < `nightly-YYYY-MM-DD-rev1` < `-rev2` <
`nightly-YYYY-MM-DD+1`
- Round-trip: `toString (ofString s) == s` for both variants
### CI workflow
- "Set Nightly" step probes existing tags on `workflow_dispatch` to find
next available `-revK`
- Scheduled nightlies retain existing behavior (skip if commit already
tagged)
- Changelog grep updated from `nightly-[-0-9]*` to `nightly-[^ ,)]*` to
match `-revK` suffixes
### `lean-bisect`
- Updated `NIGHTLY_PATTERN` regex, sort key, error messages, and help
text
### Companion PRs
- https://github.com/leanprover-community/mathlib4/pull/35220: update
`nightly_bump_and_merge.yml` tag grep and `nightly_detect_failure.yml`
warning message
-
https://github.com/leanprover-community/leanprover-community.github.io/pull/787:
update `tags_and_branches.md` documentation
🤖 Prepared with Claude Code
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This PR changes the way artifacts are transferred from the local Lake
cache to a local build path. Now, Lake will first attempt to hard link
the local build path to artifact in the cache. If this fails (e.g.,
because the cache is on a different file system or drive), it will
fallback to pre-existing approach of copying the artifact. Lake also now
marks cache artifacts as read-only to avoid corrupting the cache by
writing to a hard linked artifact.
Lake will also hard link binary artifacts into the cache. If this fails,
it will similarly fall back to copying them. Text artifacts are always
copied, not linked, as the line endings in the cache copy are
normalized.
This PR changes the alters the file format of outputs stored in the
local Lake cache to include an identifier indicating the service (if
any) the output came from. This will be used to enable lazily
downloading artifacts on-demand during builds.
This PR makes disabling the artifact cache (e.g., via
`LAKE_ARTIFACT_CACHE=false` or `enableArtifactCache = false`) now stop
Lake from fetching from the cache (whereas it previously only stopped
writing to it).
There are now 3 possible configuration of the local artifact cache for a
package:
* `true`: Artifacts will be fetched from the cache before building (if
available) and built artifacts will be cached.
* `false:`: Lake will neither fetch artifacts from the cache or store
them into it.
* **default** (no configuration set): Lake will fetch artifacts from the
cache but not store them into it. A key motivation for this is to, by
default, reuse artifacts downloaded into the cache from a remote
service.
This PR fixes a bug on Windows with `IO.Process.spawn` where setting an
environment variable to the empty string would not set the environment
variable on the subprocess.
This PR fixes a bug introduced in #12086 where a `lake build :release
--no-build` would exit with code 1 rather than the `--no-build ` code 3.
Now both the bug from #12086 and this bug are fixed.
This PR revives the ability to specify modules in dependencies via the
basic `+mod` target key.
Implementation-wise, this removes deprecation of `BuildKey.module` and
once again uses it for `+mod` target keys. It also adds a test for
depending on a module of a dependency via `needs`.
This PR fixes the `lake query` output for targets which produce an
`Array` or `List` of a value with a custom `QueryText` or `QueryJson`
instance (e.g., `deps` and `transDeps`).
This PR fixes two Lake cache issues: a bug where a failed upload would
not produce an error and a mistake in the `--wfail` checks of the cache
commands.
This PR fixes a bug where a `lake build --no-build` would exit with code
`3` if the optional job to fetch a GitHub or Reservoir release for a
package failed (even if nothing else needed rebuilding).
This PR adds `lake shake` as a built-in Lake command, moving the shake
functionality from `script/Shake.lean` into the Lake CLI.
## Motivation
Per discussion with @Kha and @tydeu, having shake as a top-level Lake
command is preferable to `lake exe shake` because:
- Avoids the awkwardness of accessing core tools via `lake exe`
- Compiles shake into the Lake binary, avoiding lakefile issues
- No benefit to lazy compilation on user machines for this tool
## Changes
- Move shake logic from `script/Shake.lean` to
`src/lake/Lake/CLI/Shake.lean`
- Add `lake shake` command dispatch in `Lake/CLI/Main.lean`
- Add help text in `Lake/CLI/Help.lean`
- Remove the standalone shake executable from `script/lakefile.toml`
## Usage
```
lake shake [OPTIONS] [<MODULE>...]
```
See `lake shake --help` for full documentation.
🤖 Prepared with Claude Code
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Mac Malone <mac@lean-fro.org>
Co-authored-by: Sebastian Ullrich <sebasti@nullri.ch>
This PR changes the default of `enableArtifactCache` to use the
workspace's `enableArtifactCache` setting if the package is a dependency
and `LAKE_ARTIFACT_CACHE` is not set. This means that dependencies of a
project with `enableArtifactCache` set will also, by default, use Lake's
local artifact cache.
This PR fixes an inconsistency in the way Lake and Lean view the
transitivity of a `meta import`. Lake now works as Lean expects and
includes the meta segment of all transitive imports of a `meta import`
in its transitive trace.
This PR moves Lake's `tests/lake/examples/targets` test from `examples`
to `tests` (and thus disabling it by default).
It is being
[flaky](https://github.com/leanprover/lean4/actions/runs/20111185289/attempts/1)
for some unknown reason, so I am disabling until I have a better
opportunity to debug it.
