This PR adds support to the server for the new module setup process by
changing how `lake setup-file` is used.
In the new server setup, `lake setup-file` is invoked with the file name
of the edited module passed as a CLI argument and with the parsed header
passed to standard input in JSON form. Standard input is used to avoid
potentially exceeding the CLI length limits on Windows. Lake will build
the module's imports along with any other dependencies and then return
the module's workspace configuration via JSON (now in the form of
`ModuleSetup`). The server then post-processes this configuration a bit
and returns it back to the Lean language processor.
The server's header is currently only fully respected by Lake for
external modules (files that are not part of any workspace library). For
workspace modules, the saved module header is currently used to build
imports (as has been done since #7909). A follow-up Lake PR will align
both cases to follow the server's header.
Lean search paths (e.g., `LEAN_PATH`, `LEAN_SRC_PATH`) are no longer
negotiated between the server and Lake. These environment variables are
already configured during sever setup by `lake serve` and do not change
on a per-file basis. Lake can also pre-resolve the `.olean` files of
imports via the `importArts` field of `ModuleSetup`, limiting the
potential utility of communicating `LEAN_PATH`.
This PR fixes a bug where the unknown identifier code actions wouldn't
work correctly for some unknown identifier error spans and adjusts
several unknown identifier spans to actually end on the identifier in
question.
The following additional adjustments are made:
- The fallback mechanism of the unknown identifier code actions is
removed, since it could produce severely incorrect suggestions for
unknown identifier errors on fields.
- A performance bug when using the code action to import all unknown
identifiers is fixed.
- A bug that occurs when the elaborator produces multiple overlapping
completion infos is fixed.
- A bug in the snapshot selection that could cause it to wait for
snapshots in snapshots with non-canonical syntax is fixed.
- Some invariants of the snapshot tree are documented.
- The snapshot tree formatting is adjusted to display the final info
tree again.
This PR adjusts the experimental module system to not export the bodies
of `def`s unless opted out by the new attribute `@[expose]` on the `def`
or on a surrounding `section`.
---------
Co-authored-by: Markus Himmel <markus@lean-fro.org>
This PR adds the `--setup` option to the `lean` CLI. It takes a path to
a JSON file containing information about a module's imports and
configuration, superseding that in the module's own file header. This
will be used by Lake to specify paths to module artifacts (e.g., oleans
and ileans) separate from the `LEAN_PATH` schema.
To facilitate JSON serialization of the header data structure, `NameMap`
JSON instances have been added to core, and `LeanOptions` now makes use
of them.
This PR adds support for the following import variants to the
experimental module system:
* `private import`: Makes the imported constants available only in
non-exported contexts such as proofs. In particular, the import will not
be loaded, or required to exist at all, when the current module is
imported into other modules.
* `import all`: Makes non-exported information such as proofs of the
imported module available in non-exported contexts in the current
module. Main purpose is to allow for reasoning about imported
definitions when they would otherwise be opaque. TODO: adjust name
resolution so that imported `private` decls are accessible through
syntax.
They can be combined into `private import all`, which will likely be the
most common usage of `import all`.
This PR fixes a regression where elaboration of a previous document
version is not cancelled on changes to the document.
Done by removing the default from `SnapshotTask.cancelTk?` and
consistently passing the current thread's token for synchronous
elaboration steps.
This PR ensures info tree users such as linters and request handlers
have access to info subtrees created by async elab task by introducing
API to leave holes filled by such tasks.
**Breaking change**: other metaprogramming users of
`Command.State.infoState` may need to call `InfoState.substituteLazy` on
it manually to fill all holes.
This PR addresses a performance regression noticed at
https://github.com/leanprover/lean4/pull/7366#issuecomment-2708162029.
It also ensures that we also consider the current message log when
logging the goals accomplished message.
`Language.Lean.internal.cmdlineSnapshots` in `Lean.Language.Lean` is
moved to `Lean.internal.cmdlineSnapshots` in `Lean.CoreM` to make the
option available in the elaborator.
This PR fixes an `Elab.async` regression where elaboration tasks are
cancelled on document edit even though their result may be reused in the
new document version, reporting an incomplete result.
While this PR fixes the functional regression, it does so as an
over-approximation by never cancelling such tasks. A follow-up PR will
implement the correct behavior of only cancelling the tasks that are not
reused.
This PR fixes a bug where the goal state selection would sometimes
select incomplete incremental snapshots on whitespace, leading to an
incorrect "no goals" response. Fixes#6594, a regression that was
originally introduced in 4.11.0 by #4727.
The fundamental cause of #6594 was that the snapshot selection would
always select the first snapshot with a range that contains the cursor
position. For tactics, whitespace had to be included in this range.
However, in the test case of #6594, this meant that the snapshot
selection would also sometimes pick a snapshot before the cursor that
still contains the cursor in its whitespace, but which also does not
necessarily contain all the information needed to produce a correct goal
state. Specifically, at the `InfoTree`-level, when the cursor is in
whitespace, we distinguish competing goal states by their level of
indentation. The snapshot selection did not have access to this
information, so it necessarily had to do the wrong thing in some cases.
This PR fixes the issue by adjusting the snapshot selection for goals to
explicitly account for whitespace and indentation, and refactoring the
language processor architecture to thread enough information through to
the snapshot selection so that it can decide which snapshots to use
without having to force too many tasks, which would destroy
incrementality in goal state requests.
Specifically, this PR makes the following adjustments:
- Refactor `SnapshotTask` to contain both a `Syntax` and a `Range`.
Before, `SnapshotTask`s had a single range that was used both for
displaying file progress information and for selecting snapshots in
server requests. For most snapshots, this range did not include
whitespace, though for tactics it did. Now, the `reportingRange` field
of `SnapshotTask` is intended exclusively for reporting file progress
information, and the `Syntax` is used for selecting snapshots in server
requests. Importantly, the `Syntax` contains the full range information
of the snapshot, i.e. its regular range and its range including
whitespace.
- Adjust all call-sites of `SnapshotTask` to produce a reasonable
`Syntax`.
- Adjust the goal snapshot selection to account for whitespace and
indentation, as the `InfoTree` goal selection does.
- Fix a bug in the snapshot tree tracing that would cause it to render
the `Info` of a snapshot at the wrong location when `trace.Elab.info`
was also set.
This PR is based on #6329.
This PR enables the language server to present multiple disjoint line
ranges as being worked on. Even before parallelism lands, we make use of
this feature to show post-elaboration tasks such as kernel checking on
the first line of a declaration to distinguish them from the final
tactic step.
This PR adds support for plugins to the frontend and server.
Implementation-wise, this adds a `plugins` argument to `runFrontend`,
`processHeader`, amd `importModules`, a `plugins` field to
`SetupImportsResult` and `FileSetupResult`. and a `pluginsPath` field to
`LeanPaths`, and then threads the value through these.
Avoids build time overhead until the option is proven to speed up
average projects. Adds Init.Prelude (many tiny declarations, "worst
case") and Init.List.Sublist (many nontrivial theorems, "best case")
under -DElab.async=true as new benchmarks for tracking.
This PR adds core metaprogramming functions for forking off background
tasks from elaboration such that their results are visible to reporting
and the language server
This PR adds support for displaying multiple threads in the trace
profiler output.
`TraceState.tid` needs to be adjusted for this purpose, which is not
done yet by the Lean elaborator as it is still single-threaded.
Between #3106 and this, it was possible that reparsing the file up to
the current position was stuck waiting in the threadpool queue,
displaying a yellow bar and not displaying any info on the unchanged
prefix.
In #4976, I forgot that we do need info trees eventually on the cmdline
for .ilean generation. Unfortunately, not reporting them incrementally
would require an API change, so let's see what the impact of incremental
reporting is
#4976 moved resolution of a promise to an earlier point, but that led to
object being marked MT earlier, so we need to move the code that
minimizes those objects earlier too to revert the performance
regression.
Extends Lean's incremental reporting and reuse between commands into
various steps inside declarations:
* headers and bodies of each (mutual) definition/theorem
* `theorem ... := by` for each contained tactic step, including
recursively inside supported combinators currently consisting of
* `·` (cdot), `case`, `next`
* `induction`, `cases`
* macros such as `next` unfolding to the above
*Incremental reuse* means not recomputing any such steps if they are not
affected by a document change. *Incremental reporting* includes the
parts seen in the recording above: the progress bar and messages. Other
language server features such as hover etc. are *not yet* supported
incrementally, i.e. they are shown only when the declaration has been
fully processed as before.
---------
Co-authored-by: Scott Morrison <scott.morrison@gmail.com>
As [reported on
Zulip](https://leanprover.zulipchat.com/#narrow/stream/341532-lean4-dev/topic/Find.20references.20broken.20in.20lean.20core/near/437051935).
The `mainModuleName` was being set incorrectly when browsing lean core
sources, resulting in failure of cross-file server requests like "Find
References". Because the `srcSearchPath` is generated asynchronously, we
store it as a `Task Name` which is resolved some time before the header
is finished parsing. (I don't think the `.get` here will ever block,
because the srcSearchPath will be ready by the time the initial command
snap is requested.)
---------
Co-authored-by: Sebastian Ullrich <sebasti@nullri.ch>
