This PR modifies dot identifier notation so that `(.a : T)` resolves
`T.a` with respect to the root namespace, like for generalized field
notation. This lets the notation refer to private names, follow aliases,
and also use open namespaces. The LSP completions are improved to follow
how dot ident notation is resolved, but it doesn't yet take into account
aliases or open namespaces.
Closes#9629
This PR removes uses of `Lean.RBMap` in Lean itself.
Furthermore some massaging of the import graph is done in order to avoid
having `Std.Data.TreeMap.AdditionalOperations` (which is quite
expensive) be the critical path for a large chunk of Lean. In particular
we can build `Lean.Meta.Simp` and `Lean.Meta.Grind` without it thanks to
these changes.
We did previously not conduct this change as `Std.TreeMap` was not
outperforming `Lean.RBMap` yet, however this has changed with the new
code generator.
This PR adjusts the experimental module system to make `private` the
default visibility modifier in `module`s, introducing `public` as a new
modifier instead. `public section` can be used to revert the default for
an entire section, though this is more intended to ease gradual adoption
of the new semantics such as in `Init` (and soon `Std`) where they
should be replaced by a future decl-by-decl re-review of visibilities.
This PR adds support to the server for the new module setup process by
changing how `lake setup-file` is used.
In the new server setup, `lake setup-file` is invoked with the file name
of the edited module passed as a CLI argument and with the parsed header
passed to standard input in JSON form. Standard input is used to avoid
potentially exceeding the CLI length limits on Windows. Lake will build
the module's imports along with any other dependencies and then return
the module's workspace configuration via JSON (now in the form of
`ModuleSetup`). The server then post-processes this configuration a bit
and returns it back to the Lean language processor.
The server's header is currently only fully respected by Lake for
external modules (files that are not part of any workspace library). For
workspace modules, the saved module header is currently used to build
imports (as has been done since #7909). A follow-up Lake PR will align
both cases to follow the server's header.
Lean search paths (e.g., `LEAN_PATH`, `LEAN_SRC_PATH`) are no longer
negotiated between the server and Lake. These environment variables are
already configured during sever setup by `lake serve` and do not change
on a per-file basis. Lake can also pre-resolve the `.olean` files of
imports via the `importArts` field of `ModuleSetup`, limiting the
potential utility of communicating `LEAN_PATH`.
This PR adds support for throwing named errors with associated error
explanations. In particular, it adds elaborators for the syntax defined
in #8649, which use the error-explanation infrastructure added in #8651.
This includes completions, hovers, and jump-to-definition for error
names.
Note that another stage0 rebuild will be required to define explanations
using `register_error_explanation`.
---------
Co-authored-by: Joachim Breitner <mail@joachim-breitner.de>
Co-authored-by: Marc Huisinga <mhuisi@protonmail.com>
This PR reworks the `simp` set around the `Id` monad, to not elide or
unfold `pure` and `Id.run`
In particular, it stops encoding the "defeq abuse" of `Id X = X` in the
statements of theorems, instead using `Id.run` and `pure` to pass back
and forth between these two spellings. Often when writing these with
`pure`, they generalize to other lawful monads; though such changes were
split off to other PRs.
This fixes the problem with the current simp set where `Id.run (pure x)`
is simplified to `Id.run x`, instead of the desirable `x`.
This is particularly bad because the` x` is sometimes inferred with type
`Id X` instead of `X`, which prevents other `simp` lemmas about `X` from
firing.
Making `Id` reducible instead is not an option, as then the `Monad`
instances would have nothing to key on.
---------
Co-authored-by: Sebastian Graf <sg@lean-fro.org>
Co-authored-by: Kim Morrison <kim@tqft.net>
Co-authored-by: Paul Reichert <6992158+datokrat@users.noreply.github.com>
This PR changes namespace completion to use the same algorithm as
declaration identifier completion, which makes it use the short name
(last name component) for completions instead of the full name, avoiding
namespace duplications.
Closes#5654
This PR improves the performance of the workspace symbol request.
In my testing on my machine, the time to respond to the workspace symbol
request containing just `c` in Mathlib has been reduced to ~1200ms from
~11000ms.
We also serve the nearest-matching 1000 symbols instead of just the
first 100 now and use the length of the symbol as a tie-breaker for when
the fuzzy matching score is equal.
Some further improvements might be gained in the future when #8087 is
fixed and we can switch back to `qsort`.
This PR adds additional infrastructure for error message formatting.
Specifically, it adds convenience formatters for hints and notes,
including the ability to attach code actions to hint messages using a
"Try This"-like widget, along with several convenience formatters for
message data.
---------
Co-authored-by: Joachim Breitner <mail@joachim-breitner.de>
This PR fixes a number of bugs related to the handling of the source
search path in the language server, where deleting files could cause
several features to stop functioning and both untitled files and files
that don't exist on disc could have conflicting module names.
In detail, it makes the following adjustments:
- The URI <-> module name conversion was adjusted to produce no name
collisions.
- File URIs in the search path yield a module name relative to the
search path, as before.
- File URIs not in the search path, non-file URIs and non-`.lean` files
yield a `«external:<full uri>»` module name.
- To avoid the issue of the URI -> module name conversion failing when a
file is deleted from disc, we now cache the result of this conversion in
the watchdog and the file worker when the file is first opened.
- All of the URI <-> module name conversions now consistently go through
`Server.documentUriFromModule?` and `moduleFromDocumentUri` to ensure
that we don't have minor deviations for this conversion all over the
place.
- The threading of the source search path through the file worker (from
`lake setup-file`) is removed. It turns out that `lake serve` already
sets the correct source search path in the environment, so we can just
always use the search path from the environment.
- Since we can now answer more requests that need the .ileans in
untitled files, a lot of the tests that test 'Go to definition' needed
to be adjusted so that they use the information from the watchdog, not
the file worker. As we load references asynchronously, this PR adds an
internal `$/lean/waitForILeans` request that tests can use to wait for
all .ilean files to be loaded and for the ilean references from the file
worker for the current document version to be finalized.
- As part of this PR, we noticed that the .ileans aren't available in
the NixOS setup, so @Kha adjusted the Nix CI to fix this.
### Breaking changes
- `Server.documentUriFromModule` has been renamed to
`Server.documentUriFromModule?` and doesn't take a `SearchPath` argument
anymore, as the `SearchPath` is now computed from the `LEAN_SRC_PATH`
environment variable. It has also been moved from `Lean.Server.GoTo` to
`Lean.Server.Utils`.
- `Server.moduleFromDocumentUri` does not take a `SearchPath` argument
anymore and won't return an `Option` anymore. It has also been moved
from `Lean.Server.GoTo` to `Lean.Server.Utils`.
- The `System.SearchPath.searchModuleNameOfUri` function has been
removed. It is recommended to use `Server.moduleFromDocumentUri`
instead.
- The `initSrcSearchPath` function has been renamed to
`getSrcSearchPath` and has been moved from `Lean.Util.Paths` to
`Lean.Util.Path`. It also doesn't need to take a `pkgSearchPath`
argument anymore.
---------
Co-authored-by: Sebastian Ullrich <sebasti@nullri.ch>
This PR adds support for code actions that resolve 'unknown identifier'
errors by either importing the missing declaration or by changing the
identifier to one from the environment.
<details>
<summary>Demo (Click to open)</summary>
</details>
Specifically, the following kinds of code actions are added by this PR,
all of which are triggered on 'unknown identifier' errors:
- A code action to import the module containing the identifier at the
text cursor position.
- A code action to change the identifier at the text cursor position to
one from the environment.
- A source action to import the modules for all unambiguous identifiers
in the file.
### Details
When clicking on an identifier with an 'unknown identifier' diagnostic,
after a debounce delay of 1000ms, the language server looks up the
(potentially partial) identifier at the position of the cursor in the
global reference data structure by fuzzy-matching against all
identifiers and collects the 10 closest matching entries. This search
accounts for open namespaces at the position of the cursor, including
the namespace of the type / expected type when using dot notation. The
10 closest matching entries are then offered to the user as code
actions:
- If the suggested identifier is not contained in the environment, a
code action that imports the module that the identifier is contained in
and changes the identifier to the suggested one is offered. The
suggestion is inserted in a "minimal" manner, i.e. by accounting for
open namespaces.
- If the suggested identifier is contained in the environment, a code
action that only changes the identifier to the suggested one is offered.
- If the suggested identifier is not contained in the environment and
the suggested identifier is a perfectly unambiguous match, a source
action to import all unambiguous in the file is offered.
The source action to import all unambiguous identifiers can also always
be triggered by right-clicking in the document and selecting the 'Source
Action...' entry.
At the moment, for large projects, the search for closely matching
identifiers in the global reference data structure is still a bit slow.
I hope to optimize it next quarter.
### Implementation notes
- Since the global reference data structure is in the watchdog process,
whereas the elaboration information is in the file worker process, this
PR implements support for file worker -> watchdog requests, including a
new `$/lean/queryModule` request that can be used by the file worker to
request global identifier information.
- To identify 'unknown identifier' errors, several 'unknown identifier'
errors in the elaborator are tagged with a new tag.
- The debounce delay of 1000ms is necessary because VS Code will
re-request code actions while editing an unknown identifier and also
while hovering over the identifier.
- We also implement cancellation for these 'unknown identifier' code
actions. Once the file worker responds to the request as having been
cancelled, the watchdog cancels its computation of all corresponding
file worker -> watchdog requests, too.
- Aliases (i.e. `export`) are currently not accounted for. I've found
that we currently don't handle them correctly in auto-completion, too,
so we will likely add support for this later when fixing the
corresponding auto-completion issue.
- The new code actions added by this request support incrementality.
This PR significantly improves the performance of auto-completion by
optimizing individual requests by a factor of ~2 and by giving language
clients like VS Code the opportunity to reuse the state of previous
completion requests, thus greatly reducing the latency for the
auto-completion list to update when adding more characters to an
identifier.
In my testing:
- The latency of completing `C` in a file with `import Mathlib` was
reduced from ~1650ms to ~800ms
- The latency of completing `Cat` in a file with `import Mathlib` was
reduced from ~800ms to ~430ms
- The latency of completing dot notation was mostly unaffected
- Successive completions are now practically instant, e.g. if we were to
complete `C` and then type it out to `Cat`, before it would take roughly
~1650ms + ~800ms, whereas now there is only a significant latency for
completing `C` (~800ms) and the completion list is updated practically
instantly when typing out `Cat`.
<details>
<summary>(Video) Auto-completion latency before this PR</summary>
</details>
<details>
<summary>(Video) Auto-completion latency after this PR</summary>
</details>
In detail, this PR makes the following changes:
- Set `isIncomplete` to `false` in non-synthetic completion responses so
that the client can re-use these completion states.
- Replace the server side fuzzy matching with a simple and fast check
that all characters in the identifier thus far are present in the same
order in the declaration to match against. There are some examples where
the simple and fast check yields a completion item that the fuzzy
matching would filter, but since VS Code filters the completion items
with its own fuzzy matching after that anyways, these extra completion
items are never actually displayed to the user.
- Remove all notions of scoring and sorting completion items from the
language server. We now rely entirely on the client to sort the
completion items as it sees fit. In my testing, the only significant
change as a result of this is that while the language server would
sometimes penalize namespaces with lots of components, VS Code instead
uses a strictly alphabetic order. Even before this change, we never
actually really prioritized local variables over global variables, so
the penalty wasn't very helpful in practice. We might add some small
form of local variable prioritization in the future, though.
- Remove the empty completion list hack that was introduced in #1885. It
does not appear to be necessary anymore.
This PR moves away from using `List.get` / `List.get?` / `List.get!` and
`Array.get!`, in favour of using the `GetElem` mediated getters. In
particular it deprecates `List.get?`, `List.get!` and `Array.get?`. Also
adds `Array.back`, taking a proof, matching `List.getLast`.
This PR adds language server support for request cancellation to the
following expensive requests: Code actions, auto-completion, document
symbols, folding ranges and semantic highlighting. This means that when
the client informs the language server that a request is stale (e.g.
because it belongs to a previous state of the document), the language
server will now prematurely cancel the computation of the response in
order to reduce the CPU load for requests that will be discarded by the
client anyways.
This PR uniformizes the naming of `enum`/`enumFrom` (on `List`) and
`zipWithIndex` (on `Array` on `Vector`), replacing all with `zipIdx`. At
the same time, we generalize to add an optional `Nat` parameter for the
initial value of the index (which previously existed, only for `List`,
as the separate function `enumFrom`).
This PR fixes a significant auto-completion performance regression that
was introduced in #5666, i.e. v4.14.0.
#5666 introduced tactic docstrings, which were attempted to be collected
for every single completion item. This is slow for hundreds of thousands
of completion items. To fix this, this PR moves the docstring
computation into the completion item resolution, which is only called
when users select a specific completion item in the UI.
A downside of this approach is that we currently can't test completion
item resolution, so we lose a few tests that cover docstrings in
completions in this PR.
This PR fixes a bug in structure instance field completion that caused
it to not function correctly for bracketed structure instances written
in Mathlib style.
