This PR improves the 'Go to Definition' UX, specifically:
- Using 'Go to Definition' on a type class projection will now extract
the specific instances that were involved and provide them as locations
to jump to. For example, using 'Go to Definition' on the `toString` of
`toString 0` will yield results for `ToString.toString` and `ToString
Nat`.
- Using 'Go to Definition' on a macro that produces syntax with type
class projections will now also extract the specific instances that were
involved and provide them as locations to jump to. For example, using
'Go to Definition' on the `+` of `1 + 1` will yield results for
`HAdd.hAdd`, `HAdd α α α` and `Add Nat`.
- Using 'Go to Declaration' will now provide all the results of 'Go to
Definition' in addition to the elaborator and the parser that were
involved. For example, using 'Go to Declaration' on the `+` of `1 + 1`
will yield results for `HAdd.hAdd`, `HAdd α α α`, `Add Nat`,
``macro_rules | `($x + $y) => ...`` and `infixl:65 " + " => HAdd.hAdd`.
- Using 'Go to Type Definition' on a value with a type that contains
multiple constants will now provide 'Go to Definition' results for each
constant. For example, using 'Go to Type Definition' on `x` for `x :
Array Nat` will yield results for `Array` and `Nat`.
### Details
'Go to Definition' for type class projections was first implemented by
#1767, but there were still a couple of shortcomings with the
implementation. E.g. in order to jump to the instance in `toString 0`,
one had to add another space within the application and then use 'Go to
Definition' on that, or macros would block instances from being
displayed. Then, when the .ilean format was added, most 'Go to
Definition' requests were already handled using the .ileans in the
watchdog process, and so the file worker never received them to handle
them with the semantic information that it has available.
This PR resolves most of the issues with the previous implementation and
refactors the 'Go to Definition' control flow so that 'Go to Definition'
requests are always handled by the file worker, with the watchdog merely
using its .ilean position information to update the positions in the
response to a more up-to-date state. This is necessary because the file
worker obtains its position information from the .oleans, which need to
be rebuilt in order to be up-to-date, while the watchdog always receives
.ilean update notifications from each active file worker with the
current position information in the editor.
Finally, all of the 'Go to Definition' code is refactored to be easier
to maintain.
### Breaking changes
`InfoTree.hoverableInfoAt?` has been generalized to
`InfoTree.hoverableInfoAtM?` and now takes a general `filter` argument
instead of several boolean flags, as was the case before.
This PR removes uses of `Lean.RBMap` in Lean itself.
Furthermore some massaging of the import graph is done in order to avoid
having `Std.Data.TreeMap.AdditionalOperations` (which is quite
expensive) be the critical path for a large chunk of Lean. In particular
we can build `Lean.Meta.Simp` and `Lean.Meta.Grind` without it thanks to
these changes.
We did previously not conduct this change as `Std.TreeMap` was not
outperforming `Lean.RBMap` yet, however this has changed with the new
code generator.
This PR migrates usages of `Std.Range` to the new polymorphic ranges.
This PR unfortunately increases the transitive imports for
frequently-used parts of `Init` because the ranges now rely on iterators
in order to provide their functionality for types other than `Nat`.
However, iteration over ranges in compiled code is as efficient as
before in the examples I checked. This is because of a special
`IteratorLoop` implementation provided in the PR for this purpose.
There were two issues that were uncovered during migration:
* In `IndPredBelow.lean`, migrating the last remaining range causes
`compilerTest1.lean` to break. I have minimized the issue and came to
the conclusion it's a compiler bug. Therefore, I have not replaced said
old range usage yet (see #9186).
* In `BRecOn.lean`, we are publicly importing the ranges. Making this
import private should theoretically work, but there seems to be a
problem with the module system, causing the build to panic later in
`Init.Data.Grind.Poly` (see #9185).
* In `FuzzyMatching.lean`, inlining fails with the new ranges, which
would have led to significant slowdown. Therefore, I have not migrated
this file either.
This PR adjusts the experimental module system to make `private` the
default visibility modifier in `module`s, introducing `public` as a new
modifier instead. `public section` can be used to revert the default for
an entire section, though this is more intended to ease gradual adoption
of the new semantics such as in `Init` (and soon `Std`) where they
should be replaced by a future decl-by-decl re-review of visibilities.
This PR replaces all usages of `[:]` slice notation in `src` with the
new `[...]` notation in production code, tests and comments. The
underlying implementation of the `Subarray` functions stays the same.
Notation cheat sheet:
* `*...*` is the doubly-unbounded range.
* `*...a` or `*...<a` contains all elements that are less than `a`.
* `*...=a` contains all elements that are less than or equal to `a`.
* `a...*` contains all elements that are greater than or equal to `a`.
* `a...b` or `a...<b` contains all elements that are greater than or
equal to `a` and less than `b`.
* `a...=b` contains all elements that are greater than or equal to `a`
and less than or equal to `b`.
* `a<...*` contains all elements that are greater than `a`.
* `a<...b` or `a<...<b` contains all elements that are greater than `a`
and less than `b`.
* `a<...=b` contains all elements that are greater than `a` and less
than or equal to `b`.
Benchmarks have shown that importing the iterator-backed parts of the
polymorphic slice library in `Init` impacts build performance. This PR
avoids this problem by separating those parts of the library that do not
rely on iterators from those those that do. Whereever the new slice
notation is used, only the iterator-independent files are imported.
This PR fixes a bug where semantic highlighting would only highlight
keywords that started with an alphanumeric character. Now, it uses
`Lean.isIdFirst`.
This PR adds support to the server for the new module setup process by
changing how `lake setup-file` is used.
In the new server setup, `lake setup-file` is invoked with the file name
of the edited module passed as a CLI argument and with the parsed header
passed to standard input in JSON form. Standard input is used to avoid
potentially exceeding the CLI length limits on Windows. Lake will build
the module's imports along with any other dependencies and then return
the module's workspace configuration via JSON (now in the form of
`ModuleSetup`). The server then post-processes this configuration a bit
and returns it back to the Lean language processor.
The server's header is currently only fully respected by Lake for
external modules (files that are not part of any workspace library). For
workspace modules, the saved module header is currently used to build
imports (as has been done since #7909). A follow-up Lake PR will align
both cases to follow the server's header.
Lean search paths (e.g., `LEAN_PATH`, `LEAN_SRC_PATH`) are no longer
negotiated between the server and Lake. These environment variables are
already configured during sever setup by `lake serve` and do not change
on a per-file basis. Lake can also pre-resolve the `.olean` files of
imports via the `importArts` field of `ModuleSetup`, limiting the
potential utility of communicating `LEAN_PATH`.
This PR adds support for throwing named errors with associated error
explanations. In particular, it adds elaborators for the syntax defined
in #8649, which use the error-explanation infrastructure added in #8651.
This includes completions, hovers, and jump-to-definition for error
names.
Note that another stage0 rebuild will be required to define explanations
using `register_error_explanation`.
---------
Co-authored-by: Joachim Breitner <mail@joachim-breitner.de>
Co-authored-by: Marc Huisinga <mhuisi@protonmail.com>
This PR implements signature help support. When typing a function
application, editors with support for signature help will now display a
popup that designates the current (remaining) function type. This
removes the need to remember the function signature while typing the
function application, or having to constantly cycle between hovering
over the function identifier and typing the application. In VS Code, the
signature help can be triggered manually using `Ctrl+Shift+Space`.
### Other changes
- In order to support signature help for the partial syntax `f a <|` or
`f a $`, these notations now elaborate as `f a`, not `f a .missing`.
- The logic in `delabConstWithSignature` that delaborates parameters is
factored out into a function `delabForallParamsWithSignature` so that it
can be used for arbitrary `forall`s, not just constants.
- The `InfoTree` formatter is adjusted to produce output where it is
easier to identify the kind of `Info` in the `InfoTree`.
- A bug in `InfoTree.smallestInfo?` is fixed so that it doesn't panic
anymore when its predicate `p` does not ensure that both `pos?` and
`tailPos?` of the `Info` are present.
This PR adds support for server-sided `RpcRef` reuse and fixes a bug
where trace nodes in the InfoView would close while the file was still
being processed.
The core of the trace node issue is that the server always serves new
RPC references in every single response to the client, which means that
the client is forced to reset its UI state.
In a previous attempt at fixing this (#8056), the server would memorize
the RPC-encoded JSON of interactive diagnostics (which includes RPC
references) and serve it for as long as it could reuse the snapshot
containing the diagnostics, so that RPC references are reused. The
problem with this was that the client then had multiple finalizers
registered for the same RPC reference (one for every reused RPC
reference that was served), and once the first reference was
garbage-collected, all other reused references would point into the
void.
This PR takes a different approach to resolve the issue: The meaning of
`$/lean/rpc/release` is relaxed from "Free the object pointed to by this
RPC reference" to "Decrement the RPC reference count of the object
pointed to by this RPC reference", and the server now maintains a
reference count to track how often a given `RpcRef` was served. Only
when every single served instance of the `RpcRef` has been released, the
object is freed. Additionally, the reuse mechanism is generalized from
being only supported for interactive diagnostics, to being supported for
any object using `WithRpcRef`. In order to make use of reusable RPC
references, downstream users still need to memorize the `WithRpcRef`
instances accordingly.
Closes#8053.
### Breaking changes
Since `WithRpcRef` is now capable of tracking its identity to decide
which `WithRpcRef` usage constitutes a reuse, the constructor of
`WithRpcRef` has been made `private` to discourage downstream users from
creating `WithRpcRef` instances with manually-set `id`s. Instead,
`WithRpcRef.mk` (which lives in `BaseIO`) is now the preferred way to
create `WithRpcRef` instances.
This PR fixes a bug where the unknown identifier code actions wouldn't
work correctly for some unknown identifier error spans and adjusts
several unknown identifier spans to actually end on the identifier in
question.
The following additional adjustments are made:
- The fallback mechanism of the unknown identifier code actions is
removed, since it could produce severely incorrect suggestions for
unknown identifier errors on fields.
- A performance bug when using the code action to import all unknown
identifiers is fixed.
- A bug that occurs when the elaborator produces multiple overlapping
completion infos is fixed.
- A bug in the snapshot selection that could cause it to wait for
snapshots in snapshots with non-canonical syntax is fixed.
- Some invariants of the snapshot tree are documented.
- The snapshot tree formatting is adjusted to display the final info
tree again.
This PR reverts #8056 because the implementation there has a bug that is
best fixed with a different approach, and which we should preferably
only merge next release cycle.
This PR fixes a bug where the trace nodes in the InfoView would close
while the file was still being elaborated.
Closes#8053.
The cause of this bug was that we didn't memorize interactive
diagnostics correctly, so the server would generate new RPC pointers in
every single `getInteractiveDiagnostics` RPC request, which lead to the
client resetting the UI.
This PR fixes an issue where `let n : Nat := sorry` in the Infoview
pretty prints as ``n : ℕ := sorry `«Foo:17:17»``. This was caused by
top-level expressions being pretty printed with the same rules as
Infoview hovers. Closes#6715. Refactors `Lean.Widget.ppExprTagged`; now
it takes a delaborator, and downstream users should configure their own
pretty printer option overrides if necessary if they used the `explicit`
argument (see `Lean.Widget.makePopup.ppExprForPopup` for an example).
Breaking change: `ppExprTagged` does not set `pp.proofs` on the root
expression.
This PR fixes a number of bugs related to the handling of the source
search path in the language server, where deleting files could cause
several features to stop functioning and both untitled files and files
that don't exist on disc could have conflicting module names.
In detail, it makes the following adjustments:
- The URI <-> module name conversion was adjusted to produce no name
collisions.
- File URIs in the search path yield a module name relative to the
search path, as before.
- File URIs not in the search path, non-file URIs and non-`.lean` files
yield a `«external:<full uri>»` module name.
- To avoid the issue of the URI -> module name conversion failing when a
file is deleted from disc, we now cache the result of this conversion in
the watchdog and the file worker when the file is first opened.
- All of the URI <-> module name conversions now consistently go through
`Server.documentUriFromModule?` and `moduleFromDocumentUri` to ensure
that we don't have minor deviations for this conversion all over the
place.
- The threading of the source search path through the file worker (from
`lake setup-file`) is removed. It turns out that `lake serve` already
sets the correct source search path in the environment, so we can just
always use the search path from the environment.
- Since we can now answer more requests that need the .ileans in
untitled files, a lot of the tests that test 'Go to definition' needed
to be adjusted so that they use the information from the watchdog, not
the file worker. As we load references asynchronously, this PR adds an
internal `$/lean/waitForILeans` request that tests can use to wait for
all .ilean files to be loaded and for the ilean references from the file
worker for the current document version to be finalized.
- As part of this PR, we noticed that the .ileans aren't available in
the NixOS setup, so @Kha adjusted the Nix CI to fix this.
### Breaking changes
- `Server.documentUriFromModule` has been renamed to
`Server.documentUriFromModule?` and doesn't take a `SearchPath` argument
anymore, as the `SearchPath` is now computed from the `LEAN_SRC_PATH`
environment variable. It has also been moved from `Lean.Server.GoTo` to
`Lean.Server.Utils`.
- `Server.moduleFromDocumentUri` does not take a `SearchPath` argument
anymore and won't return an `Option` anymore. It has also been moved
from `Lean.Server.GoTo` to `Lean.Server.Utils`.
- The `System.SearchPath.searchModuleNameOfUri` function has been
removed. It is recommended to use `Server.moduleFromDocumentUri`
instead.
- The `initSrcSearchPath` function has been renamed to
`getSrcSearchPath` and has been moved from `Lean.Util.Paths` to
`Lean.Util.Path`. It also doesn't need to take a `pkgSearchPath`
argument anymore.
---------
Co-authored-by: Sebastian Ullrich <sebasti@nullri.ch>
This PR adds support for code actions that resolve 'unknown identifier'
errors by either importing the missing declaration or by changing the
identifier to one from the environment.
<details>
<summary>Demo (Click to open)</summary>
</details>
Specifically, the following kinds of code actions are added by this PR,
all of which are triggered on 'unknown identifier' errors:
- A code action to import the module containing the identifier at the
text cursor position.
- A code action to change the identifier at the text cursor position to
one from the environment.
- A source action to import the modules for all unambiguous identifiers
in the file.
### Details
When clicking on an identifier with an 'unknown identifier' diagnostic,
after a debounce delay of 1000ms, the language server looks up the
(potentially partial) identifier at the position of the cursor in the
global reference data structure by fuzzy-matching against all
identifiers and collects the 10 closest matching entries. This search
accounts for open namespaces at the position of the cursor, including
the namespace of the type / expected type when using dot notation. The
10 closest matching entries are then offered to the user as code
actions:
- If the suggested identifier is not contained in the environment, a
code action that imports the module that the identifier is contained in
and changes the identifier to the suggested one is offered. The
suggestion is inserted in a "minimal" manner, i.e. by accounting for
open namespaces.
- If the suggested identifier is contained in the environment, a code
action that only changes the identifier to the suggested one is offered.
- If the suggested identifier is not contained in the environment and
the suggested identifier is a perfectly unambiguous match, a source
action to import all unambiguous in the file is offered.
The source action to import all unambiguous identifiers can also always
be triggered by right-clicking in the document and selecting the 'Source
Action...' entry.
At the moment, for large projects, the search for closely matching
identifiers in the global reference data structure is still a bit slow.
I hope to optimize it next quarter.
### Implementation notes
- Since the global reference data structure is in the watchdog process,
whereas the elaboration information is in the file worker process, this
PR implements support for file worker -> watchdog requests, including a
new `$/lean/queryModule` request that can be used by the file worker to
request global identifier information.
- To identify 'unknown identifier' errors, several 'unknown identifier'
errors in the elaborator are tagged with a new tag.
- The debounce delay of 1000ms is necessary because VS Code will
re-request code actions while editing an unknown identifier and also
while hovering over the identifier.
- We also implement cancellation for these 'unknown identifier' code
actions. Once the file worker responds to the request as having been
cancelled, the watchdog cancels its computation of all corresponding
file worker -> watchdog requests, too.
- Aliases (i.e. `export`) are currently not accounted for. I've found
that we currently don't handle them correctly in auto-completion, too,
so we will likely add support for this later when fixing the
corresponding auto-completion issue.
- The new code actions added by this request support incrementality.
This PR ensures info tree users such as linters and request handlers
have access to info subtrees created by async elab task by introducing
API to leave holes filled by such tasks.
**Breaking change**: other metaprogramming users of
`Command.State.infoState` may need to call `InfoState.substituteLazy` on
it manually to fill all holes.
This PR adds a canonical syntax for linking to sections in the language
reference along with formatting of examples in docstrings according to
the docstring style guide.
Docstrings are now pre-processed as follows:
* Output included as part of examples is shown with leading line comment
indicators in hovers
* URLs of the form `lean-manual://section/section-id` are rewritten to
links that point at the corresponding section in the Lean reference
manual. The reference manual's base URL is configured when Lean is built
and can be overridden with the `LEAN_MANUAL_ROOT` environment variable.
This way, releases can point documentation links to the correct
snapshot, and users can use their own, e.g. for offline reading.
Manual URLs in docstrings are validated when the docstring is added. The
presence of a URL starting with `lean-manual://` that is not a
syntactically valid section link causes the docstring to be rejected.
This allows for future extensibility to the set of allowed links. There
is no validation that the linked-to section actually exists. To provide
the best possible error messages in case of validation failures,
`Lean.addDocString` now takes a `TSyntax ``docComment` instead of a
string; clients should adapt by removing the step that extracts the
string, or by calling the lower-level `addDocStringCore` in cases where
the docstring in question is obtained from the environment and has thus
already had its links validated.
A stage0 update is required to make the documentation site configurable
at build time and for releases. A local commit on top of a stage0 update
that will be sent in a followup PR includes the configurable reference
manual root and updates to the release checklist.
---------
Co-authored-by: Marc Huisinga <mhuisi@protonmail.com>
This PR fixes a race condition in the language server that would
sometimes cause it to drop requests and never respond to them when
editing the header of a file. This in turn could cause semantic
highlighting to stop functioning in VS Code, as VS Code would stop
emitting requests when a prior request was dropped, and also cause the
InfoView to become defective. It would also cause import auto-completion
to feel a bit wonky, since these requests were sometimes dropped. This
race condition has been present in the language server since its first
version in 2020.
This PR also reverts the futile fix attempt in #7130.
The specific race condition was that if the file worker crashed or had
to be restarted while a request was in flight in the file worker, then
we wouldn't correctly replay it in our watchdog crash-restart logic.
This PR adjusts this logic to fix this.
This PR fixes an issue where the language server would run into an inlay
hint assertion violation when deleting a file that is still open in the
language server.
This PR mitigates an issue where inserting an inlay hint in VS Code by
double-clicking would insert the inlay hint at the wrong position right
after an edit.
This bug was originally reported by @plp127 at
https://leanprover.zulipchat.com/#narrow/channel/113488-general/topic/v4.2E18.2E0.20-.20inlay.20hints/near/503362330.
The cause of this bug is that when VS Code hasn't yet received a new set
of inlay hints for a new document state, it will happily move around the
displayed inlay hint, but it won't move around any of the other
position-dependent properties of the inlay hint, like the property
describing where to insert the inlay hint. Since we delay responses
after an edit by an edit delay of 3000ms to prevent inlay hint
flickering while typing, the window for this bug is relatively large.
To work around this bug, we now always immediately respond to the first
inlay hint request after an edit with the old state of the inlay hints,
which we already update correctly on edits on the server-side so that we
can serve old inlay hints for parts of the file that are still
in-progress. Essentially, we are just telling VS Code how it should have
moved all position-dependent properties of each inlay hint.
Even with this mitigation, there is still a small window for this bug to
occur, namely the window from an edit to when VS Code receives the old
inlay hints from the server. In practice, this window should be a couple
of milliseconds at most, so I'd hope it doesn't cause many problems.
There's nothing we can do about this in either vscode-lean4 or the
language server, unfortunately.
This PR fixes several inlay hint race conditions that could result in a
violation of the monotonic progress assumption, introduced in #7149.
Specifically:
- In rare circumstances, it could happen that stateful LSP requests were
executed out-of-order with their `didChange` handlers, as both requests
and the `didChange` handlers waited on `lake setup-file` to complete,
with the latter running those handlers in a dedicated task afterwards.
This meant that a request could be added to the stateful LSP handler
request queue before the corresponding `didChange` call that actually
came before it. This PR resolves this issue by folding the task that
waits for `lake setup-file` into the `RequestContext`, which ensures
that we only need to wait for it when actually executing the request
handler.
- While #7164 fixed the monotonic progress assertion violation that was
caused by `$/cancelRequest`, it did not account for our internal notion
of silent request cancellation in stateful LSP requests, which we use to
cancel the inlay hint edit delay when VS Code fails to emit a
`$/cancelRequest` notification. This issue is resolved by always
producing the full finished prefix of the command snapshot queue, even
on cancellation. Additionally, this also fixes an issue where in the
same circumstances, the language server could produce an empty inlay
hint response when a request was cancelled by our internal notion of
silent request cancellation.
- For clients that use `fullChange` `didChange` notifications (e.g. not
VS Code), we would get several aspects of stateful LSP request
`didChange` state handling wrong, which is also addressed by this PR.
This PR changes the server to run `lake setup-file` on Lake
configuration files (e.g., `lakefile.lean`).
This is needed to support Lake passing the server its own Lake plugin to
load when elaborating the configuration file.
This PR ensures that all tasks in the language server either use
dedicated tasks or reuse an existing thread from the thread pool. This
ensures that elaboration tasks cannot prevent language server tasks from
being scheduled. This is especially important with parallelism right
around the corner and elaboration becoming more likely to starve the
language server of computation, which could drive up language server
latencies significantly on machines with few cores.
Specifically, all language server tasks are refactored to use a new thin
`ServerTask` API wrapper with a single "costly" vs "cheap" dimension,
where costly tasks are always scheduled as dedicated tasks, and cheap
tasks are always made to either run on the calling thread or to reuse
the thread of the task being mapped on by using the `sync` flag.
ProofWidgets4 adaption PR:
https://github.com/leanprover-community/ProofWidgets4/pull/106
### Other changes
- This PR makes several tasks dedicated that weren't dedicated before,
and uses `sync := true` for some others. The rules for this are
described in the module docstring of `ServerTask.lean`.
- Most notably, the reporting task in the file worker was *not* a
dedicated task before this PR, which could easily lead to thread pool
starvation on successive changes. It also did not support cancellation.
This PR ensures that it does.
### Breaking changes
- `RequestTask` and the request-oriented snapshot API are refactored to
use `ServerTask` instead of `Task`. All functions in `Task` have close
analogues in `ServerTask`, and functions on `RequestTask` now need to
distinguish between whether a `map` or a `bind` is cheap or costly. This
affects all downstream users of `RequestM`, e.g. tools that extend the
language server with their own requests, or some users of the RPC
mechanism.
- The following unused functions of the `AsyncList` API have been
deleted: `append`, `unfoldAsync`, `getAll`, `waitHead?`, `cancel`
This PR adds a fast path to the inlay hint request that makes it re-use
already computed inlay hints from previous requests instead of
re-computing them. This is necessary because for some reason VS Code
emits an inlay hint request for every line you scroll, so we need to be
able to respond to these requests against the same document state
quickly. Otherwise, every single scrolled line would result in a request
that can take a few dozen ms to be responded to in long files, putting
unnecessary pressure on the CPU.
It also filters the result set by the inlay hints that have been
requested.
This PR significantly improves the performance of auto-completion by
optimizing individual requests by a factor of ~2 and by giving language
clients like VS Code the opportunity to reuse the state of previous
completion requests, thus greatly reducing the latency for the
auto-completion list to update when adding more characters to an
identifier.
In my testing:
- The latency of completing `C` in a file with `import Mathlib` was
reduced from ~1650ms to ~800ms
- The latency of completing `Cat` in a file with `import Mathlib` was
reduced from ~800ms to ~430ms
- The latency of completing dot notation was mostly unaffected
- Successive completions are now practically instant, e.g. if we were to
complete `C` and then type it out to `Cat`, before it would take roughly
~1650ms + ~800ms, whereas now there is only a significant latency for
completing `C` (~800ms) and the completion list is updated practically
instantly when typing out `Cat`.
<details>
<summary>(Video) Auto-completion latency before this PR</summary>
</details>
<details>
<summary>(Video) Auto-completion latency after this PR</summary>
</details>
In detail, this PR makes the following changes:
- Set `isIncomplete` to `false` in non-synthetic completion responses so
that the client can re-use these completion states.
- Replace the server side fuzzy matching with a simple and fast check
that all characters in the identifier thus far are present in the same
order in the declaration to match against. There are some examples where
the simple and fast check yields a completion item that the fuzzy
matching would filter, but since VS Code filters the completion items
with its own fuzzy matching after that anyways, these extra completion
items are never actually displayed to the user.
- Remove all notions of scoring and sorting completion items from the
language server. We now rely entirely on the client to sort the
completion items as it sees fit. In my testing, the only significant
change as a result of this is that while the language server would
sometimes penalize namespaces with lots of components, VS Code instead
uses a strictly alphabetic order. Even before this change, we never
actually really prioritized local variables over global variables, so
the penalty wasn't very helpful in practice. We might add some small
form of local variable prioritization in the future, though.
- Remove the empty completion list hack that was introduced in #1885. It
does not appear to be necessary anymore.
This PR moves away from using `List.get` / `List.get?` / `List.get!` and
`Array.get!`, in favour of using the `GetElem` mediated getters. In
particular it deprecates `List.get?`, `List.get!` and `Array.get?`. Also
adds `Array.back`, taking a proof, matching `List.getLast`.
This PR adds language server support for request cancellation to the
following expensive requests: Code actions, auto-completion, document
symbols, folding ranges and semantic highlighting. This means that when
the client informs the language server that a request is stale (e.g.
because it belongs to a previous state of the document), the language
server will now prematurely cancel the computation of the response in
order to reduce the CPU load for requests that will be discarded by the
client anyways.
This PR fixes a bug where the goal state selection would sometimes
select incomplete incremental snapshots on whitespace, leading to an
incorrect "no goals" response. Fixes#6594, a regression that was
originally introduced in 4.11.0 by #4727.
The fundamental cause of #6594 was that the snapshot selection would
always select the first snapshot with a range that contains the cursor
position. For tactics, whitespace had to be included in this range.
However, in the test case of #6594, this meant that the snapshot
selection would also sometimes pick a snapshot before the cursor that
still contains the cursor in its whitespace, but which also does not
necessarily contain all the information needed to produce a correct goal
state. Specifically, at the `InfoTree`-level, when the cursor is in
whitespace, we distinguish competing goal states by their level of
indentation. The snapshot selection did not have access to this
information, so it necessarily had to do the wrong thing in some cases.
This PR fixes the issue by adjusting the snapshot selection for goals to
explicitly account for whitespace and indentation, and refactoring the
language processor architecture to thread enough information through to
the snapshot selection so that it can decide which snapshots to use
without having to force too many tasks, which would destroy
incrementality in goal state requests.
Specifically, this PR makes the following adjustments:
- Refactor `SnapshotTask` to contain both a `Syntax` and a `Range`.
Before, `SnapshotTask`s had a single range that was used both for
displaying file progress information and for selecting snapshots in
server requests. For most snapshots, this range did not include
whitespace, though for tactics it did. Now, the `reportingRange` field
of `SnapshotTask` is intended exclusively for reporting file progress
information, and the `Syntax` is used for selecting snapshots in server
requests. Importantly, the `Syntax` contains the full range information
of the snapshot, i.e. its regular range and its range including
whitespace.
- Adjust all call-sites of `SnapshotTask` to produce a reasonable
`Syntax`.
- Adjust the goal snapshot selection to account for whitespace and
indentation, as the `InfoTree` goal selection does.
- Fix a bug in the snapshot tree tracing that would cause it to render
the `Info` of a snapshot at the wrong location when `trace.Elab.info`
was also set.
This PR is based on #6329.
This PR fixes a bug where both the inlay hint change invalidation logic
and the inlay hint edit delay logic were broken in untitled files.
Thanks to @Julian for spotting this!
This PR implements a number of refinements for the auto-implicit inlay
hints implemented in #6768.
Specifically:
- In #6768, there was a bug where the inlay hint edit delay could
accumulate on successive edits, which meant that it could sometimes take
much longer for inlay hints to show up. This PR implements the basic
infrastructure for request cancellation and implements request
cancellation for semantic tokens and inlay hints to resolve the issue.
With this edit delay bug fixed, it made more sense to increase the edit
delay slightly from 2000ms to 3000ms.
- In #6768, we applied the edit delay to every single inlay hint request
in order to reduce the amount of inlay hint flickering. This meant that
the edit delay also had a significant effect on how far inlay hints
would lag behind the file progress bar. This PR adjusts the edit delay
logic so that it only affects requests sent directly after a
corresponding `didChange` notification. Once the edit delay is used up,
all further semantic token requests are responded to without delay, so
that the only latency that affects how far the inlay hints lag behind
the progress bar is how often we emit refresh requests and how long VS
Code takes to respond to them.
- For inlay hints, refresh requests are now emitted 500ms after a
response to an inlay hint request, not 2000ms, which means that after
the edit delay, inlay hints should only lag behind the progress bar by
about up to 500ms. This is justifiable for inlay hints because the
response should be much smaller than e.g. is the case for semantic
tokens.
- In #6768, 'Restart File' did not prompt a refresh, but it does now.
- VS Code does not immediately remove old inlay hints from the document
when they are applied. In #6768, this meant that inlay hints would
linger around for a bit once applied. To mitigate this issue, this PR
adjusts the inlay hint edit delay logic to identify edits sent from the
client as being inlay hint applications, and sets the edit delay to 0ms
for the inlay hint requests following it. This means that inlay hints
are now applied immediately.
- In #6768, hovering over single-letter auto-implicit inlay hints was a
bit finicky because VS Code uses the regular cursor icon on inlay hints,
not the thin text cursor icon, which means that it is easy to put the
cursor in the wrong spot. We now add the separation character (` ` or
`{`) preceding an auto-implicit to the hover range as well, which makes
hovering over inlay hints much smoother.
This PR adds support for plugins to the frontend and server.
Implementation-wise, this adds a `plugins` argument to `runFrontend`,
`processHeader`, amd `importModules`, a `plugins` field to
`SetupImportsResult` and `FileSetupResult`. and a `pluginsPath` field to
`LeanPaths`, and then threads the value through these.
This PR adds preliminary support for inlay hints, as well as support for
inlay hints that denote the auto-implicits of a function. Hovering over
an auto-implicit displays its type and double-clicking the auto-implicit
inserts it into the text document.
This PR is an extension of #3910.
### Known issues
- In VS Code, when inserting an inlay hint, the inlay hint may linger
for a couple of seconds before it disappears. This is a defect of the VS
Code implementation of inlay hints and cannot adequately be resolved by
us.
- When making a change to the document, it may take a couple of seconds
until the inlay hints respond to the change. This is deliberate and
intended to reduce the amount of inlay hint flickering while typing. VS
Code has a mechanism of its own for this, but in my experience it is
still far too sensitive without additional latency.
- Inserting an auto-implicit inlay hint that depends on an auto-implicit
meta-variable causes a "failed to infer binder type" error. We can't
display these meta-variables in the inlay hint because they don't have a
user-displayable name, so it is not clear how to resolve this problem.
- Inlay hints are currently always resolved eagerly, i.e. we do not
support the `textDocument/inlayHint/resolve` request yet. Implementing
support for this request is future work.
### Other changes
- Axioms did not support auto-implicits due to an oversight in the
implementation. This PR ensures they do.
- In order to reduce the amount of inlay hint flickering when making a
change to the document, the language server serves old inlay hints for
parts of the file that have not been processed yet. This requires LSP
request handler state (that sometimes must be invalidated on
`textDocument/didChange`), so this PR introduces the notion of a
stateful LSP request handler.
- The partial response mechanism that we use for semantic tokens, where
we simulate incremental LSP responses by periodically emitting refresh
requests to the client, is generalized to accommodate both inlay hints
and semantic tokens. Additionally, it is made more robust to ensure that
we never emit refresh requests while a corresponding request is in
flight, which causes VS Code to discard the respond of the request, as
well as to ensure that we keep prompting VS Code to send another request
if it spuriously decides not to respond to one of our refresh requests.
- The synthetic identifier of an `example` had the full declaration as
its (non-canonical synthetic) range. Since we need a reasonable position
for the identifier to insert an inlay hint for the auto-implicits of an
`example`, we change the (canonical synthetic) range of the synthetic
identifier to that of the `example` keyword.
- The semantic highlighting request handling is moved to a separate
file.
### Breaking changes
- The semantic highlighting request handler is not a pure request
handler anymore, but a stateful one. Notably, this means that clients
that extend the semantic highlighting of the Lean language server with
the `chainLspRequestHandler` function must now use the
`chainStatefulLspRequestHandler` function instead.
This PR makes it harder to create "fake" theorems about definitions that
are stubbed-out with `sorry` by ensuring that each `sorry` is not
definitionally equal to any other. For example, this now fails:
```lean
example : (sorry : Nat) = sorry := rfl -- fails
```
However, this still succeeds, since the `sorry` is a single
indeterminate `Nat`:
```lean
def f (n : Nat) : Nat := sorry
example : f 0 = f 1 := rfl -- succeeds
```
One can be more careful by putting parameters to the right of the colon:
```lean
def f : (n : Nat) → Nat := sorry
example : f 0 = f 1 := rfl -- fails
```
Most sources of synthetic sorries (recall: a sorry that originates from
the elaborator) are now unique, except for elaboration errors, since
making these unique tends to cause a confusing cascade of errors. In
general, however, such sorries are labeled. This enables "go to
definition" on `sorry` in the Infoview, which brings you to its origin.
The option `set_option pp.sorrySource true` causes the pretty printer to
show source position information on sorries.
**Details:**
* Adds `Lean.Meta.mkLabeledSorry`, which creates a sorry that is labeled
with its source position. For example, `(sorry : Nat)` might elaborate
to
```
sorryAx (Lean.Name → Nat) false
`lean.foo.12.8.12.13.8.13._sorry._@.lean.foo._hyg.153
```
It can either be made unique (like the above) or merely labeled. Labeled
sorries use an encoding that does not impact defeq:
```
sorryAx (Unit → Nat) false (Function.const Lean.Name ()
`lean.foo.14.7.13.7.13.69._sorry._@.lean.foo._hyg.174)
```
* Makes the `sorry` term, the `sorry` tactic, and every elaboration
failure create labeled sorries. Most are unique sorries, but some
elaboration errors are labeled sorries.
* Renames `OmissionInfo` to `DelabTermInfo` and adds configuration
options to control LSP interactions. One field is a source position to
use for "go to definition". This is used to implement "go to definition"
on labeled sorries.
* Makes hovering over a labeled `sorry` show something friendlier than
that full `sorryAx` expression. Instead, the first hover shows the
simplified ``sorry `«lean.foo:48:11»``. Hovering over that hover shows
the full `sorryAx`. Setting `set_option pp.sorrySource true` makes
`sorry` always start with printing with this source position
information.
* Removes `Lean.Meta.mkSyntheticSorry` in favor of `Lean.Meta.mkSorry`
and `Lean.Meta.mkLabeledSorry`.
* Changes `sorryAx` so that the `synthetic` argument is no longer
optional.
* Gives `addPPExplicitToExposeDiff` awareness of labeled sorries. It can
set `pp.sorrySource` when source positions differ.
* Modifies the delaborator framework so that delaborators can set Info
themselves without it being overwritten.
Incidentally closes#4972.
Inspired by [this Zulip
thread](https://leanprover.zulipchat.com/#narrow/channel/287929-mathlib4/topic/Is.20a.20.60definition_wanted.60.20keyword.20possible.3F/near/477260277).
This PR enables tactic completion in the whitespace of a tactic proof
and adds tactic docstrings to the completion menu.
Future work:
- A couple of broken tactic completions: This is due to tactic
completion now using @david-christiansen's `Tactic.Doc.allTacticDocs` to
obtain the tactic docstrings and should be fixed soon.
- Whitespace tactic completion in tactic combinators: This requires
changing the syntax of tactic combinators to produce a syntax node that
makes it clear that a tactic is expected at the given position.
Closes#1651.
This PR enables the use of incrementality for completion in tactic
blocks. Consider the following example:
```lean
example : True := by
have : True := T
sleep 10000
```
Before this PR, in order to respond to a completion request after `T`,
`sleep 10000` has to complete first since the command must be fully
elaborated. After this PR, the completion request is responded to
immediately.
