This PR enables the language server to present multiple disjoint line
ranges as being worked on. Even before parallelism lands, we make use of
this feature to show post-elaboration tasks such as kernel checking on
the first line of a declaration to distinguish them from the final
tactic step.
This PR fixes a bug where both the inlay hint change invalidation logic
and the inlay hint edit delay logic were broken in untitled files.
Thanks to @Julian for spotting this!
This PR implements a number of refinements for the auto-implicit inlay
hints implemented in #6768.
Specifically:
- In #6768, there was a bug where the inlay hint edit delay could
accumulate on successive edits, which meant that it could sometimes take
much longer for inlay hints to show up. This PR implements the basic
infrastructure for request cancellation and implements request
cancellation for semantic tokens and inlay hints to resolve the issue.
With this edit delay bug fixed, it made more sense to increase the edit
delay slightly from 2000ms to 3000ms.
- In #6768, we applied the edit delay to every single inlay hint request
in order to reduce the amount of inlay hint flickering. This meant that
the edit delay also had a significant effect on how far inlay hints
would lag behind the file progress bar. This PR adjusts the edit delay
logic so that it only affects requests sent directly after a
corresponding `didChange` notification. Once the edit delay is used up,
all further semantic token requests are responded to without delay, so
that the only latency that affects how far the inlay hints lag behind
the progress bar is how often we emit refresh requests and how long VS
Code takes to respond to them.
- For inlay hints, refresh requests are now emitted 500ms after a
response to an inlay hint request, not 2000ms, which means that after
the edit delay, inlay hints should only lag behind the progress bar by
about up to 500ms. This is justifiable for inlay hints because the
response should be much smaller than e.g. is the case for semantic
tokens.
- In #6768, 'Restart File' did not prompt a refresh, but it does now.
- VS Code does not immediately remove old inlay hints from the document
when they are applied. In #6768, this meant that inlay hints would
linger around for a bit once applied. To mitigate this issue, this PR
adjusts the inlay hint edit delay logic to identify edits sent from the
client as being inlay hint applications, and sets the edit delay to 0ms
for the inlay hint requests following it. This means that inlay hints
are now applied immediately.
- In #6768, hovering over single-letter auto-implicit inlay hints was a
bit finicky because VS Code uses the regular cursor icon on inlay hints,
not the thin text cursor icon, which means that it is easy to put the
cursor in the wrong spot. We now add the separation character (` ` or
`{`) preceding an auto-implicit to the hover range as well, which makes
hovering over inlay hints much smoother.
This PR adds support for plugins to the frontend and server.
Implementation-wise, this adds a `plugins` argument to `runFrontend`,
`processHeader`, amd `importModules`, a `plugins` field to
`SetupImportsResult` and `FileSetupResult`. and a `pluginsPath` field to
`LeanPaths`, and then threads the value through these.
This PR adds preliminary support for inlay hints, as well as support for
inlay hints that denote the auto-implicits of a function. Hovering over
an auto-implicit displays its type and double-clicking the auto-implicit
inserts it into the text document.
This PR is an extension of #3910.
### Known issues
- In VS Code, when inserting an inlay hint, the inlay hint may linger
for a couple of seconds before it disappears. This is a defect of the VS
Code implementation of inlay hints and cannot adequately be resolved by
us.
- When making a change to the document, it may take a couple of seconds
until the inlay hints respond to the change. This is deliberate and
intended to reduce the amount of inlay hint flickering while typing. VS
Code has a mechanism of its own for this, but in my experience it is
still far too sensitive without additional latency.
- Inserting an auto-implicit inlay hint that depends on an auto-implicit
meta-variable causes a "failed to infer binder type" error. We can't
display these meta-variables in the inlay hint because they don't have a
user-displayable name, so it is not clear how to resolve this problem.
- Inlay hints are currently always resolved eagerly, i.e. we do not
support the `textDocument/inlayHint/resolve` request yet. Implementing
support for this request is future work.
### Other changes
- Axioms did not support auto-implicits due to an oversight in the
implementation. This PR ensures they do.
- In order to reduce the amount of inlay hint flickering when making a
change to the document, the language server serves old inlay hints for
parts of the file that have not been processed yet. This requires LSP
request handler state (that sometimes must be invalidated on
`textDocument/didChange`), so this PR introduces the notion of a
stateful LSP request handler.
- The partial response mechanism that we use for semantic tokens, where
we simulate incremental LSP responses by periodically emitting refresh
requests to the client, is generalized to accommodate both inlay hints
and semantic tokens. Additionally, it is made more robust to ensure that
we never emit refresh requests while a corresponding request is in
flight, which causes VS Code to discard the respond of the request, as
well as to ensure that we keep prompting VS Code to send another request
if it spuriously decides not to respond to one of our refresh requests.
- The synthetic identifier of an `example` had the full declaration as
its (non-canonical synthetic) range. Since we need a reasonable position
for the identifier to insert an inlay hint for the auto-implicits of an
`example`, we change the (canonical synthetic) range of the synthetic
identifier to that of the `example` keyword.
- The semantic highlighting request handling is moved to a separate
file.
### Breaking changes
- The semantic highlighting request handler is not a pure request
handler anymore, but a stateful one. Notably, this means that clients
that extend the semantic highlighting of the Lean language server with
the `chainLspRequestHandler` function must now use the
`chainStatefulLspRequestHandler` function instead.
This PR uniformizes the naming of `enum`/`enumFrom` (on `List`) and
`zipWithIndex` (on `Array` on `Vector`), replacing all with `zipIdx`. At
the same time, we generalize to add an optional `Nat` parameter for the
initial value of the index (which previously existed, only for `List`,
as the separate function `enumFrom`).
This PR fixes a significant auto-completion performance regression that
was introduced in #5666, i.e. v4.14.0.
#5666 introduced tactic docstrings, which were attempted to be collected
for every single completion item. This is slow for hundreds of thousands
of completion items. To fix this, this PR moves the docstring
computation into the completion item resolution, which is only called
when users select a specific completion item in the UI.
A downside of this approach is that we currently can't test completion
item resolution, so we lose a few tests that cover docstrings in
completions in this PR.
Avoids build time overhead until the option is proven to speed up
average projects. Adds Init.Prelude (many tiny declarations, "worst
case") and Init.List.Sublist (many nontrivial theorems, "best case")
under -DElab.async=true as new benchmarks for tracking.
This PR allows the dot ident notation to resolve to the current
definition, or to any of the other definitions in the same mutual block.
Existing code that uses dot ident notation may need to have `nonrec`
added if the ident has the same name as the definition.
Closes#6601
This PR makes it harder to create "fake" theorems about definitions that
are stubbed-out with `sorry` by ensuring that each `sorry` is not
definitionally equal to any other. For example, this now fails:
```lean
example : (sorry : Nat) = sorry := rfl -- fails
```
However, this still succeeds, since the `sorry` is a single
indeterminate `Nat`:
```lean
def f (n : Nat) : Nat := sorry
example : f 0 = f 1 := rfl -- succeeds
```
One can be more careful by putting parameters to the right of the colon:
```lean
def f : (n : Nat) → Nat := sorry
example : f 0 = f 1 := rfl -- fails
```
Most sources of synthetic sorries (recall: a sorry that originates from
the elaborator) are now unique, except for elaboration errors, since
making these unique tends to cause a confusing cascade of errors. In
general, however, such sorries are labeled. This enables "go to
definition" on `sorry` in the Infoview, which brings you to its origin.
The option `set_option pp.sorrySource true` causes the pretty printer to
show source position information on sorries.
**Details:**
* Adds `Lean.Meta.mkLabeledSorry`, which creates a sorry that is labeled
with its source position. For example, `(sorry : Nat)` might elaborate
to
```
sorryAx (Lean.Name → Nat) false
`lean.foo.12.8.12.13.8.13._sorry._@.lean.foo._hyg.153
```
It can either be made unique (like the above) or merely labeled. Labeled
sorries use an encoding that does not impact defeq:
```
sorryAx (Unit → Nat) false (Function.const Lean.Name ()
`lean.foo.14.7.13.7.13.69._sorry._@.lean.foo._hyg.174)
```
* Makes the `sorry` term, the `sorry` tactic, and every elaboration
failure create labeled sorries. Most are unique sorries, but some
elaboration errors are labeled sorries.
* Renames `OmissionInfo` to `DelabTermInfo` and adds configuration
options to control LSP interactions. One field is a source position to
use for "go to definition". This is used to implement "go to definition"
on labeled sorries.
* Makes hovering over a labeled `sorry` show something friendlier than
that full `sorryAx` expression. Instead, the first hover shows the
simplified ``sorry `«lean.foo:48:11»``. Hovering over that hover shows
the full `sorryAx`. Setting `set_option pp.sorrySource true` makes
`sorry` always start with printing with this source position
information.
* Removes `Lean.Meta.mkSyntheticSorry` in favor of `Lean.Meta.mkSorry`
and `Lean.Meta.mkLabeledSorry`.
* Changes `sorryAx` so that the `synthetic` argument is no longer
optional.
* Gives `addPPExplicitToExposeDiff` awareness of labeled sorries. It can
set `pp.sorrySource` when source positions differ.
* Modifies the delaborator framework so that delaborators can set Info
themselves without it being overwritten.
Incidentally closes#4972.
Inspired by [this Zulip
thread](https://leanprover.zulipchat.com/#narrow/channel/287929-mathlib4/topic/Is.20a.20.60definition_wanted.60.20keyword.20possible.3F/near/477260277).
This PR moves `IO.Channel` and `IO.Mutex` from `Init` to `Std.Sync` and
renames them to `Std.Channel` and `Std.Mutex`.
Note that the original files are retained and the deprecation is written
manually as we cannot import `Std` from `Init` so this is the only way
to deprecate without a hard breaking change. In particular we do not yet
move `Std.Queue` from `Init` to `Std` both because it needs to be
retained for this deprecation to work but also because it is already
within the `Std` namespace and as such we cannot maintain two copies of
the file at once. After the deprecation period is finished `Std.Queue`
will find a new home in `Std.Data.Queue`.
This PR fixes a bug in structure instance field completion that caused
it to not function correctly for bracketed structure instances written
in Mathlib style.
This PR fixes a performance issue where the Lean language server would
walk the full project file tree every time a file was saved, blocking
the processing of all other requests and notifications and significantly
increasing overall language server latency after saving.
This issue was originally reported at
https://leanprover.zulipchat.com/#narrow/channel/270676-lean4/topic/Compiling.20too.20slow/near/484386515
and uncovered in a lengthy investigation. The performance bug that
causes the Lean language server to walk the full project file tree when
the file watcher for .ilean files is triggered was introduced when the
.ileans were first introduced, whereas the specific issue of file saving
also triggering the walk was introduced by #3247 in 4.8.0 and the use of
the file watcher for .lean files, which would then also trigger the
directory walk. Combining this with VS Code's auto-save feature causes
the language server to walk the full project file tree on every change
of the document.
It somehow hasn't really been much of an issue until now, but we still
do way too much work in the watchdog main loop. I'll look into resolving
that more general issue in the future.
This now occurs for some large completions downstream of `import
Mathlib`. I'd like to get rid of this `whnf` call entirely in the
future, but this is a decent quick mitigation.
* Now `getPathToBaseStructure?` can navigate to all parent structures,
not just through subobjects.
* Adds a "resolution order" for methods. This is the order that
generalized field notation visits parent structures when trying to
resolve names. The algorithm to compute a resolution order is the
commonly used C3 (used for instance by Python). By default we use a
relaxed version of the algorithm that tolerates inconsistencies. Using
`set_option structure.strictResolutionOrder true` makes inconsistent
parent orderings into warnings.
* This makes generalized field notation be able to resolve names for all
parent structures, not just those that are embedded as subobjects.
Closes#3467. (And addresses side note in #1881.)
* Modifies `getAllParentStructures` to return *all* parents. This
improves dot completion in the editor.
It's difficult to distinguish theorems from regular definitions in the
completion menu, which is annoying when using completion for searching
one or the other. This PR makes theorem completions use the "Eureka!"
icon ()
to distinguish them more clearly from other completions.
NB: We are very limited in terms of which icons we can pick here since
[the completion kinds provided by LSP / VS
Code](https://code.visualstudio.com/docs/editor/intellisense#_types-of-completions)
are optimized for object-oriented programming languages, but I think
this choice strikes a nice balance between being easy to identify,
having some visual connection to theorem proving and not being used a
lot in other languages and thus not clashing with pre-existing
associations.
This PR simplifies the signature of `Array.mapIdx`, to take a function
`f : Nat \to \a \to \b` rather than a function `f : Fin as.size \to \a
\to \b`.
Lean doesn't actually use the extra generality anywhere (so in fact this
change *simplifies* all the call sites of `Array.mapIdx`, since we no
longer need to throw away the proof).
This change would make the function signature equivalent to
`List.mapIdx`, hence making it easier to write verification lemmas.
We keep the original behaviour as `Array.mapFinIdx`.
Should ensure we visit at most as many expr nodes as in the final expr
instead of many possibly overlapping mvar assignments. This is likely
the only way we can ensure acceptable performance in all cases.
---------
Co-authored-by: Kim Morrison <kim@tqft.net>
This PR ensures that deprecated declarations are displayed with a
strikethrough markup in the completion popup of VS Code and that the
docstring of a completion item denotes the meta-data of the deprecation.
This PR enables tactic completion in the whitespace of a tactic proof
and adds tactic docstrings to the completion menu.
Future work:
- A couple of broken tactic completions: This is due to tactic
completion now using @david-christiansen's `Tactic.Doc.allTacticDocs` to
obtain the tactic docstrings and should be fixed soon.
- Whitespace tactic completion in tactic combinators: This requires
changing the syntax of tactic combinators to produce a syntax node that
makes it clear that a tactic is expected at the given position.
Closes#1651.
This PR fixes three problems:
- When the language server is being stopped in a non-normal way without
going through the regular LSP shutdown protocol (e.g. by closing VS
Code), it could sometimes happen that both the watchdog and the file
worker were not properly terminated and lingered around forever,
resulting in zombie processes (#5296)
- When the file worker crashes and the user restarts it by making a
change to the document, the file worker would produce incorrect
diagnostics for the document until the file is restarted.
- (Minor) When the file worker would crash during initialization, the
error diagnostic would be reported on stderr instead of stdout
The deadlock-induced termination issue from #5296 should be resolved by
the following measures:
- The watchdog main task is always terminated with `IO.Process.exit` to
ensure that it terminates even if some other tasks in the process are
still running.
- The file worker communication task in the watchdog no longer waits for
the file worker process to terminate when writing to the client fails,
only when reading from the file worker fails.
- When the watchdog shuts down (either as a result of an orderly or a
non-normal shutdown), instead of waiting for the file worker
communication tasks to complete, it kills the file worker process. The
rationale behind this is that the file worker currently should have no
essential work to complete if the server is being stopped anyways, and
so waiting for the communication task is not necessary.
The file worker diagnostic desync after a crash was caused by us
tracking changes to the document of a crashed file worker twice: Once as
part of the document, and once as part of the queued messages to the
file worker. This meant that when the file worker was restarted, it
would receive the changes made to the document while the file worker was
crashed twice, leading to a desynced document state.
(Probably) fixes#5296.
Macros sometimes create auxiliary types and instances about them, and
they rely on the instance name generate to create unique names in that
case.
This modifies the automatic name generator to add a fresh macro scope to
the generated name if any of the constants in the type of the instance
themselves have macro scopes.
Closes#2044
On a document edit, it may be the case that the first nontrivial
snapshot is e.g. for a macro-generated tactic call that does not have
range information. In that case, instead of just displaying nothing, we
should fall back to a previous range, in this case of the original
tactic macro.
When the elaborator doesn't provide us with any `CompletionInfo`, we
currently provide no completions whatsoever. But in many cases, we can
still provide some helpful identifier completions without elaborator
information. This PR adds a fallback mode for this situation.
There is more potential here, but this should be a good start.
In principle, this issue alleviates #5172 (since we now provide
completions in these contexts). I'll leave it up to an elaboration
maintainer whether we also want to ensure that the completion infos are
provided correctly in these cases.
Fixes#4455, fixes#4705, fixes#5219
Also fixes a minor bug where a dot in brackets would report incorrect
completions instead of no completions.
---------
Co-authored-by: Sebastian Ullrich <sebasti@nullri.ch>
This PR enables the use of incrementality for completion in tactic
blocks. Consider the following example:
```lean
example : True := by
have : True := T
sleep 10000
```
Before this PR, in order to respond to a completion request after `T`,
`sleep 10000` has to complete first since the command must be fully
elaborated. After this PR, the completion request is responded to
immediately.
This PR roughly halves the time needed to load the .ilean files by
optimizing the JSON parser and the conversion from JSON to Lean data
structures.
The code is optimized roughly as follows:
- String operations are inlined more aggressively
- Parsers are changed to use new `String.Iterator` functions `curr'` and
`next'` that receive a proof and hence do not need to perform an
additional check
- The `RefIdent` of .ilean files now uses a `String` instead of a `Name`
to avoid the expensive parse step from `String` to `Name` (despite the
fact that we only very rarely actually need a `Name` in downstream code)
- Instead of `List`s and `Subarray`s, the JSON to Lean conversion now
directly passes around arrays and array indices to avoid redundant
boxing
- Parsec's `peek?` sometimes generates redundant `Option` wrappers
because the generation of basic blocks interferes with the ctor-match
optimization, so it is changed to use an `isEof` check where possible
- Early returns and inline-do-blocks cause the code generator to
generate new functions, which then interfere with optimizations, so they
are now avoided
- Mutual defs are used instead of unspecialized passing of higher-order
functions to generate faster code
- The object parser is made tail-recursive
This PR also fixes a stack overflow in `Lean.Json.compress` that would
occur with long lists and adds a benchmark for the .ilean roundtrip
(compressed pretty-printing -> parsing).
This PR fixes a small bug where over time, "import out of data" messages
would accumulate in files when their size changed before restarting its
file worker.
