This PR adds configuration options for
`decide`/`decide!`/`native_decide` and refactors the tactics to be
frontends to the same backend. Adds a `+revert` option that cleans up
the local context and reverts all local variables the goal depends on,
along with indirect propositional hypotheses. Makes `native_decide` fail
at elaboration time on failure without sacrificing performance (the
decision procedure is still evaluated just once). Now `native_decide`
supports universe polymorphism.
Closes#2072
As far as I can tell, the ability to pass a structure instance to a
deriving handler is not actually used in practice. It didn't seem to be
used in the test suite, at least.
Do we want to remove this, or do we want to use and document it? This PR
removes it, but that's not something I feel strongly about - but seeing
if it breaks Mathlib is a useful data point.
This command comes from Lean 3, which I had previously ported and
contributed to Batteries (née Std). In this new version, `#where`
produces actual command Syntax for all features of a top-level scope
(rather than splicing together strings), and it also now reports
included variables.
---------
Co-authored-by: Kim Morrison <kim@tqft.net>
This PR simplifies the signature of `Array.mapIdx`, to take a function
`f : Nat \to \a \to \b` rather than a function `f : Fin as.size \to \a
\to \b`.
Lean doesn't actually use the extra generality anywhere (so in fact this
change *simplifies* all the call sites of `Array.mapIdx`, since we no
longer need to throw away the proof).
This change would make the function signature equivalent to
`List.mapIdx`, hence making it easier to write verification lemmas.
We keep the original behaviour as `Array.mapFinIdx`.
The `decide!` tactic is like `decide`, but when it tries reducing the
`Decidable` instance it uses kernel reduction rather than the
elaborator's reduction.
The kernel ignores transparency, so it can unfold all definitions (for
better or for worse). Furthermore, by using kernel reduction we can
cache the result as an auxiliary lemma — this is more efficient than
`decide`, which needs to reduce the instance twice: once in the
elaborator to check whether the tactic succeeds, and once again in the
kernel during final typechecking.
While RFC #5629 proposes a `decide!` that skips checking altogether
during elaboration, with this PR's `decide!` we can use `decide!` as
more-or-less a drop-in replacement for `decide`, since the tactic will
fail if kernel reduction fails.
This PR also includes two small fixes:
- `blameDecideReductionFailure` now uses `withIncRecDepth`.
- `Lean.Meta.zetaReduce` now instantiates metavariables while zeta
reducing.
Some profiling:
```lean
set_option maxRecDepth 2000
set_option trace.profiler true
set_option trace.profiler.threshold 0
theorem thm1 : 0 < 1 := by decide!
theorem thm1' : 0 < 1 := by decide
theorem thm2 : ∀ x < 400, x * x ≤ 160000 := by decide!
theorem thm2' : ∀ x < 400, x * x ≤ 160000 := by decide
/-
[Elab.command] [0.003655] theorem thm1 : 0 < 1 := by decide!
[Elab.command] [0.003164] theorem thm1' : 0 < 1 := by decide
[Elab.command] [0.133223] theorem thm2 : ∀ x < 400, x * x ≤ 160000 := by decide!
[Elab.command] [0.252310] theorem thm2' : ∀ x < 400, x * x ≤ 160000 := by decide
-/
```
---------
Co-authored-by: Joachim Breitner <mail@joachim-breitner.de>
This refactors and improves the `#eval` command, introducing some new
features.
* Now evaluated results can be represented using `ToExpr` and pretty
printing. This means **hoverable output**. If `ToExpr` fails, it then
tries `Repr` and then `ToString`. The `eval.pp` option controls whether
or not to try `ToExpr`.
* There is now **auto-derivation** of `Repr` instances, enabled with the
`pp.derive.repr` option (default to **true**). For example:
```lean
inductive Baz
| a | b
#eval Baz.a
-- Baz.a
```
It simply does `deriving instance Repr for Baz` when there's no way to
represent `Baz`. If core Lean gets `ToExpr` derive handlers, they could
be used here as well.
* The option `eval.type` controls whether or not to include the type in
the output. For now the default is false.
* Now things like `#eval do return 2` work. It tries using
`CommandElabM`, `TermElabM`, or `IO` when the monad is unknown.
* Now there is no longer `Lean.Eval` or `Lean.MetaEval`. These each used
to be responsible for both adapting monads and printing results. The
concerns have been split into two. (1) The `MonadEval` class is
responsible for adapting monads for evaluation (it is similar to
`MonadLift`, but instances are allowed to use default data when
initializing state) and (2) finding a way to represent results is
handled separately.
* Error messages about failed instance synthesis are now more precise.
Once it detects that a `MonadEval` class applies, then the error message
will be specific about missing `ToExpr`/`Repr`/`ToString` instances.
* Fixes a bug where `Repr`/`ToString` instances can't be found by
unfolding types "under the monad". For example, this works now:
```lean
def Foo := List Nat
def Foo.mk (l : List Nat) : Foo := l
#eval show Lean.CoreM Foo from do return Foo.mk [1,2,3]
```
* Elaboration errors now abort evaluation. This eliminates some
not-so-relevant error messages.
* Now evaluating a value of type `m Unit` never prints a blank message.
* Fixes bugs where evaluating `MetaM` and `CoreM` wouldn't collect log
messages.
The `run_cmd`, `run_elab`, and `run_meta` commands are now frontends for
`#eval`.
Where before we had
```lean
#check fun x : Nat => ?a
-- fun x ↦ ?m.7 x : (x : Nat) → ?m.6 x
```
Now by default we have
```lean
#check fun x : Nat => ?a
-- fun x => ?a : (x : Nat) → ?m.6 x
```
In particular, delayed assignment metavariables such as `?m.7` pretty
print using the name of the metavariable they are delayed assigned to,
suppressing the bound variables used in the delayed assignment (hence
`?a` rather than `?a x`). Hovering over `?a` shows `?m.7 x`.
The benefit is that users can see the user-provided name in local
contexts. A justification for this pretty printing choice is that `?m.7
x` is supposed to stand for `?a`, and furthermore it is just as opaque
to assignment in defeq as `?a` is (however, when synthetic opaque
metavariables are made assignable, delayed assignments can be a little
less assignable than true synthetic opaque metavariables).
The original pretty printing behavior can be recovered using `set_option
pp.mvars.delayed true`.
This PR also extends the documentation for holes and synthetic holes,
with some technical details about what delayed assignments are. This
likely should be moved to the reference manual, but for now it is
included in this docstring.
(This PR is a simplified version of #3494, which has a round-trippable
notation for delayed assignments. The pretty printing in this PR is
unlikely to round trip, but it is better than the current situation,
which is that delayed assignment metavariables never round trip, and
plus it does not require introducing a new notation.)
Obviously a link to the web docs isn't ideal, but having hovers
available on the symbol is much better than nothing.
---------
Co-authored-by: David Thrane Christiansen <david@davidchristiansen.dk>
Co-authored-by: Sebastian Ullrich <sebasti@nullri.ch>
This solves the issue where certain subexpressions are lacking syntax
hovers because the hover text is not "builtin" - it only shows up if the
`Parser` constant is imported in the environment. For top level syntaxes
this is not a problem because `builtin_term_parser` will automatically
add this doc information, but nested syntaxes don't get the same
treatment.
We could walk the expression and add builtin docs recursively, but this
is somewhat expensive and unnecessary given that it's a fixed list of
declarations in lean core. Moreover, there are reasons to want to
control which syntax nodes actually get hovers, and while a better
system for that is forthcoming, for now it can be achieved by
strategically not applying the `@[builtin_doc]` attribute.
Fixes#3842
in principle we'd like to use the existing parser
```
"?" >> (ident <|> hole)
```
but somehow annotate it so that hovering the `hole` will not show the
hole's hover. But for now it was easier to just change the parser to
```
"?" >> (ident <|> "_")
```
and be done with it.
Fixes#5021
previously, `#eval` would happily evaluate expressions that contain
`sorry`, either explicitly or because of failing tactics. In conjunction
with operations like array access this can lead to the lean process
crashing, which isn't particularly great.
So how `#eval` will refuse to run code that (transitively) depends on
the `sorry` axiom (using the same code as `#print axioms`).
If the user really wants to run it, they can use `#eval!`.
Closes#1697
Also extends existing definition for `getScope`/`getScopes` and
clarifies that the `end` command is optional at the end of a file.
---------
Co-authored-by: Kyle Miller <kmill31415@gmail.com>
the internal constructions for structural and well-founded recursion
use plenty of `PProd` and `MProd`, and reading these, deeply
nested and in prefix notation, is unnecessarily troublesome.
Therefore this introduces notations
```
a ×ₚ b -- PProd a b
a ×ₘ b -- MProd a b
()ₚ -- PUnit.unit
(x,y,z)ₚ -- PProd.mk x (PProd.mk y z)
(x,y,z)ₘ -- MProd.mk x (MProd.mk y z)
```
(This is part 1, the rest will follow in #4730 after a stage0 update.)
This implements the `termination_by structural` syntax proposed in
#3909.
I went with `termination_by structural` over, say,
`termination_by (config := {method := .structural})` mainly because it
was
easier to get going (otherwise I’d have to look into how to define
recursive
parsers, as `Parser.config` depends on `term` and `termination_by` is
part of
term. But also because I find it more ergonomic and aesthetic as a user.
But syntax can still change.
The `termination_by?` syntax will no longer force well-founded
recursion,
and instead the inferred `termination_by structurally` annotation will
be shown
if structural termination is possible.
While I was it, this fixes#4546 the easy way (log errors about but
otherwise
ignore incomplete `termination_by` sets for mutual recursion). Maybe we
get
multiple replacements (#4551), but even then this this good behavior.
Involves a bit of shuffling around `TerimationHints` (now validated for
a
clique already by `PreDefinition.main`) and `TerminationArguments` (now
lifted
out of the `WF` namespace, and a bit simplified).
Fixes#3909
---------
Co-authored-by: Richard Kiss <him@richardkiss.com>
This is the groundwork for a tactic index in generated documentation, as
there was in Lean 3. There are a few challenges to getting this to work
well in Lean 4:
* There's no natural notion of *tactic identity* - a tactic may be
specified by multiple syntax rules (e.g. the pattern-matching version of
`intro` is specified apart from the default version, but both are the
same from a user perspective)
* There's no natural notion of *tactic name* - here, we take the
pragmatic choice of using the first keyword atom in the tactic's syntax
specification, but this may need to be overridable someday.
* Tactics are extensible, but we don't want to allow arbitrary imports
to clobber existing tactic docstrings, which could become unpredictable
in practice.
For tactic identity, this PR introduces the notion of a *tactic
alternative*, which is a `syntax` specification that is really "the same
as" an existing tactic, but needs to be separate for technical reasons.
This provides a notion of tactic identity, which we can use as the basis
of a tactic index in generated documentation. Alternative forms of
tactics are specified using a new `@[tactic_alt IDENT]` attribute,
applied to the new tactic syntax. It is an error to declare a tactic
syntax rule to be an alternative of another one that is itself an
alternative. Documentation hovers now take alternatives into account,
and display the docs for the canonical name.
*Tactic tags*, created with the `register_tactic_tag` command, specify
tags that may be applied to tactics. This is intended to be used by
doc-gen and Verso. Tags may be applied using the `@[tactic_tag TAG1 TAG2
...]` attribute on a canonical tactic parser, which may be used in any
module to facilitate downstream projects introducing tags that apply to
pre-existing tactics. Tags may not be removed, but it's fine to
redundantly add them. The collection of tags, and the tactics to which
they're applied, can be seen using the `#print tactic tags` command.
*Extension documentation* provides a structured way to document
extensions to tactics. The resulting documentation is gathered into a
bulleted list at the bottom of the tactic's docstring. Extensions are
added using the `tactic_extension TAC` command. This can be used when
adding new interpretations of a tactic via `macro_rules`, when extending
some table or search index used by the tactic, or in any other way. It
is a command to facilitate its flexible use with various extension
mechanisms.
Extends Lean's incremental reporting and reuse between commands into
various steps inside declarations:
* headers and bodies of each (mutual) definition/theorem
* `theorem ... := by` for each contained tactic step, including
recursively inside supported combinators currently consisting of
* `·` (cdot), `case`, `next`
* `induction`, `cases`
* macros such as `next` unfolding to the above
*Incremental reuse* means not recomputing any such steps if they are not
affected by a document change. *Incremental reporting* includes the
parts seen in the recording above: the progress bar and messages. Other
language server features such as hover etc. are *not yet* supported
incrementally, i.e. they are shown only when the declaration has been
fully processed as before.
---------
Co-authored-by: Scott Morrison <scott.morrison@gmail.com>
To eliminate parsing differences between Windows and other platforms,
the frontend now normalizes all CRLF line endings to LF, like [in
Rust](https://github.com/rust-lang/rust/issues/62865).
Effects:
- This makes Lake hashes be faithful to what Lean sees (Lake already
normalizes line endings before computing hashes).
- Docstrings now have normalized line endings. In particular, this fixes
`#guard_msgs` failing multiline tests for Windows users using CRLF.
- Now strings don't have different lengths depending on the platform.
Before this PR, the following theorem is true for LF and false for CRLF
files.
```lean
example : "
".length = 1 := rfl
```
Note: the normalization will take `\r\r\n` and turn it into `\r\n`. In
the elaborator, we reject loose `\r`'s that appear in whitespace. Rust
instead takes the approach of making the normalization routine fail.
They do this so that there's no downstream confusion about any `\r\n`
that appears.
Implementation note: the LSP maintains its own copy of a source file
that it updates when edit operations are applied. We are assuming that
edit operations never split or join CRLFs. If this assumption is not
correct, then the LSP copy of a source file can become slightly out of
sync. If this is an issue, there is some discussion
[here](https://github.com/leanprover/lean4/pull/3903#discussion_r1592930085).
even when rewriting the type of `h` becuase there is no expected type.
(When there is an expected type, it already tried both orientations.)
Also feeble attempt to include this information in the docstring without
writing half a manual chapter.
Go-to-def on `@[builtin_term_parser]` should go to the line
```lean
builtin_initialize registerBuiltinParserAttribute `builtin_term_parser ``Category.term
```
not
```lean
/-- `term` is the builtin syntax category for terms. ... -/
def term : Category := {}
```
While implementing #3925, I noticed that the performance of the
`textDocument/semanticTokens/full` request is *extremely* bad due to a
quadratic implementation. Specifically, on my machine, computing the
full semantic tokens for `Lean/Elab/Do.lean` took a full 5s. In
practice, this means that while elaborating the file, one core is
entirely busy with computing the semantic tokens for the file.
This PR fixes this performance bug by re-implementing the semantic token
handling, reducing the latency for `Lean/Elab/Do.lean` from 5s to 60ms.
As a result, the overly cautious refresh latency of 5s in #3925 can
easily be reduced to 2s again.
Since the previous semantic tokens implementation used a very brittle
hack to identify projections, this PR also changes the projection
notation elaboration to augment the `InfoTree` syntax for the field of a
projection with a special syntax node of kind
`Lean.Parser.Term.identProjKind`. With this syntax kind, projection
fields can now easily be identified in the `InfoTree`.
This fixes an issue where the completion would use info nodes before the
cursor for computing completions.
Fixes https://github.com/leanprover/lean4/issues/3462.
ToDo:
- [x] Fix test failures for completions that previously worked by
accident (cc: @Kha)
- [x] stage0 update
---------
Co-authored-by: Sebastian Ullrich <sebasti@nullri.ch>
Commands that can optionally parse an `ident` or parse any number of
`ident`s generally should require that the `ident` use `colGt`. This
keeps typos in commands from being interpreted as identifiers.
For example, without this rule,
```
universe u
Open Lean
````
parses the same as `universe u Open Lean`. It would be better to get an
error on `Open`.
This PR adds `checkColGt` to `section`, `namespace`, `end`, `variable`,
and `universe`.
Closes#2684
