This PR adjusts the results of `inferInstanceAs` and the `def` `deriving` handler to conform to recently strengthened restrictions on reducibility. This change ensures that when deriving or inferring an instance for a semireducible type definition, the definition's RHS is not leaked when the instance is reduced at lower than semireducible transparency.
More specifically, given the "source type" and "target type" (the given and expected type for `inferInstanceAs`, the right-hand side and applied left-hand side of the `def` for `deriving`), we synthesize an instance for the source type and then unfold and rewrap its components (fields, nested instances) as necessary to make them compatible with the target type. The individual steps are represented by the following options, which all default to enabled and can be disabled to help with porting:
- `backward.inferInstanceAs.wrap`: master switch for instance adjustment in both `inferInstanceAs` and the default `deriving` handler
- `backward.inferInstanceAs.wrap.reuseSubInstances`: reuse existing instances for the target type for sub-instance fields to avoid non-defeq instance diamonds
- `backward.inferInstanceAs.wrap.instances`: wrap non-reducible instances in auxiliary definitions
- `backward.inferInstanceAs.wrap.data`: wrap data fields in auxiliary definitions (proof fields are always wrapped)
This PR is an extension and rewrite of prior work in Mathlib: https://github.com/leanprover-community/mathlib4/pull/36420
Last(?) part of fix for #9077🤖 Prepared with Claude Code
# Breaking changes
Proofs that relied on the prior "defeq abuse" of these instance or that depended on their specific structure may need adjustments. As `inferInstanceAs A` now needs to know the source and target types exactly before it can continue, it cannot be used anymore as a synonym for `(inferInstance : A)`, use the latter instead when source and target type are identical.
This PR further enforces that all modules used in compile-time execution
must be meta imported in preparation for enabling
https://github.com/leanprover/lean4/pull/10291
# Breaking changes
Metaprograms that call `compileDecl` directly may now need to call
`markMeta` first where appropriate, possibly based on the value of
`isMarkedMeta` of existing decls. `addAndCompile` should be split into
`addDecl` and `compileDecl` for this in order to insert the call in
between.
This PR adds `optType` support to the `doPatDecl` parser, allowing
`let ⟨width, height⟩ : Nat × Nat ← action` in do-notation. Previously,
only
the less ergonomic `let ⟨width, height⟩ : Nat × Nat := ← action`
workaround
was available. The type annotation is propagated to the monadic action
as an
expected type, matching `doIdDecl`'s existing behavior.
Both the legacy and new (BuiltinDo) elaborators are updated.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This PR adjusts the module parser to set the leading whitespace of the
first token to the whitespace up to that token. If there are no actual
tokens in the file, the leading whitespace is set on the final (empty)
EOI token. This ensures that we do not lose the initial whitespace (e.g.
comments) of a file in `Syntax`.
(Tests generated/adjusted by Claude)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This PR adds `deriving noncomputable instance Foo for Bar` syntax so
that delta-derived instances can be marked noncomputable. Previously,
when the underlying instance was noncomputable, `deriving instance`
would fail with an opaque async compilation error.
Now:
- `deriving noncomputable instance Foo for Bar` marks the generated
instance as noncomputable (using `addDecl` + `addNoncomputable` instead
of `addAndCompile`)
- `deriving instance Foo for Bar` pre-checks for noncomputable
dependencies and gives an actionable error with a "Try this:" suggestion
pointing to the noncomputable variant
- For handler-based deriving (inductives/structures), `noncomputable`
sets `isNoncomputable` on the scope
The `optDefDeriving` and `optDeriving` trailing parsers are updated with
`notSymbol "noncomputable"` to prevent them from stealing the parse of
`deriving noncomputable instance ...`.
🤖 Prepared with Claude Code
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This PR adds the pretty printer option `pp.mdata`, which causes the
pretty printer to annotate terms with any metadata that is present. For
example,
```lean
set_option pp.mdata true
/-- info: [mdata noindex:true] 2 : Nat -/
#guard_msgs in #check no_index 2
```
The `[mdata ...] e` syntax is only for pretty printing.
Thanks to @Rob23oba for an initial version.
Closes#10929
This PR adds the experimental `idbg e`, a new do-element (and term)
syntax for live debugging between the language server and a running
compiled Lean program.
When placed in a `do` block, `idbg` captures all local variables in
scope and expression `e`, then:
- **In the language server**: starts a TCP server on localhost waiting
for the running program to
connect; the editor will mark this part of the program as "in progress"
during this wait but that
will not block `lake build` of the project.
- **In the compiled program**: on first execution of the `idbg` call
site, connects to the server,
receives the expression, compiles and evaluates it using the program's
actual runtime values, and
sends the `repr` result back.
The result is displayed as an info diagnostic on the `idbg` keyword. The
expression `e` can be
edited while the program is running - each edit triggers re-elaboration
of `e`, a new TCP exchange,
and an updated result. This makes `idbg` a live REPL for inspecting and
experimenting with
program state at a specific point in execution. Only when `idbg` is
inserted, moved, or removed does
the program need to be recompiled and restarted.
# Known Limitations
* The program will poll for the server for up to 10 minutes and needs to
be killed manually
otherwise.
* Use of multiple `idbg` at once untested, likely too much overhead from
overlapping imports without
further changes.
* `LEAN_PATH` must be properly set up so compiled program can import its
origin module.
* Untested on Windows and macOS.
This PR derives the linear order on string positions (`String.Pos.Raw`,
`String.Pos`, `String.Slice.Pos`) via `Std.LinearOrderPackage`, which
ensures that all data-carrying and propositional instances are present.
Previously, we were misssing some, like `Ord`.
This PR adds a new, extensible `do` elaborator. Users can opt into the
new elaborator by unsetting the option `backward.do.legacy`.
New elaborators for the builtin `doElem` syntax category can be
registered with attribute `doElem_elab`. For new syntax, additionally a
control info handler must be registered with attribute
`doElem_control_info` that specifies whether the new syntax `return`s
early, `break`s, `continue`s and which `mut` vars it reassigns.
Do elaborators have type ``TSyntax `doElem → DoElemCont → DoElabM
Expr``, where `DoElabM` is essentially `TermElabM` and the `DoElemCont`
represents how the rest of the `do` block is to be elaborated. Consult
the docstrings for more details.
Breaking Changes:
* The syntax for `let pat := rhs | otherwise` and similar now scope over
the `doSeq` that follows. Furthermore, `otherwise` and the sequence that
follows are now `doSeqIndented` in order not to steal syntax from record
syntax.
Breaking Changes when opting into the new `do` elaborator by unsetting
`backward.do.legacy`:
* `do` notation now always requires `Pure`.
* `do match` is now always non-dependent. There is `do match (dependent
:= true)` that expands to a
term match as a workaround for some dependent uses.
This PR treats the first character of the first line of a docstring as
being in the leftmost column, even if it physically is not. This allows
left-column items like headers to be used even after spaces. It also
detects the indentation of the entire docstring, using it as the
zero-point for indentation sensitive syntax such as headers.
Closes#12067.
This PR fixes poor error reporting from Verso docstrings. Before, if the
Verso parser didn't consume the whole docstring, then Lean would try to
parse the closing -/ and fail; this would lead to backtracking and an
assumption that the docstring must be non-Verso, with only the non-Verso
commands like #guard_msgs as possibilities. Now, the input is always
consumed.
Closes#12118.
This PR adds the option `doc.verso.module`. If set, it controls whether
module docstrings use Verso syntax. If not set, it defaults to the value
of the `doc.verso` option.
Closes#12070.
This PR makes the automatic first token detection in tactic docs much
more robust, in addition to making it work in modules and other contexts
where builtin tactics are not in the environment. It also adds the
ability to override the tactic's first token as the user-visible name.
Previously, first token detection would look up the parser descriptor in
the environment and process its syntax. This would be incorrect for
builtin parsers, as well as for modules in which the definition is not
loaded. Now, it instead consults the Pratt parsing table for the
`tactic` syntax category. Tests are added that ensure this keeps working
in modules, and also that the first token of all tactics that ship with
Lean are either detected unambiguously or annotated to remove ambiguity.
Closes#12038.
Drastically speeds up `isTracingEnabledFor` in the common case, which
has evolved from "no options set" to "`Elab.async` and probably some
linter options set but no `trace`".
## Breaking changes
`Lean.Options` is now an opaque type. The basic but not all of the
`KVMap` API has been redefined on top of it.
This PR upstreams dependency-management commands from Mathlib:
- `#import_path Foo` prints the transitive import chain that brings
`Foo` into scope
- `assert_not_exists Foo` errors if declaration `Foo` exists (for
dependency management)
- `assert_not_imported Module` warns if `Module` is transitively
imported
- `#check_assertions` verifies all pending assertions are eventually
satisfied
These commands help maintain the independence of different parts of a
library by catching unintended transitive dependencies early.
### Example usage
```lean
-- Find out how Nat got into scope
#import_path Nat
-- Declaration Nat is imported via
-- Init.Prelude,
-- which is imported by Init.Coe,
-- which is imported by Init.Notation,
-- ...
-- which is imported by this file.
-- Assert that a declaration should not be in scope yet
assert_not_exists SomeAdvancedType
-- Assert that a module should not be imported
assert_not_imported Some.Heavy.Module
-- Verify all assertions are eventually satisfied
#check_assertions
```
Addresses
https://lean-fro.zulipchat.com/#narrow/channel/398861-general/topic/path.20of.20an.20import🤖 Prepared with Claude Code
---------
Co-authored-by: Claude <noreply@anthropic.com>
This PR updates the `foldr`, `all`, `any` and `contains` functions on
`String` to be defined in terms of their `String.Slice` counterparts.
This is the last one in a long series of PRs. After this, all `String`
operations are polymorphic in the pattern, and no `String` operation
falls back to `String.Pos.Raw` internally (except those in the
`String.Pos.Raw` and `String.Substring.Raw` namespaces of course, which
still play a role in metaprogramming and will stay for the foreseeable
future).
This PR changes the interface of the `ForIn`, `ForIn'`, and `ForM`
typeclasses to not take a `Monad m` parameter. This is a breaking change
for most downstream `instance`s, which will will now need to assume
`[Monad m]`.
The rationale is that if the provider of an instance requires `m` to be
a Monad, they should assume this up front. This makes it possible for
the instanve to assume `LawfulMonad m` or some other stronger
requirement, and also to provided a concrete instance for a particular
`m` without assuming a non-canonical `Monad` structure on it.
Zulip: [#lean4 > Monad assumptions in fields of other typeclasses @
💬](https://leanprover.zulipchat.com/#narrow/channel/270676-lean4/topic/Monad.20assumptions.20in.20fields.20of.20other.20typeclasses/near/537102158)
This PR upstreams the `with_weak_namespace` command from Mathlib:
`with_weak_namespace <id> <cmd>` changes the current namespace to `<id>`
for the duration of executing command `<cmd>`, without causing scoped
things to go out of scope. This is in preparation for upstreaming the
`scoped[Foo.Bar]` syntax from Mathlib, which will be useful now that we
are adding `grind` annotations in scopes.
This PR removes all code that sets the `Option.Decl.group` field, which
is unused and has no clearly documented meaning.
The actual removal of the field would be #11305.
This PR introduces a function `String.split` which is based on
`String.Slice.split` and therefore supports all pattern types and
returns a `Std.Iter String.Slice`.
This supersedes the functions `String.splitOn` and `String.splitToList`,
and we remove all all uses of these functions from core. They will be
deprecated in a future PR.
Migrating from `String.splitOn` and `String.splitToList` is easy: we
introduce functions `Iter.toStringList` and `Iter.toStringArray` that
can be used to conveniently go from `Std.Iter String.Slice` to `List
String` and `Array String`, so for example `s.splitOn "foo"` can be
replaced by `s.split "foo" |>.toStringList`.
This PR redefines `String.take` and variants to operate on
`String.Slice`. While previously functions returning a substring of the
input sometimes returned `String` and sometimes returned
`Substring.Raw`, they now uniformly return `String.Slice`.
This is a BREAKING change, because many functions now have a different
return type. So for example, if `s` is a string and `f` is a function
accepting a string, `f (s.drop 1)` will no longer compile because
`s.drop 1` is a `String.Slice`. To fix this, insert a call to `copy` to
restore the old behavior: `f (s.drop 1).copy`.
Of course, in many cases, there will be more efficient options. For
example, don't write `f <| s.drop 1 |>.copy |>.dropEnd 1 |>.copy`, write
`f <| s.drop 1 |>.dropEnd 1 |>.copy` instead. Also, instead of `(s.drop
1).copy = "Hello"`, write `s.drop 1 == "Hello".toSlice` instead.
This PR renames `Substring` to `Substring.Raw`.
This is to signify its status as a second-class citizen (not deprecated,
but no real plans for verification, like `String.Pos.Raw`) and to free
up the name `Substring` for a possible future type `String.Substring :
String -> Type` so that `s.Substring` is the type of substrings of `s`.
The functions `String.toSubstring` and `String.toSubstring'` will remain
for now for bootstrapping reasons.
This PR implements the following improvements to the `#grind_lint`
command:
1. More informative messages when the number of instances exceeds the
minimum threshold.
2. A code action for `#grind_lint inspect` that inserts
`set_option trace.grind.ematch.instance true` whenever the number of
instances exceeds
the minimum threshold.
3. Displaying doc strings for `grind` configuration options in
`#grind_lint`.
4. Improve doc strings for `#grind_lint inspect` and `#grind_lint
check`.
Example:
```lean
/--
info: instantiating `Array.filterMap_some` triggers more than 100 additional `grind` theorem instantiations
---
info: Array.filterMap_some
[thm] instances
[thm] Array.filterMap_filterMap ↦ 94
[thm] Array.size_filterMap_le ↦ 5
[thm] Array.filterMap_some ↦ 1
---
info: Try this to display the actual theorem instances:
[apply] set_option trace.grind.ematch.instance true in
#grind_lint inspect Array.filterMap_some
-/
#guard_msgs in
#grind_lint inspect Array.filterMap_some
```
This PR adds a new, inactive and unused `doElem_elab` attribute that
will allow users to register custom elaborators for `doElem`s in the
form of the new type `DoElab`. The old `do` elaborator is active by
default but can be switched off by disabling the new option
`backward.do.legacy`.
This PR renames `String.endPos` to `String.rawEndPos`, as in a future
release the name `String.endPos` will be taken by the function that is
currently called `String.endValidPos`.
This PR moves many operations involving `String.Pos.Raw` to a the
`String.Pos.Raw` namespace with the eventual aim of freeing up the
`String` namespace to contain operations using `String.ValidPos` (to be
renamed to `String.Pos`) instead.
This PR adds the `String.ValidPos.set` and `String.ValidPos.modify`
functions.
After this PR, `String.pos_lt_eq` is no longer a `simp` lemma. Add
`String.Pos.Raw.lt_iff` as a `simp` lemma if your proofs break.
This PR introduces the `backward.privateInPublic` option to aid in
porting projects to the module system by temporarily allowing access to
private declarations from the public scope, even across modules. A
warning will be generated by such accesses unless
`backward.privateInPublic.warn` is disabled.
This PR enforces rules around arithmetic of `String.Pos.Raw`.
Specifically, it adopts the following conventions:
- Byte indices ("ordinals") in strings should be represented using
`String.Pos.Raw`
- Amounts of bytes ("cardinals") in strings should be represented using
`Nat`.
For example, `String.Slice.utf8ByteSize` now returns `Nat` instead of
`String.Pos.Raw`, and there is a new function `String.Slice.rawEndPos`.
Finally, the `HAdd` and `HSub` instances for `String.Pos.Raw` are
reorganized. This is a **breaking change**.
The `HAdd/HSub String.Pos.Raw String.Pos.Raw String.Pos.Raw` instances
have been removed. For the use case of tracking positions relative to
some other position, we instead provide `offsetBy` and `unoffsetBy`
functions. For the use case of advancing/unadvancing a position by an
arbitrary number of bytes, we instead provide `increaseBy` and
`decreaseBy` functions. For
offsetting/unoffsetting/advancing/unadvancing a position `p` by the size
of a string `s` (resp. character `c`), use `s + p`/`p - s`/`p + s`/`p -
s` (resp. `c + p`/`p - c`/`p + c`/`p - c`).
This PR adds a new helper parser for implementing parsers that contain
hexadecimal numbers. We are going to use it to implement anchors in the
`grind` interactive mode.
This PR adds auto-completion for identifiers after `end`. It also fixes
a bug where completion in the whitespace after `set_option` would not
yield the full option list.
Closes#3885.
### Breaking changes
The `«end»` syntax is adjusted to take an `identWithPartialTrailingDot`
instead of an `ident`.
This PR introduces a `coinductive` keyword, that can be used to define
coinductive predicates via a syntax identical to the one for `inductive`
keyword. The machinery relies on the implementation of elaboration of
inductive types and extracts an endomap on the appropriate space of the
predicates from the definition that is then fed to the
`PartialFixpoint`. Upon elaborating definitions, all the constructors
are declared through automatically generated lemmas.
For example, infinite sequence of transitions in a relation, can be
given by the following:
```lean4
section
variable (α : Type)
coinductive infSeq (r : α → α → Prop) : α → Prop where
| step : r a b → infSeq r b → infSeq r a
/--
info: infSeq.coinduct (α : Type) (r : α → α → Prop) (pred : α → Prop) (hyp : ∀ (x : α), pred x → ∃ b, r x b ∧ pred b)
(x✝ : α) : pred x✝ → infSeq α r x✝
-/
#guard_msgs in
#check infSeq.coinduct
/--
info: infSeq.step (α : Type) (r : α → α → Prop) {a b : α} : r a b → infSeq α r b → infSeq α r a
-/
#guard_msgs in
#check infSeq.step
end
```
The machinery also supports `mutual` blocks, as well as mixing inductive
and coinductive predicate definitions:
```lean4
mutual
coinductive tick : Prop where
| mk : ¬tock → tick
inductive tock : Prop where
| mk : ¬tick → tock
end
/--
info: tick.mutual_induct (pred_1 pred_2 : Prop) (hyp_1 : pred_1 → pred_2 → False) (hyp_2 : (pred_1 → False) → pred_2) :
(pred_1 → tick) ∧ (tock → pred_2)
-/
#guard_msgs in
#check tick.mutual_induct
```
---------
Co-authored-by: Joachim Breitner <mail@joachim-breitner.de>
This PR records extra mod uses that previously caused wrong unnecessary
import reports from shake.
---------
Co-authored-by: Sebastian Ullrich <sebasti@nullri.ch>
This PR adds the necessary infrastructure for recording elaboration
dependencies that may not be apparent from the resulting environment
such as notations and other metaprograms. An adapted version of `shake`
from Mathlib is added to `script/` but may be moved to another location
or repo in the future.
This PR changes the way that scientific numerals are parsed in order to
give better error messages for (invalid) syntax like `32.succ`.
Example:
```lean4
#check 32.succ
```
Before, the error message is:
```
unexpected identifier; expected command
```
This is because `32.` parses as a complete float, and `#check 32.`
parses as a complete command, so `succ` is being read as the start of a
new command.
With this change, the error message will move from the `succ` token to
the `32` token (which isn't totally ideal from my perspective) but gives
a less misleading error message and corresponding suggestion:
```
unexpected identifier after decimal point; consider parenthesizing the number
```
This also exposed an issue with `#guard_msgs` in Verso mode where the
docstring would log parse errors as if it contained Verso, even though
it actually worked. This has been fixed, and error messages improved as
well.
