Commit graph

11600 commits

Author SHA1 Message Date
Henrik Böving
2a28cd98fc
feat: allow bv_decide users to configure the SAT solver (#11847)
This PR adds a new `solverMode` field to `bv_decide`'s configuration,
allowing users to configure
the SAT solver for different kinds of workloads.
2025-12-30 13:17:20 +00:00
Leonardo de Moura
bba35e4532
perf: add performance comparison tests for SymM vs MetaM (#11838)
This PR adds performance comparison tests between the new `SymM` monad
and the standard `MetaM` for `intros`/`apply` operations.

The tests solve problems of the form:
```lean
let z := 0; ∀ x, ∃ y, x = z + y ∧ let z := z + x; ∀ x, ∃ y, x = z + y ∧ ... ∧ True
```
using repeated `intros` and `apply` with `Exists.intro`, `And.intro`,
`Eq.refl`, and `True.intro`.

**Results show 10-20x speedup:**

| Size | MetaM | SymM | Speedup |
|------|-------|------|---------|
| 1000 | 226ms | 21ms | 10.8x |
| 2000 | 582ms | 44ms | 13.2x |
| 3000 | 1.08s | 72ms | 15.0x |
| 4000 | 1.72s | 101ms | 17.0x |
| 5000 | 2.49s | 125ms | 19.9x |
| 6000 | 3.45s | 157ms | 22.0x |
2025-12-30 02:42:04 +00:00
Leonardo de Moura
17581a2628
feat: add backward chaining rule application to Sym (#11837)
This PR adds `BackwardRule` for efficient goal transformation via
backward chaining in `SymM`.

`BackwardRule` stores a theorem expression, precomputed pattern for
fast unification, and argument indices that become new subgoals. The
subgoal ordering lists non-dependent goals first to match the behavior
of `MetaM.apply`.

`BackwardRule.apply` unifies the goal type with the rule's pattern,
assigns the goal metavariable to the theorem application, and returns
new subgoals for unassigned arguments.
2025-12-30 00:23:08 +00:00
Paul Reichert
05664b15a3
fix: update naming of FinitenessRelation fields in the sigmaIterator.lean benchmark (#11836)
This PR fixes a broken benchmark that uses an outdated naming of
`FinitenessRelation` and `ProductivenessRelation`'s fields.
2025-12-29 23:13:13 +00:00
Leonardo de Moura
4e1a2487b7
feat: add optional binder limit to mkPatternFromTheorem (#11834)
This PR adds `num?` parameter to `mkPatternFromTheorem` to control how
many leading quantifiers are stripped when creating a pattern. This
enables matching theorems where only some quantifiers should be
converted to pattern variables.

For example, to match `mk_forall_and : (∀ x, P x) → (∀ x, Q x) → (∀ x, P
x ∧ Q x)` against a goal `∀ x, q x 0 ∧ q (f (f x)) y`, we use
`mkPatternFromTheorem ``mk_forall_and (some 5)` to create the pattern `∀
x, ?P x ∧ ?Q x`, keeping the outermost `∀` in the pattern rather than
converting it to a pattern variable.
2025-12-29 17:38:50 +00:00
Leonardo de Moura
2bca310bea
feat: efficient pattern matching and unification for the symbolic simulation framework (#11825)
This PR completes the new pattern matching and unification procedures
for the symbolic simulation framework using a two-phase approach.

**Phase 1 (Syntactic Matching):**
- Patterns use de Bruijn indices for expression variables and renamed
level params for universe variables
- Purely structural matching after reducible definitions are unfolded
- Universe levels treat `max`/`imax` as uninterpreted functions
- Proof arguments skipped via proof irrelevance
- Instance and binder constraints deferred to Phase 2

**Phase 2 (Pending Constraints):**
- Level constraints: structural equality with mvar assignment
- Instance constraints: `isDefEqI` (full `isDefEq` for TC synthesis)
- Expression constraints: `isDefEqS` with Miller pattern support
- Unassigned instance pattern variables synthesized via
`trySynthInstance`

**`isDefEqS` (Structural DefEq):**
- Miller pattern detection and assignment (`?m x y z := rhs` → `?m :=
fun x y z => rhs`)
- Scope checking via `maxFVar` to prevent out-of-scope assignments
- Optional zeta-delta reduction for let-declarations
- Proof irrelevance and instance delegation to `isDefEqI`

**Key optimizations:**
- `abstractFVars` skips metavariables and uses `maxFVar` for early
cutoff
- Per-pattern `ProofInstInfo` cache for fast argument classification
- Maximal sharing.
2025-12-29 05:18:16 +00:00
Leonardo de Moura
5042c8cc37
feat: isDefEqS, a lightweight structural definitional equality for the symbolic simulation framework (#11824)
This PR implements `isDefEqS`, a lightweight structural definitional
equality for the symbolic simulation framework. Unlike the full
`isDefEq`, it avoids expensive operations while still supporting Miller
pattern unification.

**Key features:**
- Structural matching with optional zeta-delta reduction for
let-declarations
- Miller pattern detection and assignment (`?m x y z := rhs` → `?m :=
fun x y z => rhs`)
- Scope checking via `maxFVar` to prevent out-of-scope assignments
- Proof arguments skipped via proof irrelevance
- Instance arguments delegated to full `isDefEq` (need TC machinery)
- Universe levels treated structurally (`max`/`imax` as uninterpreted)
2025-12-29 03:17:18 +00:00
Leonardo de Moura
1e99ff1dba
feat: optimized abstractFVars and abstractFVarsRange (#11820)
This PR adds optimized `abstractFVars` and `abstractFVarsRange` for
converting free variables to de Bruijn indices during pattern
matching/unification.

**Optimizations:**
- Metavariables are skipped (their contexts must not include abstracted
fvars)
- Subterms whose `maxFVar` is below the minimal abstracted fvar are
skipped via early cutoff
- Results are maximally shared via `AlphaShareBuilderM`

These optimizations are sound for Miller pattern matching where
metavariables are created before entering binders.
2025-12-28 23:12:21 +00:00
Leonardo de Moura
18702bdd47
feat: add instantiateRevBetaS (#11814)
This PR implements `instantiateRevBetaS`, which is similar to
`instantiateRevS` but beta-reduces nested applications whose function
becomes a lambda after substitution.

For example, if `e` contains a subterm `#0 a` and we apply the
substitution `#0 := fun x => x + 1`, then `instantiateRevBetaS` produces
`a + 1` instead of `(fun x => x + 1) a`.

This is useful when applying theorems. For example, when applying
`Exists.intro`:
```lean
Exists.intro.{u} {α : Sort u} {p : α → Prop} (w : α) (h : p w) : Exists p
```
to a goal of the form `∃ x : Nat, p x ∧ q x`, we create metavariables
`?w` and `?h`. With `instantiateRevBetaS`, the type of `?h` becomes `p
?w ∧ q ?w` instead of `(fun x => p x ∧ q x) ?w`.
2025-12-28 03:28:15 +00:00
Leonardo de Moura
4eaaadf1c1
feat: add pattern matching/unification for symbolic simulation (#11813)
This PR introduces a fast pattern matching and unification module for
the symbolic simulation framework (`Sym`). The design prioritizes
performance by using a two-phase approach:

**Phase 1 (Syntactic Matching)**
- Patterns use de Bruijn indices for expression variables and renamed
level params (`_uvar.0`, `_uvar.1`, ...) for universe variables
- Matching is purely structural after reducible definitions are unfolded
during preprocessing
- Universe levels treat `max` and `imax` as uninterpreted functions (no
AC reasoning)
- Binders and term metavariables are deferred to Phase 2

**Phase 2 (Pending Constraints)** [WIP]
- Handles binders (Miller patterns) and metavariable unification
- Converts remaining de Bruijn variables to metavariables
- Falls back to `isDefEq` when necessary

**Key design decisions:**
- Preprocessing unfolds reducible definitions and performs beta/zeta
reduction
- Kernel projections are expected to be folded as projection
applications before matching
- Assignment conflicts are deferred to pending rather than invoking
`isDefEq` inline
- `instantiateRevS` ensures maximal sharing of result expressions

**TODO:**
- Skip instance arguments during matching, synthesize later
- Skip proof arguments (proof irrelevance)
- Implement `processPending` for Phase 2 constraints
2025-12-28 01:44:36 +00:00
Robert J. Simmons
f483c6c10f
refactor: move error explanation text to the manual (#11688)
This PR removes error explanation text from the manual, as this content
is now directly incorporated in the manual by
leanprover/reference-manual#704.
2025-12-26 17:14:58 +00:00
Leonardo de Moura
c0d5e8bc2c
feat: intro tactic for SymM (#11803)
This PR implements `intro` (and its variants) for `SymM`. These versions
do not use reduction or infer types, and ensure expressions are
maximally shared.
2025-12-26 03:45:33 +00:00
Leonardo de Moura
c02f570b76
feat: add instantiateS and variants (#11802)
This PR adds the function `Sym.instantiateS` and its variants, which are
similar to `Expr.instantiate` but assumes the input is maximally shared
and ensures the output is also maximally shared.
2025-12-25 23:02:16 +00:00
Leonardo de Moura
19d16ff9b7
feat: add replaceS, liftLooseBVarsS, and lowerBVarsS (#11800)
This PR adds the function `Sym.replaceS`, which is similar to
`replace_fn` available in the kernel but assumes the input is maximally
shared and ensures the output is also maximally shared. The PR also
generalizes the `AlphaShareBuilder` API.
2025-12-25 20:16:45 +00:00
Leonardo de Moura
b3b33e85d3
feat: add Sym.getMaxFVar? (#11794)
This PR implements the function `getMaxFVar?` for implementing `SymM`
primitives.
2025-12-25 02:24:00 +00:00
Leonardo de Moura
723acce2a7
feat: add AlphaShareBuilder (#11793)
This PR adds functions for creating maximally shared terms from
maximally shared terms. It is more efficient than creating an expression
and then invoking `shareCommon`. We are going to use these functions for
implementing the symbolic simulation primitives.
2025-12-25 00:05:03 +00:00
Leonardo de Moura
ce56e2139e
feat: support for incrementally processing hypotheses in grind (#11787)
This PR adds support for incrementally processing local declarations in
`grind`. Instead of processing all hypotheses at once during goal
initialization, `grind` now tracks which local declarations have been
processed via `Goal.nextDeclIdx` and provides APIs to process new
hypotheses incrementally.
This feature will be used by the new `SymM` monad for efficient symbolic
simulation.
2025-12-24 02:50:22 +00:00
Leonardo de Moura
f2c9fcc0b2
feat: add optional start position to PersistentArray.forM (#11784)
This PR just adds an optional start position argument to
`PersistentArray.forM`
2025-12-23 22:12:02 +00:00
Sebastian Ullrich
950a2b7896
chore: ensure every pkg/ test has a correct lean-toolchain file (#11782) 2025-12-23 17:17:22 +00:00
Henrik Böving
4d2647f9c7
fix: foldlM mismatch part 2 (#11779)
This PR fixes an oversight in the initial #11772 PR.

Closes #11778.
2025-12-23 10:29:20 +00:00
Leonardo de Moura
a471f005d6
feat: add [grind norm] and [grind unfold] attributes (#11776)
This PR adds the attributes `[grind norm]` and `[grind unfold]` for
controlling the `grind` normalizer/preprocessor.

The `norm` modifier instructs `grind` to use a theorem as a
normalization rule. That is, the theorem is applied during the
preprocessing step. This feature is meant for advanced users who
understand how the preprocessor and `grind`'s search procedure interact
with each other.
New users can still benefit from this feature by restricting its use to
theorems that completely eliminate a symbol from the goal. Example:
```lean
theorem max_def : max n m = if n ≤ m then m else n
```
For a negative example, consider:
```lean
opaque f : Int → Int → Int → Int
theorem fax1 : f x 0 1 = 1 := sorry
theorem fax2 : f 1 x 1 = 1 := sorry
attribute [grind norm] fax1
attribute [grind =] fax2

example (h : c = 1) : f c 0 c = 1 := by
  grind -- fails
```
In this example, `fax1` is a normalization rule, but it is not
applicable to the input goal since `f c 0 c` is not an instance of `f x
0 1`. However, `f c 0 c` matches the pattern `f 1 x 1` modulo the
equality `c = 1`. Thus, `grind` instantiates `fax2` with `x := 0`,
producing the equality `f 1 0 1 = 1`, which the normalizer simplifies to
`True`. As a result, nothing useful is learned. In the future, we plan
to include linters to automatically detect issues like these. Example:
```lean
opaque f : Nat → Nat
opaque g : Nat → Nat

@[grind norm] axiom fax : f x = x + 2
@[grind norm ←] axiom fg : f x = g x

example : f x ≥ 2 := by grind
example : f x ≥ g x := by grind
example : f x + g x ≥ 4 := by grind
```

The `unfold` modifier instructs `grind` to unfold the given definition
during the preprocessing step. Example:
```lean
@[grind unfold] def h (x : Nat) := 2 * x
example : 6 ∣ 3*h x := by grind
```
2025-12-23 03:54:35 +00:00
Henrik Böving
a847b13b1a
fix: implemented_by Array.foldlM behavior when stop > start (#11774)
This PR fixes a mismatch between the behavior of `foldlM` and
`foldlMUnsafe` in the three array
types. This mismatch is only exposed when manually specifying a `stop`
value greater than the size
of the array and only exploitable through `native_decide`.

The mismatch was introduced as part of
4ba21ea10c which introduced
`foldlMUnsafe` and thus likely a mistake when building the `unsafe`
implementation instead of a
specification mistake.

Closes #11773
2025-12-22 23:46:45 +00:00
Leonardo de Moura
186a81627b
fix: Array.foldlMUnsafe bug (#11772)
This PR a bug in the optimized and unsafe implementation of
`Array.foldlM`.

Issue was reported here:

https://leanprover.zulipchat.com/#narrow/channel/113488-general/topic/Array.2Efoldl.20bug.20.28can.20prove.20False.29/near/565077432
2025-12-22 23:00:16 +00:00
Leonardo de Moura
72f9b725aa
feat: user attribute at grind_pattern (#11770)
This PR implements support for user-defined attributes at
`grind_pattern`. Suppose we have declared the `grind` attribute

```lean
register_grind_attr my_grind
```

Then, we can now write

```lean
opaque f : Nat → Nat
opaque g : Nat → Nat
axiom fg : g (f x) = x

grind_pattern [my_grind] fg => g (f x)
```
2025-12-22 20:07:02 +00:00
Leonardo de Moura
dc53fac626
chore: use extensible grind attribute framework to implement [grind] itself (#11769)
This PR uses the new support for user-defined `grind` attributes to
implement the default `[grind]` attribute.

A manual update-stage0 is required because it affects the .olean files.
2025-12-22 10:07:30 -08:00
Leonardo de Moura
0d2a574f96
feat: user-defined grind attributes (#11765)
This PR implements user-defined `grind` attributes. They are useful for
users that want to implement tactics using the `grind` infrastructure
(e.g., `progress*` in Aeneas). New `grind` attributes are declared using
the command
```lean
register_grind_attr my_grind
```
The command is similar to `register_simp_attr`. After the new attribute
is declared. Recall that similar to `register_simp_attr`, the new
attribute cannot be used in the same file it is declared.
```lean
opaque f : Nat → Nat
opaque g : Nat → Nat

@[my_grind] theorem fax : f (f x) = f x := sorry

example theorem fax2 : f (f (f x)) = f x := by
  fail_if_success grind
  grind [my_grind]
```

TODO: remove leftovers after update stage0
2025-12-22 02:57:25 +00:00
Kim Morrison
a7562bc578
feat: add guarded grind_pattern to List.eq_nil_of_length_eq_zero (#11760)
This PR allows `grind` to use `List.eq_nil_of_length_eq_zero` (and
`Array.eq_empty_of_size_eq_zero`), but only when it has already proved
the length is zero.
2025-12-22 00:05:58 +00:00
Kim Morrison
c86b10d141
chore: add grind pattern guide for Sublist.eq_of_length_le (#11762)
This PR moves the grind pattern from `Sublist.eq_of_length` to the
slightly more general `Sublist.eq_of_length_le`, and adds a grind
pattern guard so it only activates if we have a proof of the hypothesis.
2025-12-22 00:01:33 +00:00
Kim Morrison
54a88e941f
chore: followup tests for #11745 (#11764)
This PR adds additional test coverage for #11758 (fix for #11745:
nonstandard instances in grind and simp +arith).

The existing test `grind_11745.lean` only covers Int LE with `grind
-order` and `lia -order`. This adds tests for:

- LT instances (Int and Nat)
- Nat LE instances
- Mixed canonical and non-canonical instances in the same goal
- Equality derived from two LE constraints
- `simp +arith` with non-canonical instances

🤖 Prepared with Claude Code

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-21 22:31:53 +00:00
Kim Morrison
eb990538ae
fix: allow exact? to suggest local private declarations (part 2) (#11759)
This PR contains changes that were meant to be part of #11736, but I
accidentally merged without pushing my final local changes.
2025-12-21 20:03:10 +00:00
Joachim Breitner
4c0765fc07
fix: grind using congr equation of private imported matcher (#11756)
This PR fixes an issue where `grind` fails when trying to unfold a
definition by pattern matching imported by `import all` (or from a
non-`module`).

Fixes #11715

---------

Co-authored-by: Sebastian Ullrich <sebasti@nullri.ch>
2025-12-21 17:59:52 +00:00
Leonardo de Moura
5e24120dba
fix: nonstandard instances in grind and simp +arith (#11758)
This PR improves support for nonstandard `Int`/`Nat` instances in
`grind` and `simp +arith`.

Closes #11745
2025-12-21 17:56:49 +00:00
Sebastian Ullrich
f317e28d84
fix: realizeValue should default to the private scope (#11748)
This PR fixes an edge case where some tactics did not allow access to
private declarations inside private proofs under the module system

Fixes #11747
2025-12-21 01:22:19 +00:00
Leonardo de Moura
5440bf724d
fix: case-splitting selection in grind (#11749)
This PR fixes a bug in the function `selectNextSplit?` used in `grind`.
It was incorrectly computing the generation of each candidate.

Closes #11697
2025-12-20 20:17:09 +00:00
Kim Morrison
cee149cc1f
feat: add #import_path, assert_not_exists, assert_not_imported commands (#11726)
This PR upstreams dependency-management commands from Mathlib:

- `#import_path Foo` prints the transitive import chain that brings
`Foo` into scope
- `assert_not_exists Foo` errors if declaration `Foo` exists (for
dependency management)
- `assert_not_imported Module` warns if `Module` is transitively
imported
- `#check_assertions` verifies all pending assertions are eventually
satisfied

These commands help maintain the independence of different parts of a
library by catching unintended transitive dependencies early.

### Example usage

```lean
-- Find out how Nat got into scope
#import_path Nat
-- Declaration Nat is imported via
-- Init.Prelude,
--   which is imported by Init.Coe,
--   which is imported by Init.Notation,
--   ...
--   which is imported by this file.

-- Assert that a declaration should not be in scope yet
assert_not_exists SomeAdvancedType

-- Assert that a module should not be imported
assert_not_imported Some.Heavy.Module

-- Verify all assertions are eventually satisfied
#check_assertions
```

Addresses
https://lean-fro.zulipchat.com/#narrow/channel/398861-general/topic/path.20of.20an.20import

🤖 Prepared with Claude Code

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-19 04:09:33 +00:00
Kim Morrison
c74d24aaaa
fix: allow exact? to suggest local private declarations (#11736)
This PR fixes an issue where `exact?` would not suggest private
declarations defined in the current module.

## Problem

When using `exact?` in a file with private declarations, those private
declarations were not being suggested even though they are valid and
accessible:

```lean
module

axiom P : Prop
private axiom p : P
example : P := by exact? -- error: could not find lemma
```

The problem was that `blacklistInsertion` in `LazyDiscrTree` was
filtering out all declarations whose names matched `isInternalDetail`,
which includes private names due to their `_private.Module.0.name`
structure.

## Solution

The fix adds a helper function `isPrivateNameOf` that checks if a
private declaration belongs to a specific module. The
`blacklistInsertion` function now allows private declarations belonging
to the current module (`env.header.mainModule`) to pass through the
filter.

Private declarations from imported modules are still filtered out, as
they may reference internal declarations that aren't accessible (which
would cause processing errors).

Zulip discussion:
https://leanprover.zulipchat.com/#narrow/channel/270676-lean4/topic/.60exact.3F.60.20and.20private.20declarations/near/564586152

🤖 Prepared with Claude Code

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-19 04:05:54 +00:00
Henrik Böving
2db0a98b7c
fix: internalize all arguments to Quot.lift during LCNF conversion (#11729)
This PR internalizes all arguments of Quot.lift during LCNF conversion,
preventing panics in certain
non trivial programs that use quotients.

Fixes #11719.
2025-12-18 09:31:48 +00:00
Paul Reichert
4e656ea8e9
refactor: move Std.Range to Std.Legacy.Range (#11438)
This PR renames the namespace `Std.Range` to `Std.Legacy.Range`. Instead
of using `Std.Range` and `[a:b]` notation, the new range type `Std.Rco`
and its corresponding `a...b` notation should be used. There are also
other ranges with open/closed/infinite boundary shapes in
`Std.Data.Range.Polymorphic` and the new range notation also works for
`Int`, `Int8`, `UInt8`, `Fin` etc.
2025-12-18 02:07:33 +00:00
Paul Reichert
5ef0207a85
refactor: remove IteratorCollect (#11706)
This PR removes the `IteratorCollect` type class and hereby simplifies
the iterator API. Its limited advantages did not justify the complexity
cost.
2025-12-17 23:02:33 +00:00
Paul Reichert
a1b8ffe31b
feat: improve MPL support for loops over iterators, fix MPL spec priorities (#11716)
This PR adds more MPL spec lemmas for all combinations of `for` loops,
`fold(M)` and the `filter(M)/filterMap(M)/map(M)` iterator combinators.
These kinds of loops over these combinators (e.g. `it.mapM`) are first
transformed into loops over their base iterators (`it`), and if the base
iterator is of type `Iter _` or `IterM Id _`, then another spec lemma
exists for proving Hoare triples about it using an invariant and the
underlying list (`it.toList`). The PR also fixes a bug that MPL always
assigns the default priority to spec lemmas if `Std.Tactic.Do.Syntax` is
not imported and a bug that low-priority lemmas are preferred about
high-priority ones.

For context, the MPL bug was related to the fact that the `Attr.spec`
syntax is not built-in. Therefore, Lean falls back to the `Attr.simple`
syntax, which *basically* also works, but which stores the priority at a
different position. The routine to extract the priority does not
consider this and so it falls back to the default priority given an
`Attr.simple` syntax object.
2025-12-17 22:49:42 +00:00
Joachim Breitner
1918d4f0dc
chore: add test for #11655 (#11718)
This PR adds a test for issue #11655, which it seems was fixed by #11695

Fixes #11655
2025-12-17 15:54:16 +00:00
Robert J. Simmons
08c87b2ad3
feat: focused error messages for named examples (#11714)
This PR gives a focused error message when a user tries to name an
example, and tweaks error messages for attempts to define multiple
opaque names at once.

## Example errors

```
example x : 1 == 1 := by grind
```

Current message:
```
Failed to infer type of binder `x`

Note: Because this declaration's type has been explicitly provided, all parameter types and holes (e.g., `_`) in its header are resolved before its body is processed; information from the declaration body cannot be used to infer what these values should be
```

New message:
```
Failed to infer type of binder `x`

Note: Examples don't have names. The identifier `x` is being interpreted as a parameter `(x : _)`.
```

## Plural-aware identifier lists

Both the example errors and opaque errors understand pluralization and
use oxford commas.

```
opaque a b c : Nat
```

Current message:
```
Failed to infer type of binder `c`

Note: Multiple constants cannot be declared in a single declaration. The identifier(s) `b`, `c` are being interpreted as parameters `(b : _)`, `(c : _)`.
```

New message:
```
Failed to infer type of binder `c`

Note: Multiple constants cannot be declared in a single declaration. The identifiers `b` and `c` are being interpreted as parameters `(b : _)` and `(c : _)`.```
2025-12-17 14:54:41 +00:00
Paul Reichert
489f8acd77
feat: get-elem tactic support for subarrays (#11710)
This PR extends the get-elem tactic for ranges so that it supports
subarrays. Example:
```lean
example {a : Array Nat} (h : a.size = 28) : Id Unit := do
  let mut x := 0
  for h : i in *...(3 : Nat) do
    x := a[1...4][i]
```
2025-12-17 13:44:17 +00:00
Henrik Böving
fe96911368
feat: proper recursive specialization (#11479)
This PR enables the specializer to also recursively specialize in some
non trivial higher order situations.

The main motivation for this change is the upcoming changes to do
notation by sgraf. In there he uses combinators such as
```lean
@[specialize, expose]
def List.newForIn {α β γ} (l : List α) (b : β) (kcons : α → (β → γ) → β → γ) (knil : β → γ) : γ :=
  match l with
  | []     => knil b
  | a :: l => kcons a (l.newForIn · kcons knil) b
```
in programs such as
```lean
def testing :=
  let x := 42;
  List.newForIn (β := Nat) (γ := Id Nat)
    [1,2,3]
    x
    (fun i kcontinue s =>
      let x := s;
      List.newForIn
        [i:10].toList x
        (fun j kcontinue s =>
          let x := s;
          let x := x + i + j;
          kcontinue x)
        kcontinue)
    pure
```
inspecting this IR right before we get to the specializer in the current
compiler we get:
```
[Compiler.eagerLambdaLifting] size: 22
    def testing : Nat :=
      fun _f.1 _y.2 : Nat :=
        return _y.2;
      let x := 42;
      let _x.3 := 1;
      fun _f.4 i kcontinue s : Nat :=
        fun _f.5 j kcontinue s : Nat :=
          let _x.6 := Nat.add s i;
          let x := Nat.add _x.6 j;
          let _x.7 := kcontinue x;
          return _x.7;
        let _x.8 := 10;
        let _x.9 := Nat.sub _x.8 i;
        let _x.10 := Nat.add _x.9 _x.3;
        let _x.11 := 1;
        let _x.12 := Nat.sub _x.10 _x.11;
        let _x.13 := Nat.mul _x.3 _x.12;
        let _x.14 := Nat.add i _x.13;
        let _x.15 := @List.nil _;
        let _x.16 := List.range'TR.go _x.3 _x.12 _x.14 _x.15;
        let _x.17 := @List.newForIn _ _ _ _x.16 s _f.5 kcontinue;
        return _x.17;
      let _x.18 := 2;
      let _x.19 := 3;
      let _x.20 := @List.nil _;
      let _x.21 := @List.cons _ _x.19 _x.20;
      let _x.22 := @List.cons _ _x.18 _x.21;
      let _x.23 := @List.cons _ _x.3 _x.22;
      let _x.24 := @List.newForIn _ _ _ _x.23 x _f.4 _f.1;
      return _x.24 
```
Here the `kcontinue` higher order functions pose a special challenge
because they delay the discovery of new specialization opportunities.
Inspecting the IR after the current specializer (and a cleanup simp
step) we get functions that look as follows:
```
 [simp] size: 7
      def List.newForIn._at_.testing.spec_0 i kcontinue l b : Nat :=
        cases l : Nat
        | List.nil =>
          let _x.1 := kcontinue b;
          return _x.1
        | List.cons head.2 tail.3 =>
          let _x.4 := Nat.add b i;
          let x := Nat.add _x.4 head.2;
          let _x.5 := List.newForIn._at_.testing.spec_0 i kcontinue tail.3 x;
          return _x.5 
  [simp] size: 14
      def List.newForIn._at_.List.newForIn._at_.testing.spec_1.spec_1 _x.1 l b : Nat :=
        cases l : Nat
        | List.nil =>
          return b
        | List.cons head.2 tail.3 =>
          fun _f.4 x.5 : Nat :=
            let _x.6 := List.newForIn._at_.List.newForIn._at_.testing.spec_1.spec_1 _x.1 tail.3 x.5;
            return _x.6;
          let _x.7 := 10;
          let _x.8 := Nat.sub _x.7 head.2;
          let _x.9 := Nat.add _x.8 _x.1;
          let _x.10 := 1;
          let _x.11 := Nat.sub _x.9 _x.10;
          let _x.12 := Nat.mul _x.1 _x.11;
          let _x.13 := Nat.add head.2 _x.12;
          let _x.14 := @List.nil _;
          let _x.15 := List.range'TR.go _x.1 _x.11 _x.13 _x.14;
          let _x.16 := List.newForIn._at_.testing.spec_0 head.2 _f.4 _x.15 b;
          return _x.16
```
Observe that the specializer decided to abstract over `kcontinue`
instead of specializing further recursively. Thus this tight loop is now
going through an indirect call.

This PR now changes the specializer somewhat fundamentally to handle
situations like this. The most notable change is going to a fixpoint
loop of:
1. Specialize all current declarations in the worklist
2. If a declaration
- succeeded in specializing run the simplifier on it and put it back
onto the worklist
    - if it didn't don't put it back onto the worklist anymore
3. Put all newly generated specialisations on the worklist
4. Recompute fixed parameters for the current SCC
5. Repeat until the worklist is empty

Furthermore, declarations that were already specialized:
- only consider `fixedHO` parameters for specialization, in order to
avoid termination issues with repeated specialization and abstraction of
type class parameters under binders
- recursively specialized declarations only allow specialization if at
least one of their fixedHO arguments is not a parameter itself. The
reason for allowing this in first generation specialization is that we
refrain from specializing inside the body of a declaration marked as
`@[specialize]`. Thus we need to specialize them even if their arguments
don't actually contain anything of interest in order to ensure that type
classes etc. are correctly cleaned up within their bodies.

There is one last trade-off to consider. When specializing code
generated by the new do elaborator we sometimes generate intermediate
specializations that are not actually part of any call graph after we
are done specializing. We could in principle detect these functions and
delete them but having them in cache is potentially helpful for further
specializations later. Once the new do elaborator lands we plan to test
this trade-off.

Closes #10924
2025-12-17 11:05:24 +00:00
Paul Reichert
3ac9bbb3d8
feat: MPL specs for loops over iterators (#11693)
This PR makes it possible to verify loops over iterators. It provides
MPL spec lemmas about `for` loops over pure iterators. It also provides
spec lemmas that rewrite loops over `mapM`, `filterMapM` or `filterM`
iterator combinators into loops over their base iterator.
2025-12-17 09:36:44 +00:00
Joachim Breitner
118160bf07
refactor: handle irrefutable patterns in match compilation individually (#11695)
This PR refactors match compilation, to handle “side-effect free”
patterns (`.var`, `.inaccessible`, `.as`) eagerly and for each
alternative separately. The idea is that there should be less interplay
between different alternatives, and prepares the ground for #11105.

This may cause some corner case match statements to compiler or fail
compile that behaved differently before. For example, it can now use a
sparse case where previously was using a full case, and pattern
completeness may not be clear to lean now. On the other hand, using a
sparse case can mean that match statements mixing matching in indicies
with matching on the indexed datatype can work.
2025-12-17 09:02:17 +00:00
Kim Morrison
0708024c46
fix: support dot notation on declarations in grind lemma list (#11691)
This PR fixes `grind` to support dot notation on declarations in the
lemma list.

When using `grind only [foo.le]` where `foo.le` is dot notation applying
`LT.lt.le` to a theorem `foo`, grind previously failed with "Unknown
constant `foo.le`" because it tried to look up `foo.le` as a constant
name rather than elaborating it as a term.

The fix adds a fallback in `processParam`: when constant lookup fails,
it now falls back to `processTermParam` which elaborates the identifier
as a term. This allows dot notation expressions like `log_two_lt_d9.le`
to work correctly.

Closes #11690

🤖 Prepared with Claude Code

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-17 03:17:46 +00:00
Paul Reichert
e2617903f8
feat: MonadAttach (#11532)
This PR adds the new operation `MonadAttach.attach` that attaches a
proof that a postcondition holds to the return value of a monadic
operation. Most non-CPS monads in the standard library support this
operation in a nontrivial way. The PR also changes the `filterMapM`,
`mapM` and `flatMapM` combinators so that they attach postconditions to
the user-provided monadic functions passed to them. This makes it
possible to prove termination for some of these for which it wasn't
possible before. Additionally, the PR adds many missing lemmas about
`filterMap(M)` and `map(M)` that were needed in the course of this PR.
2025-12-16 18:57:00 +00:00
Sebastian Graf
5f4d724c2d
feat: abstract metavariables when generalizing match motives (#8099) (#11696)
This PR improves `match` generalization such that it abstracts
metavariables in types of local variables and in the result type of the
match over the match discriminants. Previously, a metavariable in the
result type would silently default to the behavior of `generalizing :=
false`, and a metavariable in the type of a free variable would lead to
an error (#8099). Example of a `match` that elaborates now but
previously wouldn't:
```lean
example (a : Nat) (ha : a = 37) :=
    (match a with | 42 => by contradiction | n => n) = 37
```
This is because the result type of the `match` is a metavariable that
was not abstracted over `a` and hence generalization failed; the result
is that `contradiction` cannot pick up the proof `ha : 42 = 37`.
The old behavior can be recovered by passing `(generalizing := false)`
to the `match`.

Furthermore, programs such as the following can now be elaborated:
```lean
example (n : Nat) : Id (Fin (n + 1)) :=
  have jp : ?m := ?rhs
  match n with
  | 0 => ?jmp1
  | n + 1 => ?jmp2
  where finally
  case m => exact Fin (n + 1) → Id (Fin (n + 1))
  case jmp1 => exact jp ⟨0, by decide⟩
  case jmp2 => exact jp ⟨n, by omega⟩
  case rhs => exact pure
```
This is useful for the `do` elaborator.

Fixes #8099.
2025-12-16 14:34:29 +00:00
Sebastian Graf
98616529fd
fix: early return after simplifying discriminants in mvcgen (#11687) (#11698)
This PR makes `mvcgen` early return after simplifying discriminants,
avoiding a rewrite on an ill-formed `match`.

Closes #11687.
2025-12-16 11:36:45 +00:00