This PR adds a `linter.redundantExpose` option (default `true`) that
warns when `@[expose]` or `@[no_expose]` attributes have no effect:
- `@[expose]` on `abbrev` (always exposed) or non-Prop `instance`
(always exposed)
- `@[expose]` on a `def` inside an `@[expose] section` (already exposed
by the section)
- `@[expose]`/`@[no_expose]` in a non-`module` file (no module system)
- `@[no_expose]` on a declaration that wouldn't be exposed by default
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This PR verifies the `String.dropWhile` and `String.takeWhile`
functions.
It also includes a refactor of the `PatternModel` class so that the
`not_matches_empty` condition is moved into a separate typeclass
`StrictPatternModel`. This allows string patterns to implement
`LawfulForwardPatternModel` unconditionally, which means that more of
the general theory about patterns directly applies to string patterns
without having to do a case distinction for empty strings.
This PR also includes a study of the `PatternModel` machinery given to
slices `s` and `t` such that `s.copy = t.copy`. From these results, we
deduce statements like `s.copy.startsWith pat = s.startsWith pat` (which
is far from obvious!).
This PR fixes several issues in `Init.Data.String`, most of them typos.
We also move the remaining material out of `Init.Data.String.Lemmas` to
`Init.Data.String.Lemmas.StringOrder`, which shortens the pole.
This PR adds `EquivBEq` and `LawfulHashable` instances to
`String.Slice`.
To this end, we redefine `String.Slice.hash`, which used to be
completely opaque, to be defined as `String.hash s.copy` (and then
`String.hash` remains opaque). We add tests that the `lean_slice_hash`
and `lean_string_hash` functions do indeed satisfy this relationship.
Of course, it would be even better to have a streaming MurmurHash64A
implementation in core that could be used to implement both of these so
that we can avoid the `opaque`, but that is a project for another day.
This PR adds the functions `Std.Iter.joinString` and
`Std.Iter.intercalateString`.
`it.intercalateString s` is a more efficient version of
`s.copy.intercalate (it.toList.map toString)`, and we have a lemmas
proving exactly that.
This PR reorganizes the instances `ToString Int` and `Repr Int` so that
they both point at a common definition `Int.repr` (the same setup is
used for `Nat`). It then verifies the functions `Int.repr`,
`String.isInt` and `String.toInt`.
In particular, for `a : Int` we get `a.repr.toInt? = some a`, which
implies that `Int.repr` is injective.
This PR introduces the functions `String.Slice.skipPrefix?`,
`String.Slice.Pos.skip?`, `String.Slice.skipPrefixWhile`,
`String.Slice.Pos.skipWhile` and redefines `String.Slice.takeWhile` and
`String.Slice.dropWhile` to use these new functions.
This PR renames the function `ForwardPattern.dropPrefix?` to
`ForwardPattern.skipPrefix`?
This function `(s : String.Slice) -> Option s.Pos` is not to be confused
with `String.Slice.dropPrefix? : (s : String.Slice) -> Option
String.Slice`.
This PR redefines the `String.isNat` function to use less state and
perform short-circuiting. It then verifies the `String.isNat` and
`String.toNat?` functions.
Recall that `isNat` and `toNat?` allow `_` as a digit separator. This is
why we get the complicated statement
```lean
public theorem isNat_iff {s : String} :
s.isNat = true ↔
s ≠ "" ∧
(∀ c ∈ s.toList, c.isDigit ∨ c = '_') ∧
¬ ['_', '_'] <:+: s.toList ∧
s.toList.head? ≠ some '_' ∧
s.toList.getLast? ≠ some '_'
```
For `toNat?`, we prove the fully general
```lean
public theorem toNat?_eq_some_ofDigitChars {s : String} (h : s.isNat = true) :
s.toNat? = some (Nat.ofDigitChars 10 (s.toList.filter (· != '_')) 0)
```
as well as the useful `(Nat.repr n).toNat? = some n` (and the corollary
that `Nat.repr` is injective).
For people implementing formatting routines that involve digit
separators, we have
```lean
public theorem isNat_of_isDigit {s : String} (hne : s ≠ "")
(hdigit : ∀ c ∈ s.toList, c.isDigit) : s.isNat = true
public theorem isNat_append_underscore_append {s t : String}
(hs : s.isNat = true) (ht : t.isNat = true) :
(s ++ "_" ++ t).isNat = true
public theorem toNat?_append_underscore_append_eq_some {s t : String} {n m : Nat}
(hs : s.toNat? = some n) (ht : t.toNat? = some m) :
(s ++ "_" ++ t).toNat? =
some (10 ^ (t.toList.filter (· != '_')).length * n + m)
```
The missing bit here is `(s.leftpad k '0').toNat? = s.toNat?`, which is
missing because we don't have `String.leftpad` (yet). For any reasonable
definition of `leftpad`, this will follow from
`toNat?_eq_some_ofDigitChars` since we prove the necessary ingredients
about `ofDigitChars`.
There are some rough edges around `ofDigitChars`, and in the future it
will be nice to connect this all to mathlib's `Nat.digits` and
`Nat.ofDigits`, which are similar but different.
This PR relates `String.split` to `List.splitOn` and `List.splitOnP`,
provided that we are splitting by a character or character predicate.
Also included: some more lemmas about `List.splitOn`, and a refactor of
the generic `split` verification to get rid of the awkward `SlicesFrom`
constuct.
This PR enables `backward.whnf.reducibleClassField` for v4.29.
The support is particularly important when the user marks a class field
as `[reducible]` and
the transparency mode is `.reducible`. For example, suppose `e` is `a ≤
b` where `a b : Nat`,
and `LE.le` is marked as `[reducible]`. Simply unfolding `LE.le` would
give `instLENat.1 a b`,
which would be stuck because `instLENat` has transparency
`[instance_reducible]`. To avoid this, when we unfold
a `[reducible]` class field, we also unfold the associated projection
`instLENat.1` using
`.instances` reducibility, ultimately returning `Nat.le a b`.
---------
Co-authored-by: Paul Reichert <6992158+datokrat@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Kim Morrison <kim@tqft.net>
This PR adds `String.Slice.Subslice`, which is an unbundled version of
`String.Slice`.
This type is of interest because it is the correct type for string
searching and splitting operations to land in.
This PR just adds the type with minimal API. Additional API and
subsequent refactoring of the searching and splitting API is left for
future PRs.
This PR reverses the relationship between the `ForwardPattern` and
`ToForwardSearcher` classes.
Previously, it was possible to derive `ForwardPattern` (i.e.,
`dropPrefix?`) from `ToForwardSearcher` (i.e., get an iterator of
`SearchStep (s)`). Now, we give the default instance in the other
direction: it is now possible to derive `ToForwardSearcher` from
`ForwardPattern`. Since it is usually much easier to provide
`ForwardPattern` than `ToForwardSearcher`, this means more shared code,
which pays off double since we will give a correctness proof for the
default implementation in an upcoming PR.
This PR also adds some string lemmas.
This adds `set_option debug.byAsSorry true` and `decreasing_by sorry` to
various files to allow bootstrapping with Config structure changes. These
changes will be restored after the bootstrap dance is complete.
This PR introduces the functions `(String|Slice).posGE` and
`(String|Slice).posGT` will full verification and deprecates
`Slice.findNextPos` in favor of `Slice.posGT`.
The KMP implementation is adapted to use these two new functions.
Various useful string and order lemmas are added along the way.
Also add a `simp` attribute to `Std.le_refl` and fix the resulting
fallout (yes, this would have been better as a separate PR).
Typos in `Init/` and `Std/`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
This PR makes the `FinitenessRelation` structure, which is helpful when
proving the finiteness of iterators, part of the public API. Previously,
it was marked internal and experimental.
This PR moves many constants of the iterator API from `Std.Iterators` to
the `Std` namespace in order to make them more convenient to use. These
constants include, but are not limited to, `Iter`, `IterM` and
`IteratorLoop`. This is a breaking change. If something breaks, try
adding `open Std` in order to make these constants available again. If
some constants in the `Std.Iterators` namespace cannot be found, they
can be found directly in `Std` now.
This PR adds `@[suggest_for]` annotations to Lean, allowing lean to
provide corrections for `.every` or `.some` methods in place of `.all`
or `.any` methods for most default-imported types (arrays, lists,
strings, substrings, and subarrays, and vectors).
Due to the need for stage0 updates for new annotations, the
`suggest_for` annotation itself was introduced in previous PRs: #11367,
#11529, and #11590.
## Example
```
example := "abc".every (! ·.isWhitespace)
```
Error message:
```
Invalid field `every`: The environment does not contain `String.every`, so it is not possible to project the field `every` from an expression
"abc"
of type `String`
Hint: Perhaps you meant `String.all` in place of `String.every`:
.e̵v̵e̵r̵y̵a̲l̲l̲
```
(the hint is added by this PR)
## Additional changes
Adds suggestions that are not currently active but that can be used to
generate autocompletion suggestions in the reference manual:
- `Either` -> `Except` and `Sum`
- `Exception` -> `Except`
- `ℕ` -> `Nat`
- `Nullable` -> `Option`
- `Maybe` -> `Option`
- `Optional` -> `Option`
- `Result` -> `Except`
This PR introduces a new fixpoint combinator,
`WellFounded.extrinsicFix`. A termination proof, if provided at all, can
be given extrinsically, i.e., looking at the term from the outside, and
is only required if one intends to formally verify the behavior of the
fixpoint. The new combinator is then applied to the iterator API.
Consumers such as `toList` or `ForIn` no longer require a proof that the
underlying iterator is finite. If one wants to ensure the termination of
them intrinsically, there are strictly terminating variants available
as, for example, `it.ensureTermination.toList` instead of `it.toList`.
This PR adds support for underscores as digit separators in
String.toNat?, String.toInt?, and related parsing functions. This makes
the string parsing functions consistent with Lean's numeric literal
syntax, which already supports underscores for readability (e.g.,
100_000_000).
The implementation validates that underscores:
- Cannot appear at the start or end of the number
- Cannot appear consecutively
- Are ignored when calculating the numeric value
This resolves a common source of friction when parsing user input from
command-line arguments, environment variables, or configuration files,
where users naturally expect to use the same numeric syntax they use in
source code.
## Examples
Before:
```lean
#eval "100_000_000".toNat? -- none
```
After:
```lean
#eval "100_000_000".toNat? -- some 100000000
#eval "1_000".toInt? -- some 1000
#eval "-1_000_000".toInt? -- some (-1000000)
```
## Testing
Added comprehensive tests in
`tests/lean/run/string_toNat_underscores.lean` covering:
- Basic underscore support
- Edge cases (leading/trailing/consecutive underscores)
- Both `toNat?` and `toInt?` functions
- String, Slice, and Substring types
All existing tests continue to pass.
Closes#11538🤖 Prepared with Claude Code
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
This PR updates the `foldr`, `all`, `any` and `contains` functions on
`String` to be defined in terms of their `String.Slice` counterparts.
This is the last one in a long series of PRs. After this, all `String`
operations are polymorphic in the pattern, and no `String` operation
falls back to `String.Pos.Raw` internally (except those in the
`String.Pos.Raw` and `String.Substring.Raw` namespaces of course, which
still play a role in metaprogramming and will stay for the foreseeable
future).
This PR cleans up the API around `String.find` and moves it uniformly to
the new position types `String.ValidPos` and `String.Slice.Pos`
Overview:
- To search for a character, character predicate, string or slice in a
string or slice `s`, use `s.find?` or `s.find`.
- To do the same, but starting at a position `p` of a string or slice,
use `p.find?` or `p.find`.
- To do the same but between two positions `p` and `q`, construct the
slice from `p` to `q` and then use `find?` or `find` on that.
- To search backwards, all of the above applies, except that the
function is called `revFind?`, there is no non-question-mark version
(use `getD` if there is a sane default return value in your specific
application), and that you can only search for characters and character
predicates, not strings or slices.
This PR redefines `front` and `back` on `String` to go through
`String.Slice` and adds the new `String` functions `front?`, `back?`,
`positions`, `chars`, `revPositions`, `revChars`, `byteIterator`,
`revBytes`, `lines`.
This PR renames `String.replaceStartEnd` to `String.slice`,
`String.replaceStart` to `String.sliceFrom`, and `String.replaceEnd` to
`String.sliceTo`, and similar for the corresponding functions on
`String.Slice`.
This PR adds `Std.Slice.Pattern` instances for `p : Char -> Prop` as
long as `DecidablePred p`, to allow things like `"hello".dropWhile (· =
'h')`.
To achieve this, we refactor `ForwardPattern` and friends to be
"non-uniform", i.e., the class is now `ForwardPattern pat`, not
`ForwardPattern ρ` (where `pat : ρ`).
This PR adds a function `String.Slice.length`, with the following
deprecation string: There is no constant-time length function on slices.
Use `s.positions.count` instead, or `isEmpty` if you only need to know
whether the slice is empty.
This PR adds the alias `String.Slice.any` for `String.Slice.contains`.
It would probably be even better to only have one, but we don't have a
good mechanism for pointing people looking for one towards the other, so
an alias it is for now.
This PR introduces a function `String.split` which is based on
`String.Slice.split` and therefore supports all pattern types and
returns a `Std.Iter String.Slice`.
This supersedes the functions `String.splitOn` and `String.splitToList`,
and we remove all all uses of these functions from core. They will be
deprecated in a future PR.
Migrating from `String.splitOn` and `String.splitToList` is easy: we
introduce functions `Iter.toStringList` and `Iter.toStringArray` that
can be used to conveniently go from `Std.Iter String.Slice` to `List
String` and `Array String`, so for example `s.splitOn "foo"` can be
replaced by `s.split "foo" |>.toStringList`.
This PR redefines `String.take` and variants to operate on
`String.Slice`. While previously functions returning a substring of the
input sometimes returned `String` and sometimes returned
`Substring.Raw`, they now uniformly return `String.Slice`.
This is a BREAKING change, because many functions now have a different
return type. So for example, if `s` is a string and `f` is a function
accepting a string, `f (s.drop 1)` will no longer compile because
`s.drop 1` is a `String.Slice`. To fix this, insert a call to `copy` to
restore the old behavior: `f (s.drop 1).copy`.
Of course, in many cases, there will be more efficient options. For
example, don't write `f <| s.drop 1 |>.copy |>.dropEnd 1 |>.copy`, write
`f <| s.drop 1 |>.dropEnd 1 |>.copy` instead. Also, instead of `(s.drop
1).copy = "Hello"`, write `s.drop 1 == "Hello".toSlice` instead.
This PR removes all uses of `String.Iterator` from core, preferring
`String.ValidPos` instead.
In an upcoming PR, `String.Iterator` will be renamed to
`String.Legacy.Iterator`.
This PR defines `String.Slice.replace` and redefines `String.replace` to
use the `Slice` version.
The new implementation is generic in the pattern, so it supports things
like `"education".replace isVowel "☃!" = "☃!d☃!c☃!t☃!☃!n"`. Since it
uses the `ForwardSearcher` infrastructure, `String` patterns are
searched using KMP, unlike the previous implementation which had
quadratic runtime. As a side effect, the behavior when replacing an
empty string now matches that of most other programming languages,
namely `"abc".replace "" "k" = "kakbkck"`.
This PR fixes a bug in `String.Slice.takeWhile` which caused it to get
its bookkeeping wrong and panic. The new version only uses safe
operations on `String.Slice.Pos`.
This PR shows that the iterators returned by `String.Slice.split` and
`String.Slice.splitInclusive` are finite as long as the forward matcher
iterator for the pattern is finite (which we already know for all of our
patterns).
At actually also completely redefines the iterators to avoid the inner
loop in `Internal.nextMatch` which generates inefficient code. Instead,
when encountering a mismach from the matcher, we `skip` the split
iterator.
This PR moves many operations involving `String.Pos.Raw` to a the
`String.Pos.Raw` namespace with the eventual aim of freeing up the
`String` namespace to contain operations using `String.ValidPos` (to be
renamed to `String.Pos`) instead.
This PR adds the `String.ValidPos.set` and `String.ValidPos.modify`
functions.
After this PR, `String.pos_lt_eq` is no longer a `simp` lemma. Add
`String.Pos.Raw.lt_iff` as a `simp` lemma if your proofs break.
This PR introduces a no-op version of `Shrink`, a type that should allow
shrinking small types into smaller universes given a proof that the type
is small enough, and uses it in the iterator library. Because this type
would require special compiler support, the current version is just a
wrapper around the inner type so that the wrapper is equivalent, but not
definitionally equivalent.
While `Shrink` is unable to shrink universes right now, but introducing
it now will allow us to generalize the universes in the iterator library
with fewer breaking changes as soon as an actual `Shrink` is possible.
This PR enforces rules around arithmetic of `String.Pos.Raw`.
Specifically, it adopts the following conventions:
- Byte indices ("ordinals") in strings should be represented using
`String.Pos.Raw`
- Amounts of bytes ("cardinals") in strings should be represented using
`Nat`.
For example, `String.Slice.utf8ByteSize` now returns `Nat` instead of
`String.Pos.Raw`, and there is a new function `String.Slice.rawEndPos`.
Finally, the `HAdd` and `HSub` instances for `String.Pos.Raw` are
reorganized. This is a **breaking change**.
The `HAdd/HSub String.Pos.Raw String.Pos.Raw String.Pos.Raw` instances
have been removed. For the use case of tracking positions relative to
some other position, we instead provide `offsetBy` and `unoffsetBy`
functions. For the use case of advancing/unadvancing a position by an
arbitrary number of bytes, we instead provide `increaseBy` and
`decreaseBy` functions. For
offsetting/unoffsetting/advancing/unadvancing a position `p` by the size
of a string `s` (resp. character `c`), use `s + p`/`p - s`/`p + s`/`p -
s` (resp. `c + p`/`p - c`/`p + c`/`p - c`).
