lean4-htt/src/Lean/Data
Mario Carneiro 62cdb51ed5
feat: UTF-8 string validation (#3958)
Previously, there was a function `opaque fromUTF8Unchecked : ByteArray
-> String` which would convert a list of bytes into a string, but as the
name implies it does not validate that the string is UTF-8 before doing
so and as a result it produces unsound results in the compiler (because
the lean model of `String` indirectly asserts UTF-8 validity). This PR
replaces that function by
```lean
opaque validateUTF8 (a : @& ByteArray) : Bool

opaque fromUTF8 (a : @& ByteArray) (h : validateUTF8 a) : String
```
so that while the function is still "unchecked", we have a proof witness
that the string is valid. To recover the original, actually unchecked
version, use `lcProof` or other unsafe methods to produce the proof
witness.

Because this was the only `ByteArray -> String` conversion function, it
was used in several places in an unsound way (e.g. reading untrusted
input from IO and treating it as UTF-8). These have been replaced by
`fromUTF8?` or `fromUTF8!` as appropriate.
2024-04-20 18:36:37 +00:00
..
Json feat: UTF-8 string validation (#3958) 2024-04-20 18:36:37 +00:00
Lsp fix: semantic tokens performance (#3932) 2024-04-18 07:48:44 +00:00
Xml chore: reorganising to reduce imports (#3790) 2024-03-27 11:15:01 +00:00
Array.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
AssocList.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
Format.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
FuzzyMatching.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
HashMap.lean perf: fix linearity in (HashSet|HashMap).erase (#3887) 2024-04-12 08:54:21 +00:00
HashSet.lean perf: fix linearity in (HashSet|HashMap).erase (#3887) 2024-04-12 08:54:21 +00:00
Json.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
JsonRpc.lean feat: snapshot trees and language processors (#3014) 2024-03-14 13:40:08 +00:00
KVMap.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
LBool.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
LOption.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
Lsp.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
Name.lean chore: remove the coercion from String to Name (#3589) 2024-03-21 23:46:03 +00:00
NameMap.lean chore: remove the coercion from String to Name (#3589) 2024-03-21 23:46:03 +00:00
NameTrie.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
OpenDecl.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
Options.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
Parsec.lean chore: reorganising to reduce imports (#3790) 2024-03-27 11:15:01 +00:00
PersistentArray.lean chore: extend GetElem with getElem! and getElem? (#3694) 2024-03-28 01:42:00 +00:00
PersistentHashMap.lean fix: simp usedSimps (#3821) 2024-04-02 00:50:06 +00:00
PersistentHashSet.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
Position.lean chore: remove FileMap.lines and add FileMap.getLine (#3237) 2024-03-25 10:33:04 +00:00
PrefixTree.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
Rat.lean chore: reorganising to reduce imports (#3790) 2024-03-27 11:15:01 +00:00
RBMap.lean chore: upstream solve_by_elim (#3408) 2024-02-21 01:16:04 +00:00
RBTree.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
SMap.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
SSet.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
Trie.lean perf: add prelude to all Lean modules 2024-02-18 14:55:17 -08:00
Xml.lean chore: add missing copyright headers (#3411) 2024-02-20 01:49:55 +00:00