Commit graph

64 commits

Author SHA1 Message Date
Sebastian Ullrich
7994e55d80
chore: try refining some benchmark settings (#8377) 2025-05-16 11:24:11 +00:00
Sebastian Ullrich
01dbbeed99
feat: do not export def bodies by default (#8221)
This PR adjusts the experimental module system to not export the bodies
of `def`s unless opted out by the new attribute `@[expose]` on the `def`
or on a surrounding `section`.

---------

Co-authored-by: Markus Himmel <markus@lean-fro.org>
2025-05-15 12:16:54 +00:00
Sebastian Ullrich
9486421fcc
chore: tame some slow benchmarks (#8352)
No single-topic benchmark should take half as long as stdlib.

Bench run time reduced from 27min to 21min.
2025-05-15 11:53:10 +00:00
Sebastian Ullrich
7feb583b9e
feat: enable experimental module system in Init (#8047) 2025-04-23 17:21:33 +00:00
Joachim Breitner
2386a3d7c7
chore: add RISC-V ast benchmark (#8035)
This PR adds a realistic large-inductive benchmark, taken from
https://github.com/opencompl/sail-riscv-lean
2025-04-21 15:46:38 +00:00
Henrik Böving
dd7ca772d8
refactor: more complete channel implementation for Std.Channel (#7819)
This PR extends `Std.Channel` to provide a full sync and async API, as
well as unbounded, zero sized and bounded channels.

A few notes on the implementation:
- the bounded channel is inspired by [Go channels on
steroids](https://docs.google.com/document/d/1yIAYmbvL3JxOKOjuCyon7JhW4cSv1wy5hC0ApeGMV9s/pub)
though currently doesn't do any of the lock-free optimizations
- @mhuisi convinced me that having a non-closable channel may be a good
idea as this alleviates the need for error handling which is very
annoying when working with `Task`. This does complicate the API a little
bit and I'm not quite sure whether this is a choice we want users to
give. An alternative to this would be to just write `send!` that panics
on sending to a closed channel (receiving from a closed channel is not
an error), this is for example the behavior that golang goes with.
2025-04-12 21:02:24 +00:00
Sebastian Ullrich
c672934f11
chore: add "Init size" benchmark (#7918) 2025-04-11 13:15:27 +00:00
Sebastian Ullrich
bd24ca3093
test: re-elaboration benchmarks (#7784)
Tests language server memory use by repeatedly re-elaborate a given file
2025-04-02 10:10:46 +00:00
Henrik Böving
359f5bfda9
perf: add a large AIG benchmark for bv_decide (#7721)
This PR adds a benchmark that produces a gigantic AIG out of a
relatively small input, allowing us to measure performance bottlenecks
in the AIG framework itself.
2025-03-29 16:04:25 +00:00
Sebastian Ullrich
7240d910d3
chore: more core proof benchmarks 2025-03-21 15:59:14 +01:00
Sebastian Ullrich
087f0b4a69
perf: optimize sorry detection in unused variables linter (#7129)
This PR optimizes the performance of the unused variables linter in the
case of a definition with a huge `Expr` representation
2025-02-22 16:43:39 +00:00
Marc Huisinga
0160aa1a89
test: identifier completion benchmark (#6796)
Adds a basic identifier completion benchmark so that bugs like the one
in #6794 are caught earlier.
2025-01-27 19:31:32 +00:00
Sebastian Ullrich
91e261da38
chore: disable Elab.async on the cmdline for now (#6722)
Avoids build time overhead until the option is proven to speed up
average projects. Adds Init.Prelude (many tiny declarations, "worst
case") and Init.List.Sublist (many nontrivial theorems, "best case")
under -DElab.async=true as new benchmarks for tracking.
2025-01-22 18:25:47 +00:00
Joachim Breitner
6b811f8c92
test: synthetic simp_arith benchmark (#6061)
This PR adds a simp_arith benchmark.

This benchmark highlights some improvable asymptotics in `Nat.Linear`,
which
will be fixed subsequently.
2024-11-13 15:49:52 +00:00
Sebastian Ullrich
9157c1f279
test: big_omega benchmark (#5817)
Extracted from #5614
2024-10-24 07:26:29 +00:00
Aaron Tomb
45b1b367ca
test: add a benchmark that is slow to elaborate (#5656)
Add an example Lean file that includes an unusually large definition
that takes a long time to elaborate.

It may be that it's difficult to process it more efficiently, but
perhaps someone will discover a way to improve it if it's in the
benchmark suite. Improved performance on this benchmark will likely make
some program analysis and verification tasks within Lean more feasible.

---------

Co-authored-by: Sebastian Ullrich <sebasti@nullri.ch>
2024-10-23 08:20:15 +00:00
Alex Keizer
94dd1d61bd
feat: bv_decide inequality regression tests (#5714)
This takes a few standalone bitvector problems, about inequalties, from
LNSym, and adds them as a benchmark to prevent further regressions with
bv_decide.

These problems are particularly interesting, because they've previously
had a bad interaction with bv_decides normalization pass, see
https://github.com/leanprover/lean4/issues/5664.

---------

Co-authored-by: Henrik Böving <hargonix@gmail.com>
2024-10-15 08:51:14 +00:00
Henrik Böving
248864c716
perf: benchmark for modulo on bv_decide (#5653)
This verifies a bit hack from here:
https://en.wikipedia.org/wiki/Lehmer_random_number_generator#Sample_C99_code

I previously ran the SMTLIB equivalent this with Bitwuzla in my crypto
class and got the following numbers:
- 22s with Bitwuzla
- Z3 and CVC5 don't yet terminate after > 2min

Now with`bv_decide` the overall timing is 33.7s, consisting of:
- 5s of checking the LRAT cert
- 5s of trimming the LRAT cert from 800k to 300k proof steps
- remainder actual solving time

So running `bv_decide` like a normal SMT solver without verifying the
result of the SAT solver would yield approximately ~24s.
2024-10-08 18:58:15 +00:00
Henrik Böving
3120c3d8f8
feat: add bv_decide benchmarks (#5203) 2024-08-29 12:45:58 +00:00
Marc Huisinga
9009c1ac91
fix: ilean loading performance (#4900)
This PR roughly halves the time needed to load the .ilean files by
optimizing the JSON parser and the conversion from JSON to Lean data
structures.

The code is optimized roughly as follows:
- String operations are inlined more aggressively
- Parsers are changed to use new `String.Iterator` functions `curr'` and
`next'` that receive a proof and hence do not need to perform an
additional check
- The `RefIdent` of .ilean files now uses a `String` instead of a `Name`
to avoid the expensive parse step from `String` to `Name` (despite the
fact that we only very rarely actually need a `Name` in downstream code)
- Instead of `List`s and `Subarray`s, the JSON to Lean conversion now
directly passes around arrays and array indices to avoid redundant
boxing
- Parsec's `peek?` sometimes generates redundant `Option` wrappers
because the generation of basic blocks interferes with the ctor-match
optimization, so it is changed to use an `isEof` check where possible
- Early returns and inline-do-blocks cause the code generator to
generate new functions, which then interfere with optimizations, so they
are now avoided
- Mutual defs are used instead of unspecialized passing of higher-order
functions to generate faster code
- The object parser is made tail-recursive

This PR also fixes a stack overflow in `Lean.Json.compress` that would
occur with long lists and adds a benchmark for the .ilean roundtrip
(compressed pretty-printing -> parsing).
2024-08-29 11:51:48 +00:00
Sebastian Ullrich
f6265e25f4
chore: fix "max dynamic symbols" metric (#4669)
As we do not build multiple shared libraries on non-Windows anymore,
count the max exported symbols per static library instead.
Unfortunately, this still does seem to match the number on Windows.
2024-07-08 09:09:19 +00:00
Joachim Breitner
4f50544242
chore: Nat.repr microbenchmark (#3888) 2024-04-17 18:10:32 +00:00
Marc Huisinga
093e1cf22a
test: add language server startup benchmark (#3558)
Benchmark to catch future regressions as the one fixed in #3552.
2024-03-04 09:01:51 +00:00
Sebastian Ullrich
c9aea32d3e
chore: speedcenter: count max symbols in shared libraries (#3418) 2024-02-20 19:25:24 +00:00
Sebastian Ullrich
8bc1a9c4ba
chore: actually include full build in benchmark (#3158)
I must have reverted too much while testing #3104
2024-01-10 14:33:27 +00:00
Sebastian Ullrich
caf7a21c6f
chore: include full build in stdlib benchmark (#3104) 2023-12-23 16:27:07 +00:00
Sebastian Ullrich
767139b235
chore: use all cores in stdlib benchmark 2023-12-21 10:37:18 +01:00
Sebastian Ullrich
b278172b7c chore: add import Lean benchmark 2023-11-07 18:46:28 +01:00
Sebastian Ullrich
00e981edcd perf: do not inhibit caching of default-level match reduction 2023-10-08 17:24:20 -07:00
tydeu
2ac782c315 test: lake: add env & dep cfg benchmarks + cleanup 2023-09-22 20:31:48 -04:00
Sebastian Ullrich
c3fd34f933 chore: disable "lake build lean" benchmark for now 2023-09-22 20:05:20 +02:00
tydeu
00efb7eaca test: add reconfigure benchmark 2023-09-22 20:05:20 +02:00
tydeu
5b2e3e2b0a test: make compatible with olean caching 2023-09-22 20:05:20 +02:00
tydeu
8dba187910 chore: inundation for configure benchmark 2023-09-22 20:05:20 +02:00
tydeu
1d51492139 test: lake: add build Init/Lean/Lake benchmark 2023-09-22 20:05:20 +02:00
tydeu
9a0e57c721 test: add lake benchmarks 2023-09-22 20:05:20 +02:00
Sebastian Ullrich
bb738796ae test: update parser benchmark, add to speedcenter suite 2023-08-08 18:40:19 +02:00
Sebastian Ullrich
899b673531 test: add binarytrees.st benchmark 2023-01-19 14:44:20 +01:00
Sebastian Ullrich
6b8fa76265 test: benchmark workspace symbols search 2022-10-13 21:41:58 +02:00
Sebastian Ullrich
5b7e6661f9 chore: more RBMap cleanup 2022-10-06 17:26:43 -07:00
Sebastian Ullrich
6f4cea6dba feat: add rbmap_fbip benchmark 2022-10-06 17:26:43 -07:00
Sebastian Ullrich
9f29967fb0 chore: add rbmap.library benchmark to bench suite 2022-09-24 12:35:08 +02:00
Sebastian Ullrich
2e47a8d5ca test: actually register new benchmark 2021-12-03 09:25:53 +01:00
Sebastian Ullrich
2e36e362cb chore: bench: apparently make doesn't like writing to /dev/null anymore? 2021-10-20 12:44:30 +02:00
Sebastian Ullrich
5f4b1b1d44 Revert "Revert "feat: reintroduce libleanshared, link lean & leanpkg against it""
This reverts commit ccbc9d00db.
2021-08-20 09:42:05 -07:00
Sebastian Ullrich
ccbc9d00db Revert "feat: reintroduce libleanshared, link lean & leanpkg against it" 2021-08-20 15:39:00 +02:00
Sebastian Ullrich
40f41a1f14 chore: adjust binary size benchmark 2021-08-18 13:54:52 +02:00
Sebastian Ullrich
d3f007b3cd chore: fix stdlib benchmarks 2021-08-12 22:41:42 +02:00
Sebastian Ullrich
48361d92dd chore: increase robustness of stdlib benchmark? 2020-12-30 17:01:05 +01:00
Sebastian Ullrich
1fc5b30fe1 chore: measure stage 2 sizes & add C lines benchmark 2020-12-18 17:47:11 +01:00