2120883307
@Kha I had some unexpected surprises, but it is a good change.
Here is the summary.
1- We could get rid of `a %ₙ b` and `ModN` class. We can use `HMod`
instead. It was a positive surprise since I didn't remember we had
this `ModN` class.
2- Coercions are never used in heterogeneous operators. This is
expected since `a * b` is now notation for `HMul.hMul a b`, and
`a` and `b` may have different types. I manually added instances such
as `HMul Nat Int Int`. However, I did not try to add generic instances
such as
```
instance [Coe a b] [Mul b] : HMul a b b where
hMul x y := mul (coe x) y
```
I will try later.
3- Give `h : cs.size > 0`, I got a type error at
```
let idx : Fin cs.size := ⟨cs.size - 1, Nat.predLt h⟩
```
`Nat.predLt h` has type `Nat.pred cs.size < cs.size`
However, `Nat.pred cs.size` doesn't unify with `cs.size - 1`.
The problem is that we can't synthesize the `HSub` instance until
we apply the default instances.
It worked before because `isDefEq` would force the pending TC
problem `Sub Nat` to be resolved, and after that we would be able
to reduce `cs.size - 1` and establish that it is definitionally
equal to `Nat.pred cs.size`.
I considered two possible workarounds
a) `let idx : Fin cs.size := ⟨cs.size - (1:Nat), Nat.predLt h⟩`
b) `let idx : Fin cs.size := ⟨cs.size - 1, by exact Nat.predLt h⟩`
The first one works because we are not providing enough information
for synthesizing the `HSub` instance. The second works because it
postpones the elaboration of `Nat.predLt h`. The default instances
will be applied before we start applying tactics.
4- The `.` notation is affected too. For example, `(x + 1).toUInt8`
doesn't work since we don't know the type of `x+1` until we apply
default instances. I fixed it by using `(x + (1:Nat)).toUInt8`.
Another possible fix is `Nat.toUInt8 (x + 1)`.
Similarly, `(x+1).fold ...` doesn't work.
5- The following code failed to be elaborated
```
indent (push s!"{ss'}\n") (some (0 - Format.getIndent (← getOptions)))
```
It was working before, but it relied on how the expected type is
propagated. The elaborator process
```
some (0 - Format.getIndent (← getOptions))
```
with expected type `(Option Int)`. So, the `-` is interpreted as
`Int.sub` although `Format.getIndent (← getOptions)` has type `Nat`.
In the new `HSub`, the expected type doesn't really influence TC
resolution since it is an `outparam`. So, we failed with the error
failed to synthesize `HSub Nat Nat Int`.
One possible fix was to add the instance `HSub Nat Nat Int` with
`Int.sub`, but I used the following fix
```
some ((0 : Int) - Format.getIndent (← getOptions))
```
which makes it clear that we want the `Int.sub` operator instead of
`Nat.sub`.
|
||
|---|---|---|
| .. | ||
| .gitignore | ||
| accumulate_profile.py | ||
| arith_eval.ml | ||
| binarytrees.ghc-6.hs | ||
| binarytrees.lean | ||
| binarytrees.lean.args | ||
| binarytrees.lean.expected.out | ||
| binarytrees.ocaml-2.ml | ||
| binarytrees.swift | ||
| compile.sh | ||
| const_fold.hs | ||
| const_fold.lean | ||
| const_fold.lean.args | ||
| const_fold.lean.expected.out | ||
| const_fold.ml | ||
| const_fold.sml | ||
| const_fold.swift | ||
| cross.nix | ||
| cross.yaml | ||
| deriv.hs | ||
| deriv.lean | ||
| deriv.lean.args | ||
| deriv.lean.expected.out | ||
| deriv.ml | ||
| deriv.sml | ||
| deriv.swift | ||
| disable-st.patch | ||
| full-stdlib.exec.yaml | ||
| ghc-gc.py | ||
| lean-gc.py | ||
| Makefile | ||
| mlkit-gc.py | ||
| nixpkgs.nix | ||
| ocaml-gc.py | ||
| perf.py | ||
| qsort.hs | ||
| qsort.lean | ||
| qsort.lean.args | ||
| qsort.lean.expected.out | ||
| qsort.ml | ||
| qsort.sml | ||
| qsort.swift | ||
| rbmap.hs | ||
| rbmap.lean | ||
| rbmap.lean.args | ||
| rbmap.lean.expected.out | ||
| rbmap.library.lean | ||
| rbmap.ml | ||
| rbmap.sml | ||
| rbmap.swift | ||
| rbmap2.lean | ||
| rbmap3.lean | ||
| rbmap4.lean | ||
| rbmap500k.lean | ||
| rbmap_checkpoint.hs | ||
| rbmap_checkpoint.lean | ||
| rbmap_checkpoint.lean.args | ||
| rbmap_checkpoint.lean.expected.out | ||
| rbmap_checkpoint.ml | ||
| rbmap_checkpoint.sml | ||
| rbmap_checkpoint.swift | ||
| rbmap_checkpoint2.lean | ||
| rbmap_checkpoint2.sml | ||
| rbmap_checkpoint_cpp_lean3.cpp | ||
| rbmap_checkpoint_cpp_std.cpp | ||
| rbmap_cpp_lean3.cpp | ||
| rbmap_cpp_std.cpp | ||
| README.md | ||
| report.py | ||
| run.sh | ||
| speedcenter.exec.velcom.yaml | ||
| speedcenter.yaml | ||
| test_single.sh | ||
| unionfind.lean | ||
| unionfind.lean.args | ||
| unionfind.lean.expected.out | ||
| unionfind_clean.lean | ||
Lean Benchmark Suites
This folder contains multiple small Lean programs for benchmarking used by two separate benchmark suites based on the temci benchmarking tool:
- The light-weight "Speedcenter" suite benchmarks the current build of Lean. It can be used for quick comparisons on the cmdline and powers the Lean Speedcenter website.
- The heavy-weight "Cross" suite benchmarks multiple Lean configurations and other functional compilers against each other and generates CSV and HTML reports from that. It was created for the paper "Counting Immutable Beans - Reference Counting Optimized for Purely Functional Programming" (IFL19).
Speedcenter Suite
Requirements:
- A local Lean build in
../../build/release. Build at least thebintarget. - temci. Using Nix, open a nix-shell in the project
root directory to add a compatible version to your PATH. Alternatively, try
pip3 install git+https://github.com/parttimenerd/temci.git.
To execute the suite and save the results in base.yaml, run (in this folder)
temci exec --config speedcenter.yaml --out base.yaml
Other interesting exec flags:
- use
--runs Nto modify the default number of 10 runs per benchmark - use
--included_blocks fastto excluded slow benchmarks like the stdlib benchmark. You can replacefastwith any benchmark name or label inspeedcenter.exec.yaml.
If you have multiple saved result files, you can compare them with
temci report --config speedcenter.yaml report1.yaml report2.yaml ...
Cross Suite
We recommend using Nix for building/obtaining all Lean variants and used compilers in a reproducible way. After installing Nix, running the benchmarks is as easy as
nix-shell cross.nix --pure --run make
This will record 50 runs for each benchmark configuration (this can be changed with runs in cross.yaml),
generate results in report_lean.csv and report_cross.csv, and print them to stdout in a tabulated format.
It will also generate HTML reports in report/ comparing the time-based benchmarks.
In order to reduce noise in the benchmarking data, you may instead want to try calling make inside a
temci shell:
nix-shell cross.nix --pure --run "temci short shell --sudo --preset usable --cpuset_active make"
Using root powers, this will temporarily configure your machine similarly to the LLVM benchmarking recommendations and move all your other processes to a single CPU core.