Lean 4 fork for HoTT-compatible kernel extensions (Path types, transport, HITs). Maintained against upstream leanprover/lean4.
Find a file
Leonardo de Moura fa40491c78
test: benchmark MetaM vs SymM (#12100)
This PR adds a comparison between `MetaM` and `SymM` for a benchmark was
proposed during the Lean@Google Hackathon.

### Benchmark description

In this benchmark, we define the semantics of a very simple imperative
language using an inductive predicate

```
Exec prog events mem lctx post
```

The predicate holds if, when executing the program `prog` with an
initial list of events `events`, memory `mem`, and local context `lctx`,
the postcondition `post` holds.

We then consider the following program:

```
input b
a := b
a := a + a
a := a - b
...
a := a + a
a := a - b
```

That is, after reading an input value `b`, the program repeatedly
updates the variable `a` by doubling it and then subtracting `b`.

We prove that, for any initial memory `m` and local context `l`, and
starting from the empty list of events, the following postcondition
holds:

```
fun t' m' l' =>
  m' = m ∧                      -- memory did not change
  ∃ v : Word,
    t' = [IOEvent.IN v] ∧       -- exactly one input event
    l'.get "a" = some v         -- `a` contains the input value
```

In other words, executing the program produces exactly one input event,
leaves the memory unchanged, and ensures that the final value of `a` is
equal to the input value.

### Symbolic simulation benchmark (problem size `n`, with `2·n + 2`
instructions)

| Problem size (n) | MetaM time (ms) | MetaM kernel (ms) | SymM time
(ms) | SymM kernel (ms) | Total speedup |

|------------------|------------------|-------------------|----------------|------------------|---------------|
| 10  | 94.83  | 6.60  | 7.04  | 6.18  | ~13.5× |
| 20  | 218.92 | 13.33 | 14.15 | 13.02 | ~15.5× |
| 30  | 375.10 | 22.95 | 26.51 | 19.81 | ~14.2× |
| 40  | 563.82 | 34.99 | 40.42 | 29.55 | ~14.0× |
| 50  | 815.89 | 53.78 | 60.84 | 42.25 | ~13.4× |
| 60  | 1081.09 | 73.46 | 80.99 | 53.52 | ~13.3× | 
| 70  | 1400.80 | 102.70 | 106.02 | 68.61 | ~13.2× | 
| 80  | 1772.19 | 126.65 | 134.23 | 87.64 | ~13.2× |
| 90  | 2203.41 | 161.68 | 168.26 | 115.52 | ~13.1× | 
| 100 | 2474.09 | 191.23 | 209.13 | 143.86 | ~11.8× |

<img width="580" height="455" alt="image"
src="https://github.com/user-attachments/assets/bc7058fa-e71a-4c2c-be28-860f39166965"
/>

 ### Symbolic simulation with extra simplification (SymM)

Problem size `n` corresponds to a program with `2·n + 2` instructions.

| n   | Total time (ms) | Kernel time (ms) | Non-kernel time (ms) |
|-----|------------------|------------------|----------------------|
| 10  | 6.33  | 3.97 | 2.36 |
| 20  | 10.30 | 5.59 | 4.71 |
| 30  | 13.72 | 7.38 | 6.34 |
| 40  | 17.85 | 8.84 | 9.01 |
| 50  | 21.90 | 10.63 | 11.27 |
| 60  | 27.00 | 12.56 | 14.44 |
| 70  | 32.02 | 14.04 | 17.98 |
| 80  | 37.25 | 15.76 | 21.49 |
| 90  | 42.55 | 17.95 | 24.60 |
| 100 | 49.30 | 20.03 | 29.27 |
| 200 | 125.56 | 38.21 | 87.36 |
| 300 | 293.58 | 66.79 | 226.79 |
| 400 | 361.87 | 78.96 | 282.91 |
| 500 | 518.51 | 102.51 | 416.00 |
| 600 | 716.63 | 122.81 | 593.82 |
2026-01-22 01:38:56 +00:00
.claude chore: add CI log retrieval guidance to CLAUDE.md (#11964) 2026-01-10 07:41:41 +00:00
.github perf: avoid allocation on lean_trace use in interpreter (#12036) 2026-01-18 19:33:48 +00:00
doc doc: write a guideline for tactic docstrings (#11406) 2026-01-06 04:40:20 +00:00
images
releases_drafts chore: add release draft for the module system (#11359) 2025-11-26 15:01:07 +00:00
script chore: modulize: work around unknown initial command (#12080) 2026-01-21 20:25:13 +00:00
src perf: avoid mkAppM in Sym.simp (#12099) 2026-01-22 00:01:43 +00:00
stage0 chore: update stage0 2026-01-21 15:30:16 +00:00
tests test: benchmark MetaM vs SymM (#12100) 2026-01-22 01:38:56 +00:00
.gitattributes doc: grove: update and add String data (#11551) 2025-12-08 16:49:37 +00:00
.gitignore chore: add .vscode/settings.json to .gitignore (#10795) 2025-10-16 07:08:41 +00:00
.gitpod.Dockerfile
.gitpod.yml
.ignore
CMakeLists.txt fix: cadical dynamic dependencies (#11475) 2025-12-02 13:54:26 +00:00
CMakePresets.json chore: CI: enable leak sanitizer again (#11339) 2025-11-27 18:32:35 +00:00
CODEOWNERS chore: make @hargoniX code owner of the compiler (#10732) 2025-10-10 04:43:38 +00:00
CONTRIBUTING.md
flake.lock
flake.nix
lean-toolchain
lean.code-workspace chore: lean.code-workspace: fix terminal cwd (#10802) 2025-10-16 20:19:12 +00:00
LICENSE
LICENSES
README.md
RELEASES.md

This is the repository for Lean 4.

About

Installation

See Install Lean.

Contributing

Please read our Contribution Guidelines first.

Building from Source

See Building Lean.