lean4-htt/tests/bench/sym
Leonardo de Moura 4c1e4a77b4
test: MetaM vs SymM on do notation (#12134)
This PR adds a new benchmark `shallow_add_sub_cancel.lean` that
demonstrates symbolic simulation using a shallow embedding into monadic
`do` notation, as opposed to the deep embedding approach in
`add_sub_cancel.lean`.

The shallow embedding approach:
- Uses Lean's `StateM` monad directly instead of a custom command
language

- Defines `Exec s k post` as a simple predicate: `post (k s).1 (k s).2`

- Proves helper theorems for reasoning about monadic operations (`pure`,
`bind`, `get`, `set`, `modify`, `ite`)

- Programs are written in actual `do`-notation rather than a custom AST

The benchmark solves goals using both the `MetaM` and `SymM` frameworks,
showing that the shallow embedding integrates well with the symbolic
simulation infrastructure. `SymM` is again way faster than `MetaM`

### Symbolic simulation benchmark — tactic time only

Problem size `n` corresponds to a program with `4·n` monadic actions.

| n   | MetaM tactic (ms) | SymM tactic (ms) | Speedup |
|-----|-------------------|------------------|---------|
| 10  | 82.10  | 11.37 | ~7.2×  |
| 20  | 176.21 | 17.71 | ~9.9×  |
| 30  | 306.47 | 25.39 | ~12.1× |
| 40  | 509.52 | 34.53 | ~14.7× |
| 50  | 689.19 | 43.51 | ~15.8× |
| 60  | 905.86 | 52.47 | ~17.3× |
| 70  | 1172.31 | 62.50 | ~18.8× |
| 80  | 1448.48 | 70.65 | ~20.5× |
| 90  | 1787.15 | 80.89 | ~22.1× |
| 100 | 2128.12 | 90.77 | ~23.5× |

<img width="580" height="455" alt="image"
src="https://github.com/user-attachments/assets/3511aaab-4d53-4520-8302-65d2d100df4a"
/>
2026-01-24 03:38:02 +00:00
..
add_sub_cancel.lean feat: improve Sym.simp APIs and new benchmark data (#12101) 2026-01-22 03:37:16 +00:00
meta_simp_1.lean chore: include free variables (#11894) 2026-01-04 18:51:18 +00:00
meta_simp_2.lean test: benchmarks for lambda-telescopes (#11929) 2026-01-08 00:20:03 +00:00
meta_simp_4.lean feat: simpForall and simpArrow in Sym.simp (#11950) 2026-01-09 06:20:04 +00:00
shallow_add_sub_cancel.lean test: MetaM vs SymM on do notation (#12134) 2026-01-24 03:38:02 +00:00
simp_1.lean perf: optimize congruence proof construction in Sym.simp (#11974) 2026-01-11 23:00:19 +00:00
simp_2.lean test: benchmarks for lambda-telescopes (#11929) 2026-01-08 00:20:03 +00:00
simp_3.lean perf: optimize kernel type-checking for have-telescope simplification in Sym.simp (#11967) 2026-01-11 02:20:47 +00:00
simp_4.lean test: implies vs Arrow Sym.simp benchmark (#11966) 2026-01-10 18:51:54 +00:00