08eb78a5b2
This PR sets up the new integrated test/bench suite. It then migrates all benchmarks and some related tests to the new suite. There's also some documentation and some linting. For now, a lot of the old tests are left alone so this PR doesn't become even larger than it already is. Eventually, all tests should be migrated to the new suite though so there isn't a confusing mix of two systems.
37 lines
1.3 KiB
Markdown
37 lines
1.3 KiB
Markdown
# The `build` benchmark
|
|
|
|
This benchmark executes a complete build of the stage3 stdlib from stage2 and
|
|
collects global and per-module metrics. This is different from most other
|
|
benchmarks, which benchmark the stage the bench suite is being executed in.
|
|
|
|
The following metrics are collected by a wrapper around the entire build process:
|
|
|
|
- `build//cycles`
|
|
- `build//instructions`
|
|
- `build//maxrss`
|
|
- `build//task-clock`
|
|
- `build//wall-clock`
|
|
|
|
The following metrics are collected from `leanc --profile` and `leanc --stat` and summed across all modules:
|
|
|
|
- `build/profile/<name>//wall-clock`
|
|
- `build/stat/<name>//amount`
|
|
- `build/stat/<name>//bytes`
|
|
|
|
The following metrics are collected from `lakeprof report`:
|
|
|
|
- `build/lakeprof/longest build path//wall-clock`
|
|
- `build/lakeprof/longest rebuild path//wall-clock`
|
|
|
|
The following metrics are collected individually for each module:
|
|
|
|
- `build/module/<name>//lines`
|
|
- `build/module/<name>//cycles`
|
|
- `build/module/<name>//instructions`
|
|
- `build/module/<name>//bytes .ilean`
|
|
- `build/module/<name>//bytes .olean`
|
|
- `build/module/<name>//bytes .olean.server`
|
|
- `build/module/<name>//bytes .olean.private`
|
|
|
|
If the file `build_upload_lakeprof_report` is present in the repo root,
|
|
the lakeprof report will be uploaded once the benchmark run concludes.
|