Commit graph

754 commits

Author SHA1 Message Date
Rob23oba
e713232623
fix: resolve symbolic links through IO.FS.realPath on windows (#8534)
This PR fixes `IO.FS.realPath` on windows to take symbolic links into
account.

Closes #810
2025-06-13 13:16:13 +00:00
Justin King
0d0da768d8
perf: update free_sized declaration to be compatible with glibc (#8661)
glibc adds `__attribute__((nothrow))` to its declarations, at least for
those related to malloc. glibc has yet to introduce `free_sized`, but
when it does it would cause compilation errors. This is due to the fact
that if a function declarations has `__attribute__((nothrow))` and it is
re-declared or implemented in C++ it must also have
`__attribute__((nothrow))` or `noexcept`, otherwise the compilation will
fail.

This is a follow up to https://github.com/leanprover/lean4/pull/6598.

Signed-off-by: Justin King <jcking@google.com>
2025-06-13 13:13:00 +00:00
Sebastian Ullrich
f0347ee719
chore: lean --stats gives number of imported bytes (#8725)
Thanks to `mmap`, startup time is not necessarily related to this
figure, but it can be used as a rough measure for that and how much data
the module depends on, i.e. the rebuild chance.

Also adds new cumulative benchmarks for this metric as well as the
number of imported constants and env ext entries.
2025-06-12 08:29:42 +00:00
Sebastian Ullrich
add3e1ae12
fix: IO.FS.removeDirAll should not follow symlinks (#8573)
This PR avoids the likely unexpected behavior of `removeDirAll` to
delete through symlinks and adds the new function
`IO.FS.symlinkMetadata`.

---------

Co-authored-by: Rob23oba <152706811+Rob23oba@users.noreply.github.com>
2025-06-02 08:44:17 +00:00
Leonardo de Moura
6940d2c4ff
fix: block adversarial exploit of non-aborting assert! (#8559)
This PR fixes an adversarial soundness attack described in #8554. The
attack exploits the fact that `assert!` no longer aborts execution, and
that users can redirect error messages.
Another PR will implement the same fix for `Expr.Data`.
2025-05-31 00:08:30 +00:00
Cameron Zwarich
575b4786f9
feat: optimize lean_nat_shiftr for scalars (#8268)
This PR optimizes lean_nat_shiftr for scalar operands. The new compiler
converts Nat divisions into right shifts, so this now shows up as hot in
some profiles.
2025-05-11 01:39:59 +00:00
Markus Himmel
925e53fcba
fix: include libuv outside of namespace (#8166)
This PR makes sure we never `#include <uv.h>` while inside a namespace,
which recent GCC versions don't seem to like.
2025-04-29 22:19:17 +00:00
Henrik Böving
7b6c16a44b
feat: implement a Selector for async UDP (#8139)
This PR is a follow up to #8055 and implements a `Selector` for async
UDP in order to allow IO multiplexing using UDP sockets.

The technical approach taken for this PR is basically a copy of #8078
but adjusted for UDP. The libuv API gives the same guarantee that was
used in that PR.
2025-04-29 21:01:14 +00:00
Rob23oba
9f06aff834
feat: optimized division without remainder for Int and Nat (#8089)
This PR adds optimized division functions for `Int` and `Nat` when the
arguments are known to be divisible (such as when normalizing
rationals). These are backed by the gmp functions `mpz_divexact` and
`mpz_divexact_ui`. See also leanprover-community/batteries#1202.
2025-04-29 07:23:35 +00:00
Henrik Böving
406bda8807
feat: implement a Selector for async TCP (#8078)
This PR is a follow up to #8055 and implements a `Selector` for async
TCP in order to allow IO multiplexing using TCP sockets.

As we must not commit to actually fetching data from the socket buffer
this cannot be implemented by just racing on `recv?`. Instead we perform
a call to `uv_read_start` and pass an `alloc_cb` that allocates no
memory at all. According to the docs of
[`uv_alloc_cb`](https://docs.libuv.org/en/v1.x/handle.html#c.uv_alloc_cb)
this is guaranteed to give us a `UV_ENOBUFS` in the relevant callback.
Thus we can first run this "zero read" and then go into one of three
cases:
1. We get cancelled before the zero read completes, in this case just
cancel the zero read and give up.
2. The zero read completes and we loose the race for completing the
`select`, in this case just don't do anything anymore
3. The zero read completes and we win the race for completing the
`select`, in this case we perform the actual read on the socket. As we
know that data is available already (since the read callback of the zero
read is only triggered if data actually is available) we know that the
subsequent actual read should complete right away.

In this way we avoid any data loss if we loose the race.
2025-04-24 16:05:35 +00:00
Sebastian Ullrich
7feb583b9e
feat: enable experimental module system in Init (#8047) 2025-04-23 17:21:33 +00:00
Mac Malone
b51115dac5
feat: IO.Process.SpawnArgs.inheritEnv (#6081)
This PR adds an `inheritEnv` field to `IO.Process.SpawnArgs`. If
`false`, the spawned process does not inherit its parent's environment.

For example, Lake will make use of this to ensure that build processes
do not use environment variables that Lake is not properly tracking with
its traces.
2025-04-16 00:25:32 +00:00
Sebastian Ullrich
0076ba03d4
fix: race condition in IO.getTaskState (#7945)
This PR fixes a potential race between `IO.getTaskState` and the task in
question finishing, resulting in undefined behavior.

All task state must be accessed under the respective lock.
2025-04-14 14:08:36 +00:00
Henrik Böving
8e9da7a1bc
feat: wait on dedicated tasks after main is finished (#7958)
This PR ensures that after `main` is finished we still wait on dedicated
tasks instead of exiting forcefully. If users wish to violently kill
their dedicated tasks at the end of main instead they can run
`IO.Process.exit` at the end of `main` instead.
2025-04-14 11:53:54 +00:00
Henrik Böving
ac738a8e81
perf: use mimalloc in compactor hashmaps (#7929)
This PR changes the compactor hashmap to use mimalloc which speeds up
olean serialization.
2025-04-14 09:11:34 +00:00
Sebastian Ullrich
5cd352588c
perf: use mimalloc with important C++ hash maps (#7868)
`unordered_map`/`unordered_set` does an allocation per insert, use
mimalloc for them for important hash maps
2025-04-11 16:23:33 +00:00
Sebastian Ullrich
69536808ca
feat: read/writeModuleDataParts API for serialization with cross-file sharing (#7854)
This PR introduces fundamental API to distribute module data across
multiple files in preparation for the module system.
2025-04-10 13:32:24 +00:00
Sebastian Ullrich
1b40c46ab1
chore: panic on blocking waits in sync tasks (#7853) 2025-04-08 14:49:26 +00:00
euprunin
2ea675369f
chore: fix spelling mistakes (#7328)
Co-authored-by: euprunin <euprunin@users.noreply.github.com>
2025-04-07 01:15:48 +00:00
Leonardo de Moura
5f684b4777
feat: support mpz in the shareCommon APIs (#7838)
This PR adds support for mpz objects (i.e., big nums) to the
`shareCommon` functions.
2025-04-06 19:52:50 +00:00
Mac Malone
7947457305
perf: Task.bind short circuit (#7795)
This PR skips scheduling the resulting task in `Task.bind` /
`BaseIO.bindTask` if the task has already finished. Instead, it just
returns its value.
2025-04-04 23:24:51 +00:00
Sebastian Ullrich
83067d67d6
chore: avoid mimalloc warnings (#7800) 2025-04-03 13:52:16 +00:00
Henrik Böving
626075ca34
feat: add Std.SharedMutex (#7770)
This PR adds a shared mutex (or read-write lock) as `Std.SharedMutex`.

In order to easily migrate a `Std.Mutex` to `Std.SharedMutex` if
necessary, the functions for obtaining exclusive access are named the
same, allowing a correct drop in to be done by just swapping types.
2025-04-03 08:30:54 +00:00
Sebastian Ullrich
bd24ca3093
test: re-elaboration benchmarks (#7784)
Tests language server memory use by repeatedly re-elaborate a given file
2025-04-02 10:10:46 +00:00
Henrik Böving
32cd701994
feat: add Std.RecursiveMutex (#7755)
This PR adds `Std.RecursiveMutex` as a recursive/reentrant equivalent to
`Std.Mutex`.
2025-04-01 07:35:36 +00:00
Henrik Böving
1b5a52a5e9
feat: Std.BaseMutex.tryLock and Mutex.tryAtomically (#7751)
This PR adds `Std.BaseMutex.tryLock` and `Std.Mutex.tryAtomically` as
well as unit tests for our locking and condition variable primitives.

---------

Co-authored-by: Markus Himmel <markus@lean-fro.org>
2025-03-31 12:19:09 +00:00
Sebastian Ullrich
2edfe2e9cf
perf: store mimalloc object size in header (#7734) 2025-03-31 06:52:56 +00:00
Sebastian Ullrich
3b85d26af7
perf: use mimalloc for all Lean allocations (#7736) 2025-03-30 23:00:17 +00:00
Sebastian Ullrich
5ebac3fa50
perf: use mimalloc by default (#7710)
This PR improves memory use of Lean, especially for longer-running
server processes, by up to 60%
2025-03-30 22:40:41 +00:00
Sofia Rodrigues
74b1c29a48
feat: UDP socket support using LibUV (#7574)
This PR introduces UDP socket support using the LibUV library, enabling
asynchronous I/O operations with it.
2025-03-26 15:04:25 +00:00
Sofia Rodrigues
ad547b56f5
feat: add network interfaces (#7578)
This PR introduces a function called `interfaceAddresses` that retrieves
an array of system’s network interfaces.
2025-03-24 17:57:05 +00:00
Sebastian Ullrich
884fa444a1
chore: fix fsanitize preset (#7651) 2025-03-24 09:17:45 +00:00
Sebastian Ullrich
f42a28f718
chore: revert "perf: avoid taking mutex on task deactivation" (#7590)
Likely introduced segfaults.

Reverts leanprover/lean4#7572
2025-03-20 07:04:50 +00:00
Sebastian Ullrich
a5348f4bdc
perf: avoid taking mutex on task deactivation (#7572) 2025-03-19 15:59:45 +00:00
Sofia Rodrigues
d7d1754e69
feat: socket support using LibUV (#6683)
This PR introduces TCP socket support using the LibUV library, enabling
asynchronous I/O operations with it.

---------

Co-authored-by: Henrik Böving <hargonix@gmail.com>
Co-authored-by: Markus Himmel <markus@himmel-villmar.de>
2025-03-19 13:54:51 +00:00
Sebastian Ullrich
044e3b1b56
fix: heartbeats from realizeConst should be ignored (#7473)
Avoids nondeterministic counting from racing threads
2025-03-13 15:10:29 +00:00
Eric Wieser
6cf3402f1c
perf: use free_sized in mpz.cpp (#6825)
The performance win here is pretty negligible (and of course irrelevant
with the small allocator enabled), but this is consistent with it being
used elsewhere.

Follow-up to #6598
2025-03-03 08:47:15 +00:00
Sebastian Ullrich
2e66341f69
feat: Environment.realizeConst (#7076)
This PR introduces the central parallelism API for ensuring that helper
declarations can be generated lazily without duplicating work or
creating conflicts across threads.
2025-02-26 19:32:21 +00:00
Eric Wieser
a57efd0a88
fix: free memory from lib_uv requests (#7151)
This PR fixes a memory leak in `IO.FS.createTempFile`
2025-02-26 07:52:34 +00:00
Cameron Zwarich
647573d269
feat: support LEAN_BACKTRACE on macOS (#7184)
This PR adds support for LEAN_BACKTRACE on macOS. This previously only
worked with glibc, but it can not be enabled for all Unix-like systems,
since e.g. Musl does not support it.
2025-02-22 15:29:37 +00:00
Mac Malone
a41fb49e25
feat: smarter plugin loading (#7090)
This PR strips `lib` prefixes and `_shared` suffixes from plugin names.
It also moves most of the dynlib processing code to Lean to make such
preprocessing more standard.
2025-02-18 23:03:52 +00:00
Sebastian Ullrich
2649d1510e
fix: race condition led to (sync := true) tasks being spawned as separate dedicated tasks (#7089)
If the first task finished between the first check and taking the task
manager lock, the second task would be enqueued as if given
`Priority.max` instead of being run inline.
2025-02-17 09:52:40 +00:00
Sebastian Ullrich
6f445a1c05
chore: Task.get block profiling (#7016)
* `--profile` now reports `blocking` time spent in `Task.get` inside
other profiling categories
* environment variable `LEAN_TRACE_TASK_GET_BLOCKED` when set makes
`lean` dump stack traces of `Task.get` blocks
2025-02-10 10:56:49 +00:00
Sebastian Ullrich
7c79f05cd4
feat: API to avoid deadlocks from dropped promises (#6958)
This PR improves the `Promise` API by considering how dropped promises
can lead to never-finished tasks.
2025-02-07 15:33:10 +00:00
Sebastian Ullrich
1248a55d32
perf: avoid taking mutex on already-resolved promises (#6984) 2025-02-07 10:14:35 +00:00
Sebastian Ullrich
ac9708051a
feat: respect Task.map/bind (sync := true) after waiting (#6976)
This PR extends the behavior of the `sync` flag for `Task.map/bind` etc.
to encompass synchronous execution even when they first have to wait on
completion of the first task, drastically lowering the overhead of such
tasks. Thus the flag is now equivalent to e.g. .NET's
`TaskContinuationOptions.ExecuteSynchronously`.
2025-02-07 09:06:57 +00:00
Sofia Rodrigues
52198837df
feat: improve some files separation and standardize error messages in UV modules (#6830)
This PR improves some files separation and standardize error messages in
UV modules
2025-02-06 23:24:42 +00:00
Eric Wieser
0d7e126a01
chore: re-land "perf: use C23's free_sized when available" (#6844)
Unreverts #6598

I'll combine #6825 into this before merging.
2025-02-04 12:43:56 +00:00
Sebastian Ullrich
a35bf7ee4c
chore: revert "perf: use C23's free_sized when available" (#6841)
Reverts leanprover/lean4#6598, which broke Windows CI
2025-01-29 09:11:23 +00:00
Eric Wieser
6aa6407af1
perf: use C23's free_sized when available (#6598)
See https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2699.htm for an
explanation of this feature.

---------

Co-authored-by: Chris Kennelly <ckennelly@google.com>
2025-01-28 10:17:15 +00:00