This PR adds a symbol to the runtime for marking `Array`
non-linearities. This should allow users to
spot them more easily in profiles or hunt them down using a debugger.
This PR improves the performance of `getLine` by coalescing the locking
of the underlying `FILE*`.
Unfortunately we cannot use `getline` or `fgets` for this as our code
needs to handle `\0` chars
and Windows.
This PR ensures that `Nat`s in `.olean` files use a deterministic
serialization in the case where `LEAN_USE_GMP` is not set.
This is a simplified version of
https://github.com/leanprover/lean4/pull/2908.
This PR moves the processing of options passed to the CLI from
`shell.cpp` to `Shell.lean`.
As with previous ports, this attempts to mirror as much of the original
behavior as possible, Benefits to be gained from the ported code can
come in later PRs. There should be no significant behavioral changes
from this port. Nonetheless, error reporting has changed some, hopefully
for the better. For instance, errors for improper argument
configurations has been made more consistent (e.g., Lean will now error
if numeric arguments fall outside the expected range for an option).
(Redo of #11345 to fix Windows issue.)
This PR moves the processing of options passed to the CLI from
`shell.cpp` to `Shell.lean`.
As with previous ports, this attempts to mirror as much of the original
behavior as possible, Benefits to be gained from the ported code can
come in later PRs. There should be no significant behavioral changes
from this port. Nonetheless, error reporting has changed some, hopefully
for the better. For instance, errors for improper argument
configurations has been made more consistent (e.g., Lean will now error
if numeric arguments fall outside the expected range for an option).
Given its run time of >2hrs, the job is added as a secondary job for
nightly releases and a primary job for full releases. A new check level
for differentiating between nightlies and full releases is added for
this.
(Trying to) reactivate lsan will happen in a follow-up PR.
This PR fixes several memory leaks in the new `String` API.
These leaks are mostly situations where we forgot to put borrowing
annotations. The single
exception is the new `String` constructor `ofByteArray`. It cannot take
the `ByteArray` as
a borrowed argument anymore and must thus free it on its own.
This PR fixes fallout of the closure allocator changes in #10982. As far
as we know
this bug only meaningfully manifests in non default build configurations
without mimalloc such as:
`cmake --preset release -DUSE_MIMALLOC=OFF`
The issue is that I forgot to update the deallocation functions for
closures. However, this only
seems to matter if we disable mimalloc which is why this slipped through
testing.
This PR fixes `ST.Ref.ptrEq` to act as described in the docs. This fixes
two bugs:
1. The recent `IO.RealWorld` elimination PR overlooked this function
(afaik this is the only one),
causing its return value to be generally wrong.
2. The implementation of `ptrEq` would previously always consider two
different cells with pointer
equivalent value to be pointer equal. However, the function is supposed
to check whether two
`Ref` are the same cell, not whether the contained elements are.
This PR fixes a memleak caused by the Lean based `IO.waitAny`
implementation by reverting it.
This the faulty Lean implementation:
```lean
def IO.waitAny (tasks : @& List (Task α)) (h : tasks.length > 0 := by exact Nat.zero_lt_succ _) :
BaseIO α := do
have : Nonempty α := ⟨tasks[0].get⟩
let promise : IO.Promise α ← IO.Promise.new
tasks.forM <| fun t => BaseIO.chainTask (sync := true) t promise.resolve
return promise.result!.get
```
In a situation where we call this function repeatedly in a loop with a
pair of tasks `[t1, t2]`
where `t2` is a long lived task that we pass every time and `t1` is
fresh a short lived task, `t2` will
accumlate more and more children from `BaseIO.chainTask` that fill
memory over time. The old C++
implementation did not have this issue so we are reverting.
This PR implements zero cost `BaseIO` by erasing the `IO.RealWorld`
parameter from argument lists and structures. This is a **major breaking
change for FFI**.
Concretely:
- `BaseIO` is defined in terms of `ST IO.RealWorld`
- `EIO` (and thus `IO`) is defined in terms of `EST IO.RealWorld`
- The opaque `Void` type is introduced and the trivial structure
optimization updated to account for it. Furthermore, arguments of type
`Void s` are removed from the argument lists of the C functions.
- `ST` is redefined as `Void s -> ST.Out s a` where `ST.Out` is a pair
of `Void s` and `a`
This together has the following major effects on our generated code:
- Functions that return `BaseIO`/`ST`/`EIO`/`IO`/`EST` now do not take
the dummy world parameter anymore. To account for this FFI code needs to
delete the dummy world parameter from the argument lists.
- Functions that return `BaseIO`/`ST` now return their wrapped value
directly. In particular `BaseIO UInt32` now returns a `uint32_t` instead
of a `lean_object*`. To account for this FFI code might have to change
the return type and does not need to call `lean_io_result_mk_ok` anymore
but can instead just `return` values right away (same with extracting
values from `BaseIO` computations.
- Functions that return `EIO`/`IO`/`EST` now only return the equivalent
of an `Except` node which reduces the allocation size. The
`lean_io_result_mk_ok`/`lean_io_result_mk_error` functions were updated
to account for this already so no change is required.
Besides improving performance by dropping allocation (sizes) we can now
also do fun new things such as:
```lean
@[extern "malloc"]
opaque malloc (size : USize) : BaseIO USize
```
This PR adds more selectors for TCP and Signals.
It also fixes a problem with `Selectors` that they cannot be closures
over a promise, otherwise it causes the waiter promise to never be
dropped.
This PR aims to fix the Timer API selector to make it finish as soon as
possible when unregistered. This change makes the `Selectable.one`
function drop the `selectables` array as soon as possible, so when
combined with finalizers that have some effects like the TCP socket
finalizer, it runs it as soon as possible.
This PR adds the `IO.FS.hardLink` function, which can be used to create
hard links.
This is implemented via libuv's `uv_fs_link` function.
Lake hopes to make use of this function to decrease the storage cost of
restoring artifacts.
This PR also fixes some C implementation issues found in nearby similar
functions.
This PR fixes deadlocking `exit` calls in the language server.
We have previously observed deadlocking calls to `exit` inside of the
language server and deemed them irrelevant. However, child processes of
these deadlocking exiting processes can continue to consume a large
amount of CPU as they try to compile a library etc. Hence, this PR
switches to the MT safe `_Exit` inside of the language server,
in order to ensure the server finishes when it is told to.
This PR adds vectored write for TCP and UDP (that helps a lot with not
copying the arrays over and over) and fix a RC issue in TCP and UDP
cancel functions with the line `lean_dec((lean_object*)udp_socket);` and
a similar one that tries to decrement the object inside of the `socket`.
This PR adds the type `Std.Internal.Parsec.Error`, which contains the
constructors `.eof` (useful for checking if parsing failed due to not
having enough input and then retrying when more input arrives that is
useful in the HTTP server) and `.other`, which describes other errors.
It also adds documentation to many functions, along with some new
functions to the `ByteArray` Parsec, such as `peekWhen?`, `octDigit`,
`takeWhile`, `takeUntil`, `skipWhile`, and `skipUntil`.
This PR makes `saveModuleData` throw an IO.Error instead of panicking,
if given something that cannot be serialized. This doesn't really matter
for saving modules, but is handy when writing tools to save auxiliary
date in olean files via Batteries' `pickle`.
The caller of this C++ function already is guarded in a `try`/`catch`
that promotes from a `lean::exception` to an `IO.userError`.
A simple test of this in the web editor is
```
import Batteries
#eval pickle "/tmp/foo.txt" fun x : Nat => x
```
which crashes before this change.
---------
Co-authored-by: Laurent Sartran <lsartran@google.com>
This PR ports more of the post-initialization C++ shell code to Lean.
All that remains is the initialization of the profiler and task manager.
As initialization tasks rather than main shell code, they were left in
C++ (where the rest of the initialization code currently is).
The `max_memory` and `timeout` Lean options used by the the `--memory`
and `--timeout` command-line options are now properly registered. The
server defaults for max memory and max heartbeats (timeout) were removed
as they were not actually used (because the `server` option that was
checked was neither set nor exists).
This PR also makes better use of the module system in `Shell.lean` and
fixes a minor bug in a previous port where the file name check was
dependent on building the `.ilean` rather than the `.c` file (as was
originally the case).
Fixes#9879.
This PR re-implements `IO.waitAny` using Lean instead of C++. This is to
reduce the size and
complexity of `task_manager` in order to ease future refactorings.
There is an import behavioral change of `IO.waitAny` in this PR.
Consider a situation where we have
two promises `p1`, `p2` and call `IO.waitAny [p1.result!, p2.result!]`
and `p1` resolves instantly.
Previously this would just return the result of `p1` and require nothing
else. With the new
implementation if `p2` is released before being resolved this can cause
a panic, even if
`IO.waitAny` has already finished. I argue that this is reasonable
behavior, given that an
invocation of `result!` promises that the promise will eventually be
resolved.
This PR introduces checks to make sure that the IO functions produce
errors when inputs contain NUL bytes (instead of ignoring everything
after the first NUL byte).
This PR continues #9644 , fixing the core build when using an older
system libuv.
This only affected users building Lean from scratch, since the lean
binaries we ship as part of toolchains statically link their own copy of
libuv 1.50+.
---------
Co-authored-by: Markus Himmel <markus@lean-fro.org>
This PR fixes the core build when using an older system libuv.
This only affected users building Lean from scratch, since the `lean`
binaries we ship as part of toolchains statically link their own copy of
libuv 1.50+.
This PR fixes the behavior of `String.prev`, aligning the runtime
implementation with the reference implementation. In particular, the
following statements hold now:
- `(s.prev p).byteIdx` is at least `p.byteIdx - 4` and at most
`p.byteIdx - 1`
- `s.prev 0 = 0`
- `s.prev` is monotone
Closes#9439
This PR fixes `undefined symbol: lean::mpz::divexact(lean::mpz const&,
lean::mpz const&)` when building without `LEAN_USE_GMP`
This fixes a regression in #8089
This PR adds DNS functions to the standard library
---------
Co-authored-by: Henrik Böving <hargonix@gmail.com>
Co-authored-by: Markus Himmel <markus@himmel-villmar.de>
This PR adds a function called `lean_setup_libuv` that initializes
required LIBUV components. It needs to be outside of
`lean_initialize_runtime_module` because it requires `argv` and `argc`
to work correctly.
---------
Co-authored-by: Markus Himmel <markus@lean-fro.org>
Co-authored-by: Eric Wieser <wieser.eric@gmail.com>
This PR skips attempting to compute a module name from the file name and
root directory (i.e., `lean -R`) if a name is already provided via `lean
--setup`.
This is accomplished by porting the rest of the frontend code in the
`try` block to Lean.
This PR removes an old workaround around non-implemented C++11 features
in the thread finalization.
This `ifdef` dates back to approximately 2015 as can be seen
[here](https://github.com/leanprover/lean3/blame/master/src/util/thread.cpp#L177),
the comments mention that it was originally implemented because not all
compilers at the time were able to support the C++11 `thread_local`
keyword. 10 years later this is hopefully the case and we can remove
this workaround.
There is an additional motivation for doing this,
`lean::initialize_thread` contains the following allocation:
```cpp
g_thread_finalizers_mgr = new thread_finalizers_manager;
```
this is supposed to be freed at some point but:
```cpp
// TODO(gabriel): race condition with thread finalizers
void delete_thread_finalizer_manager() {
// delete g_thread_finalizers_mgr;
// g_thread_finalizers_mgr = nullptr;
}
```
so `g_thread_finalizers_mgr` leaks upon repeated invocation of
`lean::initialize_thread`.
Note that Windows has already been using this alternative implementation
for a while so the alternative implementation has (hopefully) not rotten
away in the meantime.
This PR makes Lean code generation respect the module name provided
through `lean --setup`.
This is accomplished by porting to Lean the portion of `shell.cpp` that
spans running the frontend to exiting the process. This makes it easier
to load the module setup and control how its name is passed to the code
generation functions. This port attempts to minimize the changes made to
Lean. It marks the new Lean functions `private` and tries to preserve as
faithfully as possible the behavior of the original C++ code. Exposing
the new Lean interface publicly and/or further improving the code now
that is written in Lean is left for the future.
glibc adds `__attribute__((nothrow))` to its declarations, at least for
those related to malloc. glibc has yet to introduce `free_sized`, but
when it does it would cause compilation errors. This is due to the fact
that if a function declarations has `__attribute__((nothrow))` and it is
re-declared or implemented in C++ it must also have
`__attribute__((nothrow))` or `noexcept`, otherwise the compilation will
fail.
This is a follow up to https://github.com/leanprover/lean4/pull/6598.
Signed-off-by: Justin King <jcking@google.com>
