We use `nullKind` for the `group` parser combinator.
When pattern matching `nullKind` nodes, we check their arities.
So, error recovery often fails for parsers that use the `group`
combinator.
For example, we have the parser
```
def whereDecls := leading_parser "where " >> many1Indent (group (letRecDecl >> optional ";"))
```
If there is syntax error at `letRecDecl`, the node corresponding to
```
group (letRecDecl >> optional ";")
```
will contain only one child, and the pattern matching at
```
def expandWhereDecls (whereDecls : Syntax) (body : Syntax) : MacroM Syntax :=
match whereDecls with
| `(whereDecls|where $[$decls:letRecDecl $[;]?]*) => `(let rec $decls:letRecDecl,*; $body)
| _ => Macro.throwUnsupported
```
fails, and we can't elaborate the partial syntax tree for
`letRecDecl`, and auto-completion will not work there.
We address this issue by using a new kind for the `group` combinator.
The idea is to pattern match `group` as we pattern match `node`s with
proper syntax node kinds. This change is consistent with the way we
use `group` where it mainly a convenience for saving us the trouble of
defining a new parser definition that is used only once.
This kind of broken command does not have a position information, and
was producing a panic message in the server because it makes the
reasonable assumption that every command returned by `parserCommand`
has a position.
I am hitting many error recovery problems. This is an attempt to make
auto-completion more robust.
For example, we currently can't auto complete inside of parentheses.
The syntax tree for the syntactically incorrect tern `(s. , 0)` is
```
(Term.paren "(" [(Term.proj `s ".")])
```
The elaborator for `paren` fails to match this partial syntax tree and
returns an error. I considered adding code to the `else` branch of
this elaborator and handle partial trees there. However, we would
still be losing information for the other elements of the tuple.
Example: no hover information for them.
The helper parser makes sure we don't lose information.
After a parser error in a command, we are "swallowing" all command
parsing errors until a command was succesfully parsed.
This was producing counterintuitive behavior in the IDEs.
@kha We have a few parsers that invoke `tokenFn`, and return error
depending of what is on the top of the stack (e.g., `ident`).
These parsers were not restoring the stack size when reporting errord,
and messing up the error recovery. We never notice the problem because
operators such as <|> restore the stack size, and we were not trying
to elaborate syntacticly incorrect terms.
@Kha this is a fix for the example that @gebner posted on Zulip.
There are other possible workarounds, but I think this one is
"intuitive".
Here is the problem description and workaround.
We currently use the `withPosition` combinator at `matchAlts`. The
original motivation was nested `match`-expressions without
parenthesis.
We also have the `checkColGt` in the application parser.
Thus, `frob 5` is not parsed as an application. That is, the term
parser after the `then` keyword stops at `frob`.
Then, we get the weird "expected 'else'" error message at `5` :)
In this commit, I add a `checkColGe` check at the beginning of each
alternative right-hand-side.
The effect on the stdlib was minimal. It basically affected old code
that used the Lean 3 workaround for partial functions
```
partial def f : N -> N | i =>
...
```
In Lean 4, we can use the more natural
```
partial def f (i : N) : N :=
...
```
or
```
partial def f : N -> N := fun i =>
...
```
It also affected code such as
```
instance : Repr String.Iterator where
reprPrec | ⟨s, pos⟩, prec =>
Repr.addAppParen ("String.Iterator.mk " ++ reprArg s ++ " " ++ reprArg pos) prec
```
This is a workaround for a missing feature. We really wanted to write
```
instance : Repr String.Iterator where
reprPrec ⟨s, pos⟩ prec :=
Repr.addAppParen ("String.Iterator.mk " ++ reprArg s ++ " " ++ reprArg pos) prec
```
@Kha I marked the corresponding methods as `protected`.
I currently can't stand `throw_error`, and I am optimistic about
server highlighting feature you are working on :)
