| Age | Commit message (Collapse) | Author | Files |
|---|
|
Problem: Treesitter highlighting regressed on 32-bit builds because ranges that should cover the whole buffer were corrupted when passed into Lua.
Solution: Round-trip those range values through Lua and validate them so treesitter sees the same ranges on 32 and 64-bit builds.
|
|
This PR creates a C function `nts_parser_parse_buf()`
which is like `ts_parser_parse_string()` but instead can be passed
an nvim buffer number to parse.
|
|
This reverts commit 2a7cb32959b4c616bd2c76ae1933f8e068e391ad.
|
|
This commit changes `languagetree.lua` so that it creates a scratch
buffer under the hood when dealing with string parsers. This will make
it much easier to just use extmarks whenever we need to track injection
trees in `languagetree.lua`. This also allows us to remove the
`treesitter.c` code for parsing a string directly.
Note that the string parser's scratch buffer has `set noeol nofixeol` so
that the parsed source exactly matches the passed in string.
|
|
**Problem(?):** Buffers that (for whatever reason) aren't meant to have
a final newline are still parsed with a final newline in `treesitter.c`.
**Solution:** Don't add the newline to the last buffer line if it
shouldn't be there. (This more closely matches the approach of
`read_buffer_into()`.)
This allows us to, say, use a scratch buffer with `noeol` and `nofixeol`
behind the scenes in `get_string_parser()`.
...which would allow us to track injection trees with extmarks in that
case.
...which would allow us to not drop previous trees after reparsing a
different range with `get_parser():parse()`.
...which would prevent flickering when editing a buffer that has 2+
windows to it in view at a time.
...which would allow us to keep our sanity!!!
(one step at a time...)
|
|
|
|
These are not needed after #35129 but making uncrustify still play nice
with them was a bit tricky.
Unfortunately `uncrustify --update-config-with-doc` breaks strings
with backslashes. This issue has been reported upstream,
and in the meanwhile auto-update on every single run has been disabled.
|
|
Problem:
The previous fix in #34314 relies on copying the tree in `tree_root` to
ensure the `TSNode`'s tree cannot be mutated. But that causes the
problem where two calls to `tree_root` return nodes from different
copies of a tree, which do not compare as equal. This has broken at
least one plugin.
Solution:
Make all `TSTree`s on the Lua side always immutable, avoiding the need
to copy the tree in `tree_root`, and make the only mutation point,
`tree_edit`, copy the tree instead.
|
|
Problem:
TSNode contains a `const TSTree*` and a `const void *id`. The `id`
points to Tree-sitter's internal type `Subtree`, which resides inside
the `TSTree` but may be deallocated if the `TSTree` is mutated (which
is likely why it is `const`).
The Lua method `TSTree:edit()` mutates the tree, which can deallocate
`id`.
See #25254 and #31758.
Solution:
To avoid this, we now make a copy of the tree before pushing its root to
the Lua stack. This also removes the fenv from TSLuaTree, as it was only
used when pushing the tree root to the Lua stack.
We also copy the tree in `node_tree`.
`ts_tree_copy()` just increments a couple of reference counters, so it's
relatively cheap to call.
|
|
|
|
Now that we have bumped to tree-sitter 0.25.4, we no longer need to do
this since upstream does it for us when calling the regular parse
method.
|
|
"Error" in error messages is redundant. Just provide the context, don't
say "Error ...".
|
|
Apparently after parsing with options in tree-sitter, the options data
persists in the parser object, and thus successive calls to
`ts_parser_parse()` will act like `ts_parser_parse_with_options()`. This
is problematic because `languagetree.lua` makes coroutine-environment
assumptions based on if a nullptr has been returned by the parser
function. This commit makes it so that the parse options state is reset
upon a regular parse (would be nice if this was done upstream).
Fixes #33277
|
|
Remove the `set_timeout` functions for `TSParser` and instead add a timeout
parameter to the regular parse function. Remove these deprecated tree-sitter
API functions and replace them with the preferred `TSParseOptions` style.
|
|
Problem: Cannot disable individual captures and patterns in treesitter queries.
Solution:
* Expose the corresponding tree-sitter API functions for `TSQuery` object.
* Add documentation for `TSQuery`.
* Return the pattern ID from `get_captures_at_pos()` (and hence `:Inspect!`).
|
|
Problem: No way to check the version of a treesitter parser.
Solution: Add version metadata (ABI 15 parsers only) as well as parser state count and supertype information (ABI 15) in `vim.treesitter.language.inspect()`. Also graduate the `abi_version` field, as this is now the official upstream name.
---------
Co-authored-by: Christian Clason <c.clason@uni-graz.at>
|
|
|
|
Problem:
When running an initial parse, parse() returns an empty table rather
than an actual range. In `languagetree.lua`, we manually check if
a parse was incremental to determine the changed parse region.
Solution:
- Always return a range (in the C side) from parse().
- Simplify the language tree code a bit.
- Logger no longer shows empty ranges on the initial parse.
|
|
Breaking change: `ts_node_child_containing_descendant()` was removed
Breaking change: tree-sitter 0.25 (HEAD) required
|
|
|
|
**Problem:** Parsing can be slow for large files, and it is a blocking
operation which can be disruptive and annoying.
**Solution:** Provide a function for asynchronous parsing, which accepts
a callback to be run after parsing completes.
Co-authored-by: Lewis Russell <lewis6991@gmail.com>
Co-authored-by: Luuk van Baal <luukvbaal@gmail.com>
Co-authored-by: VanaIgr <vanaigranov@gmail.com>
|
|
Regression from 2a7d0ed6145bf3f8b139c2694563f460f829813a, which removed
header that is only needed if wasmtime support is enabled. Prevent this
from happening again by wrapping the include in a `HAVE_WASMTIME` check.
|
|
Result of `make iwyu` (after some "fixups").
|
|
**Problem:** Currently node names with non-alphanumeric, non
underscore/hyphen characters (only possible with anonymous nodes) are
not given a proper error message. See tree-sitter issue 3892 for more
details.
**Solution:** Apply a different scanning logic to anonymous nodes to
correctly identify the entire node name (i.e., up until the final double
quote)
|
|
|
|
**Problem:** Tree-sitter 0.24.0 introduced a new symbol type to denote
supertype nodes (`TSSymbolTypeSupertype`). Now, `language.inspect()`
(and the query `omnifunc`) return supertype symbols, but with double
quotes around them.
**Solution:** Mark a symbol as "named" based on it *not* being an
anonymous node, rather than checking that it is a regular node (which a
supertype also is not).
|
|
**Problems:**
- `vim.treesitter.language.inspect()` returns duplicate
symbol names, sometimes up to 6 of one kind in the case of `markdown`
- The list-like `symbols` table can have holes and is thus not even a
valid msgpack table anyway, mentioned in a test
**Solution:** Return symbols as a map, rather than a list, where field
names are the names of the symbol. The boolean value associated with the
field encodes whether or not the symbol is named.
Note that anonymous nodes are surrounded with double quotes (`"`) to
prevent potential collisions with named counterparts that have the same
identifier.
|
|
This commit also marks `child_containing_descendant()` as deprecated
(per upstream's documentation), and uses `child_with_descendant()` in
its place. Minimum required tree-sitter version will now be `0.24`.
|
|
Problem:
Tree cursors can only be efficient when they are re-used.
Short-lived cursors are very slow.
Solution:
Reimplement functions that use short-lived cursors.
|
|
Problem: Installing treesitter parser is hard (harder than
climbing to heaven).
Solution: Add optional support for wasm parsers with `wasmtime`.
Notes:
* Needs to be enabled by setting `ENABLE_WASMTIME` for tree-sitter and
Neovim. Build with
`make CMAKE_EXTRA_FLAGS=-DENABLE_WASMTIME=ON
DEPS_CMAKE_FLAGS=-DENABLE_WASMTIME=ON`
* Adds optional Rust (obviously) and C11 dependencies.
* Wasmtime comes with a lot of features that can negatively affect
Neovim performance due to library and symbol table size. Make sure to
build with minimal features and full LTO.
* To reduce re-compilation times, install `sccache` and build with
`RUSTC_WRAPPER=<path/to/sccache> make ...`
|
|
|
|
Problem: `has-ancestor?` is O(n²) for the depth of the tree since it iterates over each of the node's ancestors (bottom-up), and each ancestor takes O(n) time.
This happens because tree-sitter's nodes don't store their parent nodes, and the tree is searched (top-down) each time a new parent is requested.
Solution: Make use of new `ts_node_child_containing_descendant()` in tree-sitter v0.22.6 (which is now the minimum required version) to rewrite the `has-ancestor?` predicate in C to become O(n).
For a sample file, decreases the time taken by `has-ancestor?` from 360ms to 6ms.
|
|
Instead of painfully messing with timing to determine if queries were
reparsed, we can simply keep a counter next to the call to ts_query_new
Also memoization had a hidden dependency on the garbage collection of
the the key, a hash value which never is kept around in memory. this was
done intentionally as the hash does not capture all relevant state for the
query (external included files) even if actual query objects still
would be reachable in memory. To make the test fully deterministic in
CI, we explicitly control GC.
|
|
|
|
|
|
- Also address some coverity warnings
Fixes #27942
|
|
|
|
|
|
Problem:
`TSNode:_rawquery()` is complicated, has known issues and the Lua and
C code is awkwardly coupled (see logic with `active`).
Solution:
- Add `TSQueryCursor` and `TSQueryMatch` bindings.
- Replace `TSNode:_rawquery()` with `TSQueryCursor:next_capture()` and `TSQueryCursor:next_match()`
- Do more stuff in Lua
- API for `Query:iter_captures()` and `Query:iter_matches()` remains the same.
- `treesitter.c` no longer contains any logic related to predicates.
- Add `match_limit` option to `iter_matches()`. Default is still 256.
|
|
|
|
Query patterns can contain quantifiers (e.g. (foo)+ @bar), so a single
capture can map to multiple nodes. The iter_matches API can not handle
this situation because the match table incorrectly maps capture indices
to a single node instead of to an array of nodes.
The match table should be updated to map capture indices to an array of
nodes. However, this is a massively breaking change, so must be done
with a proper deprecation period.
`iter_matches`, `add_predicate` and `add_directive` must opt-in to the
correct behavior for backward compatibility. This is done with a new
"all" option. This option will become the default and removed after the
0.10 release.
Co-authored-by: Christian Clason <c.clason@uni-graz.at>
Co-authored-by: MDeiml <matthias@deiml.net>
Co-authored-by: Gregory Anders <greg@gpanders.com>
|
|
- `TSQuery`: userdata object for parsed query.
- `vim.treesitter.Query`: renamed from `Query`.
- Add a new field `lang`.
- `TSQueryInfo`:
- Move to `vim/treesitter/_meta.lua`, because C code owns it.
- Correct typing for `patterns`, should be a map from `integer`
(pattern_id) to `(integer|string)[][]` (list of predicates or
directives).
- `vim.treesitter.QueryInfo` is added.
- This currently has the same structure as `TSQueryInfo` (exported
from C code).
- Document the fields (see `TSQuery:inspect`).
- Add typing for `vim._ts_parse_query()`.
|
|
It's the "tree-sitter" project, but "treesitter" in our code and docs.
|
|
|
|
|
|
|
|
|
|
|
|
We already have an extensive suite of static analysis tools we use,
which causes a fair bit of redundancy as we get duplicate warnings. PVS
is also prone to give false warnings which creates a lot of work to
identify and disable.
|
|
Uncrustify is the source of truth where possible.
Remove any redundant checks from clint.py.
|
