The current `:tree-sitter-subtree` has a bug for field-names when the
field name belongs to an unnamed child node. Take this ruby example:
def self.method_name
true
end
The subtree given by tree-sitter-cli is:
(singleton_method [2, 0] - [4, 3]
object: (self [2, 4] - [2, 8])
name: (identifier [2, 9] - [2, 20])
body: (body_statement [3, 2] - [3, 6]
(true [3, 2] - [3, 6])))
But the `:tree-sitter-subtree` output was
(singleton_method
object: (self)
body: (identifier)
(body_statement (true)))
The `singleton_method` rule defines the `name` and `body` fields in an
unnamed helper rule `_method_rest` and the old implementation of
`pretty_print_tree_impl` would pass the `field_name` down from the
named `singleton_method` node.
To fix it we switch to the [TreeCursor] API which is recommended by
the tree-sitter docs for traversing the tree. `TreeCursor::field_name`
accurately determines the field name for the current cursor position
even when the node is unnamed.
[TreeCursor]: https://docs.rs/tree-sitter/0.20.9/tree_sitter/struct.TreeCursor.html
This fixes an edge case for completing shellwords. With a file
"a b.txt" in the current directory, the sequence `:open a\<tab>`
will result in the prompt containing `:open aa\ b.txt`. This is
because the length of the input which is trimmed when replacing with
completion is calculated on the part of the input which is parsed by
shellwords and then escaped (in a separate operation), which is lossy.
In this case it loses the trailing backslash.
The fix provided here refactors shellwords to track both the _words_
(shellwords with quotes and escapes resolved) and the _parts_ (chunks
of the input which turned into each word, with separating whitespace
removed). When calculating how much of the input to delete when
replacing with the completion item, we now use the length of the last
part.
This also allows us to eliminate the duplicate work done in the
`ends_with_whitespace` check.
* add tree sitter match limit to avoid slowdowns for larger files
Affects all tree sitter queries and should speedup both
syntax highlighting and text object queries.
This has been shown to fix significant slowdowns with textobjects
for rust files as small as 3k loc.
* Apply suggestions from code review
Co-authored-by: Blaž Hrastnik <blaz@mxxn.io>
Co-authored-by: Blaž Hrastnik <blaz@mxxn.io>
`deepest_preceding` is known to be a descendant of `node`. Repeated
calls of `Node::parent` _should_ eventually turn `deepest_preceding`
into `node`, but when the node is errored (the tree contains a syntax
error), `Node::parent` returns None.
In the typescript case:
if(true) &&true
// ^ press enter here
The tree is:
(program [0, 0] - [1, 0]
(if_statement [0, 0] - [0, 15]
condition: (parenthesized_expression [0, 2] - [0, 8]
(true [0, 3] - [0, 7]))
consequence: (expression_statement [0, 8] - [0, 15]
(binary_expression [0, 8] - [0, 15]
left: (identifier [0, 8] - [0, 8])
right: (true [0, 11] - [0, 15])))))
`node` is the `program` node and `deepest_preceding` is the
`binary_expression`. The tree is errored on the `binary_expression`
node with `(MISSING identifier [0, 8] - [0, 8])`.
In the C++ case:
; <<
// press enter after the ';'
The tree is:
(translation_unit [0, 0] - [1, 0]
(expression_statement [0, 0] - [0, 1])
(ERROR [0, 1] - [0, 4]
(identifier [0, 1] - [0, 1])))
`node` is the `translation_unit` and `deepest_preceding` is the `ERROR`
node.
In both cases, `Node::parent` on the errored node returns None.
This changes the completion items to be rendered with shellword
escaping, so a file `a b.txt` is rendered as `a\ b.txt` which matches
how it should be inputted.
8584b38cfb switched to shellwords for
completion in command-mode. This changes the conditions for choosing
whether to complete the command or use the command's completer.
This change processes the input as shellwords up-front and uses
shellword logic about whitespace to determine whether the command
or argument should be completed.
The change in d801a6693c to search for
suffixes in `file-types` is too permissive: files like the tutor or
`*.txt` files are now mistakenly interpreted as R or perl,
respectively.
This change changes the syntax for specifying a file-types entry that
matches by suffix:
```toml
file-types = [{ suffix = ".git/config" }]
```
And changes the file-type detection to first search for any non-suffix
patterns and then search for suffixes only with the file-types entries
marked explicitly as suffixes.
Just like for grammars we currently force a lower-case of the name for
some actions (like filesystem lookup). To make this consistent and less
surprising for users, we remove this lower-casing here.
Note: it is still the preferred way to name both language and grammar in
lower-case
Signed-off-by: Christian Speich <cspeich@emlix.com>
* Fix shellwords delimiter handling
This allows commands such as `:set statusline.center ["file-type"]` to
work. Before the quotes within the list would mess it up.
Also added a test to ensure correct behavior
* Rename Delimiter -> OnWhitespace
* Fix test::print for Unicode
The print function was not generating correct translations when
the input has Unicode (non-ASCII) in it. This is due to its use of
String::len, which gives the length in bytes, not chars.
* Fix multi-code point auto pairs
The current code for auto pairs is counting offsets by summing the
length of the open and closing chars with char::len_utf8. Unfortunately,
this gives back bytes, and the offset needs to be in chars.
Additionally, it was discovered that there was a preexisting bug where
the selection was not computed correctly in the case that the cursor
was:
1. a single grapheme in width
2. this grapheme was more than one char
3. the direction of the cursor is backwards
4. a secondary range
In this case, the offset was not being added into the anchor. This was
fixed.
* migrate auto pairs tests to integration
* review comments
* feat(syntax): add strategy to associate file to language through pattern
File path will match if it ends with any of the file types provided in the config.
Also used this feature to add support for the .git/config and .ssh/config files
* Add /etc/ssh/ssh_config to languages.toml
* cargo xtask docgen
* Update languages.md
* Update languages.md
* Update book/src/languages.md
Co-authored-by: Ivan Tham <pickfire@riseup.net>
* Update book/src/languages.md
Co-authored-by: Ivan Tham <pickfire@riseup.net>
Co-authored-by: Ivan Tham <pickfire@riseup.net>
Info logs don't show up in the log file by default, but this line
should: failures to load tree-sitter parser objects are useful errors.
A parser might fail to load it is misconfigured
(https://github.com/helix-editor/helix/pull/4303#discussion_r996448543)
or if the file does not exist.
This changes the behavior of operations like `]f`/`[f` to set the
direction of the new range to the direction of the action.
The original behavior was to always use the head of the next function.
This is inconsistent with the behavior of goto_next_paragraph and makes
it impossible to create extend variants of the textobject motions.
This causes a behavior change when there are nested functions. The
behavior in the parent commit is that repeated uses of `]f` will
select every function in the file even if nested. With this commit,
functions are skipped.
It's notable that it's possible to emulate the original behavior by
using the `ensure_selections_forward` (A-:) command between invocations
of `]f`.
* Split helix_core::find_root and helix_loader::find_local_config_dirs
The documentation of find_root described the following priority for
detecting a project root:
- Top-most folder containing a root marker in current git repository
- Git repository root if no marker detected
- Top-most folder containing a root marker if not git repository detected
- Current working directory as fallback
The commit contained in https://github.com/helix-editor/helix/pull/1249
extracted and changed the implementation of find_root in find_root_impl,
actually reversing its result order (since that is the order that made
sense for the local configuration merge, from innermost to outermost
ancestors).
Since the two uses of find_root_impl have different requirements (and
it's not a matter of reversing the order of results since, e.g., the top
repository dir should be used by find_root only if there's not marker in
other dirs), this PR splits the two implementations in two different
specialized functions.
In doing so, find_root_impl is removed and the implementation is moved
back in find_root, moving it closer to the documented behaviour thus
making it easier to verify it's actually correct
* helix-core: remove Option from find_root return type
It always returns some result, so Option is not needed
* Update description of `m` textobject to its actual functionality
Sometime recently the functionality of `m` was changed to match the
nearest pair to the cursor, rather than the former functionality of
matching the pair only if the cursor was on one of the brace characters
directly.
* Rename surround methods to reflect that they work on pairs
The current naming suggests that they may work generally on any
textobject, whereas their implementation really focuses on pairs.
* Change description of m textobject to match actual functionality
The current implementation of `m` no longer merely looks at the pair
character the cursor is on, but actually will search for the pair
(defined in helix-core/src/surround.rs) that encloses the cursor, and
not the entire selection.
* Accept suggested wording change
Co-authored-by: Michael Davis <mcarsondavis@gmail.com>
* Prefix pair surround for consistency
Co-authored-by: Michael Davis <mcarsondavis@gmail.com>
* Fix nondeterministic highlighting
This is done by prefering matches in the begining, ie for
`keyword.function`, `keyword` is a better match than `function`.
* Use all positions and not just leftmost
Fixes possible edgecase with something like `function.method.builtin`
and the queries `function.builtin` and `function.method`
* Switch to bitmask for slightly better performance
* Make matches from the start of string
Also change comments to match new behaviour
* Change default formatter for any language
* Fix clippy error
* Close stdin for Stdio formatters
* Better indentation and pattern matching
* Return Result<Option<...>> for fn format instead of Option
* Remove unwrap for stdin
* Handle FormatterErrors instead of Result<Option<...>>
* Use Transaction instead of LspFormatting
* Use Transaction directly in Document::format
* Perform stdin type formatting asynchronously
* Rename formatter.type values to kebab-case
* Debug format for displaying io::ErrorKind (msrv fix)
* Solve conflict?
* Use only stdio type formatters
* Remove FormatterType enum
* Remove old comment
* Check if the formatter exited correctly
* Add formatter configuration to the book
* Avoid allocations when writing to stdin and formatting errors
* Remove unused import
Co-authored-by: Gokul Soumya <gokulps15@gmail.com>
* Add workspace and document diagnostics picker
fixes#1891
* Fix some of @archseer's annotations
* Add From<&Spans> impl for String
* More descriptive parameter names.
* Adding From<Cow<str>> impls for Span and Spans
* Add new keymap entries to docs
* Avoid some clones
* Fix api change
* Update helix-term/src/application.rs
Co-authored-by: Bjorn Ove Hay Andersen <bjrnove@gmail.com>
* Fix a clippy hint
* Sort diagnostics first by URL and then by severity.
* Sort diagnostics first by URL and then by severity.
* Ignore missing lsp severity entries
* Add truncated filepath
* Typo
* Strip cwd from paths and use url-path without schema
* Make tests a doctest
* Better variable names
Co-authored-by: Falco Hirschenberger <falco.hirschenberger@itwm.fraunhofer.de>
Co-authored-by: Bjorn Ove Hay Andersen <bjrnove@gmail.com>
* feat: make `move_vertically` aware of tabs and wide characters
* refactor: replace unnecessary checked_sub with comparison
* refactor: leave pos_at_coords unchanged and introduce separate pos_at_visual_coords
* style: include comment to explain `pos_at_visual_coords` breaking condition
* refactor: use `pos_at_visual_coords` in `text_pos_at_screen_coords`
* feat: make `copy_selection_on_line` aware of wide characters
When a new View of a Document is created, a default cursor of 0, 0 is
created, and it does not get normalized to a single width cursor until
at least one movement of the cursor happens. This appears to have no
practical negative effect that I could find, but it makes tests difficult
to work with, since the initial selection is not what you expect it to be.
This changes the initial selection of a new View to be the width of the
first grapheme in the text.
* allows passing extra formatting options to LSPs
- adds optional field 'format' to [[language]] sections in 'languages.toml'
- passes specified options the LSPs via FormattingOptions
* cleaner conversion of formatting properties
* move formatting options inside lsp::Client
* cleans up formatting properties merge
* add reflow command
Users need to be able to hard-wrap text for many applications, including
comments in code, git commit messages, plaintext documentation, etc. It
often falls to the user to manually insert line breaks where appropriate
in order to hard-wrap text.
This commit introduces the "reflow" command (both in the TUI and core
library) to automatically hard-wrap selected text to a given number of
characters (defined by Unicode "extended grapheme clusters"). It handles
lines with a repeated prefix, such as comments ("//") and indentation.
* reflow: consider newlines to be word separators
* replace custom reflow impl with textwrap crate
* Sync reflow command docs with book
* reflow: add default max_line_len language setting
Co-authored-by: Vince Mutolo <vince@mutolo.org>
* log textobject query construction errors
The current behavior is that invalid queries are discarded silently
which makes it difficult to debug invalid textobjects (either invalid
syntax or an update may have come through that changed the valid set
of nodes).
* fix golang textobject query
`method_spec_list` used to be a named node but was removed (I think
for Helix, it was when updated to pull in the support for generics).
Instead of a named node for the list of method specs we have a bunch
of `method_spec` children nodes now. We can match on the set of them
with a `+` wildcard.
Example go for this query:
type Shape interface {
area() float64
perimeter() float64
}
Which is parsed as:
(source_file
(type_declaration
(type_spec
name: (type_identifier)
type: (interface_type
(method_spec
name: (field_identifier)
parameters: (parameter_list)
result: (type_identifier))
(method_spec
name: (field_identifier)
parameters: (parameter_list)
result: (type_identifier))))))
* Make textobject select last paragraph
Last paragraph shoud be selected if the cursor was placed on the
whitespace paragraph part and `map` is done, otherwise it would do
nothing useful, but now we select backwards for the last paragraph
which behaves similarly to kakoune, making `map` useful for the last
paragraph with whitespace. Example usecase is to copy and paste last
ledger cli paragraph quickly by `mapyp` to duplicate last entry.
* Fix typo in core textobject
Co-authored-by: Michael Davis <mcarsondavis@gmail.com>
Co-authored-by: Michael Davis <mcarsondavis@gmail.com>
* Add runtime language configuration (#1794)
* Add set-language typable command to change the language of current buffer.
* Add completer for available language options.
* Update set-language to refresh language server as well
* Add language id based config lookup on `syntax::Loader`.
* Add `Document::set_language3` to set programming language based on language
id.
* Update `Editor::refresh_language_server` to try language detection only if
language is not already set.
* Remove language detection from Editor::refresh_language_server
* Move document language detection to where the scratch buffer is saved.
* Rename Document::set_language3 to Document::set_language_by_language_id.
* Remove unnecessary clone in completers::language
* WIP: Rework indentation system
* Add ComplexNode for context-aware indentation (including a proof of concept for assignment statements in rust)
* Add switch statements to Go indents.toml (fixes the second half of issue #1523)
Remove commented-out code
* Migrate all existing indentation queries.
Add more options to ComplexNode and use them to improve C/C++ indentation.
* Add comments & replace Option<Vec<_>> with Vec<_>
* Add more detailed documentation for tree-sitter indentation
* Improve code style in indent.rs
* Use tree-sitter queries for indentation instead of TOML config.
Migrate existing indent queries.
* Add documentation for the new indent queries.
Change xtask docgen to look for indents.scm instead of indents.toml
* Improve code style in indent.rs.
Fix an issue with the rust indent query.
* Move indentation test sources to separate files.
Add `#not-kind-eq?`, `#same-line?` and `#not-same-line` custom predicates.
Improve the rust and c indent queries.
* Fix indent test.
Improve rust indent queries.
* Move indentation tests to integration test folder.
* Improve code style in indent.rs.
Reuse tree-sitter cursors for indentation queries.
* Migrate HCL indent query
* Replace custom loading in indent tests with a designated languages.toml
* Update indent query file name for --health command.
* Fix single-space formatting in indent queries.
* Add explanation for unwrapping.
Co-authored-by: Triton171 <triton0171@gmail.com>
This avoids costly conversions via byte_to_char (which are then
reversed back into bytes internally in Ropey).
Reduces time spent in slice/byte_to_char from ~24% to ~5%.
This is a rather large refactor that moves most of the code for
loading, fetching, and building grammars into a new helix-loader
module. This works well with the [[grammars]] syntax for
languages.toml defined earlier: we only have to depend on the types
for GrammarConfiguration in helix-loader and can leave all the
[[language]] entries for helix-core.
The vision with 'use-grammars' is to allow the long-requested feature
of being able to declare your own set of grammars that you would like.
A simple schema with only/except grammar names controls the list
of grammars that is fetched and built. It does not (yet) control which
grammars may be loaded at runtime if they already exist.
This is not strictly speaking necessary. tree_sitter_library was used by
just one grammar: llvm-mir-yaml, which uses the yaml grammar. This will
make the language more consistent, though. Each language can explicitly
say that they use Some(grammar), defaulting when None to the grammar that
has a grammar_id matching the language's language_id.
helix-syntax mostly existed for the sake of the build task which
checks and compiles the submodules. Since we won't be relying on
that process anymore, it doesn't end up making much sense to have
a very thin crate just for some functions that we could port to
helix-core.
The remaining build-related code is moved to helix-term which will
be able to provide grammar builds through the --build-grammars CLI
flag.
Here we add syntax to the languages.toml languge
[[grammar]]
name = "<name>"
source = { .. }
Which can be used to specify a tree-sitter grammar separately of
the language that defines it, and we make this distinction for
two reasons:
* In later commits, we will separate this code from helix-core
and bring it to a new helix-loader crate. Using separate schemas
for language and grammar configurations allows for a nice divide
between the types needed to be declared in helix-loader and in
helix-core/syntax
* Two different languages may use the same grammar. This is currently
the case with llvm-mir-yaml and yaml. We could accomplish a config
that works for this with just `[[languages]]`, but it gets a bit
dicey with languages depending on one another. If you enable
llvm-mir-yaml and disable yaml, does helix still need to fetch and
build tree-sitter-yaml? It could be a matter of interpretation.
* Move runtime file location definitions to core
* Add basic --health command
* Add language specific --health
* Show summary for all langs with bare --health
* Use TsFeature from xtask for --health
* cargo fmt
Co-authored-by: Blaž Hrastnik <blaz@mxxn.io>
Treesitter captures can contain multiple nodes like so:
```
(line_comment)+ @comment
```
This would match each line in a comment as a separate
`@comment` capture when what we actually want is the
whole set of contiguous `line_comment` nodes to be
captured under the `@comment` capture. This commit enables
this behaviour.
* impl auto pairs config
Implements configuration for which pairs of tokens get auto completed.
In order to help with this, the logic for when *not* to auto complete
has been generalized from a specific hardcoded list of characters to
simply testing if the next/prev char is alphanumeric.
It is possible to configure a global list of pairs as well as at the
language level. The language config will take precedence over the
global config.
* rename AutoPair -> Pair
* clean up insert_char command
* remove Rc
* remove some explicit cloning with another impl
* fix lint
* review comments
* global auto-pairs = false takes precedence over language settings
* make clippy happy
* print out editor config on startup
* move auto pairs accessor into Document
* rearrange auto pair doc comment
* use pattern in Froms
This code:
let start = ensure_grapheme_boundary_next(text, text.byte_to_char(start));
let end = ensure_grapheme_boundary_next(text, text.byte_to_char(end));
Would convert byte to char index, but then internally immediately convert back
to byte index, operate on it, then convert it to char index.
This change reduces the amount of time spent in ensure_grapheme_boundary from
29% to 2%.
* add select_next_sibling and select_prev_sibling commands
* refactor objects to use higher order functions
* address clippy feedback
* move selection cloning into commands
* add default keybindings under left/right brackets
* use [+t,]+t for selecting sibling syntax nodes
* setup Alt-{j,k,h,l} default keymaps for syntax selection commands
* reduce boilerplate of select_next/prev_sibling in commands
* import tree-sitter Node type in commands
Auto pairs were resulting in incorrect ranges in the resulting when the
line terminators are CRLF (i.e. Windows). It turns out this is because
when we were checking if the selection was a single-width cursor, it
incorrectly assumed that this would be a single char. This is not the
case, as a cursor can cover a multi-code point grapheme. Therefore,
we must instead explicitly work with and check graphemes to determine
if the cursor should move or extend the selection.
Fixes#1436
Michael Davis
dxtr85
ath3
Pascal Kuthe
Blaž Hrastnik
Jonathan LEI
Poliorcetics
Armin Ronacher
Christian Speich
A-Walrus
Skyler Hawthorne
midnightexigent
Fisher Darling
Daniel Ebert
Triton171
Riccardo Binetti
Luke Cycon
taupiqueur
Alexander Brevig
Daniel S Poulin
Matthias Deiml
Omnikar
PiergiorgioZagaria
Erasin
Philipp Mildenberger
Bob
Falco Hirschenberger
Saber Haj Rabiee
Matthew Toohey
Gokul Soumya
farwyler
Ryan Russell
Hugo
Andrey Tkachenko
EmmChriss
Vince Mutolo
chunghha
Thomas
Kirawi
Ivan Tham
Danilo Spinella
Roland Kovacs
antoyo
Slin Lee
Kevin Sjöberg