xonsh/tests/test_completion_context.py

885 lines
24 KiB
Python
Raw Normal View History

Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
import itertools
import typing as tp
2022-01-31 21:26:34 +05:30
from unittest import mock
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
import pytest
2022-01-31 21:26:34 +05:30
import xonsh.parsers.completion_context as ctx
from xonsh.parsers.completion_context import (
CommandArg,
CommandContext,
CompletionContextParser,
PythonContext,
)
2022-03-24 00:46:50 +05:30
from xonsh.pytest.tools import ON_WINDOWS
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
DEBUG = False
MISSING = object()
X = "\x00" # cursor marker
PARSER: tp.Optional[CompletionContextParser] = None
@pytest.fixture(scope="module", autouse=True)
def parser():
global PARSER
PARSER = CompletionContextParser(debug=DEBUG)
update test xsh usage (#4581) * todo * test: remove usage of DummyEnv and setting .env attribute on xession fixture one step closer to making too much of tweaking to xession during tests * test: fix tests vox and gitstatus-prompt * docs: update test-fixture usage * fix: import flake8 error * test: remove direct access to XSH in tests * test: remove usage of XSH in test files * todo * test: use tmp-dir to create stubs * refactor: use fixture factory to mock out XonshSession * refactor: remove direct access of XSH from functions * refactor: remove direct access of XSH from functions * fix: qa checks * refactor: rename variables to match their values * test: update failing tests because it had no PATH set previously * fix: remove builtins usage from pyghooks.py * style: * refactor: update tests to use fixtures * fix: env varialbe is setup per function some tests accidentally update the env variables and that is leaking into next tests * fix: failing vox tests * test: set commands_cache per test * test: fix failing tests * fix: failing tests on linux ptk-highlight * fix: failing tests on Windows cwd-prompt * test: copy env as to not affect original object * fix: lazily evaluate cmds-cache in pyghooks * test: fix failing tests * fix: qa errors import * test: set commands-cache per test while caching path results * test: speedup test_thread_local_swap * fix: failing tests on windows * refactor: Execer doesn't control session * refactor: XSH.unload will take care of reversing builtins attrs set * test: use env.update over monkeypatch * Revert "test: use env.update over monkeypatch" This reverts commit 010a5022247a098f1741966b8af1bf758663480e.
2022-01-08 04:03:22 +05:30
patcher = None
if ON_WINDOWS:
# on-windows has an option for interactive sessions. Overriding the lazyObject
patcher = mock.patch.object(
ctx,
"LINE_CONT_REPLACEMENT_DIFF",
("\\\n", "", -2),
)
patcher.start()
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
yield
PARSER = None
update test xsh usage (#4581) * todo * test: remove usage of DummyEnv and setting .env attribute on xession fixture one step closer to making too much of tweaking to xession during tests * test: fix tests vox and gitstatus-prompt * docs: update test-fixture usage * fix: import flake8 error * test: remove direct access to XSH in tests * test: remove usage of XSH in test files * todo * test: use tmp-dir to create stubs * refactor: use fixture factory to mock out XonshSession * refactor: remove direct access of XSH from functions * refactor: remove direct access of XSH from functions * fix: qa checks * refactor: rename variables to match their values * test: update failing tests because it had no PATH set previously * fix: remove builtins usage from pyghooks.py * style: * refactor: update tests to use fixtures * fix: env varialbe is setup per function some tests accidentally update the env variables and that is leaking into next tests * fix: failing vox tests * test: set commands_cache per test * test: fix failing tests * fix: failing tests on linux ptk-highlight * fix: failing tests on Windows cwd-prompt * test: copy env as to not affect original object * fix: lazily evaluate cmds-cache in pyghooks * test: fix failing tests * fix: qa errors import * test: set commands-cache per test while caching path results * test: speedup test_thread_local_swap * fix: failing tests on windows * refactor: Execer doesn't control session * refactor: XSH.unload will take care of reversing builtins attrs set * test: use env.update over monkeypatch * Revert "test: use env.update over monkeypatch" This reverts commit 010a5022247a098f1741966b8af1bf758663480e.
2022-01-08 04:03:22 +05:30
if ON_WINDOWS and patcher:
patcher.stop()
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
def parse(command, inner_index):
return PARSER.parse(command, inner_index)
def assert_match(
commandline, command_context=MISSING, python_context=MISSING, is_main_command=False
):
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
if X in commandline:
index = commandline.index(X)
commandline = commandline.replace(X, "")
else:
index = len(commandline)
context = parse(commandline, index)
if context is None:
raise SyntaxError(
"Failed to parse the commandline - set DEBUG = True in this file to see the error"
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
if is_main_command and python_context is MISSING:
python_context = PythonContext(commandline, index)
if command_context is not MISSING:
assert context.command == command_context
if python_context is not MISSING:
assert context.python == python_context
COMMAND_EXAMPLES = (
(f"comm{X}", CommandContext(args=(), arg_index=0, prefix="comm")),
(f" comm{X}", CommandContext(args=(), arg_index=0, prefix="comm")),
(f"comm{X}and", CommandContext(args=(), arg_index=0, prefix="comm", suffix="and")),
(f"command {X}", CommandContext(args=(CommandArg("command"),), arg_index=1)),
(f"{X} command", CommandContext(args=(CommandArg("command"),), arg_index=0)),
(f" command {X}", CommandContext(args=(CommandArg("command"),), arg_index=1)),
(
f"command --{X}",
CommandContext(args=(CommandArg("command"),), arg_index=1, prefix="--"),
),
(
f"command a {X}",
CommandContext(args=(CommandArg("command"), CommandArg("a")), arg_index=2),
),
(
f"command a b{X}",
CommandContext(
args=(CommandArg("command"), CommandArg("a")), arg_index=2, prefix="b"
),
),
(
f"command a b{X}",
CommandContext(
args=(CommandArg("command"), CommandArg("a")), arg_index=2, prefix="b"
),
),
(
f"command {X} a",
CommandContext(args=(CommandArg("command"), CommandArg("a")), arg_index=1),
),
(
f"command a {X} b",
CommandContext(
args=(CommandArg("command"), CommandArg("a"), CommandArg("b")), arg_index=2
),
),
(
f"command -{X} b",
CommandContext(
args=(CommandArg("command"), CommandArg("b")), arg_index=1, prefix="-"
),
),
(
f"command a {X}b",
CommandContext(
args=(CommandArg("command"), CommandArg("a")), arg_index=2, suffix="b"
),
),
(
f"command a{X}b",
CommandContext(
args=(CommandArg("command"),), arg_index=1, prefix="a", suffix="b"
),
),
(
f"'comm and' a{X}b",
CommandContext(
args=(CommandArg("comm and", opening_quote="'", closing_quote="'"),),
arg_index=1,
prefix="a",
suffix="b",
),
),
(
f"command >/dev/nul{X}",
CommandContext(
args=(CommandArg("command"), CommandArg(">", is_io_redir=True)),
arg_index=2,
prefix="/dev/nul",
),
),
(
f"command 2>/dev/nul{X}",
CommandContext(
args=(CommandArg("command"), CommandArg("2>", is_io_redir=True)),
arg_index=2,
prefix="/dev/nul",
),
),
(
f"command >>/dev/nul{X}",
CommandContext(
args=(CommandArg("command"), CommandArg(">>", is_io_redir=True)),
arg_index=2,
prefix="/dev/nul",
),
),
(
f"command </dev/nul{X}",
CommandContext(
args=(CommandArg("command"), CommandArg("<", is_io_redir=True)),
arg_index=2,
prefix="/dev/nul",
),
),
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
)
EMPTY_COMMAND_EXAMPLES = (
(f"{X}", CommandContext((), 0)),
(f" {X}", CommandContext((), 0)),
(f"{X} ", CommandContext((), 0)),
(f" {X} ", CommandContext((), 0)),
)
STRING_ARGS_EXAMPLES = (
(
f"'comm an{X}d'",
CommandContext(
args=(),
arg_index=0,
prefix="comm an",
suffix="d",
opening_quote="'",
closing_quote="'",
),
),
(
f"'comm and{X}'",
CommandContext(
args=(),
arg_index=0,
prefix="comm and",
suffix="",
opening_quote="'",
closing_quote="'",
),
),
(
f"'comm {X}'",
CommandContext(
args=(),
arg_index=0,
prefix="comm ",
suffix="",
opening_quote="'",
closing_quote="'",
),
),
(
f'"comm an{X}d"',
CommandContext(
args=(),
arg_index=0,
prefix="comm an",
suffix="d",
opening_quote='"',
closing_quote='"',
),
),
(
f"'''comm an{X}d'''",
CommandContext(
args=(),
arg_index=0,
prefix="comm an",
suffix="d",
opening_quote="'''",
closing_quote="'''",
),
),
(
f"fr'comm an{X}d'",
CommandContext(
args=(),
arg_index=0,
prefix="comm an",
suffix="d",
opening_quote="fr'",
closing_quote="'",
),
),
(
f"'()+{X}'",
CommandContext(
args=(), arg_index=0, prefix="()+", opening_quote="'", closing_quote="'"
),
),
(
f"'comm and'{X}",
CommandContext(
args=(),
arg_index=0,
prefix="comm and",
opening_quote="'",
closing_quote="'",
is_after_closing_quote=True,
),
),
(
f"'''comm and'''{X}",
CommandContext(
args=(),
arg_index=0,
prefix="comm and",
opening_quote="'''",
closing_quote="'''",
is_after_closing_quote=True,
),
),
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
)
COMMAND_EXAMPLES += STRING_ARGS_EXAMPLES
COMMAND_EXAMPLES += EMPTY_COMMAND_EXAMPLES
@pytest.mark.parametrize("commandline, context", COMMAND_EXAMPLES)
def test_command(commandline, context):
assert_match(commandline, context, is_main_command=True)
@pytest.mark.parametrize(
"commandline, context",
tuple(
(commandline, context)
for commandline, context in STRING_ARGS_EXAMPLES
if commandline.endswith("'") or commandline.endswith('"')
),
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
def test_partial_string_arg(commandline, context):
partial_commandline = commandline.rstrip("\"'")
partial_context = context._replace(closing_quote="")
assert_match(partial_commandline, partial_context, is_main_command=True)
CONT = "\\" "\n"
@pytest.mark.parametrize(
"commandline, context",
(
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
# line continuations:
(
f"echo {CONT}a {X}",
CommandContext(args=(CommandArg("echo"), CommandArg("a")), arg_index=2),
),
(
f"echo {CONT}{X}a {CONT} b",
CommandContext(
args=(CommandArg("echo"), CommandArg("b")), arg_index=1, suffix="a"
),
),
(
f"echo a{CONT}{X}b",
CommandContext(
args=(CommandArg("echo"),), arg_index=1, prefix="a", suffix="b"
),
),
(
f"echo a{X}{CONT}b",
CommandContext(
args=(CommandArg("echo"),), arg_index=1, prefix="a", suffix="b"
),
),
(
f"echo ${CONT}(a) {CONT} {X}b",
CommandContext(
args=(CommandArg("echo"), CommandArg("$(a)")), arg_index=2, suffix="b"
),
),
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
# line continuations in strings:
(
f"echo 'a{CONT}{X}b'",
CommandContext(
args=(CommandArg("echo"),),
arg_index=1,
prefix="a",
suffix="b",
opening_quote="'",
closing_quote="'",
),
),
(
f"echo '''a{CONT}{X}b'''",
CommandContext(
args=(CommandArg("echo"),),
arg_index=1,
prefix="a",
suffix="b",
opening_quote="'''",
closing_quote="'''",
),
),
(
f"echo 'a{CONT}{X}b",
CommandContext(
args=(CommandArg("echo"),),
arg_index=1,
prefix="a",
suffix="b",
opening_quote="'",
),
),
(
f"echo '''a{CONT}{X}b",
CommandContext(
args=(CommandArg("echo"),),
arg_index=1,
prefix="a",
suffix="b",
opening_quote="'''",
),
),
(
f"echo ''{CONT}'a{X}b",
CommandContext(
args=(CommandArg("echo"),),
arg_index=1,
prefix="a",
suffix="b",
opening_quote="'''",
),
),
(
f"echo '''a{CONT}{X} b",
CommandContext(
args=(CommandArg("echo"),),
arg_index=1,
prefix="a",
suffix=" b",
opening_quote="'''",
),
),
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
# triple-quoted strings:
(
f"echo '''a\nb{X}\nc'''",
CommandContext(
args=(CommandArg("echo"),),
arg_index=1,
prefix="a\nb",
suffix="\nc",
opening_quote="'''",
closing_quote="'''",
),
),
(
f"echo '''a\n b{X} \n c'''",
CommandContext(
args=(CommandArg("echo"),),
arg_index=1,
prefix="a\n b",
suffix=" \n c",
opening_quote="'''",
closing_quote="'''",
),
),
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
# partial triple-quoted strings:
(
f"echo '''a\nb{X}\nc",
CommandContext(
args=(CommandArg("echo"),),
arg_index=1,
prefix="a\nb",
suffix="\nc",
opening_quote="'''",
),
),
(
f"echo '''a\n b{X} \n c",
CommandContext(
args=(CommandArg("echo"),),
arg_index=1,
prefix="a\n b",
suffix=" \n c",
opening_quote="'''",
),
),
),
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
def test_multiline_command(commandline, context):
assert_match(commandline, context, is_main_command=True)
NESTING_EXAMPLES = (
# nesting, prefix
(f"echo $({X})", "$("),
(f"echo @$({X})", "@$("),
(f"echo $(echo $({X}))", "$("),
(f"echo @(x + $({X}))", "$("),
(f"!({X})", "!("),
(f"$[{X}]", "$["),
(f"![{X}]", "!["),
)
NESTED_SIMPLE_CMD_EXAMPLES = [
(
nesting,
f"simple {X}",
CommandContext(
args=(CommandArg("simple"),), arg_index=1, subcmd_opening=prefix
),
)
for nesting, prefix in NESTING_EXAMPLES[1:]
]
@pytest.mark.parametrize(
"nesting, commandline, context",
list(
itertools.chain(
(
# complex subcommand in a simple nested expression
(
NESTING_EXAMPLES[0][0],
commandline,
context._replace(subcmd_opening=NESTING_EXAMPLES[0][1]),
)
for commandline, context in COMMAND_EXAMPLES
),
NESTED_SIMPLE_CMD_EXAMPLES,
)
),
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
def test_nested_command(commandline, context, nesting):
nested_commandline = nesting.replace(X, commandline)
assert_match(nested_commandline, command_context=context, python_context=None)
NESTING_MALFORMATIONS = (
lambda s: s[:-1], # remove the last closing brace ')' / ']'
lambda s: s + s[-1], # add an extra closing brace ')' / ']'
lambda s: s[-1] + s,
lambda s: s + "$(",
lambda s: "$(" + s,
)
@pytest.mark.parametrize("nesting, commandline, context", NESTED_SIMPLE_CMD_EXAMPLES)
@pytest.mark.parametrize("malformation", NESTING_MALFORMATIONS)
def test_malformed_subcmd(nesting, commandline, context, malformation):
nested_commandline = nesting.replace(X, commandline)
nested_commandline = malformation(nested_commandline)
assert_match(nested_commandline, command_context=context, python_context=None)
MALFORMED_SUBCOMMANDS_NESTINGS = (
# nesting, subcmd_opening
(f"echo $(a $({X}", "$("),
(f"echo $(a $(b; {X}", ""),
(f"$(echo $(a $({X}", "$("),
(f"echo $[a $({X}]", "$("),
(f"echo $(a $[{X})", "$["),
(f"echo @(x = $({X}", "$("),
(f"echo @(a; x = $({X}", "$("),
(f"echo @(x = $(a; {X}", ""),
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
@pytest.mark.parametrize("nesting, subcmd_opening", MALFORMED_SUBCOMMANDS_NESTINGS)
@pytest.mark.parametrize("commandline, context", COMMAND_EXAMPLES[:5])
def test_multiple_malformed_subcmds(nesting, subcmd_opening, commandline, context):
nested_commandline = nesting.replace(X, commandline)
nested_context = context._replace(subcmd_opening=subcmd_opening)
assert_match(nested_commandline, nested_context, python_context=None)
def test_other_subcommand_arg():
command = "echo $(pwd) "
assert_match(
command,
CommandContext((CommandArg("echo"), CommandArg("$(pwd)")), arg_index=2),
is_main_command=True,
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
)
def test_combined_subcommand_arg():
command = f"echo file=$(pwd{X})/x"
# index inside the subproc
assert_match(
command,
CommandContext((), arg_index=0, prefix="pwd", subcmd_opening="$("),
python_context=None,
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
# index at the end of the command
assert_match(
command.replace(X, ""),
CommandContext((CommandArg("echo"),), arg_index=1, prefix="file=$(pwd)/x"),
is_main_command=True,
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
SUBCMD_BORDER_EXAMPLES = (
(f"{X}$(echo)", CommandContext((), 0, suffix="$(echo)")),
(f"${X}(echo)", CommandContext((), 0, prefix="$", suffix="(echo)")),
(f"$(echo){X}", CommandContext((), 0, prefix="$(echo)")),
(f"${X}( echo)", CommandContext((), 0, prefix="$", suffix="( echo)")),
(f"$(echo ){X}", CommandContext((), 0, prefix="$(echo )")),
)
@pytest.mark.parametrize("commandline, context", SUBCMD_BORDER_EXAMPLES)
def test_cursor_in_subcmd_borders(commandline, context):
assert_match(commandline, context, is_main_command=True)
MULTIPLE_COMMAND_KEYWORDS = (
"; ",
"\n",
" and ",
"&& ",
" or ",
"|| ",
"| ",
)
MULTIPLE_CMD_SIMPLE_EXAMPLES = [
(
keyword,
("echo hi", f"simple {X}"),
CommandContext(args=(CommandArg("simple"),), arg_index=1),
)
for keyword in MULTIPLE_COMMAND_KEYWORDS
]
EXTENSIVE_COMMAND_PAIRS = tuple(
itertools.chain(
zip(COMMAND_EXAMPLES, COMMAND_EXAMPLES[::-1]),
zip(COMMAND_EXAMPLES, EMPTY_COMMAND_EXAMPLES),
zip(EMPTY_COMMAND_EXAMPLES, COMMAND_EXAMPLES),
zip(EMPTY_COMMAND_EXAMPLES, EMPTY_COMMAND_EXAMPLES),
)
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
MULTIPLE_COMMAND_EXTENSIVE_EXAMPLES = tuple(
itertools.chain(
(
# cursor in first command
((first, second.replace(X, "")), first_context)
for (first, first_context), (
second,
second_context,
) in EXTENSIVE_COMMAND_PAIRS
),
(
# cursor in second command
((first.replace(X, ""), second), second_context)
for (first, first_context), (
second,
second_context,
) in EXTENSIVE_COMMAND_PAIRS
),
(
# cursor in middle command
((first.replace(X, ""), second, third.replace(X, "")), second_context)
for (first, _1), (second, second_context), (third, _3) in zip(
COMMAND_EXAMPLES[:3], COMMAND_EXAMPLES[3:6], COMMAND_EXAMPLES[6:9]
)
),
(
# cursor in third command
((first.replace(X, ""), second.replace(X, ""), third), third_context)
for (first, _1), (second, _2), (third, third_context) in zip(
COMMAND_EXAMPLES[:3], COMMAND_EXAMPLES[3:6], COMMAND_EXAMPLES[6:9]
)
),
)
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
@pytest.mark.parametrize(
"keyword, commands, context",
tuple(
itertools.chain(
(
(MULTIPLE_COMMAND_KEYWORDS[0], commands, context)
for commands, context in MULTIPLE_COMMAND_EXTENSIVE_EXAMPLES
),
MULTIPLE_CMD_SIMPLE_EXAMPLES,
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
),
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
def test_multiple_commands(keyword, commands, context):
joined_command = keyword.join(commands)
cursor_command = next(command for command in commands if X in command)
if cursor_command is commands[0]:
relative_index = cursor_command.index(X)
else:
absolute_index = joined_command.index(X)
relative_index = (
absolute_index
- joined_command.rindex(keyword, 0, absolute_index)
- len(keyword)
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
if keyword.endswith(" "):
# the last space is part of the command
relative_index += 1
cursor_command = " " + cursor_command
assert_match(joined_command, context, is_main_command=True)
@pytest.mark.parametrize(
"commandline",
(
f"{X};",
f"; {X}",
f"{X};;",
f"; {X};",
f";; {X}",
f";;; {X}",
),
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
def test_multiple_empty_commands(commandline):
assert_match(commandline, CommandContext((), 0), is_main_command=True)
@pytest.mark.parametrize(
"nesting, keyword, commands, context",
tuple(
(
nesting,
keyword,
commands,
context,
) # no subcmd_opening in nested multi-commands
for nesting, prefix in NESTING_EXAMPLES
for keyword, commands, context in MULTIPLE_CMD_SIMPLE_EXAMPLES
if keyword != "\n" # the lexer ignores newlines inside subcommands
),
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
def test_nested_multiple_commands(nesting, keyword, commands, context):
joined_command = keyword.join(commands)
nested_joined = nesting.replace(X, joined_command)
assert_match(nested_joined, context, python_context=None)
def test_multiple_nested_commands():
assert_match(
f"echo hi; echo $(ls{X})",
CommandContext((), 0, prefix="ls", subcmd_opening="$("),
python_context=None,
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
@pytest.mark.parametrize(
"commandline, context",
tuple(
(commandline, context)
for commandline, context in STRING_ARGS_EXAMPLES
if commandline.endswith("'") or commandline.endswith('"')
),
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
def test_multiple_partial_string_arg(commandline, context):
partial_commandline = commandline.rstrip("\"'")
partial_context = context._replace(closing_quote="")
assert_match("echo;" + partial_commandline, partial_context)
assert_match("echo $[a ;" + partial_commandline, partial_context)
@pytest.mark.parametrize(
"nesting, keyword, commands, context",
tuple(
(nesting, keyword, commands, context)
for nesting, prefix in NESTING_EXAMPLES[:1]
for keyword, commands, context in MULTIPLE_CMD_SIMPLE_EXAMPLES[:1]
),
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
@pytest.mark.parametrize("malformation", NESTING_MALFORMATIONS)
2021-09-26 21:02:24 +05:30
def test_malformed_subcmd_1(malformation, nesting, keyword, commands, context):
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
joined_command = keyword.join(commands)
nested_joined = nesting.replace(X, joined_command)
malformed_commandline = malformation(nested_joined)
assert_match(malformed_commandline, context, python_context=None)
MULTIPLE_COMMAND_BORDER_EXAMPLES = tuple(
itertools.chain(
itertools.chain(
*(
(
(
f"ls{ws1}{X}{kwd}{ws2}echo",
(
CommandContext((CommandArg("ls"),), 1)
if ws1
else CommandContext((), 0, prefix="ls")
),
),
)
for ws1, ws2, kwd in itertools.product(
("", " "), ("", " "), ("&&", ";")
)
)
),
# all keywords are treated as a normal arg if the cursor is at the edge
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
(
(
f"ls {X}and echo",
CommandContext((CommandArg("ls"), CommandArg("echo")), 1, suffix="and"),
),
(
f"ls and{X} echo",
CommandContext((CommandArg("ls"), CommandArg("echo")), 1, prefix="and"),
),
(
f"ls ||{X} echo",
CommandContext((CommandArg("ls"), CommandArg("echo")), 1, prefix="||"),
),
),
# if the cursor is inside the keyword, it's treated as a normal arg
(
(
f"ls a{X}nd echo",
CommandContext(
(CommandArg("ls"), CommandArg("echo")), 1, prefix="a", suffix="nd"
),
),
(
f"ls &{X}& echo",
CommandContext(
(CommandArg("ls"), CommandArg("echo")), 1, prefix="&", suffix="&"
),
),
),
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
)
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
@pytest.mark.parametrize(
"commandline, context",
tuple(
itertools.chain(
MULTIPLE_COMMAND_BORDER_EXAMPLES,
(
# ensure these rules work with more than one command
(f"cat | {commandline}", context)
for commandline, context in MULTIPLE_COMMAND_BORDER_EXAMPLES
),
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
),
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
def test_cursor_in_multiple_keyword_borders(commandline, context):
assert_match(commandline, context)
PYTHON_EXAMPLES = (
# commandline, context
(f"x = {X}", PythonContext("x = ", 4)),
(f"a {X}= x; b = y", PythonContext("a = x; b = y", 2)),
(f"a {X}= x\nb = $(ls)", PythonContext("a = x\nb = $(ls)", 2)),
)
PYTHON_NESTING_EXAMPLES = (
# nesting, prefix
f"echo @({X})",
f"echo $(echo @({X}))",
f"echo @(x + @({X}))", # invalid syntax, but can still be in a partial command
)
@pytest.mark.parametrize(
"nesting, commandline, context",
list(
itertools.chain(
(
# complex subcommand in a simple nested expression
(nesting, commandline, context._replace(is_sub_expression=True))
for nesting in PYTHON_NESTING_EXAMPLES[:1]
for commandline, context in PYTHON_EXAMPLES
),
(
# simple subcommand in a complex nested expression
(nesting, commandline, context._replace(is_sub_expression=True))
for nesting in PYTHON_NESTING_EXAMPLES
for commandline, context in PYTHON_EXAMPLES[:1]
),
)
),
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
def test_nested_python(commandline, context, nesting):
nested_commandline = nesting.replace(X, commandline)
assert_match(nested_commandline, command_context=None, python_context=context)
@pytest.mark.parametrize(
"commandline, context",
[
(
commandline.replace("$", "@"),
context._replace(
prefix=context.prefix.replace("$", "@"),
suffix=context.suffix.replace("$", "@"),
),
)
for commandline, context in SUBCMD_BORDER_EXAMPLES
],
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
def test_cursor_in_sub_python_borders(commandline, context):
assert_match(commandline, context, is_main_command=True)
@pytest.mark.parametrize(
"code",
(
f"""
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
x = 3
x.{X}""",
f"""
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
x = 3;
y = 4;
x.{X}""",
f"""
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
def func({X}):
return 100
""",
f"""
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
class A:
def a():
return "a{X}"
pass
exit()
""",
),
)
Completion context (#4017) * completion-context: Add CompletionContextParser placeholder Implements the xonsh (tab-)completion context parser. This parser is meant to parse a (possibly incomplete) command line. * completers: tools: Implement ``contextual_completer`` decorator This is used to mark completers that want to use the parsed completion context. * completers: Enable using contextual completers in xonsh/completer.py * completers: readline, ptk, jupyter: Enable using contextual completers Pass ``multiline_text`` and ``cursor_index`` to ``Completer.complete()`` * parsers: base: Refactor out a ``raise_parse_error`` function * tokenize: Enable ``tolerant`` mode If ``tolerant`` is True, yield ERRORTOKEN instead of throwing an exception when encountering an error. * lexer: Enable ``tolerant`` mode Tokenize without extra checks (e.g. paren matching). When True, ERRORTOKEN contains the erroneous string instead of an error msg. * tests: lexer: Test ``tolerant`` mode * completion-context: Implement simple CommandContext parsing * completion-context: tests: Test simple CommandContext parsing * completion-context: Implement parsing sub-commands * completion-context: tests: Test parsing sub-commands * completion-context: Add news file * completion-context: parser: Add parser table path to relevant locations Code-coverage, mypy ignore list, etc. * completion-context: Implement parsing partial strings and line continuations * completion-context: tests: Test parsing partial strings and line continuations * completion-context: Convert ``Span`` object to a ``slice`` * completion-context: Refactor out ``create_command`` and ``cursor_in_span`` * completion-context: Implement handling empty commands * completion-context: tests: Test handling empty commands * completion-context: Implement handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: tests: Test handling multiple commands Separated by newlines, `;`, `and`, `or`, `|`, `&&`, `||` * completion-context: Implement handling python context * completion-context: tests: Test handling python context * completers: tools: Add `contextual_command_completer` * completers: Make `complete_skipper` contextual * completers: Make `complete_from_man` contextual * completers: Make `complete_from_bash` contextual and add test * completers: Make `complete_pip` contextual and update tests * completers: Keep opening string quote if it exists * completion-context: Handle cursor after a closing quote For example - cursor at the end of ``ls "/usr/"``. 1. The closing quote will be appended to all completions. I.e the completion ``/usr/bin`` will turn into ``/usr/bin"`` 2. If not specified, lprefix will cover the closing prefix. I.e for ``ls "/usr/"``, the default lprefix will be 6 to include the closing quote. * completion-context: tests: Test handling cursor after a closing quote * completion-context: Fix bug with multiple empty commands e.g. `;;;` * completion-context: tests: Speed up tests From ~15 seconds to ~500 ms * completion-context: Expand commands and subcommands * completion-context: Simplify `commands` rules * completion-context: Simplify `sub_expression` rules * completion-context: Simplify editing a multi-command token * completion-context: Inline `create_command` * completion-context: Implement `contextual_command_completer_for` helper * completers: Make `complete_cd`/`complete_rmdir` contextual and add tests * completers: path: Don't append a double-backslash in a raw string When completing a path, if a raw string is used (e.g. `r"C:\Windows\"`), there's no reason to append a double-backslash (e.g. `r"C:\Windows\\"`). * completers: Make `complete_xonfig`/`complete_xontrib` contextual and add tests * completers: Make `complete_completer` contextual and add tests * completers: Make `complete_import` contextual and add tests * completion-context: Add python `ctx` attribute * completion: tools: Simplify `RichCompletion` attributes handling * completers: Make `base`, `python`, and `commands` contextual * Add tests * No need for `python_mode` completer anymore * completion: tools: Add `append_space` attribute to `RichCompletion` * completion-context: Get all lines in a main python context * xontrib: jedi: Make the `jedi` completer contextual * completers: tools: Remove `get_ptk_completer` and `PromptToolkitCompleter.current_document` These aren't needed anymore now that contextual completers can access the multiline code (via `PythonContext.multiline_code`). * completion-context: ptk: Expand aliases * completion-context: jupyter: Expand aliases and fix line handling * completer: Preserve custom prefix after closing quote * completers: bash: Ensure bash completion uses the complete prefix * completers: pip: Append a space after a pip command * completers: pip: Prevent bad package name completions * completers: Remove a common prefix from `RichCompletion` if `display` wasn't provided * completion-context: Treat cursor at edge of `&& || | ;` as normal args This will be used for completing a space * completers: Complete end proc keywords correctly
2021-03-30 20:37:56 +03:00
def test_multiline_python(code):
assert_match(code, is_main_command=True)