### Motivation
I noticed that when I type every letter in the prompt on the remote
server the typing is lagging. I thought it's ssh but no.
After inspection the commands cache code I found that on every key press
there is reading all 2000+ files in PATH because the code for cache
update has funny issue.
### Quiz
```xsh
def f():
print(1)
yield False
print(2)
yield True
print(3)
print(any(f()))
```
What will be in the output? Answer:
<details>
```
1
2
True
```
The execution of `print(3)` ignored because `any()` interrupts the
execution of the function.
If we call `print(list(f()))` the output will be `1 2 3 [False, True]`.
</details>
### Before
* Updating cache (read all files in PATH and list of aliases) two times
after start.
* Updating cache (read all files in PATH and list of aliases) on every
key press in prompt.
### After
* Update cache once at start.
* Update cache only when we have real changes of paths or aliases.
cc #4954#3895#5309
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Added test.
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Added py mention to docs
* Added tests
* Microfix
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
### Motivation
Add `spec.raise_subproc_error` to have an ability to use
`SpecModifierAlias` to manage the errors. Also this is the first step to
#4351.
Generally in scripts it's good to have RAISE_SUBPROC_ERROR=True to avoid
processing the error for every executed command. But in some cases (e.g.
`![]`) it's needed to avoid raising the error. To more elegant doing
this we can make an ability to create SpecModifier.
### After
```xsh
from xonsh.procs.specs import SpecModifierAlias
class SpecModifierNoErrAlias(SpecModifierAlias):
def on_modifer_added(self, spec):
spec.raise_subproc_error = False
aliases['noraise'] = SpecModifierNoErrAlias()
$RAISE_SUBPROC_ERROR = True
if ![noraise git pull]:
git add --all
```
Cc #5443
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Fix#5491:
* fix condition
* added more well organized tests
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
After #5491 there is one case unfixed:
### Before
```xsh
echo 'echo home' >> ~/.xonshrc
cd /tmp
echo 'echo script' > 1.xsh
xonsh 1.xsh
# all rc without ~/.xonshrc
# script
xonsh -i 1.xsh
# all rc without ~/.xonshrc # bug
# script
```
### After
```xsh
echo 'echo home' >> ~/.xonshrc
cd /tmp
echo 'echo script' > 1.xsh
xonsh 1.xsh
# all rc without ~/.xonshrc
# script
xonsh -i 1.xsh
# all rc with ~/.xonshrc
# home # FIX
# script
```
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
### Motivation
In #5099 was introduced the logic where all RC files are loaded in
interactive and non-interactive modes. This logic is not working good
for home based `~/.xonshrc` file.
First of all `~/.xonshrc` is the buffer/accumulator of settings focused
on interactive mode. Most tools with integration with xonsh (e.g.
`conda`, `zoxide`, etc) offer to just do `echo "init_tool()" >>
~/.xonshrc` (`conda` has around 20 lines of init code) to start using
the tool and users are doing this without any doubts.
But because of after 5099 `~/.xonshrc` is executed in non-interactive
mode the adding code to it leads to unexpected and unintended side
effects:
* If you run a script with shebang (e.g. `#!/usr/bin/env xonsh` or
[xonsh-awesome-cli-app](https://github.com/anki-code/xonsh-awesome-cli-app))
or just from `xonsh script.xsh` the code will be unexpected and
unintended slower.
* If you're using xonsh-based tools (e.g. you install them using pip)
and run them in environment that has no packages that initiated in
`~/.xonshrc` you will see epic errors.
* Additional context:
* Bash and Zsh do not load `~/.bashrc` and `~/.zshrc` in non-interactive
mode by the same reasons.
* We have welcome message `Create ~/.xonshrc file manually or use xonfig
to suppress the welcome message` and we don't want to make the process
of creating this file complex.
All of this leads to bad unexpected and unintended experience. This PR
is to fix this.
### Expectation
By doing this fix we assume that experienced user who wants to build
good repeatable run control files will use another ways to create config
files and this has [reflection in
docs](8860f2bd52/docs/xonshrc.rst).
In the nutshell if you want to create the RC files that affect every run
of code you should use one or many of these ways:
* Cross-desktop group (XDG) compliant `~/.config/xonsh/rc.xsh` control
file.
* The system-wide control file `/etc/xonsh/xonshrc` for Linux and OSX
and in `%ALLUSERSPROFILE%\xonsh\xonshrc` on Windows. It controls options
that are applied to all users of Xonsh on a given system.
* The home-based directory `~/.config/xonsh/rc.d/` and system
`/etc/xonsh/rc.d/` can contain .xsh files. They will be executed at
startup in order. This allows for drop-in configuration where your
configuration can be split across scripts and common and local
configurations more easily separated.
In your configs you need to check `$XONSH_INTERACTIVE` and
`$XONSH_LOGIN` explicitly.
### Before
`~/.xonshrc` is used in non-interactive mode.
```xsh
echo "echo RC" >> ~/.xonshrc
cd /tmp
echo "echo Script" > script.xsh
xonsh script.xsh
# RC
# Script
```
```xsh
cd /tmp
echo 'echo RC' >> ~/.xonshrc
echo '#!/usr/bin/env xonsh' > myscript
chmod +x myscript
./myscript
# RC
```
### After
`~/.xonshrc` is not used in non-interactive mode. Use `-i` if you need
it.
```xsh
echo "echo RC" >> ~/.xonshrc
cd /tmp
echo "echo Script" > script.xsh
xonsh script.xsh
# Script
xonsh -i script.xsh
# RC
# Script
```
```xsh
cd /tmp
echo 'echo RC' >> ~/.xonshrc
echo '#!/usr/bin/env xonsh' > myscript
chmod +x myscript
./myscript
```
Closes#5488#4096#5496
### Fun
I want to leave here some nice representation of how it works in
bash/sh/zsh from [twitter
post](https://twitter.com/paxx39/status/1742768007154479109):
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
<!---
Thanks for opening a PR on xonsh!
Please do this:
1. Include a news file with your PR
(https://xon.sh/devguide.html#changelog).
2. Add the documentation for your feature into `/docs`.
3. Add the example of usage or before-after behavior.
4. Mention the issue that this PR is addressing e.g. `#1234`.
-->
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
This is continuation for introduced before
`$XONSH_SUBPROC_OUTPUT_FORMAT` (#5377) so news file is not needed.
### Motivation
Having `output_format` in spec and in pipeline allows to switch output
format in `SpecModifierAlias` and during non-blocking work e.g.
```xsh
p=!(echo '1\n2\n3')
p.output_format = 'list_lines'
p.out
# ['1','2','3']
```
```xsh
from xonsh.procs.specs import SpecModifierAlias
class SpecModifierOutputLinesAlias(SpecModifierAlias):
def on_modifer_added(self, spec):
spec.output_format = 'list_lines'
aliases['xlines'] = SpecModifierOutputLinesAlias()
$(xlines echo '1\n2\n3')
# ['1','2','3']
```
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
### Motivation
This is fix for introduced before
`$XONSH_SUBPROC_OUTPUT_FORMAT='list_lines'` so news file is not needed.
We should return the list for single line output to avoid unintended
reading the string.
### Before
```xsh
$XONSH_SUBPROC_OUTPUT_FORMAT='list_lines'
mkdir -p /tmp/list_lines
cd /tmp/list_lines
touch 123
du $(ls)
# 0 123
for f in $(ls):
print(f)
# 1
# 2
# 3
touch 321
for f in $(ls):
print(f)
# 123
# 321
```
### After
```xsh
$XONSH_SUBPROC_OUTPUT_FORMAT='list_lines'
mkdir -p /tmp/list_lines
cd /tmp/list_lines
touch 123
du $(ls)
# 0 123
for f in $(ls):
print(f)
# 123
touch 321
for f in $(ls):
print(f)
# 123
# 321
```
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
Co-authored-by: a <1@1.1>
I'm going to revert #5423 because it was regress -
https://github.com/xonsh/xonsh/issues/5466 - and we need to fix parser
instead of resolver.
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
previously, this succeeded in CI, but always failed locally with the
following error:
```xsh
_____________________________________________________________________ test_raise_subproc_error_with_show_traceback[True] _____________________________________________________________________
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x76f6ba8882e0>, interactive = True
@skip_if_on_windows
@pytest.mark.parametrize("interactive", [True, False])
def test_raise_subproc_error_with_show_traceback(monkeypatch, interactive):
out, err, ret = run_xonsh(
"$COLOR_RESULTS=False\n$RAISE_SUBPROC_ERROR=False\n$XONSH_SHOW_TRACEBACK=False\nls nofile",
interactive=interactive,
single_command=True,
)
assert ret != 0
> assert re.match("ls.*No such file or directory\n", out)
E assert None
E + where None = <function match at 0x76f6bcd0cd30>('ls.*No such file or directory\n', "\x1b]0;ls | jyn@pop-os: ~/src/xonsh | xonsh\x07ls: cannot access 'nofile': No such file or directory\n")
E + where <function match at 0x76f6bcd0cd30> = re.match
tests/test_integrations.py:1190: AssertionError
```
the problem was that the following string was prepended before the error
message:
```
\x1b]0;ls | jyn@pop-os: ~/src/xonsh | xonsh\x07l
```
that in turn came from
[`BaseShell.settitle`](66c0490d37/xonsh/base_shell.py (L570)),
which always prepends at least the escape codes even when TITLE is set
to an empty string.
avoid this issue by mimicking a dumb terminal in tests, in which case
`settitle` doesn't add any title at all.
see
https://xonsh.zulipchat.com/#narrow/stream/435069-xonsh-dev/topic/Running.20tests.20locally
for more context.
<!---
Thanks for opening a PR on xonsh!
Please do this:
1. Include a news file with your PR
(https://xon.sh/devguide.html#changelog).
2. Add the documentation for your feature into `/docs`.
3. Add the example of usage or before-after behavior.
4. Mention the issue that this PR is addressing e.g. `#1234`.
-->
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
### Motivation
* We have no recommended way to force subprocess command be
(un)threadable
* #4214
* #2119
* #5003
* It's interesting opportunity to have a way to modify specs and CP
using `SpecModifierAlias`.
### Before
```xsh
!(ssh host -T "echo 1")
# output='' # EXPECTED: 1
__xonsh__.commands_cache.threadable_predictors['ssh'] = lambda *a, **kw: True
!(ssh host -T "echo 1")
```
### After
```xsh
xthread
# Mark command as threadable.
!(xthread ssh host -T "echo 1")
# output='1'
```
Closes:
* Closes#4214
* Closes#2119
* Partially closes#5003
Implementation of `SpecModifierAlias` will help in:
* #2618
JFYI #5413
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Gil Forsyth <gforsyth@users.noreply.github.com>
### Motivation
Closes#5003
### The case
Core utils in Nix are symlinks to one binary file that contains all
utils:
```xsh
docker run --rm -it nixos/nix bash
which echo
# /nix/store/k6h0vjh342kqlkq69sxjj8i5y50l6jfr-coreutils-9.3/bin/echo
ls -la /nix/store/k6h0vjh342kqlkq69sxjj8i5y50l6jfr-coreutils-9.3/bin/
# b2sum -> coreutils
# base32 -> coreutils
# ...
# All tools are symlinks to one binary file - `coreutils`.
```
When
[`default_predictor_readbin`](61bda708c9/xonsh/commands_cache.py (L392))
read `coreutils` it catches `(b'isatty', b'tcgetattr', b'tcsetattr')`
and return `threadable=False` forever.
The list of Nix coreutils tools are exactly the same as in [brew
coreutils](https://formulae.brew.sh/formula/coreutils). So I can check
the real predicts on distinct binaries and see that only 2 tools among
106 are unthreadable and they already covered by
`default_threadable_predictors` (If it's wrong please add unthreadable
tools to the
[default_threadable_predictors](61bda708c9/xonsh/commands_cache.py (L518))):
```xsh
brew install coreutils
ls /opt/homebrew/opt/coreutils/libexec/gnubin/ | wc -l
# 106
for t in p`/opt/homebrew/opt/coreutils/libexec/gnubin/.*`:
if not (th := __xonsh__.commands_cache.predict_threadable([t.name])):
print($(which @(t.name)), th)
# /opt/homebrew/opt/coreutils/libexec/gnubin/cat False
# /opt/homebrew/opt/coreutils/libexec/gnubin/yes False
defaults = __import__('xonsh').commands_cache.default_threadable_predictors().keys()
defaults['cat']
# <function xonsh.commands_cache.predict_false>
defaults['yes']
# <function xonsh.commands_cache.predict_false>
```
So the rest of we need is to check the symlink and apply default
prediction if the tools is the symlink to coreutils. This implements
this PR. Test included.
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Reading stop signals from the process and update the process state.
### The issue
Technically. In a couple of places that critical for processing signals
we have `os.waitpid()`. The function behavior is pretty unobvious and
one of things is processing return code after catching the signal. We
had no good signal processing around this and this PR fixes this. See
also `proc_untraced_waitpid` function description.
From user perspective. For example we have process that is waiting for
user input from terminal e.g. `python -c "input()"` or `fzf`. If this
process will be in captured pipeline e.g. `!(echo 1 | fzf | head)` it
will be suspended by OS and the pipeline will be in the endless loop
with future crashing and corrupting std at the end. This PR fixes this.
### The solution
Technically. The key function is `proc_untraced_waitpid` - it catches
the stop signals and updates the process state.
From user perspective. First of all we expect that users will use
captured object `!()` only for capturable processes. Because of it our
goal here is to just make the behavior in this case stable.
In this PR we detect that process in the pipeline is suspended and we
need to finish the command pipeline carefully:
* Show the message about suspended process.
* Keep suspended process in `jobs`. The same behavior we can see in
bash. This is good because we don't know what process suspended and why.
May be experienced user will want to continue it manually.
* Finish the CommandPipeline with returncode=None and suspended=True.
### Before
```xsh
!(fzf) # or !(python -c "input()")
# Hanging / Exceptions / OSError / No way to end the command.
# After exception:
$(echo 1)
# OSError / IO error
```
### After
```xsh
!(fzf) # or `!(ls | fzf | head)` or `!(python -c "input()")`
# Process ['fzf'] with pid 60000 suspended with signal 22 SIGTTOU and stay in `jobs`.
# This happends when process start waiting for input but there is no terminal attached in captured mode.
# CommandPipeline(returncode=None, suspended=True, ...)
$(echo 1)
# Success.
```
Closes#4752#4577
### Notes
* There is pretty edge case situation when the process was terminated so
fast that we can't catch pid alive and check signal
([src](67d672783d/xonsh/jobs.py (L71-L80))).
I leave it as is for now.
### Mentions
#2159
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Gil Forsyth <gforsyth@users.noreply.github.com>
### Motivation
Closes#4636
In fact`Env` manages two lists: explicitly set values and Xettings. When
we operate with env we have no detyping for Xettings. The `detype_all`
function doint detyping for all possible env variables.
### Before
```xsh
# No way to get all detyped variables e.g.
'PROMPT' in __xonsh__.env.detype()
# False
```
### After
```xsh
'PROMPT' in __xonsh__.env.detype_all()
# True
```
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
### Before
```xsh
echo 1
echo 2
# <<Crash>>
# No json history
```
### After
Closes#2657
```xsh
echo 1
echo 2
# <<Crash>>
# In json history
# echo 1
# echo 2
```
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
### Before
Case 1: Handler catches the exit signal and do not pass it forward. As
result xonsh could be suspended by OS or crash. Also exit code is wrong.
See repeatable examples with SIGHUP in
https://github.com/xonsh/xonsh/issues/5381#issuecomment-2097961804.
Case 2: From bash/zsh as login shell run xonsh. Then send quit signal to
xonsh. The terminal state will be broken: disabled SIGINT, mouse pointer
produces mouse state codes.
### After
Case 1: Xonsh exit normally with right exit code. Fixed#5381#5304#5371.
Case 2: After exiting with right exit code the state of the terminal is
good.
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
### Before
Case 1:
```xsh
xonsh --no-rc --no-env -DCOLOR_OUTPUT 0 # space between -DVAR and VALUE
```
```xsh
Traceback (most recent call last):
File "/Users/pc/.local/xonsh-env/lib/python3.12/site-packages/xonsh/main.py", line 478, in main
args = premain(argv)
^^^^^^^^^^^^^
File "/Users/pc/.local/xonsh-env/lib/python3.12/site-packages/xonsh/main.py", line 420, in premain
env.update([x.split("=", 1) for x in args.defines])
File "<frozen _collections_abc>", line 987, in update
ValueError: not enough values to unpack (expected 2, got 1)
Xonsh encountered an issue during launch
```
Case 2:
```xsh
xonsh --no-rc --no-env -DXONSH_HISTORY_BACKEND=dummy
history info
# json # EXPECTED: dummy
```
### After
Case 1:
```xsh
xonsh --no-rc --no-env -DCOLOR_OUTPUT 0
# Wrong format for -DCOLOR_OUTPUT argument. Use -DVAR=VAL form.
```
Case 2:
```xsh
xonsh --no-rc --no-env -DXONSH_HISTORY_BACKEND=dummy
history info
# dummy
```
Closes#5395
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Fixed#5387
### Before
```xsh
$XONSH_SHOW_TRACEBACK = False
$RAISE_SUBPROC_ERROR = True
ls nofile
# ls: nofile: No such file or directory
# Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
#
```
### After
```xsh
$RAISE_SUBPROC_ERROR = False
$XONSH_SHOW_TRACEBACK = False
ls nofile
# ls: nofile: No such file or directory
$RAISE_SUBPROC_ERROR = True
$XONSH_SHOW_TRACEBACK = False
ls nofile
# ls: nofile: No such file or directory
# subprocess.CalledProcessError: Command '['ls', 'nofile']' returned non-zero exit status 1.
$RAISE_SUBPROC_ERROR = True
$XONSH_SHOW_TRACEBACK = True
ls nofile
# ls: nofile: No such file or directory
# Traceback (most recent call last):
# ...
# subprocess.CalledProcessError: Command '['ls', 'nofile']' returned non-zero exit status 1.
$RAISE_SUBPROC_ERROR = False
$XONSH_SHOW_TRACEBACK = True
ls nofile
# ls: nofile: No such file or directory
```
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
### Motivation
It's very annoying to search env variable exactly by lprefix. The better
is to search by substring and sort results by the position of substring
and then alphabetically.
Closes#5386
### Before
```xsh
$TRA<Tab>
# nothing
```
### After
```xsh
$TRA<Tab>
# 'XONSH_TRACE_COMPLETIONS',
# 'XONSH_TRACE_SUBPROC',
# 'XONSH_TRACE_SUBPROC_FUNC',
# 'XONSH_TRACEBACK_LOGFILE',
# 'XONSH_SHOW_TRACEBACK',
# 'VC_GIT_INCLUDE_UNTRACKED'
```
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
### Motivation
There is annoying behavior when you run command in the loop and can't
interrupt e.g. [this
report](https://github.com/xonsh/xonsh/discussions/5371) and a bit
#5342. After diving into this I see the issue around return code.
### The essence
Basically ``p = subprocess.Popen()`` populates ``p.returncode`` after
``p.wait()``, ``p.poll()`` or ``p.communicate()``
([doc](https://docs.python.org/3/library/os.html#os.waitpid)). But if
you're using `os.waitpid()` BEFORE these functions you're capturing
return code from a signal subsystem and ``p.returncode`` will be ``0``
like success but it's not success. So after ``os.waitid`` call you need
to set return code manually ``p.returncode = -os.WTERMSIG(status)`` like
in Popen. Example:
```xsh
python # python interactive
import os, signal, subprocess as sp
p = sp.Popen(['sleep', '100'])
p.pid
# 123
p.wait()
# Ctrl+C or `kill -SIGINT 123` from another terminal
p.returncode
# -2
# BUT:
p = sp.Popen(['sleep', '100'])
p.pid
# 123
pid, status = os.waitpid(p.pid, os.WUNTRACED)
# status=2
# From another terminal:
kill -SIGINT 123
p.wait()
# 0
p.returncode
# 0
```
```xsh
from xonsh.tools import describe_waitpid_status
describe_waitpid_status(2)
# WIFEXITED - False - Return True if the process returning status exited via the exit() system call.
# WEXITSTATUS - 0 - Return the process return code from status.
# WIFSIGNALED - True - Return True if the process returning status was terminated by a signal.
# WTERMSIG - 2 - Return the signal that terminated the process that provided the status value.
# WIFSTOPPED - False - Return True if the process returning status was stopped.
# WSTOPSIG - 0 - Return the signal that stopped the process that provided the status value.
```
See also: [Helpful things for
knight](https://github.com/xonsh/xonsh/pull/5361#issuecomment-2078826181).
### Before
```xsh
$RAISE_SUBPROC_ERROR = True
sleep 100
# Ctrl+C
_.rtn
# 0 # It's wrong and RAISE_SUBPROC_ERROR ignored.
for i in range(5):
print(i)
sleep 5
# 0
# Ctrl+C # Can't interrupt
# 1
# 2
```
### After
```xsh
sleep 100
# Ctrl+C
_.rtn
# -2 # Like in default Popen
$RAISE_SUBPROC_ERROR = True
for i in range(5):
print(i)
sleep 5
# 0
# Ctrl+C
# subprocess.CalledProcessError
```
### Notes
* We need to refactor `xonsh.jobs`. It's pretty uncomfortable work with
module.
* The logic is blurry between Specs, Pipelines and Jobs. We need to
bring more clear functions.
* The `captured` variable looks like "just the way of capturing (stdout,
object)" but in fact it affects all logic very much. We need to create
table where we can see the logic difference for every type of capturing.
E.g. in `captured=stdout` mode we use `xonsh.jobs` to monitor the
command but in `captured=object` we avoid this and some logic from
`xonsh.jobs` applied in `stdout` mode but missed in `object` mode. We
need clear map.
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
### Motivation
* To have an ability to manage the output format added ``$XONSH_SUBPROC_OUTPUT_FORMAT`` to switch the way to return the output lines. Default ``stream_lines`` to return text. Alternative ``list_lines`` to return the list of lines. Also supported custom lambda function.
* Additionally the [proposal to change default behavior](https://github.com/xonsh/xonsh/pull/5377#discussion_r1587627131) for a single line case.
* Closes#3924 as soft solution.
### Before
```xsh
mkdir -p /tmp/tst && cd /tmp/tst && touch 1 2 3
$(ls)
# '1\n2\n3\n'
id $(whoami)
# id: ‘pc\n’: no such user: Invalid argument
du $(ls)
# du: cannot access '1'$'\n''2'$'\n''3'$'\n': No such file or directory
ls $(fzf)
# ls: cannot access 'FUNDING.yml'$'\n': No such file or directory
```
### After
```xsh
mkdir -p /tmp/tst && cd /tmp/tst && touch 1 2 3
$XONSH_SUBPROC_OUTPUT_FORMAT = 'list_lines'
$(ls)
# ['1', '2', '3']
[f for f in $(ls)]
# ['1', '2', '3']
id $(whoami)
# uid=501(user) gid=20(staff)
du $(ls)
# 0 1
# 0 2
# 0 3
ls $(fzf)
# FUNDING.yml
# etc
mkdir -p /tmp/@($(whoami))/dir
cat /etc/passwd | grep $(whoami)
```
### Notes
* It will be good to improve parser for cases like `mkdir -p /tmp/$(whoami)/dir`. PR is welcome!
* I named the default mode as `stream_lines` (instead of just `stream` or `raw`) because in fact we transform raw output into stream of lines and possibly reduce the length of output ([replacing `\r\n` to `\n`](c3a12b2a9c/xonsh/procs/pipelines.py (L380-L383))). May be some day we need to add raw "stream" output format.
* Now anybody can implement bash `IFS` behavior in [bashisms](https://github.com/xonsh/xontrib-bashisms).
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix parsing of redirect tokens
In the parser, wrap redirect tokens such as ">", "a>", and "2>1" in a
tuple (along with the name of the redirect file, if applicable) to
differentiate them from regular command line arguments.
This ensures that a command like ![echo ">"] is parsed correctly.
We now need 2 types of IO-redirect tokens (IOREDIRECT1 and IOREDIRECT2)
so that the parser knows whether or not an IO-redirect has a file
argument. For example, "a>" has type IOREDIRECT1, whereas "2>1" has
type IOREDIRECT2.
* update ioredir tests
* add tests
* add news
* fix multiple redirections error message
* add tests
* Fix incorrect IOREDIRECT tokens in Python mode
* add news
* Update lexer used in CompletionContextParser
The completion context lexer should start in subprocess mode so that
operators like ``2>`` are recognized as IOREDIRECT tokens.
This fixes the test case "command 2>/dev/nul{X}" for
test_completion_context.py::test_command.
`os.path.expanduser` relies on the value of `$HOME` set in `os.environ`.
This meant that a user who set `$HOME` would then not end up in that
directory with a no-argument call to `cd` unless they had set
`UPDATE_OS_ENVIRON`.
Now the value of `$HOME` is respected regardless of the setting in `UPDATE_OS_ENVIRON`
