### Motivation
I noticed that when I type every letter in the prompt on the remote
server the typing is lagging. I thought it's ssh but no.
After inspection the commands cache code I found that on every key press
there is reading all 2000+ files in PATH because the code for cache
update has funny issue.
### Quiz
```xsh
def f():
print(1)
yield False
print(2)
yield True
print(3)
print(any(f()))
```
What will be in the output? Answer:
<details>
```
1
2
True
```
The execution of `print(3)` ignored because `any()` interrupts the
execution of the function.
If we call `print(list(f()))` the output will be `1 2 3 [False, True]`.
</details>
### Before
* Updating cache (read all files in PATH and list of aliases) two times
after start.
* Updating cache (read all files in PATH and list of aliases) on every
key press in prompt.
### After
* Update cache once at start.
* Update cache only when we have real changes of paths or aliases.
cc #4954#3895#5309
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
### Motivation
Closes#5003
### The case
Core utils in Nix are symlinks to one binary file that contains all
utils:
```xsh
docker run --rm -it nixos/nix bash
which echo
# /nix/store/k6h0vjh342kqlkq69sxjj8i5y50l6jfr-coreutils-9.3/bin/echo
ls -la /nix/store/k6h0vjh342kqlkq69sxjj8i5y50l6jfr-coreutils-9.3/bin/
# b2sum -> coreutils
# base32 -> coreutils
# ...
# All tools are symlinks to one binary file - `coreutils`.
```
When
[`default_predictor_readbin`](61bda708c9/xonsh/commands_cache.py (L392))
read `coreutils` it catches `(b'isatty', b'tcgetattr', b'tcsetattr')`
and return `threadable=False` forever.
The list of Nix coreutils tools are exactly the same as in [brew
coreutils](https://formulae.brew.sh/formula/coreutils). So I can check
the real predicts on distinct binaries and see that only 2 tools among
106 are unthreadable and they already covered by
`default_threadable_predictors` (If it's wrong please add unthreadable
tools to the
[default_threadable_predictors](61bda708c9/xonsh/commands_cache.py (L518))):
```xsh
brew install coreutils
ls /opt/homebrew/opt/coreutils/libexec/gnubin/ | wc -l
# 106
for t in p`/opt/homebrew/opt/coreutils/libexec/gnubin/.*`:
if not (th := __xonsh__.commands_cache.predict_threadable([t.name])):
print($(which @(t.name)), th)
# /opt/homebrew/opt/coreutils/libexec/gnubin/cat False
# /opt/homebrew/opt/coreutils/libexec/gnubin/yes False
defaults = __import__('xonsh').commands_cache.default_threadable_predictors().keys()
defaults['cat']
# <function xonsh.commands_cache.predict_false>
defaults['yes']
# <function xonsh.commands_cache.predict_false>
```
So the rest of we need is to check the symlink and apply default
prediction if the tools is the symlink to coreutils. This implements
this PR. Test included.
## For community
⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍
comment**
---------
Co-authored-by: a <1@1.1>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* todo:
* refactor: remove usage of singleton in cmd-cache
* refactor: pass aliases/env to commands_cache on creation
reduce the usage of singletons
* fix: setting up mockups for tests
* feat: add $XONSH_CACHE_DIR
* refactor: use cache path to stire cmds-cache
* chore: todo
* feat: efficient cache commands per path and modified time
* refactor: move aliases to commands cache
now XonshSession is not needed to setup cmds-cache
* refactor: update tests
* fix: type annotation
* refactor: init aliases at commands-cache
* test: update failing tests
* fix: handle paths mtime changing
* docs: add news item
* refactor: remove $COMMANDS_CACHE_SIZE_WARNING
* fix: loading on windows fails because of setup sequence
* fix: failing tests
* todo
* test: remove usage of DummyEnv and setting .env attribute on xession fixture
one step closer to making too much of tweaking to xession during tests
* test: fix tests vox and gitstatus-prompt
* docs: update test-fixture usage
* fix: import flake8 error
* test: remove direct access to XSH in tests
* test: remove usage of XSH in test files
* todo
* test: use tmp-dir to create stubs
* refactor: use fixture factory to mock out XonshSession
* refactor: remove direct access of XSH from functions
* refactor: remove direct access of XSH from functions
* fix: qa checks
* refactor: rename variables to match their values
* test: update failing tests because it had no PATH set previously
* fix: remove builtins usage from pyghooks.py
* style:
* refactor: update tests to use fixtures
* fix: env varialbe is setup per function
some tests accidentally update the env variables and that is leaking
into next tests
* fix: failing vox tests
* test: set commands_cache per test
* test: fix failing tests
* fix: failing tests on linux
ptk-highlight
* fix: failing tests on Windows
cwd-prompt
* test: copy env as to not affect original object
* fix: lazily evaluate cmds-cache in pyghooks
* test: fix failing tests
* fix: qa errors import
* test: set commands-cache per test
while caching path results
* test: speedup test_thread_local_swap
* fix: failing tests on windows
* refactor: Execer doesn't control session
* refactor: XSH.unload will take care of reversing builtins attrs set
* test: use env.update over monkeypatch
* Revert "test: use env.update over monkeypatch"
This reverts commit 010a5022247a098f1741966b8af1bf758663480e.
* refactor: remove usage of global variables in abbrevs.py
* chore: add flake8-mutable to prevent mutable defaults
* fix: abbrevs expand test
* refactor: add xonsh session singleton
* refactor: fix circular errors when using xonshSession as singleton
* refactor: remove black magicked builtin attributes
* style: black format tests as well
* refactor: update tests to use xonsh-session singleton
* refactor: update abbrevs to not use builtins
* test: remove DummyCommandsCache and patch orig class
* fix: failing test_command_completers
* test: use monkeypatch to update xession fixture
* fix: failing test_pipelines
* fix: failing test_main
* chore: run test suit as single invocation
* test: fix tests/test_xonsh.xsh
* refactor: remove builtins from docs/conf.py
* fix: mypy error in jobs
* fix: test error from test_main
* test: close xession error in test_command_completers
* chore: use pytest-cov for reporting coverage
this will include subprocess calls, and will increase coverage
* style:
* fix: update failing commands-cache tests
in some cases, the thread updates the binaries faster than the next call
* chore: upgrade black to 21.5b0
* chore: upgrade mypy to 0.812 version
* fix: handle corrupt commands-cache file
* fix: parser-table output dir. this should be same as the defaults
* fix: flake8 error
* feat: use saved commands_cache between runs
this will speed-up startup time by a small factor (around 0.15s on my
system with ssd and ryzen5)
* test: update env-variable handling for cmds-cache tests
* refactor: add env-variable to control saving commands-cache
* refactor: mypy error fix for commands_cache
* fix: update handling max(iterable
* test: add tests for commands-cache new version
* docs: update env variable doc
* test: add more test for commands-cache caching behaviour
