mirrors/xonsh
1
0
Fork 0
mirror of https://github.com/xonsh/xonsh.git synced 2025-03-04 16:34:47 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge branch 'master' into linecont

Browse source
This commit is contained in:
Anthony Scopatz 2017-02-17 02:00:16 -05:00
commit 67d61ed0c2
41 changed files with 795 additions and 262 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.appveyor.yml
View file

@ -1,4 +1,4 @@
version: 0.5.4.{build}
version: 0.5.5.{build}
os: Windows Server 2012 R2
environment:

2
.circle.yml
View file

@ -24,3 +24,5 @@ test:
override:
- case $CIRCLE_NODE_INDEX in 0) py.test --flake8 --timeout=10 --cov=./xonsh;; 1) py.test --timeout=10 ;; esac:
parallel: true
post:
- if [[ $CIRCLE_NODE_INDEX -eq 0 ]]; then codecov; fi

56
CHANGELOG.rst
View file

@ -4,9 +4,63 @@ Xonsh Change Log
.. current developments
v0.5.4
v0.5.5
====================
**Added:**
* New ``--rc`` command line option allows users to specify paths to run control
files from the command line. This includes both xonsh-based and JSON-based
configuration.
* New ``$UPDATE_COMPLETIONS_ON_KEYPRESS`` controls whether or not completions
will automatically display and update while typing. This feature is only
available in the prompt-toolkit shell.
**Changed:**
* Xonsh scripts now report ``__file__`` and ``__name__`` when run as scripts
or sourced. These variables have the same meaning as they do in Python
scripts.
* ``$XONSHRC`` and related configuration variables now accept JSON-based
static configuration file names as elements. This unifies the two methods
of run control to a single entry point and loading system.
* The ``xonsh.shell.Shell()`` class now requires that an Execer instance
be explicitly provided to its init method. This class is no longer
responsible for creating an execer an its deprendencies.
* Moved decorators ``unthreadable``, ``uncapturable`` from
``xonsh.proc`` to ``xonsh.tools``.
* Some refactorings on jobs control.
**Deprecated:**
* The ``--config-path`` command line option is now deprecated in favor of
``--rc``.
**Removed:**
* ``xonsh.environ.DEFAULT_XONSHRC`` has been removed due to deprecation.
For this value, please check the environment instead, or call
``xonsh.environ.default_xonshrc(env)``.
**Fixed:**
* Command pipelines that end in a callable alias are now interruptable with
``^C`` and the processes that are piped into the alais have their file handles
closed. This should ensure that the entire pipeline is closed.
* Fixed issue where unthreadable subprocs were not allowed to be
captured with the ``$(cmd)`` operator.
* The ``ProcProxy`` class (unthreadable aliases) was not being executed and would
hange if the alias was capturable. This has been fixed.
* Fixed a ``tcsetattr: Interrupted system call`` issue when run xonsh scripts.
* Fixed issue with ``ValueError`` being thrown from ``inspect.signature()``
when called on C-extention callables in tab completer.
* Fixed issue that ``ls | less`` crashes on Mac.
* Threadable prediction was incorrectly based on the user input command, rather than
the version where aliases have been resolved. This has been corrected.
v0.5.4

1
docs/api/index.rst
View file

@ -43,6 +43,7 @@ For those of you who want the gritty details.
pretty
replay
diff_history
xoreutils/index
**Helpers:**

10
docs/api/xoreutils/cat.rst Normal file
View file

@ -0,0 +1,10 @@
.. _xonsh_xoreutils_cat:
===============================================
Cat Command -- :mod:`xonsh.xoreutils.cat`
===============================================
.. currentmodule:: xonsh.xoreutils.cat
.. automodule:: xonsh.xoreutils.cat
:members:

10
docs/api/xoreutils/echo.rst Normal file
View file

@ -0,0 +1,10 @@
.. _xonsh_xoreutils_echo:
===============================================
Echo Command -- :mod:`xonsh.xoreutils.echo`
===============================================
.. currentmodule:: xonsh.xoreutils.echo
.. automodule:: xonsh.xoreutils.echo
:members:

24
docs/api/xoreutils/index.rst Normal file
View file

@ -0,0 +1,24 @@
.. _api_xoreutils:
==================
Core Utilities API
==================
**Command Modules:**
.. toctree::
:maxdepth: 1
cat
echo
pwd
tee
tty
which
**Helper Modules:**
.. toctree::
:maxdepth: 1
uptime
util

10
docs/api/xoreutils/pwd.rst Normal file
View file

@ -0,0 +1,10 @@
.. _xonsh_xoreutils_pwd:
===============================================
Pwd Command -- :mod:`xonsh.xoreutils.pwd`
===============================================
.. currentmodule:: xonsh.xoreutils.pwd
.. automodule:: xonsh.xoreutils.pwd
:members:

10
docs/api/xoreutils/tee.rst Normal file
View file

@ -0,0 +1,10 @@
.. _xonsh_xoreutils_tee:
===============================================
Tee Command -- :mod:`xonsh.xoreutils.tee`
===============================================
.. currentmodule:: xonsh.xoreutils.tee
.. automodule:: xonsh.xoreutils.tee
:members:

10
docs/api/xoreutils/tty.rst Normal file
View file

@ -0,0 +1,10 @@
.. _xonsh_xoreutils_tty:
===============================================
TTY Command -- :mod:`xonsh.xoreutils.tty`
===============================================
.. currentmodule:: xonsh.xoreutils.tty
.. automodule:: xonsh.xoreutils.tty
:members:

10
docs/api/xoreutils/uptime.rst Normal file
View file

@ -0,0 +1,10 @@
.. _xonsh_xoreutils_uptime:
===============================================
System Uptime -- :mod:`xonsh.xoreutils.uptime`
===============================================
.. currentmodule:: xonsh.xoreutils.uptime
.. automodule:: xonsh.xoreutils.uptime
:members:

10
docs/api/xoreutils/util.rst Normal file
View file

@ -0,0 +1,10 @@
.. _xonsh_xoreutils_util:
======================================================
Core Utilites Utilities -- :mod:`xonsh.xoreutils.util`
======================================================
.. currentmodule:: xonsh.xoreutils.util
.. automodule:: xonsh.xoreutils.util
:members:

10
docs/api/xoreutils/which.rst Normal file
View file

@ -0,0 +1,10 @@
.. _xonsh_xoreutils_which:
===============================================
Which Command -- :mod:`xonsh.xoreutils.which`
===============================================
.. currentmodule:: xonsh.xoreutils.which
.. automodule:: xonsh.xoreutils.which
:members:

2
docs/events.rst
View file

@ -1,5 +1,5 @@
Core Events
===========
The following events are defined by xonsh itself.
The following events are defined by xonsh itself. For more information about events, see `the events tutorial <tutorial_events.html>`_.
.. include:: eventsbody

6
docs/tutorial_events.rst
View file

@ -45,8 +45,10 @@ Yes! It's even easy! In your xontrib, you just have to do something like::
This will enable users to call ``help(events.myxontrib_on_spam)`` and get useful output.
Under the Hood
==============
Further Reading
===============
For a complete list of available events, see `the events reference <events.html>`_.
If you want to know more about the gory details of what makes events tick, see
`Advanced Events <advanced_events.html>`_.

14
news/capunthread.rst
View file

@ -1,14 +0,0 @@
**Added:** None
**Changed:** None
**Deprecated:** None
**Removed:** None
**Fixed:**
* Fixed issue where unthreadable subprocs were not allowed to be
captured with the ``$(cmd)`` operator.
**Security:** None

13
news/codecov.rst Normal file
View file

@ -0,0 +1,13 @@
**Added:**
* CircleCI test post codecov run
**Changed:** None
**Deprecated:** None
**Removed:** None
**Fixed:** None
**Security:** None

15
news/compup.rst
View file

@ -1,15 +0,0 @@
**Added:**
* New ``$UPDATE_COMPLETIONS_ON_KEYPRESS`` controls whether or not completions
will automatically display and update while typing. This feature is only
available in the prompt-toolkit shell.
**Changed:** None
**Deprecated:** None
**Removed:** None
**Fixed:** None
**Security:** None

15
news/file.rst
View file

@ -1,15 +0,0 @@
**Added:** None
**Changed:**
* Xonsh scripts now report ``__file__`` and ``__name__`` when run as scripts
or sourced. These variables have the same meaning as they do in Python
scripts.
**Deprecated:** None
**Removed:** None
**Fixed:** None
**Security:** None

29
news/lr.rst
View file

@ -1,29 +0,0 @@
**Added:**
* New ``--rc`` command line option allows users to specify paths to run control
files from the command line. This includes both xonsh-based and JSON-based
configuration.
**Changed:**
* ``$XONSHRC`` and related configuration variables now accept JSON-based
static configuration file names as elements. This unifies the two methods
of run control to a single entry point and loading system.
* The ``xonsh.shell.Shell()`` class now requires that an Execer instance
be explicitly provided to its init method. This class is no longer
responsible for creating an execer an its deprendencies.
**Deprecated:**
* The ``--config-path`` command line option is now deprecated in favor of
``--rc``.
**Removed:**
* ``xonsh.environ.DEFAULT_XONSHRC`` has been removed due to deprecation.
For this value, please check the environment instead, or call
``xonsh.environ.default_xonshrc(env)``.
**Fixed:** None
**Security:** None

14
news/move-tools.rst
View file

@ -1,14 +0,0 @@
**Added:** None
**Changed:**
* Moved decorators ``unthreadable``, ``uncapturable`` from
``xonsh.proc`` to ``xonsh.tools``.
**Deprecated:** None
**Removed:** None
**Fixed:** None
**Security:** None

14
news/ppunth.rst
View file

@ -1,14 +0,0 @@
**Added:** None
**Changed:** None
**Deprecated:** None
**Removed:** None
**Fixed:**
* The ``ProcProxy`` class (unthreadable aliases) was not being executed and would
hange if the alias was capturable. This has been fixed.
**Security:** None

14
news/valerrargcomp.rst
View file

@ -1,14 +0,0 @@
**Added:** None
**Changed:** None
**Deprecated:** None
**Removed:** None
**Fixed:**
* Fixed issue with ``ValueError`` being thrown from ``inspect.signature()``
when called on C-extention callables in tab completer.
**Security:** None

13
news/xonsh-script-fix.rst
View file

@ -1,13 +0,0 @@
**Added:** None
**Changed:** None
**Deprecated:** None
**Removed:** None
**Fixed:**
* Fixed a ``tcsetattr: Interrupted system call`` issue when run xonsh scripts.
**Security:** None

16
news/xoreutils.rst Normal file
View file

@ -0,0 +1,16 @@
**Added:**
* New core utility function aliases (written in pure Python) are now
available in ``xonsh.xoreutils``. These include: ``cat``, ``echo``,
``pwd``, ``tee``, ``tty``, and ``yes``. These are not enabled by default.
Use the new ``coreutils`` xontrib to load them.
**Changed:** None
**Deprecated:** None
**Removed:** None
**Fixed:** None
**Security:** None

16
tests/test_xonsh.xsh
View file

@ -1,3 +1,6 @@
import builtins
def test_simple():
assert 1 + 1 == 2
@ -10,7 +13,12 @@ def test_envionment():
def test_xonsh_party():
x = 'xonsh'
y = 'party'
out = $(echo @(x + '-' + y)).strip()
assert out == 'xonsh-party', 'Out really was <' + out + '>, sorry.'
orig = builtins.__xonsh_env__.get('XONSH_INTERACTIVE')
builtins.__xonsh_env__['XONSH_INTERACTIVE'] = False
try:
x = 'xonsh'
y = 'party'
out = $(echo @(x + '-' + y)).strip()
assert out == 'xonsh-party', 'Out really was <' + out + '>, sorry.'
finally:
builtins.__xonsh_env__['XONSH_INTERACTIVE'] = orig

10
xonsh/__init__.py
View file

@ -1,4 +1,4 @@
__version__ = '0.5.4'
__version__ = '0.5.5'
# amalgamate exclude jupyter_kernel parser_table parser_test_table pyghooks
@ -26,8 +26,6 @@ else:
_sys.modules['xonsh.platform'] = __amalgam__
pretty = __amalgam__
_sys.modules['xonsh.pretty'] = __amalgam__
jobs = __amalgam__
_sys.modules['xonsh.jobs'] = __amalgam__
lazyimps = __amalgam__
_sys.modules['xonsh.lazyimps'] = __amalgam__
parser = __amalgam__
@ -48,18 +46,20 @@ else:
_sys.modules['xonsh.events'] = __amalgam__
foreign_shells = __amalgam__
_sys.modules['xonsh.foreign_shells'] = __amalgam__
jobs = __amalgam__
_sys.modules['xonsh.jobs'] = __amalgam__
lexer = __amalgam__
_sys.modules['xonsh.lexer'] = __amalgam__
openpy = __amalgam__
_sys.modules['xonsh.openpy'] = __amalgam__
proc = __amalgam__
_sys.modules['xonsh.proc'] = __amalgam__
xontribs = __amalgam__
_sys.modules['xonsh.xontribs'] = __amalgam__
dirstack = __amalgam__
_sys.modules['xonsh.dirstack'] = __amalgam__
inspectors = __amalgam__
_sys.modules['xonsh.inspectors'] = __amalgam__
proc = __amalgam__
_sys.modules['xonsh.proc'] = __amalgam__
shell = __amalgam__
_sys.modules['xonsh.shell'] = __amalgam__
timings = __amalgam__

40
xonsh/built_ins.py
View file

@ -8,7 +8,6 @@ import io
import os
import re
import sys
import time
import types
import shlex
import signal
@ -495,6 +494,8 @@ class SubprocSpec:
self.prep_env(kwargs)
self.prep_preexec_fn(kwargs, pipeline_group=pipeline_group)
if callable(self.alias):
if 'preexec_fn' in kwargs:
kwargs.pop('preexec_fn')
p = self.cls(self.alias, self.cmd, **kwargs)
else:
p = self._run_binary(kwargs)
@ -532,7 +533,7 @@ class SubprocSpec:
def prep_preexec_fn(self, kwargs, pipeline_group=None):
"""Prepares the 'preexec_fn' keyword argument"""
if not (ON_POSIX and self.cls is subprocess.Popen):
if not ON_POSIX:
return
if not builtins.__xonsh_env__.get('XONSH_INTERACTIVE'):
return
@ -675,7 +676,7 @@ def _update_last_spec(last):
if callable_alias:
pass
else:
thable = builtins.__xonsh_commands_cache__.predict_threadable(last.args)
thable = builtins.__xonsh_commands_cache__.predict_threadable(last.cmd)
if captured and thable:
last.cls = PopenThread
elif not thable:
@ -792,34 +793,27 @@ def run_subproc(cmds, captured=False):
"""
specs = cmds_to_specs(cmds, captured=captured)
captured = specs[-1].captured
procs = []
proc = pipeline_group = None
for spec in specs:
starttime = time.time()
proc = spec.run(pipeline_group=pipeline_group)
procs.append(proc)
if ON_POSIX and pipeline_group is None and \
spec.cls is subprocess.Popen:
pipeline_group = proc.pid
if not spec.is_proxy:
if captured == 'hiddenobject':
command = HiddenCommandPipeline(specs)
else:
command = CommandPipeline(specs)
proc = command.proc
background = command.spec.background
if not all(x.is_proxy for x in specs):
add_job({
'cmds': cmds,
'pids': [i.pid for i in procs],
'pids': [i.pid for i in command.procs],
'obj': proc,
'bg': spec.background,
'bg': background,
'pipeline': command,
'pgrp': command.term_pgid,
})
if _should_set_title(captured=captured):
# set title here to get currently executing command
pause_call_resume(proc, builtins.__xonsh_shell__.settitle)
# create command or return if backgrounding.
if spec.background:
if background:
return
if captured == 'hiddenobject':
command = HiddenCommandPipeline(specs, procs, starttime=starttime,
captured=captured)
else:
command = CommandPipeline(specs, procs, starttime=starttime,
captured=captured)
# now figure out what we should return.
if captured == 'stdout':
command.end()
@ -961,7 +955,7 @@ def convert_macro_arg(raw_arg, kind, glbs, locs, *, name='<arg>',
ctx |= set(locs.keys())
mode = mode or 'eval'
arg = execer.parse(raw_arg, ctx, mode=mode, filename=filename)
elif kind is types.CodeType or kind is compile:
elif kind is types.CodeType or kind is compile: # NOQA
mode = mode or 'eval'
arg = execer.compile(raw_arg, mode=mode, glbs=glbs, locs=locs,
filename=filename)

122
xonsh/jobs.py
View file

@ -6,12 +6,12 @@ import time
import ctypes
import signal
import builtins
import functools
import subprocess
import collections
from xonsh.lazyasd import LazyObject
from xonsh.platform import ON_DARWIN, ON_WINDOWS, ON_CYGWIN, LIBC
from xonsh.tools import unthreadable
tasks = LazyObject(collections.deque, globals(), 'tasks')
@ -28,7 +28,10 @@ if ON_DARWIN:
for pid in job['pids']:
if pid is None: # the pid of an aliased proc is None
continue
os.kill(pid, signal)
try:
os.kill(pid, signal)
except ProcessLookupError:
pass
elif ON_WINDOWS:
pass
elif ON_CYGWIN:
@ -67,7 +70,7 @@ if ON_WINDOWS:
def ignore_sigtstp():
pass
def _set_pgrp(info):
def give_terminal_to(pgid):
pass
def wait_for_active_job(last_task=None, backgrounded=False):
@ -101,65 +104,41 @@ else:
def ignore_sigtstp():
signal.signal(signal.SIGTSTP, signal.SIG_IGN)
def _set_pgrp(info):
pid = info['pids'][0]
if pid is None:
# occurs if first process is an alias
info['pgrp'] = None
return
try:
info['pgrp'] = os.getpgid(pid)
except ProcessLookupError:
info['pgrp'] = None
_shell_pgrp = os.getpgrp()
_block_when_giving = LazyObject(lambda: (signal.SIGTTOU, signal.SIGTTIN,
signal.SIGTSTP, signal.SIGCHLD),
globals(), '_block_when_giving')
# check for shell tty
@functools.lru_cache(1)
def _shell_tty():
try:
_st = sys.stderr.fileno()
if os.tcgetpgrp(_st) != os.getpgid(os.getpid()):
# we don't own it
_st = None
except OSError:
_st = None
return _st
# _give_terminal_to is a simplified version of:
# give_terminal_to is a simplified version of:
# give_terminal_to from bash 4.3 source, jobs.c, line 4030
# this will give the terminal to the process group pgid
if ON_CYGWIN:
# on cygwin, signal.pthread_sigmask does not exist in Python, even
# though pthread_sigmask is defined in the kernel. thus, we use
# ctypes to mimic the calls in the "normal" version below.
def _give_terminal_to(pgid):
st = _shell_tty()
if st is not None and os.isatty(st):
omask = ctypes.c_ulong()
mask = ctypes.c_ulong()
LIBC.sigemptyset(ctypes.byref(mask))
for i in _block_when_giving:
LIBC.sigaddset(ctypes.byref(mask), ctypes.c_int(i))
LIBC.sigemptyset(ctypes.byref(omask))
LIBC.sigprocmask(ctypes.c_int(signal.SIG_BLOCK),
ctypes.byref(mask),
ctypes.byref(omask))
LIBC.tcsetpgrp(ctypes.c_int(st), ctypes.c_int(pgid))
LIBC.sigprocmask(ctypes.c_int(signal.SIG_SETMASK),
ctypes.byref(omask), None)
def give_terminal_to(pgid):
omask = ctypes.c_ulong()
mask = ctypes.c_ulong()
LIBC.sigemptyset(ctypes.byref(mask))
for i in _block_when_giving:
LIBC.sigaddset(ctypes.byref(mask), ctypes.c_int(i))
LIBC.sigemptyset(ctypes.byref(omask))
LIBC.sigprocmask(ctypes.c_int(signal.SIG_BLOCK),
ctypes.byref(mask),
ctypes.byref(omask))
st = sys.stderr.fileno()
LIBC.tcsetpgrp(ctypes.c_int(st), ctypes.c_int(pgid))
LIBC.sigprocmask(ctypes.c_int(signal.SIG_SETMASK),
ctypes.byref(omask), None)
return True
else:
def _give_terminal_to(pgid):
st = _shell_tty()
if st is not None and os.isatty(st):
oldmask = signal.pthread_sigmask(signal.SIG_BLOCK,
_block_when_giving)
os.tcsetpgrp(st, pgid)
signal.pthread_sigmask(signal.SIG_SETMASK, oldmask)
def give_terminal_to(pgid):
oldmask = signal.pthread_sigmask(signal.SIG_BLOCK,
_block_when_giving)
os.tcsetpgrp(sys.stderr.fileno(), pgid)
signal.pthread_sigmask(signal.SIG_SETMASK, oldmask)
return True
def wait_for_active_job(last_task=None, backgrounded=False):
"""
@ -170,22 +149,14 @@ else:
active_task = get_next_task()
# Return when there are no foreground active task
if active_task is None:
_give_terminal_to(_shell_pgrp) # give terminal back to the shell
if backgrounded and hasattr(builtins, '__xonsh_shell__'):
# restoring sanity could probably be called whenever we return
# control to the shell. But it only seems to matter after a
# ^Z event. This *has* to be called after we give the terminal
# back to the shell.
builtins.__xonsh_shell__.shell.restore_tty_sanity()
return last_task
pgrp = active_task.get('pgrp', None)
obj = active_task['obj']
backgrounded = False
# give the terminal over to the fg process
if pgrp is not None:
_give_terminal_to(pgrp)
_continue(active_task)
_, wcode = os.waitpid(obj.pid, os.WUNTRACED)
try:
_, wcode = os.waitpid(obj.pid, os.WUNTRACED)
except ChildProcessError: # No child processes
return wait_for_active_job(last_task=active_task,
backgrounded=backgrounded)
if os.WIFSTOPPED(wcode):
print('^Z')
active_task['status'] = "stopped"
@ -262,7 +233,6 @@ def add_job(info):
num = get_next_job_number()
info['started'] = time.time()
info['status'] = "running"
_set_pgrp(info)
tasks.appendleft(num)
builtins.__xonsh_all_jobs__[num] = info
if info['bg']:
@ -307,9 +277,9 @@ def clean_jobs():
# newline
print()
print('xonsh: {}'.format(msg), file=sys.stderr)
print('-'*5, file=sys.stderr)
print('-' * 5, file=sys.stderr)
jobs([], stdout=sys.stderr)
print('-'*5, file=sys.stderr)
print('-' * 5, file=sys.stderr)
print('Type "exit" or press "ctrl-d" again to force quit.',
file=sys.stderr)
jobs_clean = False
@ -341,6 +311,7 @@ def jobs(args, stdin=None, stdout=sys.stdout, stderr=None):
return None, None
@unthreadable
def fg(args, stdin=None):
"""
xonsh command: fg
@ -354,32 +325,33 @@ def fg(args, stdin=None):
return '', 'Cannot bring nonexistent job to foreground.\n'
if len(args) == 0:
act = tasks[0] # take the last manipulated task by default
tid = tasks[0] # take the last manipulated task by default
elif len(args) == 1:
try:
if args[0] == '+': # take the last manipulated task
act = tasks[0]
tid = tasks[0]
elif args[0] == '-': # take the second to last manipulated task
act = tasks[1]
tid = tasks[1]
else:
act = int(args[0])
tid = int(args[0])
except (ValueError, IndexError):
return '', 'Invalid job: {}\n'.format(args[0])
if act not in builtins.__xonsh_all_jobs__:
if tid not in builtins.__xonsh_all_jobs__:
return '', 'Invalid job: {}\n'.format(args[0])
else:
return '', 'fg expects 0 or 1 arguments, not {}\n'.format(len(args))
# Put this one on top of the queue
tasks.remove(act)
tasks.appendleft(act)
tasks.remove(tid)
tasks.appendleft(tid)
job = get_task(act)
job = get_task(tid)
job['bg'] = False
job['status'] = "running"
print_one_job(act)
wait_for_active_job()
print_one_job(tid)
pipeline = job['pipeline']
pipeline.resume(job)
def bg(args, stdin=None):

7
xonsh/main.py
View file

@ -6,6 +6,7 @@ import enum
import argparse
import builtins
import contextlib
import signal
import traceback
from xonsh import __version__
@ -340,6 +341,12 @@ def main(argv=None):
def main_xonsh(args):
"""Main entry point for xonsh cli."""
if not ON_WINDOWS:
def func_sig_ttin_ttou(n, f):
pass
signal.signal(signal.SIGTTIN, func_sig_ttin_ttou)
signal.signal(signal.SIGTTOU, func_sig_ttin_ttou)
events.on_post_init.fire()
env = builtins.__xonsh_env__
shell = builtins.__xonsh_shell__

147
xonsh/proc.py
View file

@ -29,7 +29,7 @@ from xonsh.tools import (redirect_stdout, redirect_stderr, print_exception,
XonshCalledProcessError, findfirst, on_main_thread,
XonshError, format_std_prepost)
from xonsh.lazyasd import lazyobject, LazyObject
from xonsh.jobs import wait_for_active_job
from xonsh.jobs import wait_for_active_job, give_terminal_to, _continue
from xonsh.lazyimps import fcntl, termios, _winapi, msvcrt, winutils
# these decorators are imported for users back-compatible
from xonsh.tools import unthreadable, uncapturable # NOQA
@ -332,11 +332,11 @@ def populate_console(reader, fd, buffer, chunksize, queue, expandsize=None):
# I believe that there is a bug in PTK that if we reset the
# cursor position, the cursor on the next prompt is accidentally on
# the next line. If this is fixed, uncomment the following line.
#if max_offset < offset + expandsize:
# rows, max_offset, orig_posize = _expand_console_buffer(
# if max_offset < offset + expandsize:
# rows, max_offset, orig_posize = _expand_console_buffer(
# cols, max_offset, expandsize,
# orig_posize, fd)
# winutils.set_console_cursor_position(x, y, fd=fd)
# winutils.set_console_cursor_position(x, y, fd=fd)
while True:
posize = winutils.get_position_size(fd)
offset = (cols*y) + x
@ -838,7 +838,10 @@ class PopenThread(threading.Thread):
new[LFLAG] |= termios.ECHO | termios.ICANON
new[CC][termios.VMIN] = 1
new[CC][termios.VTIME] = 0
termios.tcsetattr(self.stdin_fd, termios.TCSANOW, new)
try:
termios.tcsetattr(self.stdin_fd, termios.TCSANOW, new)
except termios.error:
pass
#
# Dispatch methods
@ -1236,6 +1239,7 @@ class ProcProxyThread(threading.Thread):
self.stdout = stdout
self.stderr = stderr
self.env = env or builtins.__xonsh_env__
self._interrupted = False
if ON_WINDOWS:
if self.p2cwrite != -1:
@ -1263,9 +1267,19 @@ class ProcProxyThread(threading.Thread):
if universal_newlines:
self.stderr = io.TextIOWrapper(self.stderr)
# Set some signal handles, if we can. Must come before process
# is started to prevent deadlock on windows
self.old_int_handler = None
if on_main_thread():
self.old_int_handler = signal.signal(signal.SIGINT,
self._signal_int)
# start up the proc
super().__init__()
self.start()
def __del__(self):
self._restore_sigint()
def run(self):
"""Set up input/output streams and execute the child function in a new
thread. This is part of the `threading.Thread` interface and should
@ -1369,8 +1383,41 @@ class ProcProxyThread(threading.Thread):
def wait(self, timeout=None):
"""Waits for the process to finish and returns the return code."""
self.join()
self._restore_sigint()
return self.returncode
#
# SIGINT handler
#
def _signal_int(self, signum, frame):
"""Signal handler for SIGINT - Ctrl+C may have been pressed."""
# check if we have already be interrupted to prevent infintie recurrsion
if self._interrupted:
return
self._interrupted = True
# close file handles here to stop an processes piped to us.
handles = (self.p2cread, self.p2cwrite, self.c2pread, self.c2pwrite,
self.errread, self.errwrite)
for handle in handles:
safe_fdclose(handle)
if self.poll() is not None:
self._restore_sigint(frame=frame)
if on_main_thread():
signal.pthread_kill(threading.get_ident(), signal.SIGINT)
def _restore_sigint(self, frame=None):
old = self.old_int_handler
if old is not None:
if on_main_thread():
signal.signal(signal.SIGINT, old)
self.old_int_handler = None
if frame is not None:
if old is not None and old is not self._signal_int:
old(signal.SIGINT, frame)
if self._interrupted:
self.returncode = 1
# The code below (_get_devnull, _get_handles, and _make_inheritable) comes
# from subprocess.py in the Python 3.4.2 Standard Library
def _get_devnull(self):
@ -1554,7 +1601,11 @@ class ProcProxy(object):
if self.stdin is None:
stdin = None
else:
stdin = io.TextIOWrapper(self.stdin, encoding=enc, errors=err)
if isinstance(self.stdin, int):
inbuf = io.open(self.stdin, 'rb', -1)
else:
inbuf = self.stdin
stdin = io.TextIOWrapper(inbuf, encoding=enc, errors=err)
stdout = self._pick_buf(self.stdout, sys.stdout, enc, err)
stderr = self._pick_buf(self.stderr, sys.stderr, enc, err)
# run the actual function
@ -1629,6 +1680,17 @@ def safe_readable(handle):
return status
def update_fg_process_group(pipeline_group, background):
if background:
return False
if not ON_POSIX:
return False
env = builtins.__xonsh_env__
if not env.get('XONSH_INTERACTIVE'):
return False
return give_terminal_to(pipeline_group)
class CommandPipeline:
"""Represents a subprocess-mode command pipeline."""
@ -1639,18 +1701,12 @@ class CommandPipeline:
nonblocking = (io.BytesIO, NonBlockingFDReader, ConsoleParallelReader)
def __init__(self, specs, procs, starttime=None, captured=False):
def __init__(self, specs):
"""
Parameters
----------
specs : list of SubprocSpec
Process sepcifications
procs : list of Popen-like
Process objects.
starttime : floats or None, optional
Start timestamp.
captured : bool or str, optional
Flag for whether or not the command should be captured.
Attributes
----------
@ -1668,18 +1724,38 @@ class CommandPipeline:
A string of the standard error.
lines : list of str
The output lines
starttime : floats or None
Pipeline start timestamp.
"""
self.procs = procs
self.proc = procs[-1]
self.starttime = None
self.ended = False
self.procs = []
self.specs = specs
self.spec = specs[-1]
self.starttime = starttime or time.time()
self.captured = captured
self.ended = False
self.captured = specs[-1].captured
self.input = self._output = self.errors = self.endtime = None
self._closed_handle_cache = {}
self.lines = []
self._stderr_prefix = self._stderr_postfix = None
self.term_pgid = None
background = self.spec.background
pipeline_group = None
for spec in specs:
if self.starttime is None:
self.starttime = time.time()
try:
proc = spec.run(pipeline_group=pipeline_group)
except XonshError:
self._return_terminal()
raise
if proc.pid and pipeline_group is None and not spec.is_proxy and \
self.captured != 'object':
pipeline_group = proc.pid
if update_fg_process_group(pipeline_group, background):
self.term_pgid = pipeline_group
self.procs.append(proc)
self.proc = self.procs[-1]
def __repr__(self):
s = self.__class__.__name__ + '('
@ -1905,11 +1981,20 @@ class CommandPipeline:
#
def end(self, tee_output=True):
"""Waits for the command to complete and then runs any closing and
cleanup procedures that need to be run.
"""
End the pipeline, return the controlling terminal if needed.
Main things done in self._end().
"""
if self.ended:
return
self._end(tee_output=tee_output)
self._return_terminal()
def _end(self, tee_output):
"""Waits for the command to complete and then runs any closing and
cleanup procedures that need to be run.
"""
if tee_output:
for _ in self.tee_stdout():
pass
@ -1924,6 +2009,28 @@ class CommandPipeline:
self.ended = True
self._raise_subproc_error()
def _return_terminal(self):
if ON_WINDOWS or not ON_POSIX:
return
pgid = os.getpgid(0)
if self.term_pgid is None or pgid == self.term_pgid:
return
if give_terminal_to(pgid): # if gave term succeed
self.term_pgid = pgid
if hasattr(builtins, '__xonsh_shell__'):
# restoring sanity could probably be called whenever we return
# control to the shell. But it only seems to matter after a
# ^Z event. This *has* to be called after we give the terminal
# back to the shell.
builtins.__xonsh_shell__.shell.restore_tty_sanity()
def resume(self, job, tee_output=True):
self.ended = False
if give_terminal_to(job['pgrp']):
self.term_pgid = job['pgrp']
_continue(job)
self.end(tee_output=tee_output)
def _endtime(self):
"""Sets the closing timestamp if it hasn't been already."""
if self.endtime is None:

7
xonsh/shell.py
View file

@ -35,6 +35,13 @@ events.doc('on_postcommand', """
on_postcommand(cmd: str, rtn: int, out: str or None, ts: list) -> None
Fires just after a command is executed. The arguments are the same as history.
Parameters:
* ``cmd``: The command that was executed (after transformation)
* ``rtn``: The result of the command executed (``0`` for success)
* ``out``: If xonsh stores command output, this is the output
* ``ts``: Timestamps, in the order of ``[starting, ending]``
""")
events.doc('on_pre_prompt', """

19
xonsh/xontribs.json
View file

@ -23,6 +23,25 @@
"its behavior. To see the modifications as they are applied (in unified diff",
"format), please set ``$XONSH_DEBUG`` to ``2`` or higher."]
},
{"name": "coreutils",
"package": "xonsh",
"url": "http://xon.sh",
"description": [
"Additional core utilites that are implemened in xonsh. The current list ",
"includes:\n",
"\n",
"* cat\n",
"* echo\n",
"* pwd\n",
"* tee\n",
"* tty",
"* yes\n",
"\n",
"In many cases, these may have a lower performance overhead than the ",
"posix command line utility with the same name. This is because these ",
"tools avoid the need for a full subprocess call. Additionally, these ",
"tools are cross-platform."]
},
{"name": "distributed",
"package": "xonsh",
"url": "http://xon.sh",

106
xonsh/xoreutils/cat.py Normal file
View file

@ -0,0 +1,106 @@
"""Implements a cat command for xonsh."""
import os
from xonsh.xoreutils.util import arg_handler
def _cat_single_file(opts, fname, stdin, out, err, line_count=1):
if fname == '-':
f = stdin
elif os.path.isdir(fname):
print("cat: {}: Is a directory.".format(fname), file=err)
return True, line_count
elif not os.path.exists(fname):
print("cat: No such file or directory: {}".format(fname), file=err)
return True, line_count
else:
f = open(fname, 'rb')
sep = os.linesep.encode()
last_was_blank = False
while True:
_r = r = f.readline()
if isinstance(_r, str):
_r = r = _r.encode()
if r == b'':
break
if r.endswith(sep):
_r = _r[:-len(sep)]
this_one_blank = _r == b''
if last_was_blank and this_one_blank and opts['squeeze_blank']:
continue
last_was_blank = this_one_blank
if (opts['number_all'] or
(opts['number_nonblank'] and not this_one_blank)):
start = ("%6d " % line_count).encode()
_r = start + _r
line_count += 1
if opts['show_ends']:
_r = _r + b'$'
try:
print(_r.decode('unicode_escape'), flush=True, file=out)
except:
pass
return False, line_count
def cat(args, stdin, stdout, stderr):
"""A cat command for xonsh."""
opts = _cat_parse_args(args)
if opts is None:
print(CAT_HELP_STR, file=stdout)
return 0
line_count = 1
errors = False
if len(args) == 0:
args = ['-']
for i in args:
o = _cat_single_file(opts, i, stdin, stdout, stderr, line_count)
if o is None:
return -1
_e, line_count = o
errors = _e or errors
return int(errors)
def _cat_parse_args(args):
out = {'number_nonblank': False, 'number_all': False, 'squeeze_blank': False, 'show_ends': False}
if '--help' in args:
return
arg_handler(args, out, '-b', 'number_nonblank', True, '--number-nonblank')
arg_handler(args, out, '-n', 'number_all', True, '--number')
arg_handler(args, out, '-E', 'show_ends', True, '--show-ends')
arg_handler(args, out, '-s', 'squeeze_blank', True, '--squeeze-blank')
arg_handler(args, out, '-T', 'show_tabs', True, '--show-tabs')
return out
CAT_HELP_STR = """This version of cat was written in Python for the xonsh project: http://xon.sh
Based on cat from GNU coreutils: http://www.gnu.org/software/coreutils/
Usage: cat [OPTION]... [FILE]...
Concatenate FILE(s), or standard input, to standard output.
-b, --number-nonblank number nonempty output lines, overrides -n
-E, --show-ends display $ at end of each line
-n, --number number all output lines
-s, --squeeze-blank suppress repeated empty output lines
-T, --show-tabs display TAB characters as ^I
-u (ignored)
--help display this help and exit
With no FILE, or when FILE is -, read standard input.
Examples:
cat f - g Output f's contents, then standard input, then g's contents.
cat Copy standard input to standard output."""
# NOT IMPLEMENTED:
# -A, --show-all equivalent to -vET
# -e equivalent to -vE
# -t equivalent to -vT
# -v, --show-nonprinting use ^ and M- notation, except for LFD and TAB
# --version output version information and exit"""

44
xonsh/xoreutils/echo.py Normal file
View file

@ -0,0 +1,44 @@
"""Implements a simple echo command for xonsh."""
def echo(args, stdin, stdout, stderr):
"""A simple echo command."""
opts = _echo_parse_args(args)
if opts is None:
return
if opts['help']:
print(ECHO_HELP, file=stdout)
return 0
ender = opts['end']
args = map(str, args)
if opts['escapes']:
args = map(lambda x: x.encode().decode('unicode_escape'), args)
print(*args, end=ender, file=stdout)
def _echo_parse_args(args):
out = {'escapes': False, 'end': '\n'}
if '-e' in args:
args.remove('-e')
out['escapes'] = True
if '-E' in args:
args.remove('-E')
out['escapes'] = False
if '-n' in args:
args.remove('-n')
out['end'] = ''
if '-h' in args or '--help' in args:
out['help'] = True
return out
ECHO_HELP = """Usage: echo [OPTIONS]... [STRING]...
Echo the STRING(s) to standard output.
-n do not include the trailing newline
-e enable interpretation of backslash escapes
-E disable interpretation of backslash escapes (default)
-h --help display this message and exit
This version of echo was written in Python for the xonsh project: http://xon.sh
Based on echo from GNU coreutils: http://www.gnu.org/software/coreutils/"""

28
xonsh/xoreutils/pwd.py Normal file
View file

@ -0,0 +1,28 @@
"""A pwd implementation for xonsh."""
import os
def pwd(args, stdin, stdout, stderr):
"""A pwd implementation"""
e = __xonsh_env__['PWD']
if '-h' in args or '--help' in args:
print(PWD_HELP, file=stdout)
return 0
if '-P' in args:
e = os.path.realpath(e)
print(e, file=stdout)
return 0
PWD_HELP = """Usage: pwd [OPTION]...
Print the full filename of the current working directory.
-P, --physical avoid all symlinks
--help display this help and exit
This version of pwd was written in Python for the xonsh project: http://xon.sh
Based on pwd from GNU coreutils: http://www.gnu.org/software/coreutils/"""
# Not Implemented
# -L, --logical use PWD from environment, even if it contains symlinks

59
xonsh/xoreutils/tee.py Normal file
View file

@ -0,0 +1,59 @@
"""A tee implementation for xonsh."""
def tee(args, stdin, stdout, stderr):
"""A tee command for xonsh."""
mode = 'w'
if '-a' in args:
args.remove('-a')
mode = 'a'
if '--append' in args:
args.remove('--append')
mode = 'a'
if '--help' in args:
print(TEE_HELP, file=stdout)
return 0
if stdin is None:
msg = "tee was not piped stdin, must have input stream to read from."
print(msg, file=stderr)
return 1
errors = False
files = []
for i in args:
if i == '-':
files.append(stdout)
else:
try:
files.append(open(i, mode))
except:
print('tee: failed to open {}'.format(i), file=stderr)
errors = True
files.append(stdout)
while True:
r = stdin.read(1024)
if r == '':
break
for i in files:
i.write(r)
for i in files:
if i != stdout:
i.close()
return int(errors)
TEE_HELP = """This version of tee was written in Python for the xonsh project: http://xon.sh
Based on tee from GNU coreutils: http://www.gnu.org/software/coreutils/
Usage: tee [OPTION]... [FILE]...
Copy standard input to each FILE, and also to standard output.
-a, --append append to the given FILEs, do not overwrite
--help display this help and exit
If a FILE is -, copy again to standard output."""
# NOT IMPLEMENTED:
# -i, --ignore-interrupts ignore interrupt signals

44
xonsh/xoreutils/tty.py Normal file
View file

@ -0,0 +1,44 @@
"""A tty implementation for xonsh"""
import os
import sys
def tty(args, stdin, stdout, stderr):
"""A tty command for xonsh."""
if '--help' in args:
print(TTY_HELP, file=stdout)
return 0
silent = False
for i in ('-s', '--silent', '--quiet'):
if i in args:
silent = True
args.remove(i)
if len(args) > 0:
if not silent:
for i in args:
print('tty: Invalid option: {}'.format(i), file=stderr)
print("Try 'tty --help' for more information", file=stderr)
return 2
try:
fd = stdin.fileno()
except:
fd = sys.stdin.fileno()
if not os.isatty(fd):
if not silent:
print('not a tty', file=stdout)
return 1
if not silent:
try:
print(os.ttyname(fd), file=stdout)
except:
return 3
return 0
TTY_HELP = """Usage: tty [OPTION]...
Print the file name of the terminal connected to standard input.
-s, --silent, --quiet print nothing, only return an exit status
--help display this help and exit
This version of tty was written in Python for the xonsh project: http://xon.sh
Based on tty from GNU coreutils: http://www.gnu.org/software/coreutils/"""

19
xonsh/xoreutils/util.py Normal file
View file

@ -0,0 +1,19 @@
"""Assorted utilities for xonsh core utils."""
def arg_handler(args, out, short, key, val, long=None):
"""A simple argument handler for xoreutils."""
if short in args:
args.remove(short)
if isinstance(key, (list, tuple)):
for k in key:
out[k] = val
else:
out[key] = val
if long is not None and long in args:
args.remove(long)
if isinstance(key, (list, tuple)):
for k in key:
out[k] = val
else:
out[key] = val

25
xonsh/xoreutils/yes.py Normal file
View file

@ -0,0 +1,25 @@
"""An implementation of yes for xonsh."""
def yes(args, stdin, stdout, stderr):
"""A yes command."""
if '--help' in args:
print(YES_HELP, file=stdout)
return 0
to_print = ["y"] if len(args) == 0 else [str(i) for i in args]
while True:
print(*to_print, file=stdout)
return 0
YES_HELP = """Usage: yes [STRING]...
or: yes OPTION
Repeatedly output a line with all specified STRING(s), or 'y'.
--help display this help and exit
This version of yes was written in Python for the xonsh project: http://xon.sh
Based on yes from GNU coreutils: http://www.gnu.org/software/coreutils/"""

30
xontrib/coreutils.py Normal file
View file

@ -0,0 +1,30 @@
"""Additional core utilites that are implemented in xonsh. The current list
includes:
* cat
* echo
* pwd
* tee
* tty
* yes
In many cases, these may have a lower performance overhead than the
posix command line utility with the same name. This is because these
tools avoid the need for a full subprocess call. Additionally, these
tools are cross-platform.
"""
from xonsh.xoreutils.cat import cat
from xonsh.xoreutils.echo import echo
from xonsh.xoreutils.pwd import pwd
from xonsh.xoreutils.tee import tee
from xonsh.xoreutils.tty import tty
from xonsh.xoreutils.yes import yes
__all__ = ()
aliases['cat'] = cat
aliases['echo'] = echo
aliases['pwd'] = pwd
aliases['tee'] = tee
aliases['tty'] = tty
aliases['yes'] = yes