Merge branch 'master' of https://github.com/scopatz/xonsh

.appveyor.yml

@@ -1,7 +1,7 @@
-version: 0.4.3.{build}
+version: 0.4.4.{build}
 os: Windows Server 2012 R2
 install:
-- C:\Python35\Scripts\pip install --upgrade -r requirements-tests.txt
+- C:\Python35\Scripts\pip install -r requirements-tests.txt
 build: off
 test_script:
 - C:\Python35\Scripts\py.test

.gitignore

@@ -40,5 +40,6 @@ include/
 # Editor project files
 *.komodo*
 .cache
+.idea
 
 .coverage

.travis.yml

@@ -1,11 +1,51 @@
 language: python
-python:
-  - 3.4
-  - 3.5
-  - "nightly"
+matrix:
+    include:
+        - os: linux
+          python: "nightly"
+          env: RUN_COVERAGE=true
+        - os: linux
+          language: generic
+          env: PYTHON="3.4" MINICONDA_OS="Linux"
+        - os: linux
+          language: generic
+          env: PYTHON="3.5" MINICONDA_OS="Linux"
+        - os: osx
+          language: generic
+          env: PYTHON="3.4" MINICONDA_OS="MacOSX"
+        - os: osx
+          language: generic
+          env: PYTHON="3.5" MINICONDA_OS="MacOSX"
+
+before_install:
+  - if [[ ! "$TRAVIS_PYTHON_VERSION" == "nightly" ]]; then
+      URL="https://repo.continuum.io/miniconda/Miniconda3-latest-${MINICONDA_OS}-x86_64.sh";
+      wget "${URL}" -O miniconda.sh;
+      bash miniconda.sh -b -p $HOME/miniconda;
+      export PATH="$HOME/miniconda/bin:$PATH";
+      hash -r;
+      conda config --set always_yes yes --set changeps1 no;
+      conda update -q conda;
+      conda info -a;
+    fi
+
 install:
-  - pip install --upgrade -r requirements-tests.txt
+  - if [[ ! "$TRAVIS_PYTHON_VERSION" == "nightly" ]]; then
+      conda create -q -n test_env python=${PYTHON} pygments prompt_toolkit ply pytest pytest-timeout psutil;
+      source activate test_env;
+      python setup.py install;
+    else
+      pip install -r requirements-tests.txt;
+    fi
+
 script:
-  - py.test --flake8 --cov=. --cov-report=term
+  - if [[ $RUN_COVERAGE = true ]]; then
+      py.test --flake8 --cov=. --cov-report=term --timeout=10;
+    else
+      py.test --timeout=10;
+    fi
+
 after_success:
-  - codecov
+  - if [[ $RUN_COVERAGE = true ]]; then
+      codecov;
+    fi

CHANGELOG.rst

@@ -4,6 +4,107 @@ Xonsh Change Log
 
 .. current developments
 
+v0.4.4
+====================
+
+**Added:**
+
+* New ``lazyobject()``, ``lazydict()``, and ``lazybool()`` decorators to turn
+  functions into lazy, global objects.
+* ``vox remove`` command can remove multiple environments at once.
+* Added FreeBSD support.
+* Tab completion for pip python package manager.
+* Regular expressions for enviroment variable matching
+
+* __contains__ method on Env
+* Added news tests to enforce changelog conformity.
+* A new way to add optional items to the prompt format string has been added.
+  Instead of relying on formatter dict items being padded with a space, now the
+  padding characters are specified in the format string itself, in place of the
+  format spec (after a ``:``).
+
+  For example, previously the prompt string ``{cwd}{curr_branch} $`` would rely
+  on ``curr_branch`` giving its output prepended with a space for separation,
+  or outputting nothing if it is not applicable. Now ``curr_branch`` just
+  outputs a value or ``None``, and the prompt string has to specify the
+  surrounding characters: ``{cwd}{curr_branch: {}} $``. Here the  value of
+  ``curr_branch`` will be prepended with a space (``{}`` is a placeholder for
+  the value itself). The format string after ``:`` is applied only if the value
+  is not ``None``.
+* ``xonsh.completers`` subpackage is now amalgamated.
+* amalgamate.py will now warn if the same name is defined across multiple
+  different files.
+* xonsh_builtins, xonsh_execer fixtures in conftest.py
+* Docs on how to tweak the Windows ConHost for a better color scheme.
+* Docs: how to fix Thunar's "Open Terminal Here" action.
+* A new API class was added to Vox: ``xontrib.voxapi.Vox``. This allows programtic access to the virtual environment machinery for other xontribs. See the API documentation for details.
+* History now accepts multiple slices arguments separated by spaces
+
+
+**Changed:**
+
+* amalgamate now works on Python 2 and allows relative imports.
+* Top-level xonsh package now more lazy.
+* Show conda environement name in prompt in parentheses similiar what conda does.
+* Implementation of expandvars now uses regex
+* Because of the addition of "optional items" to the prompt format string, the
+  functions ``xonsh.environ.current_branch``, ``xonsh.environ.env_name`` and
+  formatter dict items ``curr_branch``, ``current_job``, ``env_name`` are
+  no longer padded with a separator.
+* many test cases to use fixtures and parametrization
+* Public interface in ``xonsh.ansi_colors`` module now has ``ansi_``
+  prefix to prevent name conflicts with other parts of xonsh.
+* Vox was moved to xontrib. Behaves exactly the same as before, just need to add it to your xontribs.
+* is_int_as_str and is_slice_as_str are now reimplemented in EAFP style
+
+
+**Deprecated:**
+
+* yield statements (nose style) and for loops in tests
+* is_int_or_slice
+
+
+**Removed:**
+
+* _is_in_env, _get_env_string functions on tools
+* ``xonsh.environ.format_prompt`` has been dropped; ``partial_format_prompt``
+  can be used instead.
+* for loops and yield statements in test cases, unused imports
+* is_int_or_slice
+
+
+**Fixed:**
+
+* Fixed bug on Windows preventing xonsh from changing the console title.
+* Unrecognized ``$XONSH_COLOR_STYLE`` values don't crash terminal.
+* Writing the window title will no longer accidentally answer interactive
+  questions, eg ``rm -i`` now works as expected.
+* more matching cases for envvar reference
+* Certain linux VTE terminals would not start new tabs in the previous CWD.
+  This may now be rectified by adding ``{vte_new_tab_cwd}`` somewhere to the
+  prompt.
+* Unqualified usage of Unstorable in xonsh setup wizard that was causing the
+  wizard to crash and burn
+* Bare ``except:`` was replaced with ``except Exception`` to prevent
+  accidentally catching utility exceptions such as KeyboardInterrupt, which
+  caused unexpected problems like printing out the raw $PROMPT string.
+* Fixed multiple definition of ``EQUAL``.
+* Fixed multiple definition of ``pprint``.
+* Fixed multiple definition of ``pyghooks``.
+* Fixed multiple definition of ``pygments``.
+* Fixed multiple definition of ``tokenize``.
+* redundant and 'leaky' tests in nose
+* Fix bug that prevented disabling $INTENSIFY_COLORS_ON_WIN in ``xonshrc``
+* ``LazyJSON`` will now hide failures to close, and instead rely on reference
+  counting if something goes wrong.
+* Fixed maximum recurssion error with color styles.
+* Parser tables will no longer be generated in the current directory
+  by accident.
+* Error messages when zsh or bash history file is not found
+
+
+
+
 v0.4.3
 ====================
 

CONTRIBUTING.rst

@@ -15,6 +15,33 @@ develop xonsh.
 
 .. note:: All code changes must go through the pull request review procedure.
 
+
+Making Your First Change
+========================
+
+First, install xonsh from source and open a xonsh shell in your favorite
+terminal application. See installation instructions for details.
+
+Next, make a trivial change (e.g. ``print("hello!")`` in ``main.py``).
+
+Finally, run the following commands. You should see the effects of your change
+(e.g. ``hello!``)::
+
+        $ $XONSH_DEBUG=1
+        $ xonsh
+
+The xonsh build process collapses all Python source files into a single
+``__amalgam__.py`` file. When xonsh is started with a falsy value for
+`$XONSH_DEBUG <envvars.html>`_, it imports Python modules straight from
+``__amalgam__.py``, which decreases startup times by eliminating the cost of
+runtime imports. But setting ``$ $XONSH_DEBUG=1`` will suppress amalgamated
+imports. Reloading the xonsh shell (``$ xonsh``) won't simply import the stale
+``__amalgam__.py`` file that doesn't contain your new change, but will instead
+import the unamalgamated source code which does contain your change. You can now
+load every subsequent change by reloading xonsh, and if your code changes don't
+seem to have any effect, make sure you check ``$XONSH_DEBUG`` first!
+
+
 Changelog
 =========
 Pull requests will often have CHANGELOG entries associated with. However,
@@ -169,7 +196,7 @@ Run all the tests using pytest::
 
     $ py.test -q
 
-Use "-q" to keep pytest from outputting a bunch of info for every test.  
+Use "-q" to keep pytest from outputting a bunch of info for every test.
 
 ----------------------------------
 Running the Tests - Advanced

amalgamate.py

@@ -6,9 +6,11 @@ import pprint
 from itertools import repeat
 from collections import namedtuple
 from collections.abc import Mapping
-from ast import parse, walk, literal_eval, Import, ImportFrom
+from ast import parse, walk, Import, ImportFrom
 
-ModNode = namedtuple('ModNode', ['name', 'pkgdeps', 'extdeps'])
+__version__ = '0.1.2'
+
+ModNode = namedtuple('ModNode', ['name', 'pkgdeps', 'extdeps', 'futures'])
 ModNode.__doc__ = """Module node for dependency graph.
 
 Attributes
@@ -19,6 +21,8 @@ pkgdeps : frozenset of str
     Module dependencies in the same package.
 extdeps : frozenset of str
     External module dependencies from outside of the package.
+futures : frozenset of str
+    Import directive names antecedent to 'from __future__ import'
 """
 
 
@@ -49,15 +53,166 @@ class SourceCache(Mapping):
 
 SOURCES = SourceCache()
 
-def make_node(name, pkg, allowed):
+
+class GlobalNames(object):
+    """Stores globally defined names that have been seen on ast nodes."""
+
+    impnodes = frozenset(['import', 'importfrom'])
+
+    def __init__(self, pkg='<pkg>'):
+        self.cache = {}
+        self.pkg = pkg
+        self.module = '<mod>'
+        self.topnode = None
+
+    def warn_duplicates(self):
+        s = ''
+        for key in sorted(self.cache.keys()):
+            val = self.cache[key]
+            if len(val) < 2:
+                continue
+            val = sorted(val)
+            if all([val[0][0] == x[0] for x in val[1:]]):
+                continue
+            s += 'WARNING: {0!r} defined in multiple locations:\n'.format(key)
+            for loc in val:
+                s += '  {}:{} ({})\n'.format(*loc)
+        if len(s) > 0:
+            print(s, end='', flush=True, file=sys.stderr)
+
+    def entry(self, name, lineno):
+        if name.startswith('__'):
+            return
+        topnode = self.topnode
+        e = (self.pkg + '.' + self.module, lineno, topnode)
+        if name in self.cache:
+            if topnode in self.impnodes and \
+                    all([topnode == x[2] for x in self.cache[name]]):
+                return
+            self.cache[name].add(e)
+        else:
+            self.cache[name] = set([e])
+
+    def add(self, node, istopnode=False):
+        """Adds the names from the node to the cache."""
+        nodename = node.__class__.__name__.lower()
+        if istopnode:
+            self.topnode = nodename
+        meth = getattr(self, '_add_' + nodename, None)
+        if meth is not None:
+            meth(node)
+
+    def _add_name(self, node):
+        self.entry(node.id, node.lineno)
+
+    def _add_tuple(self, node):
+        for x in node.elts:
+            self.add(x)
+
+    def _add_assign(self, node):
+        for target in node.targets:
+            self.add(target)
+
+    def _add_functiondef(self, node):
+        self.entry(node.name, node.lineno)
+
+    def _add_classdef(self, node):
+        self.entry(node.name, node.lineno)
+
+    def _add_import(self, node):
+        lineno = node.lineno
+        for target in node.names:
+            if target.asname is None:
+                name, _, _ = target.name.partition('.')
+            else:
+                name = target.asname
+            self.entry(name, lineno)
+
+    def _add_importfrom(self, node):
+        pkg, _ = resolve_package_module(node.module, self.pkg, node.level)
+        if pkg == self.pkg:
+            return
+        lineno = node.lineno
+        for target in node.names:
+            if target.asname is None:
+                name = target.name
+            else:
+                name = target.asname
+            self.entry(name, lineno)
+
+    def _add_with(self, node):
+        for item in node.items:
+            if item.optional_vars is None:
+                continue
+            self.add(item.optional_vars)
+        for child in node.body:
+            self.add(child, istopnode=True)
+
+    def _add_for(self, node):
+        self.add(node.target)
+        for child in node.body:
+            self.add(child, istopnode=True)
+
+    def _add_while(self, node):
+        for child in node.body:
+            self.add(child, istopnode=True)
+
+    def _add_if(self, node):
+        for child in node.body:
+            self.add(child, istopnode=True)
+        for child in node.orelse:
+            self.add(child, istopnode=True)
+
+    def _add_try(self, node):
+        for child in node.body:
+            self.add(child, istopnode=True)
+
+
+def module_is_package(module, pkg, level):
+    """Returns whether or not the module name refers to the package."""
+    if level == 0:
+        return module == pkg
+    elif level == 1:
+        return module is None
+    else:
+        return False
+
+
+def module_from_package(module, pkg, level):
+    """Returns whether or not a module is from the package."""
+    if level == 0:
+        return module.startswith(pkg + '.')
+    elif level == 1:
+        return True
+    else:
+        return False
+
+
+def resolve_package_module(module, pkg, level, default=None):
+    """Returns a 2-tuple of package and module name, even for relative
+    imports
+    """
+    if level == 0:
+        p, _, m = module.rpartition('.')
+    elif level == 1:
+        p = pkg
+        m = module or default
+    else:
+        p = m = None
+    return p, m
+
+
+def make_node(name, pkg, allowed, glbnames):
     """Makes a node by parsing a file and traversing its AST."""
     raw = SOURCES[pkg, name]
     tree = parse(raw, filename=name)
     # we only want to deal with global import statements
-    pkgdot = pkg + '.'
     pkgdeps = set()
     extdeps = set()
+    futures = set()
+    glbnames.module = name
     for a in tree.body:
+        glbnames.add(a, istopnode=True)
         if isinstance(a, Import):
             for n in a.names:
                 p, dot, m = n.name.rpartition('.')
@@ -66,18 +221,19 @@ def make_node(name, pkg, allowed):
                 else:
                     extdeps.add(n.name)
         elif isinstance(a, ImportFrom):
-            if a.module == pkg:
+            if module_is_package(a.module, pkg, a.level):
                 pkgdeps.update(n.name for n in a.names if n.name in allowed)
-            elif not a.module or a.module.startswith(pkgdot):
-                if a.module is None:
-                    p, dot, m = pkg, ".", a.names[0].name
-                else:
-                    p, dot, m = a.module.rpartition('.')
+            elif module_from_package(a.module, pkg, a.level):
+                p, m = resolve_package_module(a.module, pkg, a.level,
+                                              default=a.names[0].name)
                 if p == pkg and m in allowed:
                     pkgdeps.add(m)
                 else:
                     extdeps.add(a.module)
-    return ModNode(name, frozenset(pkgdeps), frozenset(extdeps))
+            elif a.module == '__future__':
+                futures.update(n.name for n in a.names)
+    return ModNode(name, frozenset(pkgdeps), frozenset(extdeps),
+                   frozenset(futures))
 
 
 def make_graph(pkg, exclude=None):
@@ -93,8 +249,10 @@ def make_graph(pkg, exclude=None):
         allowed.add(base)
     if exclude:
         allowed -= exclude
+    glbnames = GlobalNames(pkg=pkg)
     for base in allowed:
-        graph[base] = make_node(base, pkg, allowed)
+        graph[base] = make_node(base, pkg, allowed, glbnames)
+    glbnames.warn_duplicates()
     return graph
 
 
@@ -136,7 +294,8 @@ class _LazyModule(_ModuleType):
     @classmethod
     def load(cls, pkg, mod, asname=None):
         if mod in _modules:
-            return _modules[pkg]
+            key = pkg if asname is None else mod
+            return _modules[key]
         else:
             return cls(pkg, mod, asname)
 
@@ -161,6 +320,7 @@ class _LazyModule(_ModuleType):
 
 """
 
+
 def get_lineno(node, default=0):
     """Gets the lineno of a node or returns the default."""
     return getattr(node, 'lineno', default)
@@ -189,7 +349,6 @@ def format_lazy_import(names):
     lines = ''
     for _, name, asname in names:
         pkg, _, _ = name.partition('.')
-        target = asname or pkg
         if asname is None:
             line = '{pkg} = _LazyModule.load({pkg!r}, {mod!r})\n'
         else:
@@ -213,7 +372,6 @@ def format_from_import(names):
 
 def rewrite_imports(name, pkg, order, imps):
     """Rewrite the global imports in the file given the amalgamation."""
-    pkgdot = pkg + '.'
     raw = SOURCES[pkg, name]
     tree = parse(raw, filename=name)
     replacements = []  # list of (startline, stopline, str) tuples
@@ -237,27 +395,26 @@ def rewrite_imports(name, pkg, order, imps):
                     imps.add(imp)
                     keep.append(imp)
             if len(keep) == 0:
-                s = ', '.join(n.name for n in  a.names)
+                s = ', '.join(n.name for n in a.names)
                 s = '# amalgamated ' + s + '\n'
             else:
                 s = format_lazy_import(keep)
             replacements.append((start, stop, s))
         elif isinstance(a, ImportFrom):
-            if not a.module:
-                a.module = pkg
-                p, dot, m = pkg, ".", ""
-            else:
-                p, dot, m = a.module.rpartition('.')
-            if a.module == pkg:
+            p, m = resolve_package_module(a.module, pkg, a.level, default='')
+            if module_is_package(a.module, pkg, a.level):
                 for n in a.names:
                     if n.name in order:
                         msg = ('Cannot amalgamate import of '
                                'amalgamated module:\n\n  from {0} import {1}\n'
                                '\nin {0}/{2}.py').format(pkg, n.name, name)
                         raise RuntimeError(msg)
-            elif a.module.startswith(pkgdot) and p == pkg and m in order:
+            elif p == pkg and m in order:
                 replacements.append((start, stop,
-                                     '# amalgamated ' + a.module + '\n'))
+                                     '# amalgamated ' + p + '.' + m + '\n'))
+            elif a.module == '__future__':
+                replacements.append((start, stop,
+                                     '# amalgamated __future__ directive\n'))
             else:
                 keep = []
                 for n in a.names:
@@ -282,12 +439,23 @@ def rewrite_imports(name, pkg, order, imps):
     return ''.join(lines)
 
 
+def sorted_futures(graph):
+    """Returns a sorted, unique list of future imports."""
+    f = set()
+    for value in graph.values():
+        f |= value.futures
+    return sorted(f)
+
+
 def amalgamate(order, graph, pkg):
     """Create amalgamated source."""
     src = ('\"\"\"Amalgamation of {0} package, made up of the following '
            'modules, in order:\n\n* ').format(pkg)
     src += '\n* '.join(order)
     src += '\n\n\"\"\"\n'
+    futures = sorted_futures(graph)
+    if len(futures) > 0:
+        src += 'from __future__ import ' + ', '.join(futures) + '\n'
     src += LAZY_IMPORTS
     imps = set()
     for name in order:
@@ -373,7 +541,7 @@ def main(args=None):
             continue
         print('Amalgamating ' + pkg)
         exclude = read_exclude(pkg)
-        print('  excluding {}'.format(pprint.pformat(exclude)))
+        print('  excluding {}'.format(pprint.pformat(exclude or None)))
         graph = make_graph(pkg, exclude=exclude)
         order = depsort(graph)
         src = amalgamate(order, graph, pkg)

docs/api/index.rst

@@ -65,7 +65,6 @@ For those of you who want the gritty details.
     xonfig
     codecache
     contexts
-    vox
 
 
 **Xontribs:**
@@ -74,4 +73,5 @@ For those of you who want the gritty details.
     :maxdepth: 1
 
     mplhooks
+    vox
 

docs/api/proc.rst

@@ -7,3 +7,4 @@ Python Procedures as Subprocess Commands (``xonsh.proc``)
 .. automodule:: xonsh.proc
     :members:
     :undoc-members:
+    :exclude-members: msvcrt _winapi

docs/api/vox.rst

@@ -1,10 +1,10 @@
 .. _xonsh_vox:
 
 ******************************************************
-Vox (``xonsh.vox``)
+Vox (``xontrib.voxapi``)
 ******************************************************
 
-.. automodule:: xonsh.vox
+.. automodule:: xontrib.voxapi
     :members:
     :undoc-members:
     :inherited-members:

docs/conf.py

@@ -285,7 +285,7 @@ def make_xontribs():
     s = ('.. list-table::\n'
          '    :header-rows: 0\n\n')
     table = []
-    ncol = 3
+    ncol = 5
     row = '    {0} - :ref:`{1} <{2}>`'
     for i, name in enumerate(names):
         star = '*' if i%ncol == 0 else ' '
@@ -319,7 +319,7 @@ def make_xontribs():
             pd = md['packages'].get(pkgname, {})
             pkg = pkgname
             if 'url' in pd:
-                pkg = '`{0} <{1}>`_'.format(pkg, pd['url'])
+                pkg = '`{0} website <{1}>`_'.format(pkg, pd['url'])
             if 'license' in pd:
                 pkg = pkg + ', ' + pd['license']
             inst = ''

docs/index.rst

@@ -244,7 +244,9 @@ Contributing
 We highly encourage contributions to xonsh!  If you would like to contribute,
 it is as easy as forking the repository on GitHub, making your changes, and
 issuing a pull request.  If you have any questions about this process don't
-hesitate to ask the mailing list (xonsh@googlegroups.com).
+hesitate to ask the mailing list (xonsh@googlegroups.com) or the `Gitter <https://gitter.im/xonsh/xonsh>`_ channel.
+
+See the `Developer's Guide <devguide.html>`_ for more information about contributing.
 
 ==========
 Contact Us

docs/linux.rst

@@ -127,3 +127,17 @@ fix this, please add ``{vte_new_tab_cwd}`` somewhere to you prompt:
 
 This will issue the proper escape sequence to the terminal without otherwise
 affecting the displayed prompt.
+
+
+"Open Terminal Here" Action Does Not Work in Thunar
+===================================================
+
+If you use Thunar and "Open Terminal Here" action does not work,
+you can try to replace a command for this action by the following:
+
+.. code-block:: sh
+
+    exo-open --working-directory %f --launch TerminalEmulator xonsh --shell-type=best
+
+In order to do this, go to ``Edit > Configure custom actions...``,
+then choose ``Open Terminal Here`` and click on ``Edit currently selected action`` button.

docs/python_virtual_environments.rst

@@ -13,6 +13,10 @@ Luckily, xonsh ships with its own virtual environments manager called **Vox**.
 Vox
 ===
 
+First, load the vox xontrib::
+
+    $ xontrib load vox
+
 To create a new environment with vox, run ``vox new <envname>``::
 
     $ vox new myenv

docs/talks_and_articles.rst

@@ -5,9 +5,9 @@ Here are some talks, articles, and other sundry about your favorite shell.
 
 Talks
 ============
-**PyCon 2016:** presented by Anthony Scopatz
+**Python Nordeste 2016:** presented by Lucas Inojosa http://lucasicf.github.io/talks/shell_python/
 
-**Python Nordeste 2016:** presented by Lucas Inojosa <http://lucasicf.github.io/talks/shell_python/>
+**PyCon 2016:** presented by Anthony Scopatz
 
 .. raw:: html
 

docs/tutorial.rst

@@ -1305,6 +1305,7 @@ The curly brackets act as a placeholder, because the additional part is an
 ordinary format string. What we're doing here is equivalent to this expression:
 
 .. code-block:: python
+
     " [{}]".format(curr_branch()) if curr_branch() is not None else ""
 
 

news/aliascomp.rst

@@ -6,8 +6,8 @@
 
 **Removed:** None
 
-**Fixed:**
+**Fixed:** 
 
-* Unrecognized ``$XONSH_COLOR_STYLE`` values don't crash terminal.
+* Alias tab completion works again
 
 **Security:** None

news/amalgamation.rst

@@ -0,0 +1,15 @@
+**Added:** None
+
+**Changed:**
+
+* Moved ``amalgamate_source`` outside ``build_tables``
+
+* Disable amalgamation on setup develop
+
+**Deprecated:** None
+
+**Removed:** None
+
+**Fixed:** None
+
+**Security:** None

news/conda_env_paren.rst

@@ -1,13 +0,0 @@
-**Added:** None
-
-**Changed:** 
-
-* Show conda environement name in prompt in parentheses similiar what conda does. 
-
-**Deprecated:** None
-
-**Removed:** None
-
-**Fixed:** None
-
-**Security:** None

news/cs.rst

@@ -1,16 +0,0 @@
-**Added:** None
-
-**Changed:**
-
-* Public interface in ``xonsh.ansi_colors`` module now has ``ansi_``
-  prefix to prevent name conflicts with other parts of xonsh.
-
-**Deprecated:** None
-
-**Removed:** None
-
-**Fixed:**
-
-* Fixed maximum recurssion error with color styles.
-
-**Security:** None

news/enc.rst

@@ -0,0 +1,16 @@
+**Added:**
+
+* ``xon.sh`` script now sets ``$LANG=C.UTF8`` in the event that no encoding
+  is detected.
+
+**Changed:**
+
+* xonfig command now dumps more encoding related settings.
+
+**Deprecated:** None
+
+**Removed:** None
+
+**Fixed:** None
+
+**Security:** None

news/fix-name-error.rst

@@ -1,14 +0,0 @@
-**Added:** None
-
-**Changed:** None
-
-**Deprecated:** None
-
-**Removed:** None
-
-**Fixed:**
-
-* Unqualified usage of Unstorable in xonsh setup wizard that was causing the
-  wizard to crash and burn
-
-**Security:** None

news/futamal.rst

@@ -1,6 +1,6 @@
 **Added:**
 
-* Tab completion for pip python package manager.
+* amalgamate.py now properly handles ``from __future__`` imports.
 
 **Changed:** None
 

news/jupyter.rst

@@ -6,8 +6,8 @@
 
 **Removed:** None
 
-**Fixed:**
+**Fixed:** 
 
-* Fixed bug on Windows preventing xonsh from changing the console title.
+* Completions in ``jupyter_kernel.py`` now use updated completion framework
 
 **Security:** None

news/lazycomp.rst

@@ -0,0 +1,15 @@
+**Added:** None
+
+**Changed:**
+
+* ``xonsh.completers`` sub-package is now fully lazy.
+
+**Deprecated:** None
+
+**Removed:** None
+
+**Fixed:**
+
+* Fixed amalgamation of aliased imports that are already in ``sys.modules``.
+
+**Security:** None

news/lazz.rst

@@ -1,16 +0,0 @@
-**Added:**
-
-* New ``lazyobject()``, ``lazydict()``, and ``lazybool()`` decorators to turn
-  functions into lazy, global objects.
-
-**Changed:**
-
-* Top-level xonsh package now more lazy.
-
-**Deprecated:** None
-
-**Removed:** None
-
-**Fixed:** None
-
-**Security:** None

news/ljbfd.rst

@@ -1,14 +0,0 @@
-**Added:** None
-
-**Changed:** None
-
-**Deprecated:** None
-
-**Removed:** None
-
-**Fixed:**
-
-* ``LazyJSON`` will now hide failures to close, and instead rely on reference
-  counting if something goes wrong.
-
-**Security:** None

news/newstests.rst

@@ -1,13 +0,0 @@
-**Added:**
-
-* Added news tests to enforce changelog conformity.
-
-**Changed:** None
-
-**Deprecated:** None
-
-**Removed:** None
-
-**Fixed:** None
-
-**Security:** None

news/no-bare-except.rst

@@ -1,15 +0,0 @@
-**Added:** None
-
-**Changed:** None
-
-**Deprecated:** None
-
-**Removed:** None
-
-**Fixed:**
-
-* Bare ``except:`` was replaced with ``except Exception`` to prevent
-  accidentally catching utility exceptions such as KeyboardInterrupt, which
-  caused unexpected problems like printing out the raw $PROMPT string.
-
-**Security:** None

news/od.rst

@@ -1,14 +0,0 @@
-**Added:** None
-
-**Changed:** None
-
-**Deprecated:** None
-
-**Removed:** None
-
-**Fixed:**
-
-* Parser tables will no longer be generated in the current directory
-  by accident.
-
-**Security:** None

news/ply.rst

@@ -0,0 +1,14 @@
+**Added:** None
+
+**Changed:** None
+
+**Deprecated:** None
+
+**Removed:** None
+
+**Fixed:** 
+
+* Version number reported by bundled PLY
+* ``xonfig`` no longer breaks if PLY is externally installed and version 3.8
+
+**Security:** None

news/prompt-optional-items.rst

@@ -1,33 +0,0 @@
-**Added:**
-
-* A new way to add optional items to the prompt format string has been added.
-  Instead of relying on formatter dict items being padded with a space, now the
-  padding characters are specified in the format string itself, in place of the
-  format spec (after a ``:``).
-
-  For example, previously the prompt string ``{cwd}{curr_branch} $`` would rely
-  on ``curr_branch`` giving its output prepended with a space for separation,
-  or outputting nothing if it is not applicable. Now ``curr_branch`` just
-  outputs a value or ``None``, and the prompt string has to specify the
-  surrounding characters: ``{cwd}{curr_branch: {}} $``. Here the  value of
-  ``curr_branch`` will be prepended with a space (``{}`` is a placeholder for
-  the value itself). The format string after ``:`` is applied only if the value
-  is not ``None``.
-
-**Changed:**
-
-* Because of the addition of "optional items" to the prompt format string, the
-  functions ``xonsh.environ.current_branch``, ``xonsh.environ.env_name`` and
-  formatter dict items ``curr_branch``, ``current_job``, ``env_name`` are
-  no longer padded with a separator.
-
-**Deprecated:** None
-
-**Removed:**
-
-* ``xonsh.environ.format_prompt`` has been dropped; ``partial_format_prompt``
-  can be used instead.
-
-**Fixed:** None
-
-**Security:** None

news/pytest-migration.rst

@@ -1,21 +0,0 @@
-**Added:**
-
-* xonsh_builtins, xonsh_execer fixtures in conftest.py
-
-**Changed:**
-
-* many test cases to use fixtures and parametrization
-
-**Deprecated:**
-
-* yield statements (nose style) and for loops in tests
-
-**Removed:**
-
-* for loops and yield statements in test cases, unused imports
-
-**Fixed:**
-
-* redundant and 'leaky' tests in nose
-
-**Security:** None

news/relimps.rst

@@ -1,6 +1,6 @@
 **Added:**
 
-* Added FreeBSD support.
+* amalgamate.py now supports relative imports.
 
 **Changed:** None
 

news/set-title.rst

@@ -2,7 +2,7 @@
 
 **Changed:**
 
-* amalgamate now works on Python 2 and allows relative imports.
+* $TITLE now changes both icon (tab) and window title
 
 **Deprecated:** None
 

news/tools-expandvars-refactor.rst

@@ -1,21 +0,0 @@
-**Added:**
-
-* Regular expressions for enviroment variable matching
-
-* __contains__ method on Env
-
-**Changed:**
-
-* Implementation of expandvars now uses regex
-
-**Deprecated:** None
-
-**Removed:**
-
-* _is_in_env, _get_env_string functions on tools
-
-**Fixed:**
-
-* more matching cases for envvar reference
-
-**Security:** None

news/vocal-which.rst

@@ -1,6 +1,6 @@
-**Added:**
+**Added:** 
 
-* ``vox remove`` command can remove multiple environments at once.
+* ``which -v`` now calls superhelp, which will print highlighted source.
 
 **Changed:** None
 

news/vox_opts.rst

@@ -0,0 +1,14 @@
+**Added:** None
+
+**Changed:**
+
+* The vox xontrib now takes flags very similar to Python's venv tool. Use 
+  ``vox --help <command>`` to learn more.
+
+**Deprecated:** None
+
+**Removed:** None
+
+**Fixed:** None
+
+**Security:** None

news/vte.rst

@@ -1,15 +0,0 @@
-**Added:** None
-
-**Changed:** None
-
-**Deprecated:** None
-
-**Removed:** None
-
-**Fixed:**
-
-* Certain linux VTE terminals would not start new tabs in the previous CWD.
-  This may now be rectified by adding ``{vte_new_tab_cwd}`` somewhere to the
-  prompt.
-
-**Security:** None

news/win-color.rst

@@ -1,15 +0,0 @@
-**Added:** 
-
-* Docs on how to tweak the Windows ConHost for a better color scheme.
-
-**Changed:** None
-
-**Deprecated:** None
-
-**Removed:** None
-
-**Fixed:**
-
-* Fix bug that prevented disabling $INTENSIFY_COLORS_ON_WIN in ``xonshrc`` 
-
-**Security:** None

requirements-tests.txt

@@ -1,7 +1,12 @@
-ply
-pytest
-pytest-flake8
-pytest-cov
-prompt-toolkit
-pygments
-codecov
+ply==3.8
+py==1.4.31
+pyflakes==1.2.3
+pytest==2.9.2
+pytest-flake8==0.5
+pytest-cov==2.3.0
+pytest-timeout==1.0.0
+prompt-toolkit==1.0.3
+pygments==2.1.3
+codecov==2.0.5
+flake8==2.6.2
+coverage==4.2

scripts/xon.sh

@@ -1,2 +1,10 @@
 #!/bin/sh
-/usr/bin/env PYTHONUNBUFFERED=1 python3 -u -m xonsh $@
+
+# set locale if it is totally undefined
+if [ -z "${LC_ALL+x}" ] && [ -z "${LC_CTYPE+x}" ] && \
+   [ -z "${LANG+x}" ] && [ -z "${LANGUAGE+x}" ]; then
+  export LANG=C.UTF-8
+fi
+
+# run python
+exec /usr/bin/env PYTHONUNBUFFERED=1 python3 -u -m xonsh $@

setup.cfg

@@ -1,8 +1,21 @@
 [pytest]
 flake8-max-line-length = 180
 flake8-ignore =
-    # Temporary until we fix all the errors
-    *.py E301 E302 E303 W391 E402 E127 E128 E201 E731 E701 E271 E265 E266 E225 E202 E502 E231 E228 E227 E203 E122 E251 W291 E124 E261 E125 E111 E222 E272 F401 F811 F841 F821 F402
+    *.py E402
+    tests/tools.py E128
+    xonsh/ast.py F401
+    xonsh/built_ins.py F821
+    xonsh/commands_cache.py F841
+    xonsh/history.py F821
+    xonsh/pyghooks.py F821
+    xonsh/readline_shell.py F401
+    xonsh/timings.py F401
+    xonsh/tokenize.py F821 F841
+    xonsh/tools.py E731
+    xonsh/xonfig.py E731
+    xonsh/ptk/key_bindings.py F841
+    xonsh/ptk/shell.py E731
+    xontrib/vox.py F821
     __amalgam__.py ALL
     # test files should be PEP8 but a ton of errors for now
     test_*.py ALL

setup.py

@@ -35,7 +35,8 @@ except ImportError:
     HAVE_JUPYTER = False
 
 
-TABLES = ['xonsh/lexer_table.py', 'xonsh/parser_table.py', 'xonsh/__amalgam__.py']
+TABLES = ['xonsh/lexer_table.py', 'xonsh/parser_table.py',
+          'xonsh/__amalgam__.py', 'xonsh/completers/__amalgam__.py']
 
 
 def clean_tables():
@@ -49,14 +50,18 @@ def clean_tables():
 os.environ['XONSH_DEBUG'] = '1'
 from xonsh import __version__ as XONSH_VERSION
 
-def amalagamate_source():
-    """Amalgamtes source files."""
+
+def amalgamate_source():
+    """Amalgamates source files."""
+    sys.path.insert(0, os.path.dirname(__file__))
     try:
         import amalgamate
     except ImportError:
         print('Could not import amalgamate, skipping.', file=sys.stderr)
         return
-    amalgamate.main(['amalgamate', '--debug=XONSH_DEBUG', 'xonsh'])
+    amalgamate.main(['amalgamate', '--debug=XONSH_DEBUG', 'xonsh',
+                     'xonsh.completers'])
+    sys.path.pop(0)
 
 
 def build_tables():
@@ -66,7 +71,6 @@ def build_tables():
     from xonsh.parser import Parser
     Parser(lexer_table='lexer_table', yacc_table='parser_table',
            outputdir='xonsh')
-    amalagamate_source()
     sys.path.pop(0)
 
 
@@ -81,7 +85,7 @@ def install_jupyter_hook(prefix=None, root=None):
             "display_name": "Xonsh",
             "language": "xonsh",
             "codemirror_mode": "shell",
-           }
+            }
     with TemporaryDirectory() as d:
         os.chmod(d, 0o755)  # Starts off as 700, not user readable
         if sys.platform == 'win32':
@@ -116,13 +120,13 @@ def dirty_version():
         return False
     _version = _version.decode('ascii')
     try:
-        base, N, sha = _version.strip().split('-')
-    except ValueError: # on base release
+        _, N, sha = _version.strip().split('-')
+    except ValueError:  # tag name may contain "-"
         open('xonsh/dev.githash', 'w').close()
         print('failed to parse git version', file=sys.stderr)
         return False
     sha = sha.strip('g')
-    replace_version(base, N)
+    replace_version(N)
     with open('xonsh/dev.githash', 'w') as f:
         f.write(sha)
     print('wrote git version: ' + sha, file=sys.stderr)
@@ -131,14 +135,17 @@ def dirty_version():
 
 ORIGINAL_VERSION_LINE = None
 
-def replace_version(base, N):
+
+def replace_version(N):
     """Replace version in `__init__.py` with devN suffix"""
     global ORIGINAL_VERSION_LINE
     with open('xonsh/__init__.py', 'r') as f:
         raw = f.read()
     lines = raw.splitlines()
+    msg_assert = '__version__ must be the first line of the __init__.py'
+    assert '__version__' in lines[0], msg_assert
     ORIGINAL_VERSION_LINE = lines[0]
-    lines[0] = "__version__ = '{}.dev{}'".format(base, N)
+    lines[0] = lines[0].rstrip(" '") + ".dev{}'".format(N)
     upd = '\n'.join(lines) + '\n'
     with open('xonsh/__init__.py', 'w') as f:
         f.write(upd)
@@ -146,6 +153,8 @@ def replace_version(base, N):
 
 def restore_version():
     """If we touch the version in __init__.py discard changes after install."""
+    if ORIGINAL_VERSION_LINE is None:
+        return
     with open('xonsh/__init__.py', 'r') as f:
         raw = f.read()
     lines = raw.splitlines()
@@ -160,6 +169,7 @@ class xinstall(install):
     def run(self):
         clean_tables()
         build_tables()
+        amalgamate_source()
         # add dirty version number
         dirty = dirty_version()
         # install Jupyter hook
@@ -176,19 +186,18 @@ class xinstall(install):
             restore_version()
 
 
-
 class xsdist(sdist):
     """Xonsh specialization of setuptools sdist class."""
     def make_release_tree(self, basedir, files):
         clean_tables()
         build_tables()
+        amalgamate_source()
         dirty = dirty_version()
         sdist.make_release_tree(self, basedir, files)
         if dirty:
             restore_version()
 
 
-#-----------------------------------------------------------------------------
 # Hack to overcome pip/setuptools problem on Win 10.  See:
 #   https://github.com/tomduck/pandoc-eqnos/issues/6
 #   https://github.com/pypa/pip/issues/2783
@@ -199,7 +208,7 @@ class install_scripts_quoted_shebang(install_scripts):
     def write_script(self, script_name, contents, mode="t", *ignored):
         shebang = str(contents.splitlines()[0])
         if shebang.startswith('#!') and ' ' in shebang[2:].strip() \
-          and '"' not in shebang:
+           and '"' not in shebang:
             quoted_shebang = '#!"%s"' % shebang[2:].strip()
             contents = contents.replace(shebang, quoted_shebang)
         super().write_script(script_name, contents, mode, *ignored)

tests/conftest.py

@@ -1,9 +1,10 @@
 import builtins
 import pytest
-from tools import DummyShell, sp, XonshBlockError
+from tools import DummyShell, sp
 import xonsh.built_ins
 from xonsh.built_ins import ensure_list_of_strs
 from xonsh.execer import Execer
+from xonsh.tools import XonshBlockError
 import glob
 
 

tests/test_history.py

@@ -8,43 +8,39 @@ from __future__ import unicode_literals, print_function
 import io
 import os
 import sys
+import shlex
 
 from xonsh.lazyjson import LazyJSON
-from xonsh.history import History
+from xonsh.history import History, _hist_create_parser, _hist_parse_args
 from xonsh import history
 
-
-HIST_TEST_KWARGS = dict(sessionid='SESSIONID', gc=False)
+import pytest
 
 
-def test_hist_init():
+@pytest.yield_fixture
+def hist():
+    h = History(filename='xonsh-HISTORY-TEST.json', here='yup', sessionid='SESSIONID', gc=False)
+    yield h
+    os.remove(h.filename)
+
+
+def test_hist_init(hist):
     """Test initialization of the shell history."""
-    FNAME = 'xonsh-SESSIONID.json'
-    FNAME += '.init'
-    History(filename=FNAME, here='yup', **HIST_TEST_KWARGS)
-    with LazyJSON(FNAME) as lj:
+    with LazyJSON(hist.filename) as lj:
         obs = lj['here']
     assert 'yup' == obs
-    os.remove(FNAME)
 
 
-def test_hist_append(xonsh_builtins):
+def test_hist_append(hist, xonsh_builtins):
     """Verify appending to the history works."""
-    FNAME = 'xonsh-SESSIONID.json'
-    FNAME += '.append'
-    hist = History(filename=FNAME, here='yup', **HIST_TEST_KWARGS)
     xonsh_builtins.__xonsh_env__['HISTCONTROL'] = set()
     hf = hist.append({'joco': 'still alive'})
     assert hf is None
     assert 'still alive' == hist.buffer[0]['joco']
-    os.remove(FNAME)
 
 
-def test_hist_flush(xonsh_builtins):
+def test_hist_flush(hist, xonsh_builtins):
     """Verify explicit flushing of the history works."""
-    FNAME = 'xonsh-SESSIONID.json'
-    FNAME += '.flush'
-    hist = History(filename=FNAME, here='yup', **HIST_TEST_KWARGS)
     hf = hist.flush()
     assert hf is None
     xonsh_builtins.__xonsh_env__['HISTCONTROL'] = set()
@@ -53,17 +49,12 @@ def test_hist_flush(xonsh_builtins):
     assert hf is not None
     while hf.is_alive():
         pass
-    with LazyJSON(FNAME) as lj:
+    with LazyJSON(hist.filename) as lj:
         obs = lj['cmds'][0]['joco']
     assert 'still alive' == obs
-    os.remove(FNAME)
 
 
-def test_cmd_field(xonsh_builtins):
-    """Test basic history behavior."""
-    FNAME = 'xonsh-SESSIONID.json'
-    FNAME += '.cmdfield'
-    hist = History(filename=FNAME, here='yup', **HIST_TEST_KWARGS)
+def test_cmd_field(hist, xonsh_builtins):
     # in-memory
     xonsh_builtins.__xonsh_env__['HISTCONTROL'] = set()
     hf = hist.append({'rtn': 1})
@@ -79,36 +70,25 @@ def test_cmd_field(xonsh_builtins):
     assert 1 == hist.rtns[0]
     assert 1 == hist.rtns[-1]
     assert None == hist.outs[-1]
-    os.remove(FNAME)
 
+def run_show_cmd(hist_args, commands, base_idx=0, step=1):
+    """Run and evaluate the output of the given show command."""
+    stdout = sys.stdout
+    stdout.seek(0, io.SEEK_SET)
+    stdout.truncate()
+    history.history_main(hist_args)
+    stdout.seek(0, io.SEEK_SET)
+    hist_lines = stdout.readlines()
+    assert len(commands) == len(hist_lines)
+    for idx, (cmd, actual) in enumerate(zip(commands, hist_lines)):
+        expected = ' {:d}: {:s}\n'.format(base_idx + idx * step, cmd)
+        assert expected == actual
 
-def test_show_cmd(xonsh_builtins):
+def test_show_cmd(hist, xonsh_builtins):
     """Verify that CLI history commands work."""
-    FNAME = 'xonsh-SESSIONID.json'
-    FNAME += '.show_cmd'
     cmds = ['ls', 'cat hello kitty', 'abc', 'def', 'touch me', 'grep from me']
-
-    def format_hist_line(idx, cmd):
-        """Construct a history output line."""
-        return ' {:d}: {:s}\n'.format(idx, cmd)
-
-    def run_show_cmd(hist_args, commands, base_idx=0, step=1):
-        """Run and evaluate the output of the given show command."""
-        stdout.seek(0, io.SEEK_SET)
-        stdout.truncate()
-        history._hist_main(hist, hist_args)
-        stdout.seek(0, io.SEEK_SET)
-        hist_lines = stdout.readlines()
-        assert len(commands) == len(hist_lines)
-        for idx, (cmd, actual) in enumerate(zip(commands, hist_lines)):
-            expected = format_hist_line(base_idx + idx * step, cmd)
-            assert expected == actual
-
-    hist = History(filename=FNAME, here='yup', **HIST_TEST_KWARGS)
-    stdout = io.StringIO()
-    saved_stdout = sys.stdout
-    sys.stdout = stdout
-
+    sys.stdout = io.StringIO()
+    xonsh_builtins.__xonsh_history__ = hist
     xonsh_builtins.__xonsh_env__['HISTCONTROL'] = set()
     for ts,cmd in enumerate(cmds):  # populate the shell history
         hist.append({'inp': cmd, 'rtn': 0, 'ts':(ts+1, ts+1.5)})
@@ -147,14 +127,11 @@ def test_show_cmd(xonsh_builtins):
     run_show_cmd(['show', '-4:-2'],
                            cmds[-4:-2], len(cmds) - 4)
 
-    sys.stdout = saved_stdout
-    os.remove(FNAME)
+    sys.stdout = sys.__stdout__
 
-def test_histcontrol(xonsh_builtins):
+
+def test_histcontrol(hist, xonsh_builtins):
     """Test HISTCONTROL=ignoredups,ignoreerr"""
-    FNAME = 'xonsh-SESSIONID.json'
-    FNAME += '.append'
-    hist = History(filename=FNAME, here='yup', **HIST_TEST_KWARGS)
 
     xonsh_builtins.__xonsh_env__['HISTCONTROL'] = 'ignoredups,ignoreerr'
     assert len(hist.buffer) == 0
@@ -209,4 +186,25 @@ def test_histcontrol(xonsh_builtins):
     assert '/bin/ls' == hist.buffer[-1]['inp']
     assert 0 == hist.buffer[-1]['rtn']
 
-    os.remove(FNAME)
+
+@pytest.mark.parametrize('args', [ '-h', '--help', 'show -h', 'show --help'])
+def test_parse_args_help(args, capsys):
+    with pytest.raises(SystemExit):
+        args = _hist_parse_args(shlex.split(args))
+    assert 'show this help message and exit' in capsys.readouterr()[0]
+
+
+@pytest.mark.parametrize('args, exp', [
+    ('', ('show', 'session', [])),
+    ('show', ('show', 'session', [])),
+    ('show session', ('show', 'session', [])),
+    ('show session 15', ('show', 'session', ['15'])),
+    ('show bash 3:5 15:66', ('show', 'bash', ['3:5', '15:66'])),
+    ('show zsh 3 5:6 16 9:3', ('show', 'zsh', ['3', '5:6', '16', '9:3'])),
+    ])
+def test_parser_show(args, exp):
+    args = _hist_parse_args(shlex.split(args))
+    action, session, slices = exp
+    assert args.action == action
+    assert args.session == session
+    assert args.slices == slices

tests/test_tools.py

@@ -14,16 +14,15 @@ from xonsh.lexer import Lexer
 from xonsh.tools import (
     EnvPath, always_false, always_true, argvquote,
     bool_or_int_to_str, bool_to_str, check_for_partial_string,
-    dynamic_cwd_tuple_to_str, ensure_int_or_slice, ensure_string,
+    dynamic_cwd_tuple_to_str, ensure_slice, ensure_string,
     env_path_to_str, escape_windows_cmd_string, executables_in,
     expand_case_matching, find_next_break, iglobpath, is_bool, is_bool_or_int,
-    is_callable, is_dynamic_cwd_width, is_env_path, is_float, is_int,
-    is_int_as_str, is_logfile_opt, is_slice_as_str, is_string,
-    is_string_or_callable, logfile_opt_to_str, str_to_env_path,
+    is_callable, is_dynamic_cwd_width, is_env_path, is_float, is_int, is_logfile_opt,
+    is_string_or_callable, logfile_opt_to_str, str_to_env_path, is_string,
     subexpr_from_unbalanced, subproc_toks, to_bool, to_bool_or_int,
     to_dynamic_cwd_tuple, to_logfile_opt, pathsep_to_set, set_to_pathsep,
     is_string_seq, pathsep_to_seq, seq_to_pathsep, is_nonstring_seq_of_strings,
-    pathsep_to_upper_seq, seq_to_upper_pathsep, expandvars
+    pathsep_to_upper_seq, seq_to_upper_pathsep, expandvars, is_int_as_str, is_slice_as_str
     )
 from xonsh.commands_cache import CommandsCache
 from xonsh.built_ins import expand_path
@@ -348,22 +347,6 @@ def test_is_int(inp, exp):
     assert exp == obs
 
 
-@pytest.mark.parametrize('inp, exp', [
-    ('42', True),
-    ('42.0', False),
-    (42, False),
-    ([42], False),
-    ([], False),
-    (None, False),
-    ('', False),
-    (False, False),
-    (True, False),
-])
-def test_is_int_as_str(inp, exp):
-    obs = is_int_as_str(inp)
-    assert exp == obs
-
-
 @pytest.mark.parametrize('inp, exp', [
     (42.0, True),
     (42.000101010010101010101001010101010001011100001101101011100, True),
@@ -382,29 +365,6 @@ def test_is_float(inp, exp):
     assert exp == obs
 
 
-@pytest.mark.parametrize('inp, exp', [
-    (42, False),
-    (None, False),
-    ('42', False),
-    ('-42', False),
-    (slice(1,2,3), False),
-    ([], False),
-    (False, False),
-    (True, False),
-    ('1:2:3', True),
-    ('1::3', True),
-    ('1:', True),
-    (':', True),
-    ('[1:2:3]', True),
-    ('(1:2:3)', True),
-    ('r', False),
-    ('r:11', False),
-])
-def test_is_slice_as_str(inp, exp):
-    obs = is_slice_as_str(inp)
-    assert exp == obs
-
-
 def test_is_string_true():
     assert is_string('42.0')
 
@@ -853,24 +813,52 @@ def test_bool_or_int_to_str(inp, exp):
 
 
 @pytest.mark.parametrize('inp, exp', [
-        (42, 42),
+        (42, slice(42, 43)),
         (None, slice(None, None, None)),
-        ('42', 42),
-        ('-42', -42),
+        ('42', slice(42, 43)),
+        ('-42', slice(-42, -41)),
         ('1:2:3', slice(1, 2, 3)),
         ('1::3', slice(1, None, 3)),
         (':', slice(None, None, None)),
         ('1:', slice(1, None, None)),
         ('[1:2:3]', slice(1, 2, 3)),
         ('(1:2:3)', slice(1, 2, 3)),
-        ('r', False),
-        ('r:11', False),
         ])
-def test_ensure_int_or_slice(inp, exp):
-    obs = ensure_int_or_slice(inp)
+def test_ensure_slice(inp, exp):
+    obs = ensure_slice(inp)
     assert exp == obs
 
 
+@pytest.mark.parametrize('inp, error', [
+    ('42.3', ValueError),
+    ('3:asd5:1', ValueError),
+    ('test' , ValueError),
+    ('6.53:100:5', ValueError),
+    ('4:-', ValueError),
+    ('2:15-:3', ValueError),
+    ('50:-:666', ValueError),
+    (object(), TypeError),
+    ([], TypeError)
+])
+def test_ensure_slice_invalid(inp, error):
+    with pytest.raises(error):
+        obs = ensure_slice(inp)
+
+
+@pytest.mark.parametrize('inp, exp', [
+    ('42', True),
+    ('42.0', False),
+    (42, False),
+    ([42], False),
+    ([], False),
+    (None, False),
+    ('', False),
+    (False, False),
+    (True, False),
+])
+def test_is_int_as_str(inp, exp):
+    obs = is_int_as_str(inp)
+    assert exp == obs
 @pytest.mark.parametrize('inp, exp', [
     ('20', False),
     ('20%', False),
@@ -884,6 +872,29 @@ def test_is_dynamic_cwd_width(inp, exp):
     assert exp == obs
 
 
+@pytest.mark.parametrize('inp, exp', [
+    (42, False),
+    (None, False),
+    ('42', False),
+    ('-42', False),
+    (slice(1,2,3), False),
+    ([], False),
+    (False, False),
+    (True, False),
+    ('1:2:3', True),
+    ('1::3', True),
+    ('1:', True),
+    (':', True),
+    ('[1:2:3]', True),
+    ('(1:2:3)', True),
+    ('r', False),
+    ('r:11', False),
+])
+def test_is_slice_as_str(inp, exp):
+    obs = is_slice_as_str(inp)
+    assert exp == obs
+
+
 @pytest.mark.parametrize('inp, exp', [
     ('throwback.log', True),
     ('', True),

tests/test_vox.py

@@ -0,0 +1,45 @@
+"""Vox tests"""
+
+import builtins
+import stat
+import os
+from xontrib.voxapi import Vox
+
+from tools import skip_if_on_conda
+
+
+@skip_if_on_conda
+def test_crud(xonsh_builtins, tmpdir):
+    """
+    Creates a virtual environment, gets it, enumerates it, and then deletes it.
+    """
+    xonsh_builtins.__xonsh_env__['VIRTUALENV_HOME'] = str(tmpdir)
+    vox = Vox()
+    vox.create('spam')
+    assert stat.S_ISDIR(tmpdir.join('spam').stat().mode)
+
+    env, bin = vox['spam']
+    assert env == str(tmpdir.join('spam'))
+    assert os.path.isdir(bin)
+
+    assert 'spam' in vox
+
+    del vox['spam']
+
+    assert not tmpdir.join('spam').check()
+
+
+@skip_if_on_conda
+def test_activate(xonsh_builtins, tmpdir):
+    """
+    Creates a virtual environment, gets it, enumerates it, and then deletes it.
+    """
+    xonsh_builtins.__xonsh_env__['VIRTUALENV_HOME'] = str(tmpdir)
+    # I consider the case that the user doesn't have a PATH set to be unreasonable
+    xonsh_builtins.__xonsh_env__.setdefault('PATH', [])
+    vox = Vox()
+    vox.create('spam')
+    vox.activate('spam')
+    assert xonsh_builtins.__xonsh_env__['VIRTUAL_ENV'] == vox['spam'].env
+    vox.deactivate()
+    assert 'VIRTUAL_ENV' not in xonsh_builtins.__xonsh_env__

tests/tools.py

@@ -2,19 +2,15 @@
 """Tests the xonsh lexer."""
 from __future__ import unicode_literals, print_function
 import sys
-import glob
 import builtins
 import platform
 import subprocess
 from collections import defaultdict
-from contextlib import contextmanager
 
 import pytest
 
 from xonsh.environ import Env
-from xonsh.built_ins import ensure_list_of_strs
 from xonsh.base_shell import BaseShell
-from xonsh.tools import XonshBlockError
 
 
 VER_3_4 = (3, 4)
@@ -23,11 +19,15 @@ VER_MAJOR_MINOR = sys.version_info[:2]
 VER_FULL = sys.version_info[:3]
 ON_DARWIN = (platform.system() == 'Darwin')
 ON_WINDOWS = (platform.system() == 'Windows')
-
+ON_CONDA = True in [conda in pytest.__file__ for conda
+                    in ['anaconda', 'miniconda']]
 
 # pytest skip decorators
 skip_if_py34 = pytest.mark.skipif(VER_MAJOR_MINOR < VER_3_5, reason="Py3.5+ only test")
 
+skip_if_on_conda = pytest.mark.skipif(ON_CONDA,
+                        reason="Conda and virtualenv _really_ hate each other")
+
 skip_if_on_windows = pytest.mark.skipif(ON_WINDOWS, reason='Unix stuff')
 
 skip_if_on_unix = pytest.mark.skipif(not ON_WINDOWS, reason='Windows stuff')

xonsh-in-docker.py

@@ -1,10 +1,9 @@
 #!/usr/bin/env python
-import sys 
 import subprocess
 import os
 import argparse
 
-program_description = """Build and run Xonsh in a fresh, controlled 
+program_description = """Build and run Xonsh in a fresh, controlled
     environment using docker """
 
 parser = argparse.ArgumentParser(description=program_description)
@@ -14,7 +13,7 @@ parser.add_argument('--python', '-p', default='3.4', metavar='python_version')
 parser.add_argument('--ptk', '-t', default='1.00', metavar='ptk_version')
 parser.add_argument('--keep', action='store_true')
 parser.add_argument('--build', action='store_true')
-parser.add_argument('--command', '-c', default='xonsh', 
+parser.add_argument('--command', '-c', default='xonsh',
                     metavar='command')
 
 args = parser.parse_args()
@@ -30,8 +29,8 @@ WORKDIR /xonsh
 ADD ./ ./
 RUN python setup.py install
 """.format(
-        python_version = args.python,
-        ptk_version = args.ptk)
+    python_version=args.python,
+    ptk_version=args.ptk)
 
 print('Building and running Xonsh')
 print('Using python ', args.python)
@@ -42,7 +41,7 @@ with open('./Dockerfile', 'w+') as f:
 
 env_string = ' '.join(args.env)
 
-subprocess.call(['docker', 'build', '-t' , 'xonsh', '.'])
+subprocess.call(['docker', 'build', '-t', 'xonsh', '.'])
 os.remove('./Dockerfile')
 
 if not args.build:

xonsh/__init__.py

@@ -1,4 +1,4 @@
-__version__ = '0.4.3'
+__version__ = '0.4.4'
 
 # amalgamate exclude jupyter_kernel parser_table parser_test_table pyghooks
 # amalgamate exclude winutils wizard
@@ -19,8 +19,8 @@ else:
         _sys.modules['xonsh.ansi_colors'] = __amalgam__
         codecache = __amalgam__
         _sys.modules['xonsh.codecache'] = __amalgam__
-        openpy = __amalgam__
-        _sys.modules['xonsh.openpy'] = __amalgam__
+        lazyimps = __amalgam__
+        _sys.modules['xonsh.lazyimps'] = __amalgam__
         platform = __amalgam__
         _sys.modules['xonsh.platform'] = __amalgam__
         pretty = __amalgam__
@@ -37,8 +37,6 @@ else:
         _sys.modules['xonsh.tokenize'] = __amalgam__
         tools = __amalgam__
         _sys.modules['xonsh.tools'] = __amalgam__
-        vox = __amalgam__
-        _sys.modules['xonsh.vox'] = __amalgam__
         ast = __amalgam__
         _sys.modules['xonsh.ast'] = __amalgam__
         contexts = __amalgam__
@@ -49,10 +47,10 @@ else:
         _sys.modules['xonsh.dirstack'] = __amalgam__
         foreign_shells = __amalgam__
         _sys.modules['xonsh.foreign_shells'] = __amalgam__
-        inspectors = __amalgam__
-        _sys.modules['xonsh.inspectors'] = __amalgam__
         lexer = __amalgam__
         _sys.modules['xonsh.lexer'] = __amalgam__
+        openpy = __amalgam__
+        _sys.modules['xonsh.openpy'] = __amalgam__
         proc = __amalgam__
         _sys.modules['xonsh.proc'] = __amalgam__
         xontribs = __amalgam__
@@ -63,6 +61,8 @@ else:
         _sys.modules['xonsh.environ'] = __amalgam__
         history = __amalgam__
         _sys.modules['xonsh.history'] = __amalgam__
+        inspectors = __amalgam__
+        _sys.modules['xonsh.inspectors'] = __amalgam__
         base_shell = __amalgam__
         _sys.modules['xonsh.base_shell'] = __amalgam__
         replay = __amalgam__

xonsh/aliases.py

@@ -14,16 +14,16 @@ from xonsh.foreign_shells import foreign_shell_data
 from xonsh.jobs import jobs, fg, bg, clean_jobs
 from xonsh.history import history_main
 from xonsh.platform import (ON_ANACONDA, ON_DARWIN, ON_WINDOWS, ON_FREEBSD,
-    scandir)
+                            scandir)
 from xonsh.proc import foreground
 from xonsh.replay import replay_main
 from xonsh.timings import timeit_alias
 from xonsh.tools import (XonshError, argvquote, escape_windows_cmd_string,
                          to_bool)
-from xonsh.vox import Vox
 from xonsh.xontribs import xontribs_main
 from xonsh.xoreutils import _which
-from xonsh.completers._aliases import completer_alias
+
+import xonsh.completers._aliases as xca
 
 
 class Aliases(abc.MutableMapping):
@@ -143,7 +143,6 @@ class Aliases(abc.MutableMapping):
                 p.pretty(dict(self))
 
 
-
 def xonsh_exit(args, stdin=None):
     """Sends signal to exit shell."""
     if not clean_jobs():
@@ -284,9 +283,8 @@ def xexec(args, stdin=None):
     """Replaces current process with command specified and passes in the
     current xonsh environment.
     """
-    env = builtins.__xonsh_env__
-    denv = env.detype()
     if len(args) > 0:
+        denv = builtins.__xonsh_env__.detype()
         try:
             os.execvpe(args[0], args, denv)
         except FileNotFoundError as e:
@@ -298,9 +296,9 @@ def xexec(args, stdin=None):
 
 @lazyobject
 def _BANG_N_PARSER():
-    parser = argparse.ArgumentParser('!n', usage='!n <n>',
-                description="Re-runs the nth command as specified in the "
-                            "argument.")
+    parser = argparse.ArgumentParser(
+        '!n', usage='!n <n>',
+        description="Re-runs the nth command as specified in the argument.")
     parser.add_argument('n', type=int, help='the command to rerun, may be '
                                             'negative')
     return parser
@@ -327,6 +325,7 @@ def bang_bang(args, stdin=None):
 
 class AWitchAWitch(argparse.Action):
     SUPPRESS = '==SUPPRESS=='
+
     def __init__(self, option_strings, version=None, dest=SUPPRESS,
                  default=SUPPRESS, **kwargs):
         super().__init__(option_strings=option_strings, dest=dest,
@@ -349,7 +348,7 @@ def which(args, stdin=None, stdout=None, stderr=None):
     parser = argparse.ArgumentParser('which', description=desc)
     parser.add_argument('args', type=str, nargs='+',
                         help='The executables or aliases to search for')
-    parser.add_argument('-a','--all', action='store_true', dest='all',
+    parser.add_argument('-a', '--all', action='store_true', dest='all',
                         help='Show all matches in $PATH and xonsh.aliases')
     parser.add_argument('-s', '--skip-alias', action='store_true',
                         help='Do not search in xonsh.aliases', dest='skip')
@@ -398,12 +397,14 @@ def which(args, stdin=None, stdout=None, stderr=None):
         # skip alias check if user asks to skip
         if (arg in builtins.aliases and not pargs.skip):
             if pargs.plain or not pargs.verbose:
-                if isinstance(builtins.aliases[arg], list):
+                if not callable(builtins.aliases[arg]):
                     print(' '.join(builtins.aliases[arg]), file=stdout)
                 else:
                     print(arg, file=stdout)
             else:
                 print("aliases['{}'] = {}".format(arg, builtins.aliases[arg]), file=stdout)
+                if callable(builtins.aliases[arg]):
+                    builtins.__xonsh_superhelp__(builtins.aliases[arg])
             nmatches += 1
             if not pargs.all:
                 continue
@@ -458,12 +459,6 @@ def trace(args, stdin=None):
         pass
 
 
-def vox(args, stdin=None):
-    """Runs Vox environment manager."""
-    vox = Vox()
-    return vox(args, stdin=stdin)
-
-
 def showcmd(args, stdin=None):
     """usage: showcmd [-h|--help|cmd args]
 
@@ -513,10 +508,9 @@ def make_default_aliases():
         'scp-resume': ['rsync', '--partial', '-h', '--progress', '--rsh=ssh'],
         'showcmd': showcmd,
         'ipynb': ['jupyter', 'notebook', '--no-browser'],
-        'vox': vox,
         'which': which,
         'xontrib': xontribs_main,
-        'completer': completer_alias
+        'completer': xca.completer_alias
     }
     if ON_WINDOWS:
         # Borrow builtin commands from cmd.exe.

xonsh/ansi_colors.py

@@ -340,6 +340,7 @@ RGB_256 = LazyObject(lambda: {
 RE_RGB3 = LazyObject(lambda: re.compile(r'(.)(.)(.)'), globals(), 'RE_RGB3')
 RE_RGB6 = LazyObject(lambda: re.compile(r'(..)(..)(..)'), globals(), 'RE_RGB6')
 
+
 def rgb_to_ints(rgb):
     """Converts an RGB string into a tuple of ints."""
     if len(rgb) == 6:
@@ -366,8 +367,10 @@ def rgb_to_256(rgb):
             if s <= part <= b:
                 s1 = abs(s - part)
                 b1 = abs(b - part)
-                if s1 < b1: closest = s
-                else: closest = b
+                if s1 < b1:
+                    closest = s
+                else:
+                    closest = b
                 res.append(closest)
                 break
             i += 1
@@ -530,6 +533,7 @@ def _default_style():
         }
     return style
 
+
 def _monokai_style():
     style = {
         'NO_COLOR': '0',
@@ -554,9 +558,9 @@ def _monokai_style():
     return style
 
 
-#############################################################
-#############   Auto-generated below this line   ############
-#############################################################
+####################################
+# Auto-generated below this line   #
+####################################
 
 def _algol_style():
     style = {
@@ -605,6 +609,7 @@ def _algol_nu_style():
     _ansi_expand_style(style)
     return style
 
+
 def _autumn_style():
     style = {
         'BLACK': '38;5;18',
@@ -628,6 +633,7 @@ def _autumn_style():
     _ansi_expand_style(style)
     return style
 
+
 def _borland_style():
     style = {
         'BLACK': '38;5;16',
@@ -771,6 +777,7 @@ def _igor_style():
     _ansi_expand_style(style)
     return style
 
+
 def _lovelace_style():
     style = {
         'BLACK': '38;5;59',
@@ -866,6 +873,7 @@ def _native_style():
     _ansi_expand_style(style)
     return style
 
+
 def _paraiso_dark_style():
     style = {
         'BLACK': '38;5;95',
@@ -913,6 +921,7 @@ def _paraiso_light_style():
     _ansi_expand_style(style)
     return style
 
+
 def _pastie_style():
     style = {
         'BLACK': '38;5;16',
@@ -960,6 +969,7 @@ def _perldoc_style():
     _ansi_expand_style(style)
     return style
 
+
 def _rrt_style():
     style = {
         'BLACK': '38;5;09',

xonsh/ast.py

@@ -2,17 +2,19 @@
 """The xonsh abstract syntax tree node."""
 # These are imported into our module namespace for the benefit of parser.py.
 # pylint: disable=unused-import
-from ast import Module, Num, Expr, Str, Bytes, UnaryOp, UAdd, USub, Invert, \
-    BinOp, Add, Sub, Mult, Div, FloorDiv, Mod, Pow, Compare, Lt, Gt, \
-    LtE, GtE, Eq, NotEq, In, NotIn, Is, IsNot, Not, BoolOp, Or, And, \
-    Subscript, Load, Slice, ExtSlice, List, Tuple, Set, Dict, AST, NameConstant, \
-    Name, GeneratorExp, Store, comprehension, ListComp, SetComp, DictComp, \
-    Assign, AugAssign, BitXor, BitAnd, BitOr, LShift, RShift, Assert, Delete, \
-    Del, Pass, Raise, Import, alias, ImportFrom, Continue, Break, Yield, \
-    YieldFrom, Return, IfExp, Lambda, arguments, arg, Call, keyword, \
-    Attribute, Global, Nonlocal, If, While, For, withitem, With, Try, \
-    ExceptHandler, FunctionDef, ClassDef, Starred, NodeTransformer, \
+from ast import (
+    Module, Num, Expr, Str, Bytes, UnaryOp, UAdd, USub, Invert,
+    BinOp, Add, Sub, Mult, Div, FloorDiv, Mod, Pow, Compare, Lt, Gt,
+    LtE, GtE, Eq, NotEq, In, NotIn, Is, IsNot, Not, BoolOp, Or, And,
+    Subscript, Load, Slice, ExtSlice, List, Tuple, Set, Dict, AST, NameConstant,
+    Name, GeneratorExp, Store, comprehension, ListComp, SetComp, DictComp,
+    Assign, AugAssign, BitXor, BitAnd, BitOr, LShift, RShift, Assert, Delete,
+    Del, Pass, Raise, Import, alias, ImportFrom, Continue, Break, Yield,
+    YieldFrom, Return, IfExp, Lambda, arguments, arg, Call, keyword,
+    Attribute, Global, Nonlocal, If, While, For, withitem, With, Try,
+    ExceptHandler, FunctionDef, ClassDef, Starred, NodeTransformer,
     Interactive, Expression, Index, literal_eval, dump, walk, increment_lineno
+)
 from ast import Ellipsis as EllipsisNode
 # pylint: enable=unused-import
 import textwrap
@@ -464,7 +466,6 @@ class CtxAwareTransformer(NodeTransformer):
         return node
 
 
-
 def pdump(s, **kwargs):
     """performs a pretty dump of an AST node."""
     if isinstance(s, AST):
@@ -474,7 +475,7 @@ def pdump(s, **kwargs):
     lens = len(s) + 1
     if lens == 1:
         return s
-    i = min([s.find(o)%lens for o in openers])
+    i = min([s.find(o) % lens for o in openers])
     if i == lens - 1:
         return s
     closer = closers[openers.find(s[i])]
@@ -490,7 +491,7 @@ def pdump(s, **kwargs):
     return pre + mid + post
 
 
-def pprint(s, *, sep=None, end=None, file=None, flush=False, **kwargs):
+def pprint_ast(s, *, sep=None, end=None, file=None, flush=False, **kwargs):
     """Performs a pretty print of the AST nodes."""
     print(pdump(s, **kwargs), sep=sep, end=end, file=file, flush=flush)
 

xonsh/base_shell.py

@@ -7,7 +7,7 @@ import time
 import builtins
 
 from xonsh.tools import (XonshError, escape_windows_cmd_string, print_exception,
-    DefaultNotGiven)
+                         DefaultNotGiven)
 from xonsh.platform import HAS_PYGMENTS, ON_WINDOWS
 from xonsh.codecache import (should_use_cache, code_cache_name,
                              code_cache_check, get_cache_filename,
@@ -23,6 +23,7 @@ class _TeeOut(object):
         self.buffer = buf
         self.stdout = sys.stdout
         self.encoding = self.stdout.encoding
+        self.errors = self.stdout.errors
         sys.stdout = self
 
     def __del__(self):
@@ -34,7 +35,7 @@ class _TeeOut(object):
 
     def write(self, data):
         """Writes data to the original stdout and the buffer."""
-        #data = data.replace('\001', '').replace('\002', '')
+        # data = data.replace('\001', '').replace('\002', '')
         self.stdout.write(data)
         self.buffer.write(data)
 
@@ -55,6 +56,7 @@ class _TeeErr(object):
         self.buffer = buf
         self.stderr = sys.stderr
         self.encoding = self.stderr.encoding
+        self.errors = self.stderr.errors
         sys.stderr = self
 
     def __del__(self):
@@ -66,7 +68,7 @@ class _TeeErr(object):
 
     def write(self, data):
         """Writes data to the original stderr and the buffer."""
-        #data = data.replace('\001', '').replace('\002', '')
+        # data = data.replace('\001', '').replace('\002', '')
         self.stderr.write(data)
         self.buffer.write(data)
 
@@ -226,7 +228,6 @@ class BaseShell(object):
 
     def settitle(self):
         """Sets terminal title."""
-        _ = self
         env = builtins.__xonsh_env__  # pylint: disable=no-member
         term = env.get('TERM', None)
         # Shells running in emacs sets TERM to "dumb" or "eterm-color".
@@ -242,7 +243,11 @@ class BaseShell(object):
             t = escape_windows_cmd_string(t)
             os.system('title {}'.format(t))
         else:
-            os.write(1, "\x1b]2;{0}\x07".format(t).encode())
+            with open(1, 'wb', closefd=False) as f:
+                # prevent xonsh from answering interative questions
+                # on the next command by writing the title
+                f.write("\x1b]0;{0}\x07".format(t).encode())
+                f.flush()
 
     @property
     def prompt(self):
@@ -266,7 +271,6 @@ class BaseShell(object):
 
     def _append_history(self, tee_out=None, **info):
         """Append information about the command to the history."""
-        _ = self
         hist = builtins.__xonsh_history__  # pylint: disable=no-member
         info['rtn'] = hist.last_cmd_rtn
         tee_out = tee_out or None

xonsh/built_ins.py

@@ -26,19 +26,22 @@ from xonsh.environ import Env, default_env, locate_binary
 from xonsh.foreign_shells import load_foreign_aliases
 from xonsh.jobs import add_job, wait_for_active_job
 from xonsh.platform import ON_POSIX, ON_WINDOWS
-from xonsh.proc import (ProcProxy, SimpleProcProxy, ForegroundProcProxy,
-                        SimpleForegroundProcProxy, TeePTYProc,
-                        CompletedCommand, HiddenCompletedCommand)
+from xonsh.proc import (
+    ProcProxy, SimpleProcProxy, ForegroundProcProxy,
+    SimpleForegroundProcProxy, TeePTYProc, pause_call_resume, CompletedCommand,
+    HiddenCompletedCommand)
 from xonsh.tools import (
     suggest_commands, expandvars, globpath, XonshError,
     XonshCalledProcessError, XonshBlockError
 )
 from xonsh.commands_cache import CommandsCache
 
+import xonsh.completers.init
 
 BUILTINS_LOADED = False
 INSPECTOR = LazyObject(Inspector, globals(), 'INSPECTOR')
 
+
 @lazyobject
 def AT_EXIT_SIGNALS():
     sigs = (signal.SIGABRT, signal.SIGFPE, signal.SIGILL, signal.SIGSEGV,
@@ -71,6 +74,7 @@ def resetting_signal_handle(sig, f):
     once the new handle is finished.
     """
     oldh = signal.getsignal(sig)
+
     def newh(s=None, frame=None):
         f(s, frame)
         signal.signal(sig, oldh)
@@ -157,7 +161,7 @@ def pathsearch(func, s, pymode=False):
     if (not callable(func) or
             len(inspect.signature(func).parameters) != 1):
         error = "%r is not a known path search function"
-        raise XonshError(error % searchfunc)
+        raise XonshError(error % func)
     o = func(s)
     no_match = [] if pymode else [s]
     return o if len(o) != 0 else no_match
@@ -530,12 +534,10 @@ def run_subproc(cmds, captured=False):
         })
     if (env.get('XONSH_INTERACTIVE') and
             not env.get('XONSH_STORE_STDOUT') and
-            not _capture_streams):
+            not _capture_streams and
+            hasattr(builtins, '__xonsh_shell__')):
         # set title here to get current command running
-        try:
-            builtins.__xonsh_shell__.settitle()
-        except AttributeError:
-            pass
+        pause_call_resume(prev_proc, builtins.__xonsh_shell__.settitle)
     if background:
         return
     if prev_is_proxy:
@@ -569,7 +571,7 @@ def run_subproc(cmds, captured=False):
             output = output.replace('\r\n', '\n')
         else:
             hist.last_cmd_out = output
-        if captured == 'object': # get stderr as well
+        if captured == 'object':  # get stderr as well
             named = _stderr_name is not None
             unnamed = prev_proc.stderr not in {None, sys.stderr}
             if named:
@@ -711,6 +713,7 @@ def load_builtins(execer=None, config=None, login=False, ctx=None):
     builtins.__xonsh_all_jobs__ = {}
     builtins.__xonsh_ensure_list_of_strs__ = ensure_list_of_strs
     builtins.__xonsh_list_of_strs_or_callables__ = list_of_strs_or_callables
+    builtins.__xonsh_completers__ = xonsh.completers.init.default_completers()
     # public built-ins
     builtins.XonshError = XonshError
     builtins.XonshBlockError = XonshBlockError
@@ -775,6 +778,7 @@ def unload_builtins():
              '__xonsh_subproc_uncaptured__',
              '__xonsh_execer__',
              '__xonsh_commands_cache__',
+             '__xonsh_completers__',
              'XonshError',
              'XonshBlockError',
              'XonshCalledProcessError',

xonsh/codecache.py

@@ -6,6 +6,7 @@ import builtins
 
 from xonsh.lazyasd import lazyobject
 
+
 def _splitpath(path, sofar=[]):
     folder, path = os.path.split(path)
     if path == "":
@@ -35,6 +36,7 @@ def _make_if_not_exists(dirname):
     if not os.path.isdir(dirname):
         os.makedirs(dirname)
 
+
 def should_use_cache(execer, mode):
     """
     Return ``True`` if caching has been enabled for this mode (through command
@@ -42,7 +44,7 @@ def should_use_cache(execer, mode):
     """
     if mode == 'exec':
         return ((execer.scriptcache or
-                    execer.cacheall) and
+                 execer.cacheall) and
                 (builtins.__xonsh_env__['XONSH_CACHE_SCRIPTS'] or
                     builtins.__xonsh_env__['XONSH_CACHE_EVERYTHING']))
     else:

xonsh/completers/__init__.py

@@ -1,30 +1,38 @@
-import builtins
-from collections import OrderedDict
-
-from xonsh.completers.pip import complete_pip
-from xonsh.completers.man import complete_from_man
-from xonsh.completers.bash import complete_from_bash
-from xonsh.completers.base import complete_base
-from xonsh.completers.path import complete_path
-from xonsh.completers.dirs import complete_cd, complete_rmdir
-from xonsh.completers.python import (complete_python, complete_import,
-                                     complete_python_mode)
-from xonsh.completers.commands import complete_skipper
-from xonsh.completers.completer import complete_completer
-
-completers = OrderedDict([
-    ('python_mode', complete_python_mode),
-    ('base', complete_base),
-    ('completer', complete_completer),
-    ('skip', complete_skipper),
-    ('pip', complete_pip),
-    ('cd', complete_cd),
-    ('rmdir', complete_rmdir),
-    ('bash', complete_from_bash),
-    ('man', complete_from_man),
-    ('import', complete_import),
-    ('python', complete_python),
-    ('path', complete_path),
-])
-
-builtins.__xonsh_completers__ = completers
+# amalgamate exclude
+import os as _os
+if _os.getenv('XONSH_DEBUG', ''):
+    pass
+else:
+    import sys as _sys
+    try:
+        from xonsh.completers import __amalgam__
+        bash = __amalgam__
+        _sys.modules['xonsh.completers.bash'] = __amalgam__
+        completer = __amalgam__
+        _sys.modules['xonsh.completers.completer'] = __amalgam__
+        pip = __amalgam__
+        _sys.modules['xonsh.completers.pip'] = __amalgam__
+        tools = __amalgam__
+        _sys.modules['xonsh.completers.tools'] = __amalgam__
+        _aliases = __amalgam__
+        _sys.modules['xonsh.completers._aliases'] = __amalgam__
+        commands = __amalgam__
+        _sys.modules['xonsh.completers.commands'] = __amalgam__
+        man = __amalgam__
+        _sys.modules['xonsh.completers.man'] = __amalgam__
+        path = __amalgam__
+        _sys.modules['xonsh.completers.path'] = __amalgam__
+        python = __amalgam__
+        _sys.modules['xonsh.completers.python'] = __amalgam__
+        base = __amalgam__
+        _sys.modules['xonsh.completers.base'] = __amalgam__
+        dirs = __amalgam__
+        _sys.modules['xonsh.completers.dirs'] = __amalgam__
+        init = __amalgam__
+        _sys.modules['xonsh.completers.init'] = __amalgam__
+        del __amalgam__
+    except ImportError:
+        pass
+    del _sys
+del _os
+# amalgamate end

xonsh/completers/_aliases.py

@@ -1,14 +1,17 @@
 import builtins
+import collections
 
-from collections import OrderedDict
+import xonsh.lazyasd as xl
 
 from xonsh.completers.tools import justify
 
-VALID_ACTIONS = frozenset({'add', 'remove', 'list'})
+
+VALID_ACTIONS = xl.LazyObject(lambda: frozenset({'add', 'remove', 'list'}),
+                              globals(), 'VALID_ACTIONS')
 
 
 def _add_one_completer(name, func, loc='end'):
-    new = OrderedDict()
+    new = collections.OrderedDict()
     if loc == 'start':
         new[name] = func
         for (k, v) in builtins.__xonsh_completers__.items():
@@ -117,25 +120,22 @@ def completer_alias(args, stdin=None):
         func = _list_completers
     return func(args[1:], stdin=stdin)
 
-COMPLETER_LIST_HELP_STR = """
-completer list: list the active completers, in order
+COMPLETER_LIST_HELP_STR = """completer list: ordered list the active completers
 
 Usage:
     completer remove
-""".lstrip()
+"""
 
-COMPLETER_REMOVE_HELP_STR = """
-completer remove: removes a completer from xonsh
+COMPLETER_REMOVE_HELP_STR = """completer remove: removes a completer from xonsh
 
 Usage:
     completer remove NAME
 
 NAME is a unique name of a completer (run "completer list" to see the current
      completers in order)
-""".lstrip()
+"""
 
-COMPLETER_ADD_HELP_STR = """
-completer add: adds a new completer to xonsh
+COMPLETER_ADD_HELP_STR = """completer add: adds a new completer to xonsh
 
 Usage:
     completer add NAME FUNC [POS]
@@ -172,4 +172,4 @@ POS (optional) is a position into the list of completers at which the new
                  be added before the completer named KEY
 
      If POS is not provided, the default value is "start"
-""".lstrip()
+"""

xonsh/completers/base.py

@@ -1,6 +1,4 @@
 """Base completer for xonsh."""
-from collections import Sequence
-
 from xonsh.completers.path import complete_path
 from xonsh.completers.python import complete_python
 from xonsh.completers.commands import complete_command

xonsh/completers/bash.py

@@ -2,16 +2,17 @@ import os
 import re
 import shlex
 import pickle
+import hashlib
+import pathlib
 import builtins
 import subprocess
 
-import hashlib
+import xonsh.lazyasd as xl
+import xonsh.platform as xp
 
-from pathlib import Path
 
-from xonsh.platform import ON_WINDOWS
-
-RE_DASHF = re.compile(r'-F\s+(\w+)')
+RE_DASHF = xl.LazyObject(lambda: re.compile(r'-F\s+(\w+)'),
+                         globals(), 'RE_DASHF')
 
 INITED = False
 
@@ -34,13 +35,6 @@ for ((i=0;i<${{#COMPREPLY[*]}};i++)) do echo ${{COMPREPLY[i]}}; done
 """
 
 
-if ON_WINDOWS:
-    from xonsh.platform import windows_bash_command
-    BASH_COMMAND = windows_bash_command()
-else:
-    BASH_COMMAND = 'bash'
-
-
 def update_bash_completion():
     global BASH_COMPLETE_FUNCS, BASH_COMPLETE_FILES, BASH_COMPLETE_HASH
     global CACHED_FUNCS, CACHED_FILES, CACHED_HASH, INITED
@@ -115,7 +109,7 @@ def complete_from_bash(prefix, line, begidx, endidx, ctx):
         end=endidx + 1, prefix=prefix, prev=shlex.quote(prev))
     try:
         out = subprocess.check_output(
-            [BASH_COMMAND], input=script, universal_newlines=True,
+            [xp.bash_command()], input=script, universal_newlines=True,
             stderr=subprocess.PIPE, env=builtins.__xonsh_env__.detype())
     except (subprocess.CalledProcessError, FileNotFoundError):
         out = ''
@@ -157,7 +151,7 @@ def _load_bash_complete_files():
     func_files = {}
     for line in out.splitlines():
         parts = line.split()
-        if ON_WINDOWS:
+        if xp.ON_WINDOWS:
             parts = [parts[0], ' '.join(parts[2:])]
         func_files[parts[0]] = parts[-1]
     BASH_COMPLETE_FILES = {
@@ -170,8 +164,9 @@ def _load_bash_complete_files():
 def _source_completions(source):
     try:
         return subprocess.check_output(
-            [BASH_COMMAND], input='\n'.join(source), universal_newlines=True,
-            env=builtins.__xonsh_env__.detype(), stderr=subprocess.DEVNULL)
+            [xp.bash_command()], input='\n'.join(source),
+            universal_newlines=True, env=builtins.__xonsh_env__.detype(),
+            stderr=subprocess.DEVNULL)
     except FileNotFoundError:
         return ''
 
@@ -179,7 +174,7 @@ def _source_completions(source):
 def _collect_completions_sources():
     sources = []
     completers = builtins.__xonsh_env__.get('BASH_COMPLETIONS', ())
-    paths = (Path(x) for x in completers)
+    paths = (pathlib.Path(x) for x in completers)
     for path in paths:
         if path.is_file():
             sources.append('source "{}"'.format(path.as_posix()))

xonsh/completers/commands.py

@@ -1,8 +1,9 @@
 import os
 import builtins
 
-from xonsh.tools import executables_in
-from xonsh.platform import ON_WINDOWS
+import xonsh.tools as xt
+import xonsh.platform as xp
+
 from xonsh.completers.tools import get_filter_function
 
 SKIP_TOKENS = {'sudo', 'time', 'timeit', 'which', 'showcmd', 'man'}
@@ -16,14 +17,13 @@ def complete_command(cmd, line, start, end, ctx):
     out = {s + space
            for s in builtins.__xonsh_commands_cache__
            if get_filter_function()(s, cmd)}
-    if ON_WINDOWS:
-        out |= {i
-                for i in executables_in('.')
+    if xp.ON_WINDOWS:
+        out |= {i for i in xt.executables_in('.')
                 if i.startswith(cmd)}
     base = os.path.basename(cmd)
     if os.path.isdir(base):
         out |= {os.path.join(base, i)
-                for i in executables_in(base)
+                for i in xt.executables_in(base)
                 if i.startswith(cmd)}
     return out
 
@@ -51,6 +51,5 @@ def complete_skipper(cmd, line, start, end, ctx):
                          start - len(first) - 1,
                          end - len(first) - 1,
                          ctx)
-
     else:
         return set()

xonsh/completers/dirs.py

@@ -1,5 +1,3 @@
-import os
-
 from xonsh.completers.man import complete_from_man
 from xonsh.completers.path import complete_dir
 

xonsh/completers/init.py

@@ -0,0 +1,31 @@
+"""Constructor for xonsh completer objects."""
+import collections
+
+from xonsh.completers.pip import complete_pip
+from xonsh.completers.man import complete_from_man
+from xonsh.completers.bash import complete_from_bash
+from xonsh.completers.base import complete_base
+from xonsh.completers.path import complete_path
+from xonsh.completers.dirs import complete_cd, complete_rmdir
+from xonsh.completers.python import (complete_python, complete_import,
+                                     complete_python_mode)
+from xonsh.completers.commands import complete_skipper
+from xonsh.completers.completer import complete_completer
+
+
+def default_completers():
+    """Creates a copy of the default completers."""
+    return collections.OrderedDict([
+        ('python_mode', complete_python_mode),
+        ('base', complete_base),
+        ('completer', complete_completer),
+        ('skip', complete_skipper),
+        ('pip', complete_pip),
+        ('cd', complete_cd),
+        ('rmdir', complete_rmdir),
+        ('bash', complete_from_bash),
+        ('man', complete_from_man),
+        ('import', complete_import),
+        ('python', complete_python),
+        ('path', complete_path),
+        ])

xonsh/completers/man.py

@@ -4,13 +4,22 @@ import pickle
 import builtins
 import subprocess
 
+import xonsh.lazyasd as xl
+
 from xonsh.completers.tools import get_filter_function
 
 OPTIONS = None
 OPTIONS_PATH = None
 
-SCRAPE_RE = re.compile(r'^(?:\s*(?:-\w|--[a-z0-9-]+)[\s,])+', re.M)
-INNER_OPTIONS_RE = re.compile(r'-\w|--[a-z0-9-]+')
+
+@xl.lazyobject
+def SCRAPE_RE():
+    return re.compile(r'^(?:\s*(?:-\w|--[a-z0-9-]+)[\s,])+', re.M)
+
+
+@xl.lazyobject
+def INNER_OPTIONS_RE():
+    return re.compile(r'-\w|--[a-z0-9-]+')
 
 
 def complete_from_man(prefix, line, start, end, ctx):

xonsh/completers/path.py

@@ -1,25 +1,27 @@
 import os
-import re
 import ast
 import builtins
 
-from xonsh.platform import ON_WINDOWS
-from xonsh.tools import (subexpr_from_unbalanced, get_sep,
-                         check_for_partial_string, RE_STRING_START,
-                         iglobpath, levenshtein)
+import xonsh.tools as xt
+import xonsh.platform as xp
+import xonsh.lazyasd as xl
 
 from xonsh.completers.tools import get_filter_function
 
-CHARACTERS_NEED_QUOTES = ' `\t\r\n${}*()"\',?&'
-if ON_WINDOWS:
-    CHARACTERS_NEED_QUOTES += '%'
+
+@xl.lazyobject
+def CHARACTERS_NEED_QUOTES():
+    cnq = ' `\t\r\n${}*()"\',?&'
+    if xp.ON_WINDOWS:
+        cnq += '%'
+    return cnq
 
 
 def _path_from_partial_string(inp, pos=None):
     if pos is None:
         pos = len(inp)
     partial = inp[:pos]
-    startix, endix, quote = check_for_partial_string(partial)
+    startix, endix, quote = xt.check_for_partial_string(partial)
     _post = ""
     if startix is None:
         return None
@@ -33,7 +35,7 @@ def _path_from_partial_string(inp, pos=None):
             else:
                 return None
         string = partial[startix:endix]
-    end = re.sub(RE_STRING_START, '', quote)
+    end = xt.RE_STRING_START.sub('', quote)
     _string = string
     if not _string.endswith(end):
         _string = _string + end
@@ -54,19 +56,17 @@ def _normpath(p):
     and '/' at the end. On windows it does the same with backslashes
     """
     initial_dotslash = p.startswith(os.curdir + os.sep)
-    initial_dotslash |= (ON_WINDOWS and p.startswith(os.curdir + os.altsep))
+    initial_dotslash |= (xp.ON_WINDOWS and p.startswith(os.curdir + os.altsep))
     p = p.rstrip()
     trailing_slash = p.endswith(os.sep)
-    trailing_slash |= (ON_WINDOWS and p.endswith(os.altsep))
+    trailing_slash |= (xp.ON_WINDOWS and p.endswith(os.altsep))
     p = os.path.normpath(p)
     if initial_dotslash and p != '.':
         p = os.path.join(os.curdir, p)
     if trailing_slash:
         p = os.path.join(p, '')
-
-    if ON_WINDOWS and builtins.__xonsh_env__.get('FORCE_POSIX_PATHS'):
+    if xp.ON_WINDOWS and builtins.__xonsh_env__.get('FORCE_POSIX_PATHS'):
         p = p.replace(os.sep, os.altsep)
-
     return p
 
 
@@ -90,7 +90,7 @@ def _env(prefix):
 
 
 def _dots(prefix):
-    slash = get_sep()
+    slash = xt.get_sep()
     if slash == '\\':
         slash = ''
     if prefix in {'', '.'}:
@@ -108,8 +108,8 @@ def _add_cdpaths(paths, prefix):
     glob_sorted = env.get('GLOB_SORTED')
     for cdp in env.get('CDPATH'):
         test_glob = os.path.join(cdp, prefix) + '*'
-        for s in iglobpath(test_glob, ignore_case=(not csc),
-                           sort_result=glob_sorted):
+        for s in xt.iglobpath(test_glob, ignore_case=(not csc),
+                              sort_result=glob_sorted):
             if os.path.isdir(s):
                 paths.add(os.path.basename(s))
 
@@ -129,7 +129,7 @@ def _quote_paths(paths, start, end):
     space = ' '
     backslash = '\\'
     double_backslash = '\\\\'
-    slash = get_sep()
+    slash = xt.get_sep()
     orig_start = start
     orig_end = end
     for s in paths:
@@ -166,9 +166,9 @@ def _joinpath(path):
     elif len(path) == 0:
         return ''
     elif path == ('',):
-        return get_sep()
+        return xt.get_sep()
     elif path[0] == '':
-        return get_sep() + _normpath(os.path.join(*path))
+        return xt.get_sep() + _normpath(os.path.join(*path))
     else:
         return _normpath(os.path.join(*path))
 
@@ -177,7 +177,7 @@ def _splitpath(path):
     # convert a path into an intermediate tuple representation
     # if this tuple starts with '', it means that the path was an absolute path
     path = _normpath(path)
-    if path.startswith(get_sep()):
+    if path.startswith(xt.get_sep()):
         pre = ('', )
     else:
         pre = ()
@@ -227,7 +227,7 @@ def _expand_one(sofar, nextone, csc):
     glob_sorted = builtins.__xonsh_env__.get('GLOB_SORTED')
     for i in sofar:
         _glob = os.path.join(_joinpath(i), '*') if i is not None else '*'
-        for j in iglobpath(_glob, sort_result=glob_sorted):
+        for j in xt.iglobpath(_glob, sort_result=glob_sorted):
             j = os.path.basename(j)
             if subsequence_match(j, nextone, csc):
                 out.add((i or ()) + (j, ))
@@ -251,8 +251,8 @@ def complete_path(prefix, line, start, end, ctx, cdpath=True, filtfunc=None):
     env = builtins.__xonsh_env__
     csc = env.get('CASE_SENSITIVE_COMPLETIONS')
     glob_sorted = env.get('GLOB_SORTED')
-    for s in iglobpath(prefix + '*', ignore_case=(not csc),
-                       sort_result=glob_sorted):
+    for s in xt.iglobpath(prefix + '*', ignore_case=(not csc),
+                          sort_result=glob_sorted):
         paths.add(s)
     if len(paths) == 0 and env.get('SUBSEQUENCE_PATH_COMPLETION'):
         # this block implements 'subsequence' matching, similar to fish and zsh.
@@ -272,9 +272,10 @@ def complete_path(prefix, line, start, end, ctx, cdpath=True, filtfunc=None):
             paths |= {_joinpath(i) for i in matches_so_far}
     if len(paths) == 0 and env.get('FUZZY_PATH_COMPLETION'):
         threshold = env.get('SUGGEST_THRESHOLD')
-        for s in iglobpath(os.path.dirname(prefix) + '*', ignore_case=(not csc),
-                           sort_result=glob_sorted):
-            if levenshtein(prefix, s, threshold) < threshold:
+        for s in xt.iglobpath(os.path.dirname(prefix) + '*',
+                              ignore_case=(not csc),
+                              sort_result=glob_sorted):
+            if xt.levenshtein(prefix, s, threshold) < threshold:
                 paths.add(s)
     if tilde in prefix:
         home = os.path.expanduser(tilde)

xonsh/completers/pip.py

@@ -1,15 +1,15 @@
 import re
 import subprocess
 
-from xonsh.lazyasd import LazyObject, lazyobject
+import xonsh.lazyasd as xl
 
-PIP_RE = LazyObject(lambda: re.compile("pip(?:\d|\.)*"),
-                    globals(), 'PIP_RE')
-PIP_LIST_RE = LazyObject(lambda: re.compile("pip(?:\d|\.)* (?:uninstall|show)"),
-                         globals(), 'PIP_LIST_RE')
+PIP_RE = xl.LazyObject(lambda: re.compile("pip(?:\d|\.)*"),
+                       globals(), 'PIP_RE')
+PIP_LIST_RE = xl.LazyObject(lambda: re.compile("pip(?:\d|\.)* (?:uninstall|show)"),
+                            globals(), 'PIP_LIST_RE')
 
 
-@lazyobject
+@xl.lazyobject
 def ALL_COMMANDS():
     help_text = str(subprocess.check_output(['pip', '--help'],
                                             stderr=subprocess.DEVNULL))

xonsh/completers/python.py

@@ -1,27 +1,34 @@
+"""Completers for Python code"""
 import re
 import sys
 import inspect
 import builtins
 import importlib
+import collections.abc as abc
 
-from collections.abc import Iterable
+import xonsh.tools as xt
+import xonsh.lazyasd as xl
 
-from xonsh.tools import (subexpr_from_unbalanced, get_sep,
-                         check_for_partial_string, RE_STRING_START)
 from xonsh.completers.tools import get_filter_function
 
-RE_ATTR = re.compile(r'([^\s\(\)]+(\.[^\s\(\)]+)*)\.(\w*)$')
 
-XONSH_TOKENS = {
-    'and ', 'as ', 'assert ', 'break', 'class ', 'continue', 'def ', 'del ',
-    'elif ', 'else', 'except ', 'finally:', 'for ', 'from ', 'global ',
-    'import ', 'if ', 'in ', 'is ', 'lambda ', 'nonlocal ', 'not ', 'or ',
-    'pass', 'raise ', 'return ', 'try:', 'while ', 'with ', 'yield ', '+', '-',
-    '/', '//', '%', '**', '|', '&', '~', '^', '>>', '<<', '<', '<=', '>', '>=',
-    '==', '!=', '->', '=', '+=', '-=', '*=', '/=', '%=', '**=', '>>=', '<<=',
-    '&=', '^=', '|=', '//=', ',', ';', ':', '?', '??', '$(', '${', '$[', '..',
-    '...', '![', '!(', '@(', '@$(', '@'
-}
+@xl.lazyobject
+def RE_ATTR():
+    return re.compile(r'([^\s\(\)]+(\.[^\s\(\)]+)*)\.(\w*)$')
+
+
+@xl.lazyobject
+def XONSH_TOKENS():
+    return {
+        'and ', 'as ', 'assert ', 'break', 'class ', 'continue', 'def ', 'del ',
+        'elif ', 'else', 'except ', 'finally:', 'for ', 'from ', 'global ',
+        'import ', 'if ', 'in ', 'is ', 'lambda ', 'nonlocal ', 'not ', 'or ',
+        'pass', 'raise ', 'return ', 'try:', 'while ', 'with ', 'yield ', '+',
+        '-', '/', '//', '%', '**', '|', '&', '~', '^', '>>', '<<', '<', '<=',
+        '>', '>=', '==', '!=', '->', '=', '+=', '-=', '*=', '/=', '%=', '**=',
+        '>>=', '<<=', '&=', '^=', '|=', '//=', ',', ';', ':', '?', '??', '$(',
+        '${', '$[', '..', '...', '![', '!(', '@(', '@$(', '@'
+        }
 
 
 def complete_python(prefix, line, start, end, ctx):
@@ -61,9 +68,9 @@ def attr_complete(prefix, ctx, filter_func):
     if m is None:
         return attrs
     expr, attr = m.group(1, 3)
-    expr = subexpr_from_unbalanced(expr, '(', ')')
-    expr = subexpr_from_unbalanced(expr, '[', ']')
-    expr = subexpr_from_unbalanced(expr, '{', '}')
+    expr = xt.subexpr_from_unbalanced(expr, '(', ')')
+    expr = xt.subexpr_from_unbalanced(expr, '[', ']')
+    expr = xt.subexpr_from_unbalanced(expr, '{', '}')
     _ctx = None
     xonsh_safe_eval = builtins.__xonsh_execer__.eval
     try:
@@ -90,7 +97,7 @@ def attr_complete(prefix, ctx, filter_func):
         a = getattr(val, opt)
         if callable(a):
             rpl = opt + '('
-        elif isinstance(a, Iterable):
+        elif isinstance(a, abc.Iterable):
             rpl = opt + '['
         else:
             rpl = opt

xonsh/completers/tools.py

@@ -1,3 +1,4 @@
+"""Xonsh completer tools."""
 import builtins
 import textwrap
 

xonsh/diff_history.py

@@ -8,47 +8,47 @@ import argparse
 from xonsh.lazyjson import LazyJSON
 from xonsh.tools import print_color
 
-NO_COLOR = '{NO_COLOR}'
-RED = '{RED}'
-GREEN = '{GREEN}'
-BOLD_RED = '{BOLD_RED}'
-BOLD_GREEN = '{BOLD_GREEN}'
+NO_COLOR_S = '{NO_COLOR}'
+RED_S = '{RED}'
+GREEN_S = '{GREEN}'
+BOLD_RED_S = '{BOLD_RED}'
+BOLD_GREEN_S = '{BOLD_GREEN}'
 
 # intern some strings
-REPLACE = 'replace'
-DELETE = 'delete'
-INSERT = 'insert'
-EQUAL = 'equal'
+REPLACE_S = 'replace'
+DELETE_S = 'delete'
+INSERT_S = 'insert'
+EQUAL_S = 'equal'
 
 
 def bold_str_diff(a, b, sm=None):
     if sm is None:
         sm = difflib.SequenceMatcher()
-    aline = RED + '- '
-    bline = GREEN + '+ '
+    aline = RED_S + '- '
+    bline = GREEN_S + '+ '
     sm.set_seqs(a, b)
     for tag, i1, i2, j1, j2 in sm.get_opcodes():
-        if tag == REPLACE:
-            aline += BOLD_RED + a[i1:i2] + RED
-            bline += BOLD_GREEN + b[j1:j2] + GREEN
-        elif tag == DELETE:
-            aline += BOLD_RED + a[i1:i2] + RED
-        elif tag == INSERT:
-            bline += BOLD_GREEN + b[j1:j2] + GREEN
-        elif tag == EQUAL:
+        if tag == REPLACE_S:
+            aline += BOLD_RED_S + a[i1:i2] + RED_S
+            bline += BOLD_GREEN_S + b[j1:j2] + GREEN_S
+        elif tag == DELETE_S:
+            aline += BOLD_RED_S + a[i1:i2] + RED_S
+        elif tag == INSERT_S:
+            bline += BOLD_GREEN_S + b[j1:j2] + GREEN_S
+        elif tag == EQUAL_S:
             aline += a[i1:i2]
             bline += b[j1:j2]
         else:
             raise RuntimeError('tag not understood')
-    return aline + NO_COLOR + '\n' + bline + NO_COLOR +'\n'
+    return aline + NO_COLOR_S + '\n' + bline + NO_COLOR_S + '\n'
 
 
 def redline(line):
-   return '{red}- {line}{no_color}\n'.format(red=RED, line=line, no_color=NO_COLOR)
+    return '{red}- {line}{no_color}\n'.format(red=RED_S, line=line, no_color=NO_COLOR_S)
 
 
 def greenline(line):
-   return '{green}+ {line}{no_color}\n'.format(green=GREEN, line=line, no_color=NO_COLOR)
+    return '{green}+ {line}{no_color}\n'.format(green=GREEN_S, line=line, no_color=NO_COLOR_S)
 
 
 def highlighted_ndiff(a, b):
@@ -58,7 +58,7 @@ def highlighted_ndiff(a, b):
     sm.set_seqs(a, b)
     linesm = difflib.SequenceMatcher()
     for tag, i1, i2, j1, j2 in sm.get_opcodes():
-        if tag == REPLACE:
+        if tag == REPLACE_S:
             for aline, bline in itertools.zip_longest(a[i1:i2], b[j1:j2]):
                 if bline is None:
                     s += redline(aline)
@@ -66,13 +66,13 @@ def highlighted_ndiff(a, b):
                     s += greenline(bline)
                 else:
                     s += bold_str_diff(aline, bline, sm=linesm)
-        elif tag == DELETE:
+        elif tag == DELETE_S:
             for aline in a[i1:i2]:
                 s += redline(aline)
-        elif tag == INSERT:
+        elif tag == INSERT_S:
             for bline in b[j1:j2]:
                 s += greenline(bline)
-        elif tag == EQUAL:
+        elif tag == EQUAL_S:
             for aline in a[i1:i2]:
                 s += '  ' + aline + '\n'
         else:
@@ -127,7 +127,7 @@ class HistoryDiffer(object):
         s = ('{red}--- {aline}{no_color}\n'
              '{green}+++ {bline}{no_color}')
         s = s.format(aline=self._header_line(self.a), bline=self._header_line(self.b),
-                     red=RED, green=GREEN, no_color=NO_COLOR)
+                     red=RED_S, green=GREEN_S, no_color=NO_COLOR_S)
         return s
 
     def _env_both_diff(self, in_both, aenv, benv):
@@ -147,13 +147,13 @@ class HistoryDiffer(object):
         if len(only_x) == 0:
                 return ''
         if self.verbose:
-            xstr = ',\n'.join(['    {0!r}: {1!r}'.format(key, xenv[key]) \
+            xstr = ',\n'.join(['    {0!r}: {1!r}'.format(key, xenv[key])
                                for key in only_x])
             xstr = '\n' + xstr
         else:
             xstr = ', '.join(['{0!r}'.format(key) for key in only_x])
         in_x = 'These vars are only in {color}{xid}{no_color}: {{{xstr}}}\n\n'
-        return in_x.format(xid=xid, color=color, no_color=NO_COLOR, xstr=xstr)
+        return in_x.format(xid=xid, color=color, no_color=NO_COLOR_S, xstr=xstr)
 
     def envdiff(self):
         """Computes the difference between the environments."""
@@ -169,19 +169,19 @@ class HistoryDiffer(object):
             in_a = in_b = ''
         else:
             keydiff = self._env_both_diff(in_both, aenv, benv)
-            in_a = self._env_in_one_diff(akeys, bkeys, RED, self.a['sessionid'], aenv)
-            in_b = self._env_in_one_diff(bkeys, akeys, GREEN, self.b['sessionid'], benv)
+            in_a = self._env_in_one_diff(akeys, bkeys, RED_S, self.a['sessionid'], aenv)
+            in_b = self._env_in_one_diff(bkeys, akeys, GREEN_S, self.b['sessionid'], benv)
         s = 'Environment\n-----------\n' + in_a + keydiff + in_b
         return s
 
     def _cmd_in_one_diff(self, inp, i, xlj, xid, color):
         s = 'cmd #{i} only in {color}{xid}{no_color}:\n'
-        s = s.format(i=i, color=color, xid=xid, no_color=NO_COLOR)
+        s = s.format(i=i, color=color, xid=xid, no_color=NO_COLOR_S)
         lines = inp.splitlines()
         lt = '{color}{pre}{no_color} {line}\n'
-        s += lt.format(color=color, no_color=NO_COLOR, line=lines[0], pre='>>>')
+        s += lt.format(color=color, no_color=NO_COLOR_S, line=lines[0], pre='>>>')
         for line in lines[1:]:
-            s += lt.format(color=color, no_color=NO_COLOR, line=line, pre='...')
+            s += lt.format(color=color, no_color=NO_COLOR_S, line=line, pre='...')
         if not self.verbose:
             return s + '\n'
         out = xlj['cmds'][0].get('out', 'Note: no output stored')
@@ -193,16 +193,16 @@ class HistoryDiffer(object):
         aout = self.a['cmds'][i].get('out', None)
         bout = self.b['cmds'][j].get('out', None)
         if aout is None and bout is None:
-            #s += 'Note: neither output stored\n'
+            # s += 'Note: neither output stored\n'
             pass
         elif bout is None:
             aid = self.a['sessionid']
-            s += 'Note: only {red}{aid}{no_color} output stored\n'.format(red=RED,
-                                                            aid=aid, no_color=NO_COLOR)
+            s += 'Note: only {red}{aid}{no_color} output stored\n'.format(
+                red=RED_S, aid=aid, no_color=NO_COLOR_S)
         elif aout is None:
             bid = self.b['sessionid']
-            s += 'Note: only {green}{bid}{no_color} output stored\n'.format(green=GREEN,
-                                                            bid=bid, no_color=NO_COLOR)
+            s += 'Note: only {green}{bid}{no_color} output stored\n'.format(
+                green=GREEN_S, bid=bid, no_color=NO_COLOR_S)
         elif aout != bout:
             s += 'Outputs differ\n'
             s += highlighted_ndiff(aout.splitlines(), bout.splitlines())
@@ -212,13 +212,14 @@ class HistoryDiffer(object):
         brtn = self.b['cmds'][j]['rtn']
         if artn != brtn:
             s += ('Return vals {red}{artn}{no_color} & {green}{brtn}{no_color} differ\n'
-                  ).format(red=RED, green=GREEN, no_color=NO_COLOR, artn=artn, brtn=brtn)
+                  ).format(red=RED_S, green=GREEN_S, no_color=NO_COLOR_S, artn=artn, brtn=brtn)
         return s
 
     def _cmd_replace_diff(self, i, ainp, aid, j, binp, bid):
         s = ('cmd #{i} in {red}{aid}{no_color} is replaced by \n'
              'cmd #{j} in {green}{bid}{no_color}:\n')
-        s = s.format(i=i, aid=aid, j=j, bid=bid, red=RED, green=GREEN, no_color=NO_COLOR)
+        s = s.format(i=i, aid=aid, j=j, bid=bid, red=RED_S, green=GREEN_S,
+                     no_color=NO_COLOR_S)
         s += highlighted_ndiff(ainp.splitlines(), binp.splitlines())
         if not self.verbose:
             return s + '\n'
@@ -235,30 +236,30 @@ class HistoryDiffer(object):
         sm.set_seqs(ainps, binps)
         s = ''
         for tag, i1, i2, j1, j2 in sm.get_opcodes():
-            if tag == REPLACE:
+            if tag == REPLACE_S:
                 zipper = itertools.zip_longest
                 for i, ainp, j, binp in zipper(range(i1, i2), ainps[i1:i2],
                                                range(j1, j2), binps[j1:j2]):
                     if j is None:
-                        s += self._cmd_in_one_diff(ainp, i, self.a, aid, RED)
+                        s += self._cmd_in_one_diff(ainp, i, self.a, aid, RED_S)
                     elif i is None:
-                        s += self._cmd_in_one_diff(binp, j, self.b, bid, GREEN)
+                        s += self._cmd_in_one_diff(binp, j, self.b, bid, GREEN_S)
                     else:
                         self._cmd_replace_diff(i, ainp, aid, j, binp, bid)
-            elif tag == DELETE:
+            elif tag == DELETE_S:
                 for i, inp in enumerate(ainps[i1:i2], i1):
-                    s += self._cmd_in_one_diff(inp, i, self.a, aid, RED)
-            elif tag == INSERT:
+                    s += self._cmd_in_one_diff(inp, i, self.a, aid, RED_S)
+            elif tag == INSERT_S:
                 for j, inp in enumerate(binps[j1:j2], j1):
-                    s += self._cmd_in_one_diff(inp, j, self.b, bid, GREEN)
-            elif tag == EQUAL:
+                    s += self._cmd_in_one_diff(inp, j, self.b, bid, GREEN_S)
+            elif tag == EQUAL_S:
                 for i, j, in zip(range(i1, i2), range(j1, j2)):
                     odiff = self._cmd_out_and_rtn_diff(i, j)
                     if len(odiff) > 0:
                         h = ('cmd #{i} in {red}{aid}{no_color} input is the same as \n'
                              'cmd #{j} in {green}{bid}{no_color}, but output differs:\n')
-                        s += h.format(i=i, aid=aid, j=j, bid=bid, red=RED, green=GREEN,
-                                      no_color=NO_COLOR)
+                        s += h.format(i=i, aid=aid, j=j, bid=bid, red=RED_S, green=GREEN_S,
+                                      no_color=NO_COLOR_S)
                         s += odiff + '\n'
             else:
                 raise RuntimeError('tag not understood')
@@ -280,6 +281,7 @@ class HistoryDiffer(object):
 
 _HD_PARSER = None
 
+
 def _dh_create_parser(p=None):
     global _HD_PARSER
     p_was_none = (p is None)
@@ -309,4 +311,3 @@ def diff_history_main(args=None, stdin=None):
     parser = _dh_create_parser()
     ns = parser.parse_args(args)
     _dh_main_action(ns)
-

xonsh/environ.py

@@ -11,10 +11,11 @@ import string
 import pprint
 import locale
 import builtins
+import warnings
+import traceback
 import itertools
 import contextlib
 import subprocess
-import warnings
 import collections
 import collections.abc as abc
 
@@ -60,6 +61,7 @@ def LOCALE_CATS():
 
 def locale_convert(key):
     """Creates a converter for a locale key."""
+
     def lc_converter(val):
         try:
             locale.setlocale(LOCALE_CATS[key], val)
@@ -68,6 +70,7 @@ def locale_convert(key):
             msg = 'Failed to set locale {0!r} to {1!r}'.format(key, val)
             warnings.warn(msg, RuntimeWarning)
         return val
+
     return lc_converter
 
 
@@ -80,6 +83,7 @@ def to_debug(x):
         builtins.__xonsh_execer__.debug_level = val
     return val
 
+
 Ensurer = collections.namedtuple('Ensurer', ['validate', 'convert', 'detype'])
 Ensurer.__doc__ = """Named tuples whose elements are functions that
 represent environment variable validation, conversion, detyping.
@@ -104,8 +108,9 @@ DEFAULT_ENSURERS = LazyObject(lambda: {
     'GLOB_SORTED': (is_bool, to_bool, bool_to_str),
     'HISTCONTROL': (is_string_set, csv_to_set, set_to_csv),
     'IGNOREEOF': (is_bool, to_bool, bool_to_str),
-    'INTENSIFY_COLORS_ON_WIN':(always_false, intensify_colors_on_win_setter,
-                               bool_to_str),
+    'INTENSIFY_COLORS_ON_WIN': (always_false, intensify_colors_on_win_setter,
+                                bool_to_str),
+    'LANG': (is_string, ensure_string, ensure_string),
     'LC_COLLATE': (always_false, locale_convert('LC_COLLATE'), ensure_string),
     'LC_CTYPE': (always_false, locale_convert('LC_CTYPE'), ensure_string),
     'LC_MESSAGES': (always_false, locale_convert('LC_MESSAGES'), ensure_string),
@@ -145,7 +150,8 @@ DEFAULT_ENSURERS = LazyObject(lambda: {
     'XONSH_STORE_STDIN': (is_bool, to_bool, bool_to_str),
     'XONSH_TRACEBACK_LOGFILE': (is_logfile_opt, to_logfile_opt,
                                 logfile_opt_to_str)
-    }, globals(), 'DEFAULT_ENSURERS')
+}, globals(), 'DEFAULT_ENSURERS')
+
 
 #
 # Defaults
@@ -155,6 +161,7 @@ def default_value(f):
     f._xonsh_callable_default = True
     return f
 
+
 def is_callable_default(x):
     """Checks if a value is a callable default."""
     return callable(x) and getattr(x, '_xonsh_callable_default', False)
@@ -181,6 +188,7 @@ def default_prompt():
 DEFAULT_PROMPT = LazyObject(default_prompt, globals(), 'DEFAULT_PROMPT')
 DEFAULT_TITLE = '{current_job:{} | }{user}@{hostname}: {cwd} | xonsh'
 
+
 @default_value
 def xonsh_data_dir(env):
     """Ensures and returns the $XONSH_DATA_DIR"""
@@ -218,6 +226,7 @@ def default_xonshrc():
 
 DEFAULT_XONSHRC = LazyObject(default_xonshrc, globals(), 'DEFAULT_XONSHRC')
 
+
 # Default values should generally be immutable, that way if a user wants
 # to set them they have to do a copy and write them to the environment.
 # try to keep this sorted.
@@ -245,6 +254,7 @@ def DEFAULT_VALUES():
         'IGNOREEOF': False,
         'INDENT': '    ',
         'INTENSIFY_COLORS_ON_WIN': True,
+        'LANG': 'C.UTF-8',
         'LC_CTYPE': locale.setlocale(locale.LC_CTYPE),
         'LC_COLLATE': locale.setlocale(locale.LC_COLLATE),
         'LC_TIME': locale.setlocale(locale.LC_TIME),
@@ -294,7 +304,7 @@ def DEFAULT_VALUES():
         'XONSH_STORE_STDIN': False,
         'XONSH_STORE_STDOUT': False,
         'XONSH_TRACEBACK_LOGFILE': None
-        }
+    }
     if hasattr(locale, 'LC_MESSAGES'):
         dv['LC_MESSAGES'] = locale.setlocale(locale.LC_MESSAGES)
     return dv
@@ -332,7 +342,7 @@ DEFAULT_DOCS = LazyObject(lambda: {
         'full path only (without the cd command).'),
     'AUTO_PUSHD': VarDocs(
         'Flag for automatically pushing directories onto the directory stack.'
-        ),
+    ),
     'AUTO_SUGGEST': VarDocs(
         'Enable automatic command suggestions based on history, like in the fish '
         'shell.\n\nPressing the right arrow key inserts the currently '
@@ -342,16 +352,16 @@ DEFAULT_DOCS = LazyObject(lambda: {
         'completion files may be found. The default values are platform '
         'dependent, but sane. To specify an alternate list, do so in the run '
         'control file.', default=(
-        "Normally this is:\n\n"
-        "    ('/etc/bash_completion',\n"
-        "     '/usr/share/bash-completion/completions/git')\n\n"
-        "But, on Mac it is:\n\n"
-        "    ('/usr/local/etc/bash_completion',\n"
-        "     '/opt/local/etc/profile.d/bash_completion.sh')\n\n"
-        "And on Arch Linux it is:\n\n"
-        "    ('/usr/share/bash-completion/bash_completion',\n"
-        "     '/usr/share/bash-completion/completions/git')\n\n"
-        "Other OS-specific defaults may be added in the future.")),
+            "Normally this is:\n\n"
+            "    ('/etc/bash_completion',\n"
+            "     '/usr/share/bash-completion/completions/git')\n\n"
+            "But, on Mac it is:\n\n"
+            "    ('/usr/local/etc/bash_completion',\n"
+            "     '/opt/local/etc/profile.d/bash_completion.sh')\n\n"
+            "And on Arch Linux it is:\n\n"
+            "    ('/usr/share/bash-completion/bash_completion',\n"
+            "     '/usr/share/bash-completion/completions/git')\n\n"
+            "Other OS-specific defaults may be added in the future.")),
     'CASE_SENSITIVE_COMPLETIONS': VarDocs(
         'Sets whether completions should be case sensitive or case '
         'insensitive.', default='True on Linux, False otherwise.'),
@@ -381,7 +391,8 @@ DEFAULT_DOCS = LazyObject(lambda: {
         "$COMPLETIONS_DISPLAY is 'single' or 'multi'. This only affects the "
         'prompt-toolkit shell.'),
     'DIRSTACK_SIZE': VarDocs('Maximum size of the directory stack.'),
-    'DYNAMIC_CWD_WIDTH': VarDocs('Maximum length in number of characters '
+    'DYNAMIC_CWD_WIDTH': VarDocs(
+        'Maximum length in number of characters '
         'or as a percentage for the `cwd` prompt variable. For example, '
         '"20" is a twenty character width and "10%" is ten percent of the '
         'number of columns available.'),
@@ -414,12 +425,15 @@ DEFAULT_DOCS = LazyObject(lambda: {
         store_as_str=True),
     'IGNOREEOF': VarDocs('Prevents Ctrl-D from exiting the shell.'),
     'INDENT': VarDocs('Indentation string for multiline input'),
-    'INTENSIFY_COLORS_ON_WIN': VarDocs('Enhance style colors for readability '
+    'INTENSIFY_COLORS_ON_WIN': VarDocs(
+        'Enhance style colors for readability '
         'when using the default terminal (cmd.exe) on Windows. Blue colors, '
         'which are hard to read, are replaced with cyan. Other colors are '
         'generally replaced by their bright counter parts.',
         configurable=ON_WINDOWS),
-    'LOADED_CONFIG': VarDocs('Whether or not the xonsh config file was loaded',
+    'LANG': VarDocs('Fallback locale setting for systems where it matters'),
+    'LOADED_CONFIG': VarDocs(
+        'Whether or not the xonsh config file was loaded',
         configurable=False),
     'LOADED_RC_FILES': VarDocs(
         'Whether or not any of the xonsh run control files were loaded at '
@@ -436,14 +450,14 @@ DEFAULT_DOCS = LazyObject(lambda: {
         'Prompt text for 2nd+ lines of input, may be str or function which '
         'returns a str.'),
     'OLDPWD': VarDocs('Used to represent a previous present working directory.',
-        configurable=False),
+                      configurable=False),
     'PATH': VarDocs(
         'List of strings representing where to look for executables.'),
     'PATHEXT': VarDocs('Sequence of extention strings (eg, ".EXE") for '
                        'filtering valid executables by. Each element must be '
                        'uppercase.'),
     'PRETTY_PRINT_RESULTS': VarDocs(
-            'Flag for "pretty printing" return values.'),
+        'Flag for "pretty printing" return values.'),
     'PROMPT': VarDocs(
         'The prompt text. May contain keyword arguments which are '
         "auto-formatted, see 'Customizing the Prompt' at "
@@ -461,7 +475,8 @@ DEFAULT_DOCS = LazyObject(lambda: {
         'This is most useful in xonsh scripts or modules where failures '
         'should cause an end to execution. This is less useful at a terminal. '
         'The error that is raised is a subprocess.CalledProcessError.'),
-    'RIGHT_PROMPT': VarDocs('Template string for right-aligned text '
+    'RIGHT_PROMPT': VarDocs(
+        'Template string for right-aligned text '
         'at the prompt. This may be parameterized in the same way as '
         'the $PROMPT variable. Currently, this is only available in the '
         'prompt-toolkit shell.'),
@@ -521,10 +536,12 @@ DEFAULT_DOCS = LazyObject(lambda: {
         "in the same manner as $PROMPT, see 'Customizing the Prompt' "
         'http://xon.sh/tutorial.html#customizing-the-prompt.',
         default='xonsh.environ.DEFAULT_TITLE'),
-    'UPDATE_OS_ENVIRON': VarDocs("If True os.environ will always be updated "
+    'UPDATE_OS_ENVIRON': VarDocs(
+        "If True os.environ will always be updated "
         "when the xonsh environment changes. The environment can be reset to "
         "the default value by calling '__xonsh_env__.undo_replace_env()'"),
-    'VC_BRANCH_TIMEOUT': VarDocs('The timeout (in seconds) for version control '
+    'VC_BRANCH_TIMEOUT': VarDocs(
+        'The timeout (in seconds) for version control '
         'branch computations. This is a timeout per subprocess call, so the '
         'total time to compute will be larger than this in many cases.'),
     'VI_MODE': VarDocs(
@@ -550,8 +567,8 @@ DEFAULT_DOCS = LazyObject(lambda: {
         'A tuple of the locations of run control files, if they exist.  User '
         'defined run control file will supersede values set in system-wide '
         'control file if there is a naming collision.', default=(
-        "On Linux & Mac OSX: ('/etc/xonshrc', '~/.xonshrc')\n"
-        "On Windows: ('%ALLUSERSPROFILE%\\xonsh\\xonshrc', '~/.xonshrc')")),
+            "On Linux & Mac OSX: ('/etc/xonshrc', '~/.xonshrc')\n"
+            "On Windows: ('%ALLUSERSPROFILE%\\xonsh\\xonshrc', '~/.xonshrc')")),
     'XONSH_CACHE_SCRIPTS': VarDocs(
         'Controls whether the code for scripts run from xonsh will be cached'
         ' (``True``) or re-compiled each time (``False``).'),
@@ -582,7 +599,8 @@ DEFAULT_DOCS = LazyObject(lambda: {
         "(https://docs.python.org/3/library/codecs.html#error-handlers) "
         'for more information and available options.',
         default="'surrogateescape'"),
-    'XONSH_HISTORY_FILE': VarDocs('Location of history file (deprecated).',
+    'XONSH_HISTORY_FILE': VarDocs(
+        'Location of history file (deprecated).',
         configurable=False, default="'~/.xonsh_history'"),
     'XONSH_HISTORY_SIZE': VarDocs(
         'Value and units tuple that sets the size of history after garbage '
@@ -619,7 +637,8 @@ DEFAULT_DOCS = LazyObject(lambda: {
         'XONSH_SHOW_TRACEBACK has been set. Its value must be a writable file '
         'or None / the empty string if traceback logging is not desired. '
         'Logging to a file is not enabled by default.'),
-    }, globals(), 'DEFAULT_DOCS')
+}, globals(), 'DEFAULT_DOCS')
+
 
 #
 # actual environment
@@ -649,7 +668,7 @@ class Env(abc.MutableMapping):
         self._defaults = DEFAULT_VALUES
         self._docs = DEFAULT_DOCS
         if len(args) == 0 and len(kwargs) == 0:
-            args = (os.environ, )
+            args = (os.environ,)
         for key, val in dict(*args, **kwargs).items():
             self[key] = val
         if 'PATH' not in self._d:
@@ -880,8 +899,8 @@ def get_git_branch():
             inp = 'source {}; __git_ps1 "${{1:-%s}}"'.format(script)
             try:
                 branch = subprocess.check_output(['bash'], cwd=cwd, input=inp,
-                            stderr=subprocess.PIPE, timeout=vcbt, env=denv,
-                            universal_newlines=True)
+                                                 stderr=subprocess.PIPE, timeout=vcbt, env=denv,
+                                                 universal_newlines=True)
                 break
             except subprocess.TimeoutExpired as e:
                 branch = e
@@ -893,7 +912,7 @@ def get_git_branch():
         cmd = ['git', 'rev-parse', '--abbrev-ref', 'HEAD']
         try:
             s = subprocess.check_output(cmd, cwd=cwd, timeout=vcbt, env=denv,
-                    stderr=subprocess.PIPE, universal_newlines=True)
+                                        stderr=subprocess.PIPE, universal_newlines=True)
             if ON_WINDOWS and len(s) == 0:
                 # Workaround for a bug in ConEMU/cmder, retry without redirection
                 s = subprocess.check_output(cmd, cwd=cwd, timeout=vcbt,
@@ -942,7 +961,7 @@ def get_hg_branch(cwd=None, root=None):
         with open(bookmark_path, 'r') as bookmark_file:
             active_bookmark = bookmark_file.read()
         branch = "{0}, {1}".format(*(b.strip(os.linesep) for b in
-                                   (branch, active_bookmark)))
+                                     (branch, active_bookmark)))
     else:
         branch = branch.strip(os.linesep)
     return branch
@@ -950,6 +969,7 @@ def get_hg_branch(cwd=None, root=None):
 
 _FIRST_BRANCH_TIMEOUT = True
 
+
 def _first_branch_timeout_message():
     global _FIRST_BRANCH_TIMEOUT
     sbtm = builtins.__xonsh_env__['SUPPRESS_BRANCH_TIMEOUT_MESSAGE']
@@ -1026,8 +1046,8 @@ def hg_dirty_working_directory():
     denv['HGRCPATH'] = ''
     try:
         s = subprocess.check_output(['hg', 'identify', '--id'],
-                stderr=subprocess.PIPE, cwd=cwd, timeout=vcbt,
-                universal_newlines=True, env=denv)
+                                    stderr=subprocess.PIPE, cwd=cwd, timeout=vcbt,
+                                    universal_newlines=True, env=denv)
         return s.strip(os.linesep).endswith('+')
     except (subprocess.CalledProcessError, subprocess.TimeoutExpired,
             FileNotFoundError):
@@ -1095,14 +1115,16 @@ def _replace_home(x):
         return x
 
 
-_replace_home_cwd = lambda: _replace_home(builtins.__xonsh_env__['PWD'])
+def _replace_home_cwd():
+    return _replace_home(builtins.__xonsh_env__['PWD'])
+
 
 def _collapsed_pwd():
     sep = get_sep()
     pwd = _replace_home_cwd().split(sep)
     l = len(pwd)
-    leader = sep if l>0 and len(pwd[0])==0 else ''
-    base = [i[0] if ix != l-1 else i for ix,i in enumerate(pwd) if len(i) > 0]
+    leader = sep if l > 0 and len(pwd[0]) == 0 else ''
+    base = [i[0] if ix != l - 1 else i for ix, i in enumerate(pwd) if len(i) > 0]
     return leader + sep.join(base)
 
 
@@ -1124,7 +1146,7 @@ def _dynamically_collapsed_pwd():
     # Reserve space for separators
     remaining_space_for_text = remaining_space - len(pwd)
     parts = []
-    for i  in range(len(pwd)):
+    for i in range(len(pwd)):
         part = pwd[i]
         part_len = int(min(len(part), max(1, remaining_space_for_text // (len(pwd) - i))))
         remaining_space_for_text -= part_len
@@ -1199,8 +1221,7 @@ FORMATTER_DICT = LazyObject(lambda: dict(
     current_job=_current_job,
     env_name=env_name,
     vte_new_tab_cwd=vte_new_tab_cwd,
-    ), globals(), 'FORMATTER_DICT')
-
+), globals(), 'FORMATTER_DICT')
 
 _FORMATTER = LazyObject(string.Formatter, globals(), '_FORMATTER')
 
@@ -1303,6 +1324,7 @@ def _format_value(val, spec, conv):
 RE_HIDDEN = LazyObject(lambda: re.compile('\001.*?\002'), globals(),
                        'RE_HIDDEN')
 
+
 def multiline_prompt(curr=''):
     """Returns the filler text for the prompt in multiline scenarios."""
     line = curr.rsplit('\n', 1)[1] if '\n' in curr else curr
@@ -1354,7 +1376,8 @@ BASE_ENV = LazyObject(lambda: {
     'BASH_COMPLETIONS': list(DEFAULT_VALUES['BASH_COMPLETIONS']),
     'FORMATTER_DICT': dict(DEFAULT_VALUES['FORMATTER_DICT']),
     'XONSH_VERSION': XONSH_VERSION,
-    }, globals(), 'BASE_ENV')
+}, globals(), 'BASE_ENV')
+
 
 def load_static_config(ctx, config=None):
     """Loads a static configuration file from a given context, rather than the
@@ -1364,7 +1387,7 @@ def load_static_config(ctx, config=None):
     env['XDG_CONFIG_HOME'] = ctx.get('XDG_CONFIG_HOME',
                                      DEFAULT_VALUES['XDG_CONFIG_HOME'])
     env['XONSH_CONFIG_DIR'] = ctx['XONSH_CONFIG_DIR'] if 'XONSH_CONFIG_DIR' in ctx \
-                              else xonsh_config_dir(env)
+        else xonsh_config_dir(env)
     if config is not None:
         env['XONSHCONFIG'] = ctx['XONSHCONFIG'] = config
     elif 'XONSHCONFIG' in ctx:
@@ -1391,7 +1414,7 @@ def load_static_config(ctx, config=None):
                 print_exception()
                 # JSONDecodeError was added in Python v3.5
                 jerr = json.JSONDecodeError \
-                       if hasattr(json, 'JSONDecodeError') else ValueError
+                    if hasattr(json, 'JSONDecodeError') else ValueError
                 if isinstance(e, jerr):
                     msg = 'Xonsh config file is not valid JSON.'
                 else:
@@ -1423,13 +1446,15 @@ def xonshrc_context(rcfiles=None, execer=None, initial=None):
             loaded.append(True)
         except SyntaxError as err:
             loaded.append(False)
-            msg = 'syntax error in xonsh run control file {0!r}: {1!s}'
-            warnings.warn(msg.format(rcfile, err), RuntimeWarning)
+            exc = traceback.format_exc()
+            msg = '{0}\nsyntax error in xonsh run control file {1!r}: {2!s}'
+            warnings.warn(msg.format(exc, rcfile, err), RuntimeWarning)
             continue
         except Exception as err:
             loaded.append(False)
-            msg = 'error running xonsh run control file {0!r}: {1!s}'
-            warnings.warn(msg.format(rcfile, err), RuntimeWarning)
+            exc = traceback.format_exc()
+            msg = '{0}\nerror running xonsh run control file {1!r}: {2!s}'
+            warnings.warn(msg.format(exc, rcfile, err), RuntimeWarning)
             continue
     return env
 

xonsh/execer.py

@@ -1,6 +1,5 @@
 # -*- coding: utf-8 -*-
 """Implements the xonsh executer."""
-import re
 import sys
 import types
 import inspect
@@ -111,7 +110,7 @@ class Execer(object):
         return code
 
     def eval(self, input, glbs=None, locs=None, stacklevel=2,
-                transform=True):
+             transform=True):
         """Evaluates (and returns) xonsh code."""
         if isinstance(input, types.CodeType):
             code = input
@@ -127,7 +126,7 @@ class Execer(object):
         return eval(code, glbs, locs)
 
     def exec(self, input, mode='exec', glbs=None, locs=None, stacklevel=2,
-                transform=True):
+             transform=True):
         """Execute xonsh code."""
         if isinstance(input, types.CodeType):
             code = input
@@ -205,7 +204,7 @@ class Execer(object):
                         # or for some other syntax error
                         raise original_error
                 elif sbpline[last_error_col:].startswith('![![') or \
-                     sbpline.lstrip().startswith('![!['):
+                        sbpline.lstrip().startswith('![!['):
                     # if we have already wrapped this in subproc tokens
                     # and it still doesn't work, adding more won't help
                     # anything

xonsh/foreign_shells.py

@@ -1,7 +1,6 @@
 # -*- coding: utf-8 -*-
 """Tools to help interface with foreign shells, such as Bash."""
 import os
-import sys
 import re
 import json
 import shlex
@@ -16,7 +15,6 @@ from xonsh.lazyasd import LazyObject
 from xonsh.tools import to_bool, ensure_string
 from xonsh.platform import ON_WINDOWS, ON_CYGWIN
 
-
 COMMAND = """{seterrprevcmd}
 {prevcmd}
 echo __XONSH_ENV_BEG__
@@ -89,55 +87,55 @@ CANON_SHELL_NAMES = LazyObject(lambda: {
     '/usr/bin/zsh': 'zsh',
     'cmd': 'cmd',
     'cmd.exe': 'cmd',
-    }, globals(), 'CANON_SHELL_NAMES')
+}, globals(), 'CANON_SHELL_NAMES')
 
 DEFAULT_ENVCMDS = LazyObject(lambda: {
     'bash': 'env',
     'zsh': 'env',
     'cmd': 'set',
-    }, globals(), 'DEFAULT_ENVCMDS')
+}, globals(), 'DEFAULT_ENVCMDS')
 
 DEFAULT_ALIASCMDS = LazyObject(lambda: {
     'bash': 'alias',
     'zsh': 'alias -L',
     'cmd': '',
-    }, globals(), 'DEFAULT_ALIASCMDS')
+}, globals(), 'DEFAULT_ALIASCMDS')
 
 DEFAULT_FUNCSCMDS = LazyObject(lambda: {
     'bash': DEFAULT_BASH_FUNCSCMD,
     'zsh': DEFAULT_ZSH_FUNCSCMD,
     'cmd': '',
-    }, globals(), 'DEFAULT_FUNCSCMDS')
+}, globals(), 'DEFAULT_FUNCSCMDS')
 
 DEFAULT_SOURCERS = LazyObject(lambda: {
     'bash': 'source',
     'zsh': 'source',
     'cmd': 'call',
-    }, globals(), 'DEFAULT_SOURCERS')
+}, globals(), 'DEFAULT_SOURCERS')
 
 DEFAULT_TMPFILE_EXT = LazyObject(lambda: {
     'bash': '.sh',
     'zsh': '.zsh',
     'cmd': '.bat',
-    }, globals(), 'DEFAULT_TMPFILE_EXT')
+}, globals(), 'DEFAULT_TMPFILE_EXT')
 
 DEFAULT_RUNCMD = LazyObject(lambda: {
     'bash': '-c',
     'zsh': '-c',
     'cmd': '/C',
-    }, globals(), 'DEFAULT_RUNCMD')
+}, globals(), 'DEFAULT_RUNCMD')
 
 DEFAULT_SETERRPREVCMD = LazyObject(lambda: {
     'bash': 'set -e',
     'zsh': 'set -e',
     'cmd': '@echo off',
-    }, globals(), 'DEFAULT_SETERRPREVCMD')
+}, globals(), 'DEFAULT_SETERRPREVCMD')
 
 DEFAULT_SETERRPOSTCMD = LazyObject(lambda: {
     'bash': '',
     'zsh': '',
     'cmd': 'if errorlevel 1 exit 1',
-    }, globals(), 'DEFAULT_SETERRPOSTCMD')
+}, globals(), 'DEFAULT_SETERRPOSTCMD')
 
 
 @functools.lru_cache()
@@ -224,9 +222,9 @@ def foreign_shell_data(shell, interactive=True, login=False, envcmd=None,
     tmpfile_ext = DEFAULT_TMPFILE_EXT.get(shkey, 'sh') if tmpfile_ext is None else tmpfile_ext
     runcmd = DEFAULT_RUNCMD.get(shkey, '-c') if runcmd is None else runcmd
     seterrprevcmd = DEFAULT_SETERRPREVCMD.get(shkey, '') \
-                        if seterrprevcmd is None else seterrprevcmd
+        if seterrprevcmd is None else seterrprevcmd
     seterrpostcmd = DEFAULT_SETERRPOSTCMD.get(shkey, '') \
-                        if seterrpostcmd is None else seterrpostcmd
+        if seterrpostcmd is None else seterrpostcmd
     command = COMMAND.format(envcmd=envcmd, aliascmd=aliascmd, prevcmd=prevcmd,
                              postcmd=postcmd, funcscmd=funcscmd,
                              seterrprevcmd=seterrprevcmd,
@@ -246,7 +244,7 @@ def foreign_shell_data(shell, interactive=True, login=False, envcmd=None,
     try:
         s = subprocess.check_output(cmd, stderr=subprocess.PIPE, env=currenv,
                                     # start new session to avoid hangs
-                                    #(doesn't work on Cygwin though)
+                                    # (doesn't work on Cygwin though)
                                     start_new_session=(not ON_CYGWIN),
                                     universal_newlines=True)
     except (subprocess.CalledProcessError, FileNotFoundError):
@@ -267,7 +265,7 @@ ENV_RE = LazyObject(lambda: re.compile('__XONSH_ENV_BEG__\n(.*)'
                                        '__XONSH_ENV_END__', flags=re.DOTALL),
                     globals(), 'ENV_RE')
 ENV_SPLIT_RE = LazyObject(lambda: re.compile('^([^=]+)=([^=]*|[^\n]*)$',
-                                             flags=re.DOTALL|re.MULTILINE),
+                                             flags=re.DOTALL | re.MULTILINE),
                           globals(), 'ENV_SPLIT_RE')
 
 
@@ -342,7 +340,7 @@ def parse_funcs(s, shell, sourcer=None):
         warnings.warn(msg.format(exc, shell, s, g1), RuntimeWarning)
         return {}
     sourcer = DEFAULT_SOURCERS.get(shell, 'source') if sourcer is None \
-                                                    else sourcer
+        else sourcer
     funcs = {}
     for funcname, filename in namefiles.items():
         if funcname.startswith('_'):
@@ -377,7 +375,7 @@ class ForeignShellFunctionAlias(object):
             Command to source foreing files with.
         """
         sourcer = DEFAULT_SOURCERS.get(shell, 'source') if sourcer is None \
-                                                        else sourcer
+            else sourcer
         self.name = name
         self.shell = shell
         self.filename = filename
@@ -385,7 +383,7 @@ class ForeignShellFunctionAlias(object):
 
     def __eq__(self, other):
         if not hasattr(other, 'name') or not hasattr(other, 'shell') or \
-           not hasattr(other, 'filename') or not hasattr(other, 'sourcer'):
+                not hasattr(other, 'filename') or not hasattr(other, 'sourcer'):
             return NotImplemented
         return (self.name == other.name) and (self.shell == other.shell) and \
                (self.filename == other.filename) and (self.sourcer == other.sourcer)
@@ -417,10 +415,11 @@ class ForeignShellFunctionAlias(object):
 
 
 VALID_SHELL_PARAMS = LazyObject(lambda: frozenset([
-                                    'shell', 'interactive', 'login', 'envcmd',
-                                    'aliascmd', 'extra_args', 'currenv', 'safe',
-                                    'prevcmd', 'postcmd', 'funcscmd', 'sourcer',
-                                    ]), globals(), 'VALID_SHELL_PARAMS')
+    'shell', 'interactive', 'login', 'envcmd',
+    'aliascmd', 'extra_args', 'currenv', 'safe',
+    'prevcmd', 'postcmd', 'funcscmd', 'sourcer',
+]), globals(), 'VALID_SHELL_PARAMS')
+
 
 def ensure_shell(shell):
     """Ensures that a mapping follows the shell specification."""
@@ -437,17 +436,17 @@ def ensure_shell(shell):
         shell['login'] = to_bool(shell['login'])
     if 'envcmd' in shell_keys:
         shell['envcmd'] = None if shell['envcmd'] is None \
-                               else ensure_string(shell['envcmd'])
+            else ensure_string(shell['envcmd'])
     if 'aliascmd' in shell_keys:
         shell['aliascmd'] = None if shell['aliascmd'] is None \
-                                 else ensure_string(shell['aliascmd'])
+            else ensure_string(shell['aliascmd'])
     if 'extra_args' in shell_keys and not isinstance(shell['extra_args'], tuple):
         shell['extra_args'] = tuple(map(ensure_string, shell['extra_args']))
     if 'currenv' in shell_keys and not isinstance(shell['currenv'], tuple):
         ce = shell['currenv']
         if isinstance(ce, abc.Mapping):
             ce = tuple([(ensure_string(k), v) for k, v in ce.items()])
-        elif isinstance(ce, Sequence):
+        elif isinstance(ce, abc.Sequence):
             ce = tuple([(ensure_string(k), v) for k, v in ce])
         else:
             raise RuntimeError('unrecognized type for currenv')
@@ -460,16 +459,16 @@ def ensure_shell(shell):
         shell['postcmd'] = ensure_string(shell['postcmd'])
     if 'funcscmd' in shell_keys:
         shell['funcscmd'] = None if shell['funcscmd'] is None \
-                                 else ensure_string(shell['funcscmd'])
+            else ensure_string(shell['funcscmd'])
     if 'sourcer' in shell_keys:
         shell['sourcer'] = None if shell['sourcer'] is None \
-                                 else ensure_string(shell['sourcer'])
+            else ensure_string(shell['sourcer'])
     if 'seterrprevcmd' in shell_keys:
         shell['seterrprevcmd'] = None if shell['seterrprevcmd'] is None \
-                                 else ensure_string(shell['seterrprevcmd'])
+            else ensure_string(shell['seterrprevcmd'])
     if 'seterrpostcmd' in shell_keys:
         shell['seterrpostcmd'] = None if shell['seterrpostcmd'] is None \
-                                 else ensure_string(shell['seterrpostcmd'])
+            else ensure_string(shell['seterrpostcmd'])
     return shell
 
 

xonsh/history.py

@@ -15,9 +15,9 @@ import threading
 import collections
 import collections.abc as abc
 
-from xonsh.lazyasd import LazyObject
+from xonsh.lazyasd import lazyobject
 from xonsh.lazyjson import LazyJSON, ljdump, LJNode
-from xonsh.tools import (ensure_int_or_slice, to_history_tuple,
+from xonsh.tools import (ensure_slice, to_history_tuple,
                          expanduser_abs_path)
 from xonsh.diff_history import _dh_create_parser, _dh_main_action
 
@@ -115,7 +115,6 @@ class HistoryGC(threading.Thread):
         This is sorted by the last closed time. Returns a list of (timestamp,
         file) tuples.
         """
-        _ = self  # this could be a function but is intimate to this class
         # pylint: disable=no-member
         xdd = builtins.__xonsh_env__.get('XONSH_DATA_DIR')
         xdd = expanduser_abs_path(xdd)
@@ -269,7 +268,7 @@ def _find_histfile_var(file_list=None, default=None):
     return hist_file
 
 
-def _all_xonsh_parser(*args):
+def _all_xonsh_parser(**kwargs):
     """
     Returns all history as found in XONSH_DATA_DIR.
 
@@ -295,7 +294,7 @@ def _all_xonsh_parser(*args):
     return [(c, t, ind) for ind, (c, t) in enumerate(commands)]
 
 
-def _curr_session_parser(hist=None):
+def _curr_session_parser(hist=None, **kwargs):
     """
     Take in History object and return command list tuple with
     format: (name, start_time, index)
@@ -311,14 +310,14 @@ def _curr_session_parser(hist=None):
     return [(c, t, ind) for ind, (c, t) in commands]
 
 
-def _zsh_hist_parser(location=None):
+def _zsh_hist_parser(location=None, **kwargs):
     default_location = os.path.join('~', '.zsh_history')
     location_list = [os.path.join('~', '.zshrc'),
                      os.path.join('~', '.zprofile')]
     if location is None:
         location = _find_histfile_var(location_list, default_location)
     z_hist_formatted = []
-    if os.path.isfile(location):
+    if location and os.path.isfile(location):
         with open(location, 'r', errors='backslashreplace') as z_file:
             z_txt = z_file.read()
             z_hist = z_txt.splitlines()
@@ -337,18 +336,17 @@ def _zsh_hist_parser(location=None):
                 return z_hist_formatted
 
     else:
-        print("No zsh history file found at: {}".format(location),
-              file=sys.stderr)
+        print("No zsh history file found", file=sys.stderr)
 
 
-def _bash_hist_parser(location=None):
+def _bash_hist_parser(location=None, **kwargs):
     default_location = os.path.join('~', '.bash_history')
     location_list = [os.path.join('~', '.bashrc'),
                      os.path.join('~', '.bash_profile')]
     if location is None:
         location = _find_histfile_var(location_list, default_location)
     bash_hist_formatted = []
-    if os.path.isfile(location):
+    if location and os.path.isfile(location):
         with open(location, 'r', errors='backslashreplace') as bash_file:
             b_txt = bash_file.read()
             bash_hist = b_txt.splitlines()
@@ -357,10 +355,7 @@ def _bash_hist_parser(location=None):
                     bash_hist_formatted.append((command, 0.0, ind))
                 return bash_hist_formatted
     else:
-        import ipdb
-        ipdb.set_trace()
-        print("No bash history file found at: {}".format(location),
-              file=sys.stderr)
+        print("No bash history file", file=sys.stderr)
 
 
 @functools.lru_cache()
@@ -375,38 +370,10 @@ def _hist_create_parser():
     show.add_argument('-r', dest='reverse', default=False,
                       action='store_true',
                       help='reverses the direction')
-    show.add_argument('n', nargs='?', default=None,
-                      help='display n\'th history entry if n is a simple '
-                           'int, or range of entries if it is Python '
-                           'slice notation')
-    # all action
-    xonsh = subp.add_parser('xonsh', aliases=['all'],
-                            help='displays history from all sessions')
-    xonsh.add_argument('-r', dest='reverse', default=False,
-                       action='store_true',
-                       help='reverses the direction')
-    xonsh.add_argument('n', nargs='?', default=None,
-                       help='display n\'th history entry if n is a '
-                            'simple int, or range of entries if it '
-                            'is Python slice notation')
-    # zsh action
-    zsh = subp.add_parser('zsh', help='displays history from zsh sessions')
-    zsh.add_argument('-r', dest='reverse', default=False,
-                     action='store_true',
-                     help='reverses the direction')
-    zsh.add_argument('n', nargs='?', default=None,
-                     help='display n\'th history entry if n is a '
-                     'simple int, or range of entries if it '
-                     'is Python slice notation')
-    # bash action
-    bash = subp.add_parser('bash', help='displays history from bash sessions')
-    bash.add_argument('-r', dest='reverse', default=False,
-                      action='store_true',
-                      help='reverses the direction')
-    bash.add_argument('n', nargs='?', default=None,
-                      help='display n\'th history entry if n is a '
-                      'simple int, or range of entries if it '
-                      'is Python slice notation')
+    show.add_argument('session', nargs='?', choices=_HIST_SESSIONS.keys(), default='session',
+                      help='Choose a history session, defaults to current session')
+    show.add_argument('slices', nargs=argparse.REMAINDER, default=[],
+                      help='display history entries or range of entries')
     # 'id' subcommand
     subp.add_parser('id', help='displays the current session id')
     # 'file' subcommand
@@ -441,7 +408,7 @@ def _hist_create_parser():
 
 
 def _hist_show(ns=None, hist=None, start_index=None, end_index=None,
-          start_time=None, end_time=None, location=None):
+               start_time=None, end_time=None, location=None):
     """Show the requested portion of shell history.
     Accepts multiple history sources (xonsh, bash, zsh)
 
@@ -454,29 +421,25 @@ def _hist_show(ns=None, hist=None, start_index=None, end_index=None,
     can be supplied as a str with the follow options::
 
         session - returns xonsh history from current session
-        all     - returns xonsh history from all sessions
+        xonsh   - returns xonsh history from all sessions
+        all     - alias of xonsh
         zsh     - returns all zsh history
         bash    - returns all bash history
     """
     # Check if ns is a string, meaning it was invoked from
-    # __xonsh_history__
+    if hist is None:
+        hist = bultins.__xonsh_history__
     alias = True
-    valid_formats = {'session': functools.partial(_curr_session_parser, hist),
-                     'show': functools.partial(_curr_session_parser, hist),
-                     'all': _all_xonsh_parser,
-                     'xonsh': _all_xonsh_parser,
-                     'zsh': functools.partial(_zsh_hist_parser, location),
-                     'bash': functools.partial(_bash_hist_parser, location)}
-    if isinstance(ns, str) and ns in valid_formats.keys():
+    if isinstance(ns, str) and ns in _HIST_SESSIONS:
         ns = _hist_create_parser().parse_args([ns])
         alias = False
     if not ns:
-        ns = _hist_create_parser().parse_args(['all'])
+        ns = _hist_create_parser().parse_args(['show', 'xonsh'])
         alias = False
     try:
-        commands = valid_formats[ns.action]()
+        commands = _HIST_SESSIONS[ns.session](hist=hist, location=location)
     except KeyError:
-        print("{} is not a valid history format".format(ns.action))
+        print("{} is not a valid history session".format(ns.action))
         return None
     if not commands:
         return None
@@ -496,18 +459,23 @@ def _hist_show(ns=None, hist=None, start_index=None, end_index=None,
             print("Invalid end time, must be float or datetime.")
     idx = None
     if ns:
-        idx = ensure_int_or_slice(ns.n)
-        if idx is False:
-            print("{} is not a valid input.".format(ns.n),
-                  file=sys.stderr)
-            return
-        elif isinstance(idx, int):
+        _commands = []
+        for s in ns.slices:
             try:
-                commands = [commands[idx]]
-            except IndexError:
-                err = "Index likely not in range. Only {} commands."
-                print(err.format(len(commands)))
+                s = ensure_slice(s)
+            except (ValueError, TypeError):
+                print('{!r} is not a valid slice format'.format(s), file=sys.stderr)
                 return
+            if s:
+                try:
+                    _commands.extend(commands[s])
+                except IndexError:
+                    err = "Index likely not in range. Only {} commands."
+                    print(err.format(len(commands)), file=sys.stderr)
+                    return
+        else:
+            if _commands:
+                commands = _commands
     else:
         idx = slice(start_index, end_index)
 
@@ -537,7 +505,22 @@ def _hist_show(ns=None, hist=None, start_index=None, end_index=None,
 # Interface to History
 #
 class History(object):
-    """Xonsh session history."""
+    """Xonsh session history.
+
+    Attributes
+    ----------
+    rtns : sequence of ints
+        The return of the command (ie, 0 on success)
+    inps : sequence of strings
+        The command as typed by the user, including newlines
+    tss : sequence of two-tuples of floats
+        The timestamps of when the command started and finished, including
+        fractions
+    outs : sequence of strings
+        The output of the command, if xonsh is configured to save it
+
+    In all of these sequences, index 0 is the oldest and -1 (the last item) is the newest.
+    """
 
     def __init__(self, filename=None, sessionid=None, buffersize=100, gc=True,
                  **meta):
@@ -648,8 +631,7 @@ class History(object):
         Valid options:
             `session` - returns xonsh history from current session
             `show`    - alias of `session`
-            `all`     - returns xonsh history from all sessions
-            `xonsh`   - alias of `all`
+            `xonsh`   - returns xonsh history from all sessions
             `zsh`     - returns all zsh history
             `bash`    - returns all bash history
         """
@@ -680,39 +662,44 @@ def _hist_gc(ns, hist):
             continue
 
 
-_HIST_MAIN_ACTIONS = LazyObject(lambda: {
-    'show': _hist_show,
-    'xonsh': _hist_show,
-    'zsh': _hist_show,
-    'bash': _hist_show,
-    'session': _hist_show,
-    'all': _hist_show,
-    'id': lambda ns, hist: print(hist.sessionid),
-    'file': lambda ns, hist: print(hist.filename),
-    'info': _hist_info,
-    'diff': _dh_main_action,
-    'gc': _hist_gc,
-    }, globals(), '_HIST_MAIN_ACTIONS')
+@lazyobject
+def _HIST_SESSIONS():
+    return {'session': _curr_session_parser,
+            'xonsh': _all_xonsh_parser,
+            'all': _all_xonsh_parser,
+            'zsh': _zsh_hist_parser,
+            'bash': _bash_hist_parser}
 
 
-def _hist_main(hist, args):
-    """This implements the history CLI."""
-    if not args or args[0] in ['-r'] or ensure_int_or_slice(args[0]):
+@lazyobject
+def _HIST_MAIN_ACTIONS():
+    return {
+        'show': _hist_show,
+        'id': lambda ns, hist: print(hist.sessionid),
+        'file': lambda ns, hist: print(hist.filename),
+        'info': _hist_info,
+        'diff': _dh_main_action,
+        'gc': _hist_gc,
+    }
+
+
+def _hist_parse_args(args):
+    """Parse arguments using the history argument parser."""
+    parser = _hist_create_parser()
+    if not args:
+        args = ['show']
+    elif args[0] not in ['-h', '--help'] and args[0] not in _HIST_MAIN_ACTIONS:
         args.insert(0, 'show')
-    elif args[0] not in list(_HIST_MAIN_ACTIONS) + ['-h', '--help']:
-        print("{} is not a valid input.".format(args[0]),
-              file=sys.stderr)
-        return
-    if (args[0] in ['show', 'xonsh', 'zsh', 'bash', 'session', 'all'] and
-        len(args) > 1 and args[-1].startswith('-') and
-            args[-1][1].isdigit()):
-        args.insert(-1, '--')  # ensure parsing stops before a negative int
-    ns = _hist_create_parser().parse_args(args)
-    if ns.action is None:  # apply default action
-        ns = _hist_create_parser().parse_args(['show'] + args)
-    _HIST_MAIN_ACTIONS[ns.action](ns, hist)
+    if (args[0] == 'show'
+       and len(args) > 1
+       and args[1] not in ['-h', '--help', '-r']
+       and args[1] not in _HIST_SESSIONS):
+        args.insert(1, 'session')
+    return parser.parse_args(args)
 
 
 def history_main(args=None, stdin=None):
     """This is the history command entry point."""
-    _hist_main(builtins.__xonsh_history__, args)  # pylint: disable=no-member
+    hist = builtins.__xonsh_history__
+    ns = _hist_parse_args(args)
+    _HIST_MAIN_ACTIONS[ns.action](ns, hist)

xonsh/inspectors.py

@@ -15,7 +15,6 @@ import types
 import inspect
 import itertools
 import linecache
-import importlib
 import collections
 
 from xonsh.lazyasd import LazyObject
@@ -24,11 +23,7 @@ from xonsh.openpy import read_py_file
 from xonsh.tools import (cast_unicode, safe_hasattr, indent,
                          print_color, format_color)
 from xonsh.platform import HAS_PYGMENTS, PYTHON_VERSION_INFO
-
-pygments = LazyObject(lambda: importlib.import_module('pygments'),
-                      globals(), 'pygments')
-pyghooks = LazyObject(lambda: importlib.import_module('xonsh.pyghooks'),
-                      globals(), 'pyghooks')
+from xonsh.lazyimps import pygments, pyghooks
 
 
 # builtin docstrings to ignore
@@ -307,6 +302,7 @@ if PYTHON_VERSION_INFO < (3, 5, 0):
     FrameInfo = collections.namedtuple('FrameInfo', ['frame', 'filename',
                                                      'lineno', 'function',
                                                      'code_context', 'index'])
+
     def getouterframes(frame, context=1):
         """Wrapper for getouterframes so that it acts like the Python v3.5 version."""
         return [FrameInfo(*f) for f in inspect.getouterframes(frame, context=context)]
@@ -634,8 +630,8 @@ class Inspector(object):
                 str_head = 'string_form'
                 if not detail_level and len(ostr) > string_max:
                     ostr = ostr[:shalf] + ' <...> ' + ostr[-shalf:]
-                    ostr = ("\n" + " " * len(str_head.expandtabs())).\
-                           join(q.strip() for q in ostr.split("\n"))
+                    ostr = ("\n" + " " * len(str_head.expandtabs())). \
+                        join(q.strip() for q in ostr.split("\n"))
                 out[str_head] = ostr
             except:  # pylint:disable=bare-except
                 pass

xonsh/jobs.py

@@ -129,7 +129,6 @@ else:
             _st = None
         return _st
 
-
     # _give_terminal_to is a simplified version of:
     #    give_terminal_to from bash 4.3 source, jobs.c, line 4030
     # this will give the terminal to the process group pgid
@@ -164,7 +163,6 @@ else:
                 os.tcsetpgrp(st, pgid)
                 signal.pthread_sigmask(signal.SIG_SETMASK, oldmask)
 
-
     def wait_for_active_job():
         """
         Wait for the active job to finish, to be killed by SIGINT, or to be

xonsh/jupyter_kernel.py

@@ -9,10 +9,12 @@ from ipykernel.kernelbase import Kernel
 from xonsh import __version__ as version
 from xonsh.main import main_context
 from xonsh.tools import redirect_stdout, redirect_stderr, swap
+from xonsh.completer import Completer
 
 
 MAX_SIZE = 8388608  # 8 Mb
 
+
 class XonshKernel(Kernel):
     """Xonsh xernal for Jupyter"""
     implementation = 'Xonsh ' + version
@@ -27,6 +29,10 @@ class XonshKernel(Kernel):
                      'file_extension': '.xsh',
                      }
 
+    def __init__(self, **kwargs):
+        self.completer = Completer()
+        super().__init__(**kwargs)
+
     def do_execute(self, code, silent, store_history=True, user_expressions=None,
                    allow_stdin=False):
         """Execute user code."""
@@ -95,8 +101,14 @@ class XonshKernel(Kernel):
     def do_complete(self, code, pos):
         """Get completions."""
         shell = builtins.__xonsh_shell__
-        comps, beg, end = shell.completer.find_and_complete(code, pos, shell.ctx)
-        message = {'matches': comps, 'cursor_start': beg, 'cursor_end': end+1,
+        line = code.split('\n')[-1]
+        line = builtins.aliases.expand_alias(line)
+        prefix = line.split(' ')[-1]
+        endidx = pos
+        begidx = pos - len(prefix)
+        rtn, _ = self.completer.complete(prefix, line, begidx,
+                                         endidx, shell.ctx)
+        message = {'matches': rtn, 'cursor_start': begidx, 'cursor_end': endidx,
                    'metadata': {}, 'status': 'ok'}
         return message
 

xonsh/lazyasd.py

@@ -10,6 +10,9 @@ import importlib
 import importlib.util
 import collections.abc as abc
 
+__version__ = '0.1.1'
+
+
 class LazyObject(object):
 
     def __init__(self, load, ctx, name):
@@ -115,7 +118,6 @@ def lazyobject(f):
     return LazyObject(f, f.__globals__, f.__name__)
 
 
-
 class LazyDict(abc.MutableMapping):
 
     def __init__(self, loaders, ctx, name):
@@ -249,7 +251,7 @@ class BackgroundModuleProxy(types.ModuleType):
             }
 
     def __getattribute__(self, name):
-        passthrough = frozenset({'__dct__','__class__', '__spec__'})
+        passthrough = frozenset({'__dct__', '__class__', '__spec__'})
         if name in passthrough:
             return super().__getattribute__(name)
         dct = self.__dct__
@@ -278,14 +280,19 @@ class BackgroundModuleLoader(threading.Thread):
 
     def run(self):
         # wait for other modules to stop being imported
-        i = 0
-        last = -6
-        hist = [-5, -4, -3, -2, -1]
-        while not all(last == x for x in hist):
+        # We assume that module loading is finished when sys.modules doesn't
+        # get longer in 5 consecutive 1ms waiting steps
+        counter = 0
+        last = -1
+        while counter < 5:
+            new = len(sys.modules)
+            if new == last:
+                counter += 1
+            else:
+                last = new
+                counter = 0
             time.sleep(0.001)
-            last = hist[i%5] = len(sys.modules)
-            i += 1
-        # now import pkg_resources properly
+        # now import module properly
         modname = importlib.util.resolve_name(self.name, self.package)
         if isinstance(sys.modules[modname], BackgroundModuleProxy):
             del sys.modules[modname]

xonsh/lazyimps.py

@@ -0,0 +1,9 @@
+"""Lazy imports that may apply across the xonsh package."""
+import importlib
+
+from xonsh.lazyasd import LazyObject
+
+pygments = LazyObject(lambda: importlib.import_module('pygments'),
+                      globals(), 'pygments')
+pyghooks = LazyObject(lambda: importlib.import_module('xonsh.pyghooks'),
+                      globals(), 'pyghooks')

xonsh/lazyjson.py

@@ -68,8 +68,7 @@ def index(obj, sort_keys=False):
     return s, idx
 
 
-JSON_FORMAT = \
-"""{{"locs": [{iloc:>10}, {ilen:>10}, {dloc:>10}, {dlen:>10}],
+JSON_FORMAT = """{{"locs": [{iloc:>10}, {ilen:>10}, {dloc:>10}, {dlen:>10}],
  "index": {index},
  "data": {data}
 }}
@@ -248,4 +247,3 @@ class LazyJSON(LJNode):
 
     def __exit__(self, exc_type, exc_value, traceback):
         self.close()
-

xonsh/lexer.py

@@ -11,11 +11,11 @@ try:
 except ImportError:
     from xonsh.ply.lex import LexToken
 
-from xonsh.lazyasd import LazyObject, lazyobject
+from xonsh.lazyasd import lazyobject
 from xonsh.platform import PYTHON_VERSION_INFO
 from xonsh.tokenize import (OP, IOREDIRECT, STRING, DOLLARNAME, NUMBER,
-    SEARCHPATH, NEWLINE, INDENT, DEDENT, NL, COMMENT, ENCODING,
-    ENDMARKER, NAME, ERRORTOKEN, tokenize, TokenError)
+                            SEARCHPATH, NEWLINE, INDENT, DEDENT, NL, COMMENT, ENCODING,
+                            ENDMARKER, NAME, ERRORTOKEN, tokenize, TokenError)
 
 
 @lazyobject
@@ -169,6 +169,7 @@ def _make_matcher_handler(tok, typ, pymode, ender, handlers):
     matcher = (')' if tok.endswith('(') else
                '}' if tok.endswith('{') else
                ']' if tok.endswith('[') else None)
+
     def _inner_handler(state, token):
         state['pymode'].append((pymode, tok, matcher, token.start))
         state['last'] = token

xonsh/main.py

@@ -5,26 +5,19 @@ import sys
 import enum
 import argparse
 import builtins
-import importlib
 import contextlib
 
 from xonsh import __version__
-from xonsh.lazyasd import LazyObject, lazyobject
-from xonsh.environ import DEFAULT_VALUES
+from xonsh.lazyasd import lazyobject
 from xonsh.shell import Shell
-from xonsh.pretty import pprint, pretty
+from xonsh.pretty import pretty
 from xonsh.proc import HiddenCompletedCommand
 from xonsh.jobs import ignore_sigtstp
 from xonsh.tools import setup_win_unicode_console, print_color
 from xonsh.platform import HAS_PYGMENTS, ON_WINDOWS
 from xonsh.codecache import run_script_with_cache, run_code_with_cache
 from xonsh.xonfig import xonfig_main
-
-
-pygments = LazyObject(lambda: importlib.import_module('pygments'),
-                      globals(), 'pygments')
-pyghooks = LazyObject(lambda: importlib.import_module('xonsh.pyghooks'),
-                      globals(), 'pyghooks')
+from xonsh.lazyimps import pygments, pyghooks
 
 
 def get_setproctitle():
@@ -195,8 +188,7 @@ def premain(argv=None):
         args.mode = XonshMode.interactive
         shell_kwargs['completer'] = True
         shell_kwargs['login'] = True
-    from xonsh import imphooks
-    shell = builtins.__xonsh_shell__ = Shell(**shell_kwargs)
+    builtins.__xonsh_shell__ = Shell(**shell_kwargs)
     env = builtins.__xonsh_env__
     env['XONSH_LOGIN'] = shell_kwargs['login']
     if args.defines is not None:
@@ -264,7 +256,3 @@ def main_context(argv=None):
     args = premain(argv)
     yield builtins.__xonsh_shell__
     postmain(args)
-
-
-if __name__ == '__main__':
-    main()

xonsh/openpy.py

@@ -13,17 +13,17 @@ This file was forked from the IPython project:
 * Copyright (c) 2001, Nathaniel Gray <n8gray@caltech.edu>
 """
 import io
-import os
 import re
-import tokenize
 
 from xonsh.lazyasd import LazyObject
+from xonsh.tokenize import detect_encoding, tokopen
 
 
 cookie_comment_re = LazyObject(
     lambda: re.compile(r"^\s*#.*coding[:=]\s*([-\w.]+)", re.UNICODE),
     globals(), 'cookie_comment_re')
 
+
 def source_to_unicode(txt, errors='replace', skip_encoding_cookie=True):
     """Converts a bytes string with python source code to unicode.
 
@@ -39,7 +39,7 @@ def source_to_unicode(txt, errors='replace', skip_encoding_cookie=True):
     else:
         buf = txt
     try:
-        encoding, _ = tokenize.detect_encoding(buf.readline)
+        encoding, _ = detect_encoding(buf.readline)
     except SyntaxError:
         encoding = "ascii"
     buf.seek(0)
@@ -65,7 +65,6 @@ def strip_encoding_cookie(filelike):
             yield second
     except StopIteration:
         return
-
     for line in it:
         yield line
 
@@ -86,7 +85,7 @@ def read_py_file(filename, skip_encoding_cookie=True):
     -------
     A unicode string containing the contents of the file.
     """
-    with tokenize.tokopen(filename) as f:  # the open function defined in this module.
+    with tokopen(filename) as f:  # the open function defined in this module.
         if skip_encoding_cookie:
             return "".join(strip_encoding_cookie(f))
         else:

xonsh/parser.py

@@ -3,6 +3,7 @@
 from xonsh.lazyasd import lazyobject
 from xonsh.platform import PYTHON_VERSION_INFO
 
+
 @lazyobject
 def Parser():
     if PYTHON_VERSION_INFO < (3, 5, 0):

xonsh/parsers/base.py

@@ -21,6 +21,7 @@ from xonsh.lazyasd import LazyObject
 RE_SEARCHPATH = LazyObject(lambda: re.compile(SearchPath), globals(),
                            'RE_SEARCHPATH')
 
+
 class Location(object):
     """Location in a file."""
 
@@ -274,7 +275,7 @@ class BaseParser(object):
         yacc_kwargs['outputdir'] = outputdir
         self.parser = None
         YaccLoader(self, yacc_kwargs)
-        #self.parser = yacc.yacc(**yacc_kwargs)
+        # self.parser = yacc.yacc(**yacc_kwargs)
 
         # Keeps track of the last token given to yacc (the lookahead token)
         self._last_yielded_token = None

xonsh/parsers/v34.py

@@ -101,7 +101,6 @@ class Parser(BaseParser):
                     self._set_arg(p0, arg)
             self._set_arg(p0, p[6], ensure_kw=True)
         elif lenp == 8:
-            kwkey = 'kwargs'
             p0['starargs'], p4 = p[3], p[4]
             if p1 is not None:
                 for arg in p1:

xonsh/platform.py

@@ -10,7 +10,7 @@ import functools
 import subprocess
 import importlib.util
 
-from xonsh.lazyasd import LazyObject, LazyBool, lazyobject, lazybool
+from xonsh.lazyasd import LazyBool, lazyobject, lazybool
 
 
 @lazyobject
@@ -227,8 +227,7 @@ def git_for_windows_path():
 
 @functools.lru_cache(1)
 def windows_bash_command():
-    """Determines teh command for Bash on windows."""
-    import winreg
+    """Determines the command for Bash on windows."""
     # Check that bash is on path otherwise try the default directory
     # used by Git for windows
     wbc = 'bash'
@@ -248,6 +247,17 @@ def windows_bash_command():
 # Environment variables defaults
 #
 
+
+@functools.lru_cache(1)
+def bash_command():
+    """Determines the command for Bash on the current plaform."""
+    if ON_WINDOWS:
+        bc = windows_bash_command()
+    else:
+        bc = 'bash'
+    return bc
+
+
 @lazyobject
 def BASH_COMPLETIONS_DEFAULT():
     """A possibly empty tuple with default paths to Bash completions known for
@@ -266,11 +276,11 @@ def BASH_COMPLETIONS_DEFAULT():
                '/opt/local/etc/profile.d/bash_completion.sh')
     elif ON_WINDOWS and git_for_windows_path():
         bcd = (os.path.join(git_for_windows_path(),
-                    'usr\\share\\bash-completion'),
+                            'usr\\share\\bash-completion'),
                os.path.join(git_for_windows_path(),
-                    'usr\\share\\bash-completion\\completions'),
+                            'usr\\share\\bash-completion\\completions'),
                os.path.join(git_for_windows_path(),
-                     'mingw64\\share\\git\\completion\\git-completion.bash'))
+                            'mingw64\\share\\git\\completion\\git-completion.bash'))
     else:
         bcd = ()
     return bcd
@@ -292,7 +302,7 @@ def PATH_DEFAULT():
     elif ON_WINDOWS:
         import winreg
         key = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE,
-                r'SYSTEM\CurrentControlSet\Control\Session Manager\Environment')
+                             r'SYSTEM\CurrentControlSet\Control\Session Manager\Environment')
         pd = tuple(winreg.QueryValueEx(key, 'Path')[0].split(os.pathsep))
     else:
         pd = ()

xonsh/ply/__init__.py

@@ -1,5 +1,5 @@
 # PLY package
 # Author: David Beazley (dave@dabeaz.com)
 
-__version__ = '3.7'
+__version__ = '3.8'
 __all__ = ['lex','yacc']

xonsh/pretty.py

@@ -17,8 +17,8 @@ Example Usage
 
 To directly print the representation of an object use `pprint`::
 
-    from pretty import pprint
-    pprint(complex_object)
+    from pretty import pretty_print
+    pretty_pprint(complex_object)
 
 To get a string of the output use `pretty`::
 
@@ -88,13 +88,14 @@ import collections
 
 from xonsh.lazyasd import LazyObject, lazyobject
 
-
-__all__ = ['pretty', 'pprint', 'PrettyPrinter', 'RepresentationPrinter',
+__all__ = [
+    'pretty', 'pretty_print', 'PrettyPrinter', 'RepresentationPrinter',
     'for_type', 'for_type_by_name']
 
 
 MAX_SEQ_LENGTH = 1000
 
+
 def _safe_getattr(obj, attr, default=None):
     """Safe version of getattr.
 
@@ -121,9 +122,9 @@ def pretty(obj, verbose=False, max_width=79, newline='\n', max_seq_length=MAX_SE
     return stream.getvalue()
 
 
-def pprint(obj, verbose=False, max_width=79, newline='\n', max_seq_length=MAX_SEQ_LENGTH):
+def pretty_print(obj, verbose=False, max_width=79, newline='\n', max_seq_length=MAX_SEQ_LENGTH):
     """
-    Like `pretty` but print to stdout.
+    Like pretty() but print to stdout.
     """
     printer = RepresentationPrinter(sys.stdout, verbose, max_width, newline, max_seq_length=max_seq_length)
     printer.pretty(obj)
@@ -133,7 +134,6 @@ def pprint(obj, verbose=False, max_width=79, newline='\n', max_seq_length=MAX_SE
 
 
 class _PrettyPrinterBase(object):
-
     @contextlib.contextmanager
     def indent(self, indent):
         """with statement support for indenting/dedenting."""
@@ -233,7 +233,6 @@ class PrettyPrinter(_PrettyPrinterBase):
         self.output_width = self.indentation
         self.buffer_width = 0
 
-
     def begin_group(self, indent=0, open=''):
         """
         Begin a group.  If you want support for python < 2.5 which doesn't has
@@ -321,8 +320,8 @@ class RepresentationPrinter(PrettyPrinter):
     """
 
     def __init__(self, output, verbose=False, max_width=79, newline='\n',
-        singleton_pprinters=None, type_pprinters=None, deferred_pprinters=None,
-        max_seq_length=MAX_SEQ_LENGTH):
+                 singleton_pprinters=None, type_pprinters=None, deferred_pprinters=None,
+                 max_seq_length=MAX_SEQ_LENGTH):
 
         PrettyPrinter.__init__(self, output, max_width, newline, max_seq_length=max_seq_length)
         self.verbose = verbose
@@ -398,13 +397,11 @@ class RepresentationPrinter(PrettyPrinter):
 
 
 class Printable(object):
-
     def output(self, stream, output_width):
         return output_width
 
 
 class Text(Printable):
-
     def __init__(self):
         self.objs = []
         self.width = 0
@@ -420,7 +417,6 @@ class Text(Printable):
 
 
 class Breakable(Printable):
-
     def __init__(self, seq, width, pretty):
         self.obj = seq
         self.width = width
@@ -442,7 +438,6 @@ class Breakable(Printable):
 
 
 class Group(Printable):
-
     def __init__(self, depth):
         self.depth = depth
         self.breakables = collections.deque()
@@ -450,7 +445,6 @@ class Group(Printable):
 
 
 class GroupQueue(object):
-
     def __init__(self, *groups):
         self.queue = []
         for group in groups:
@@ -532,6 +526,7 @@ def _seq_pprinter_factory(start, end, basetype):
     Factory that returns a pprint function useful for sequences.  Used by
     the default pprint for tuples, dicts, and lists.
     """
+
     def inner(obj, p, cycle):
         typ = type(obj)
         if basetype is not None and typ is not basetype and typ.__repr__ != basetype.__repr__:
@@ -551,6 +546,7 @@ def _seq_pprinter_factory(start, end, basetype):
             # Special case for 1-item tuples.
             p.text(',')
         p.end_group(step, end)
+
     return inner
 
 
@@ -558,6 +554,7 @@ def _set_pprinter_factory(start, end, basetype):
     """
     Factory that returns a pprint function useful for sets and frozensets.
     """
+
     def inner(obj, p, cycle):
         typ = type(obj)
         if basetype is not None and typ is not basetype and typ.__repr__ != basetype.__repr__:
@@ -586,6 +583,7 @@ def _set_pprinter_factory(start, end, basetype):
                     p.breakable()
                 p.pretty(x)
             p.end_group(step, end)
+
     return inner
 
 
@@ -594,6 +592,7 @@ def _dict_pprinter_factory(start, end, basetype=None):
     Factory that returns a pprint function used by the default pprint of
     dicts and dict proxies.
     """
+
     def inner(obj, p, cycle):
         typ = type(obj)
         if basetype is not None and typ is not basetype and typ.__repr__ != basetype.__repr__:
@@ -619,6 +618,7 @@ def _dict_pprinter_factory(start, end, basetype=None):
             p.text(': ')
             p.pretty(obj[key])
         p.end_group(1, end)
+
     return inner
 
 
@@ -648,7 +648,7 @@ def _re_pattern_pprint(obj, p, cycle):
         p.breakable()
         done_one = False
         for flag in ('TEMPLATE', 'IGNORECASE', 'LOCALE', 'MULTILINE', 'DOTALL',
-            'UNICODE', 'VERBOSE', 'DEBUG'):
+                     'UNICODE', 'VERBOSE', 'DEBUG'):
             if obj.flags & getattr(re, flag):
                 if done_one:
                     p.text('|')
@@ -689,7 +689,7 @@ def _repr_pprint(obj, p, cycle):
     """A pprint that just redirects to the normal repr function."""
     # Find newlines and replace them with p.break_()
     output = repr(obj)
-    for idx,output_line in enumerate(output.splitlines()):
+    for idx, output_line in enumerate(output.splitlines()):
         if idx:
             p.break_()
         p.text(output_line)
@@ -719,40 +719,38 @@ def _exception_pprint(obj, p, cycle):
     p.end_group(step, ')')
 
 
-
-
 @lazyobject
 def _type_pprinters():
     #: printers for builtin types
     tp = {
-        int:                        _repr_pprint,
-        float:                      _repr_pprint,
-        str:                        _repr_pprint,
-        tuple:                      _seq_pprinter_factory('(', ')', tuple),
-        list:                       _seq_pprinter_factory('[', ']', list),
-        dict:                       _dict_pprinter_factory('{', '}', dict),
-        set:                        _set_pprinter_factory('{', '}', set),
-        frozenset:                  _set_pprinter_factory('frozenset({', '})', frozenset),
-        super:                      _super_pprint,
-        type(re.compile('')):       _re_pattern_pprint,
-        type:                       _type_pprint,
-        types.FunctionType:         _function_pprint,
-        types.BuiltinFunctionType:  _function_pprint,
-        types.MethodType:           _repr_pprint,
-        datetime.datetime:          _repr_pprint,
-        datetime.timedelta:         _repr_pprint,
-        }
+        int: _repr_pprint,
+        float: _repr_pprint,
+        str: _repr_pprint,
+        tuple: _seq_pprinter_factory('(', ')', tuple),
+        list: _seq_pprinter_factory('[', ']', list),
+        dict: _dict_pprinter_factory('{', '}', dict),
+        set: _set_pprinter_factory('{', '}', set),
+        frozenset: _set_pprinter_factory('frozenset({', '})', frozenset),
+        super: _super_pprint,
+        type(re.compile('')): _re_pattern_pprint,
+        type: _type_pprint,
+        types.FunctionType: _function_pprint,
+        types.BuiltinFunctionType: _function_pprint,
+        types.MethodType: _repr_pprint,
+        datetime.datetime: _repr_pprint,
+        datetime.timedelta: _repr_pprint,
+    }
     #: the exception base
     try:
-       _exception_base = BaseException
+        _exception_base = BaseException
     except NameError:
         _exception_base = Exception
-    tp[_exception_base] =  _exception_pprint
+    tp[_exception_base] = _exception_pprint
     try:
         tp[types.DictProxyType] = _dict_pprinter_factory('<dictproxy {', '}>')
         tp[types.ClassType] = _type_pprint
         tp[types.SliceType] = _repr_pprint
-    except AttributeError: # Python 3
+    except AttributeError:  # Python 3
         tp[slice] = _repr_pprint
     try:
         tp[xrange] = _repr_pprint
@@ -763,6 +761,7 @@ def _type_pprinters():
         tp[bytes] = _repr_pprint
     return tp
 
+
 #: printers for types specified by name
 @lazyobject
 def _deferred_type_pprinters():
@@ -802,8 +801,8 @@ def for_type_by_name(type_module, type_name, func, dtp=None):
 
 #: printers for the default singletons
 _singleton_pprinters = LazyObject(lambda: dict.fromkeys(
-                                    map(id, [None, True, False, Ellipsis,
-                                             NotImplemented]), _repr_pprint),
+    map(id, [None, True, False, Ellipsis,
+             NotImplemented]), _repr_pprint),
                                   globals(), '_singleton_pprinters')
 
 
@@ -844,4 +843,3 @@ def _counter_pprint(obj, p, cycle):
             p.text('...')
         elif len(obj):
             p.pretty(dict(obj))
-

xonsh/proc.py

@@ -11,6 +11,7 @@ import io
 import os
 import sys
 import time
+import signal
 import builtins
 import functools
 import threading
@@ -19,17 +20,24 @@ import subprocess
 import collections
 import collections.abc as abc
 
-from xonsh.tools import (redirect_stdout, redirect_stderr, ON_WINDOWS, ON_LINUX,
-                         fallback, print_exception, XonshCalledProcessError)
+from xonsh.platform import ON_WINDOWS, ON_LINUX, ON_POSIX
+from xonsh.tools import (redirect_stdout, redirect_stderr, fallback,
+                         print_exception, XonshCalledProcessError)
 from xonsh.teepty import TeePTY
-from xonsh.lazyasd import LazyObject
-
+from xonsh.lazyasd import LazyObject, lazyobject
 
 # force some lazy imports so we don't have errors on non-windows platforms
 _winapi = LazyObject(lambda: importlib.import_module('_winapi'),
                      globals(), '_winapi')
-msvcrt = LazyObject(lambda: importlib.import_module('msvcrt'),
-                    globals(), 'msvcrt')
+
+
+@lazyobject
+def msvcrt():
+    if ON_WINDOWS:
+        import msvcrt as m
+    else:
+        m = None
+    return m
 
 
 class Handle(int):
@@ -58,6 +66,7 @@ class ProcProxy(threading.Thread):
     """
     Class representing a function to be run as a subprocess-mode command.
     """
+
     def __init__(self, f, args,
                  stdin=None,
                  stdout=None,
@@ -330,6 +339,7 @@ class ProcProxy(threading.Thread):
 def wrap_simple_command(f, args, stdin, stdout, stderr):
     """Decorator for creating 'simple' callable aliases."""
     bgable = getattr(f, '__xonsh_backgroundable__', True)
+
     @functools.wraps(f)
     def wrapped_simple_command(args, stdin, stdout, stderr):
         try:
@@ -356,6 +366,7 @@ def wrap_simple_command(f, args, stdin, stdout, stderr):
         except Exception:
             print_exception()
             return 1  # returncode for failure
+
     return wrapped_simple_command
 
 
@@ -372,6 +383,7 @@ class SimpleProcProxy(ProcProxy):
         f = wrap_simple_command(f, args, stdin, stdout, stderr)
         super().__init__(f, args, stdin, stdout, stderr, universal_newlines)
 
+
 #
 # Foreground Process Proxies
 #
@@ -437,6 +449,7 @@ def foreground(f):
     f.__xonsh_backgroundable__ = False
     return f
 
+
 #
 # Pseudo-terminal Proxies
 #
@@ -444,7 +457,6 @@ def foreground(f):
 
 @fallback(ON_LINUX, subprocess.Popen)
 class TeePTYProc(object):
-
     def __init__(self, args, stdin=None, stdout=None, stderr=None, preexec_fn=None,
                  env=None, universal_newlines=False):
         """Popen replacement for running commands in teed psuedo-terminal. This
@@ -457,8 +469,8 @@ class TeePTYProc(object):
         self.args = args
         self.universal_newlines = universal_newlines
         xenv = builtins.__xonsh_env__ if hasattr(builtins, '__xonsh_env__') \
-                                      else {'XONSH_ENCODING': 'utf-8',
-                                            'XONSH_ENCODING_ERRORS': 'strict'}
+            else {'XONSH_ENCODING': 'utf-8',
+                  'XONSH_ENCODING_ERRORS': 'strict'}
 
         if not os.access(args[0], os.F_OK):
             raise FileNotFoundError('command {0!r} not found'.format(args[0]))
@@ -536,9 +548,9 @@ def _wcode_to_popen(code):
 
 
 _CCTuple = collections.namedtuple("_CCTuple", ["stdin", "stdout", "stderr",
-                "pid", "returncode", "args", "alias", "stdin_redirect",
-                "stdout_redirect", "stderr_redirect", "timestamp",
-                "executed_cmd"])
+                                               "pid", "returncode", "args", "alias", "stdin_redirect",
+                                               "stdout_redirect", "stderr_redirect", "timestamp",
+                                               "executed_cmd"])
 
 
 class CompletedCommand(_CCTuple):
@@ -561,7 +573,6 @@ class CompletedCommand(_CCTuple):
             yield pre
             pre = post
 
-
     def itercheck(self):
         yield from self
         if self.returncode:
@@ -590,9 +601,32 @@ class CompletedCommand(_CCTuple):
         """Alias to return code."""
         return self.returncode
 
+
 CompletedCommand.__new__.__defaults__ = (None,) * len(CompletedCommand._fields)
 
 
 class HiddenCompletedCommand(CompletedCommand):
     def __repr__(self):
         return ''
+
+
+def pause_call_resume(p, f, *args, **kwargs):
+    """For a process p, this will call a function f with the remaining args and
+    and kwargs. If the process cannot accept signals, the function will be called.
+
+    Parameters
+    ----------
+    p : Popen object or similar
+    f : callable
+    args : remaining arguments
+    kwargs : keyword arguments
+    """
+    can_send_signal = hasattr(p, 'send_signal') and ON_POSIX
+    if can_send_signal:
+        p.send_signal(signal.SIGSTOP)
+    try:
+        f(*args, **kwargs)
+    except Exception:
+        pass
+    if can_send_signal:
+        p.send_signal(signal.SIGCONT)

xonsh/ptk/__init__.py

@@ -1,4 +1,5 @@
 # must come before ptk / pygments imports
 from xonsh.lazyasd import load_module_in_background
+
 load_module_in_background('pkg_resources', debug='XONSH_DEBUG',
-    replacements={'pygments.plugin': 'pkg_resources'})
+                          replacements={'pygments.plugin': 'pkg_resources'})

xonsh/ptk/completer.py

@@ -6,8 +6,6 @@ import builtins
 from prompt_toolkit.layout.dimension import LayoutDimension
 from prompt_toolkit.completion import Completer, Completion
 
-from xonsh.platform import ptk_version
-
 
 class PromptToolkitCompleter(Completer):
     """Simple prompt_toolkit Completer object.
@@ -32,6 +30,7 @@ class PromptToolkitCompleter(Completer):
                 endidx = document.cursor_position_col
                 begidx = line[:endidx].rfind(' ') + 1 if line[:endidx].rfind(' ') >= 0 else 0
                 prefix = line[begidx:endidx]
+                line = builtins.aliases.expand_alias(line)
                 completions, l = self.completer.complete(prefix,
                                                          line,
                                                          begidx,
@@ -52,6 +51,7 @@ class PromptToolkitCompleter(Completer):
             h = window.render_info.content_height
             r = builtins.__xonsh_env__.get('COMPLETIONS_MENU_ROWS')
             size = h + r
+
             def comp_height(cli):
                 # If there is an autocompletion menu to be shown, make sure that o
                 # layout has at least a minimal height in order to display it.

xonsh/ptk/key_bindings.py

@@ -49,7 +49,7 @@ def carriage_return(b, cli):
         b.newline(copy_margin=True)
     elif (b.document.char_before_cursor == '\\' and
             not (not builtins.__xonsh_env__.get('FORCE_POSIX_PATHS')
-                and ON_WINDOWS)):
+                 and ON_WINDOWS)):
         b.newline()
     elif (b.document.find_next_word_beginning() is not None and
             (any(not _is_blank(i)
@@ -74,6 +74,7 @@ class TabShouldInsertIndentFilter(Filter):
 
         return bool(before_cursor.isspace())
 
+
 class BeginningOfLine(Filter):
     """
     Check if cursor is at beginning of a line other than the first line
@@ -85,6 +86,7 @@ class BeginningOfLine(Filter):
         return bool(len(before_cursor) == 0
                     and not cli.current_buffer.document.on_first_line)
 
+
 class EndOfLine(Filter):
     """
     Check if cursor is at the end of a line other than the last line
@@ -177,7 +179,6 @@ def load_xonsh_bindings(key_bindings_manager):
         b.cursor_left(count=abs(relative_begin_index))
         b.cursor_down(count=1)
 
-
     @handle(Keys.ControlI, filter=insert_mode)
     def generate_completions(event):
         """
@@ -203,6 +204,8 @@ def load_xonsh_bindings(key_bindings_manager):
             second_tab()
         else:
             event.cli.start_completion(insert_common_part=True,
-                                    select_first=False)
+                                       select_first=False)
+
+
 def _is_blank(l):
     return len(l.strip()) == 0

xonsh/ptk/shell.py

@@ -6,7 +6,6 @@ from prompt_toolkit.key_binding.manager import KeyBindingManager
 from prompt_toolkit.auto_suggest import AutoSuggestFromHistory
 from prompt_toolkit.layout.lexers import PygmentsLexer
 from prompt_toolkit.shortcuts import print_tokens
-from prompt_toolkit.filters import Condition
 from prompt_toolkit.styles import PygmentsStyle
 from pygments.styles import get_all_styles
 from pygments.token import Token
@@ -14,7 +13,6 @@ from pygments.token import Token
 from xonsh.base_shell import BaseShell
 from xonsh.tools import print_exception
 from xonsh.environ import partial_format_prompt
-from xonsh.platform import ptk_version, ptk_version_info
 from xonsh.pyghooks import (XonshLexer, partial_color_tokenize,
                             xonsh_style_proxy)
 from xonsh.ptk.completer import PromptToolkitCompleter

xonsh/ptk/shortcuts.py

@@ -6,7 +6,7 @@ from prompt_toolkit.interface import CommandLineInterface
 from prompt_toolkit.enums import EditingMode
 from prompt_toolkit.utils import DummyContext
 from prompt_toolkit.shortcuts import (create_prompt_application,
-    create_eventloop, create_asyncio_eventloop, create_output)
+                                      create_eventloop, create_asyncio_eventloop, create_output)
 
 from xonsh.platform import ptk_version_info
 
@@ -32,7 +32,6 @@ class Prompter(object):
         return self
 
     def __exit__(self, exc_type, exc_value, traceback):
-        #self.reset()
         pass
 
     def prompt(self, message='', **kwargs):

xonsh/pyghooks.py

@@ -4,15 +4,15 @@ import os
 import re
 import string
 import builtins
-import importlib
 from warnings import warn
 from collections import ChainMap
 from collections.abc import MutableMapping
 
 # must come before pygments imports
 from xonsh.lazyasd import load_module_in_background
+
 load_module_in_background('pkg_resources', debug='XONSH_DEBUG',
-    replacements={'pygments.plugin': 'pkg_resources'})
+                          replacements={'pygments.plugin': 'pkg_resources'})
 
 from pygments.lexer import inherit, bygroups, using, this
 from pygments.lexers.shell import BashLexer
@@ -23,7 +23,7 @@ from pygments.style import Style
 from pygments.styles import get_style_by_name
 import pygments.util
 
-from xonsh.lazyasd import LazyObject, lazyobject, LazyDict
+from xonsh.lazyasd import LazyObject, LazyDict
 from xonsh.tools import (ON_WINDOWS, intensify_colors_for_cmd_exe,
                          expand_gray_colors_for_cmd_exe)
 from xonsh.tokenize import SearchPath
@@ -104,6 +104,7 @@ def norm_name(name):
     """Normalizes a color name."""
     return name.replace('#', 'HEX').replace('BGHEX', 'BACKGROUND_HEX')
 
+
 def color_by_name(name, fg=None, bg=None):
     """Converts a color name to a color token, foreground name,
     and background name.  Will take into consideration current foreground
@@ -633,9 +634,9 @@ def _monokai_style():
     return style
 
 
-#############################################################
-#############   Auto-generated below this line   ############
-#############################################################
+######################################
+#   Auto-generated below this line   #
+######################################
 def _algol_style():
     style = {
         Color.BLACK: '#666',
@@ -659,6 +660,7 @@ def _algol_style():
     _expand_style(style)
     return style
 
+
 def _algol_nu_style():
     style = {
         Color.BLACK: '#666',
@@ -682,6 +684,7 @@ def _algol_nu_style():
     _expand_style(style)
     return style
 
+
 def _autumn_style():
     style = {
         Color.BLACK: '#000080',

xonsh/readline_shell.py

@@ -27,14 +27,11 @@ from xonsh.ansi_colors import ansi_partial_color_format, ansi_color_style_names,
 from xonsh.environ import partial_format_prompt, multiline_prompt
 from xonsh.tools import print_exception
 from xonsh.platform import ON_WINDOWS, ON_CYGWIN, ON_DARWIN
+from xonsh.lazyimps import pygments, pyghooks
 
-pygments = LazyObject(lambda: importlib.import_module('pygments'),
-                      globals(), 'pygments')
-terminal256 = LazyObject(lambda: importlib.import_module(
-                                    'pygments.formatters.terminal256'),
-                      globals(), 'terminal')
-pyghooks = LazyObject(lambda: importlib.import_module('xonsh.pyghooks'),
-                      globals(), 'pyghooks')
+terminal256 = LazyObject(
+    lambda: importlib.import_module('pygments.formatters.terminal256'),
+    globals(), 'terminal')
 
 readline = None
 RL_COMPLETION_SUPPRESS_APPEND = RL_LIB = RL_STATE = None
@@ -95,16 +92,16 @@ def setup_readline():
     # as discussed at http://stackoverflow.com/a/7116997
     if uses_libedit and ON_DARWIN:
         readline.parse_and_bind("bind ^I rl_complete")
-        print('\n'.join(['', "*"*78,
-            "libedit detected - readline will not be well behaved, including but not limited to:",
-            "   * crashes on tab completion",
-            "   * incorrect history navigation",
-            "   * corrupting long-lines",
-            "   * failure to wrap or indent lines properly",
-            "",
-            "It is highly recommended that you install gnureadline, which is installable with:",
-            "     pip install gnureadline",
-            "*"*78]), file=sys.stderr)
+        print('\n'.join(['', "*" * 78,
+                         "libedit detected - readline will not be well behaved, including but not limited to:",
+                         "   * crashes on tab completion",
+                         "   * incorrect history navigation",
+                         "   * corrupting long-lines",
+                         "   * failure to wrap or indent lines properly",
+                         "",
+                         "It is highly recommended that you install gnureadline, which is installable with:",
+                         "     pip install gnureadline",
+                         "*" * 78]), file=sys.stderr)
     else:
         readline.parse_and_bind("tab: complete")
     # try to load custom user settings
@@ -203,9 +200,11 @@ def rl_variable_value(variable):
 
 def _insert_text_func(s, readline):
     """Creates a function to insert text via readline."""
+
     def inserter():
         readline.insert_text(s)
         readline.redisplay()
+
     return inserter
 
 
@@ -256,6 +255,7 @@ class ReadlineShell(BaseShell, cmd.Cmd):
         rl_completion_suppress_append()  # this needs to be called each time
         _rebind_case_sensitive_completions()
 
+        line = builtins.aliases.expand_alias(line)
         mline = line.partition(' ')[2]
         offs = len(mline) - len(text)
         if self.completer is None:
@@ -335,7 +335,7 @@ class ReadlineShell(BaseShell, cmd.Cmd):
             if intro is not None:
                 self.intro = intro
             if self.intro:
-                self.stdout.write(str(self.intro)+"\n")
+                self.stdout.write(str(self.intro) + "\n")
             stop = None
             while not stop:
                 line = None
@@ -345,7 +345,7 @@ class ReadlineShell(BaseShell, cmd.Cmd):
                     exec_now = line.endswith('\n')
                 if self.use_rawinput and not exec_now:
                     inserter = None if line is None \
-                                    else _insert_text_func(line, readline)
+                        else _insert_text_func(line, readline)
                     if inserter is not None:
                         readline.set_pre_input_hook(inserter)
                     try:
@@ -404,7 +404,6 @@ class ReadlineShell(BaseShell, cmd.Cmd):
             # This is needed to support some system where line-wrapping doesn't
             # work. This is a bug in upstream Python, or possibly readline.
             RL_LIB.rl_reset_screen_size()
-        #return super().prompt
         if self.need_more_lines:
             if self.mlprompt is None:
                 try:
@@ -432,7 +431,7 @@ class ReadlineShell(BaseShell, cmd.Cmd):
         """
         hide = hide if self._force_hide is None else self._force_hide
         return ansi_partial_color_format(string, hide=hide,
-                    style=builtins.__xonsh_env__.get('XONSH_COLOR_STYLE'))
+                                         style=builtins.__xonsh_env__.get('XONSH_COLOR_STYLE'))
 
     def print_color(self, string, hide=False, **kwargs):
         if isinstance(string, str):
@@ -452,12 +451,11 @@ class ReadlineShell(BaseShell, cmd.Cmd):
 
     def color_style(self):
         """Returns the current color map."""
-        style = style=builtins.__xonsh_env__.get('XONSH_COLOR_STYLE')
+        style = style = builtins.__xonsh_env__.get('XONSH_COLOR_STYLE')
         return ansi_color_style(style=style)
 
 
 class ReadlineHistoryAdder(threading.Thread):
-
     def __init__(self, wait_for_gc=True, *args, **kwargs):
         """Thread responsible for adding inputs from history to the current readline
         instance. May wait for the history garbage collector to finish.
@@ -480,8 +478,8 @@ class ReadlineHistoryAdder(threading.Thread):
         for _, _, f in files:
             try:
                 lj = LazyJSON(f, reopen=False)
-                for cmd in lj['cmds']:
-                    inp = cmd['inp'].splitlines()
+                for command in lj['cmds']:
+                    inp = command['inp'].splitlines()
                     for line in inp:
                         if line == 'EOF':
                             continue

xonsh/replay.py

@@ -52,8 +52,7 @@ class Replayer(object):
         new_env = self._merge_envs(merge_envs, re_env)
         new_hist = History(env=new_env.detype(), locked=True, ts=[time.time(), None],
                            gc=False, filename=target)
-        with swap(builtins, '__xonsh_env__', new_env), \
-             swap(builtins, '__xonsh_history__', new_hist):
+        with swap(builtins, '__xonsh_env__', new_env), swap(builtins, '__xonsh_history__', new_hist):
             for cmd in self._lj['cmds']:
                 inp = cmd['inp']
                 shell.default(inp)
@@ -79,6 +78,7 @@ class Replayer(object):
 
 _REPLAY_PARSER = None
 
+
 def _rp_create_parser(p=None):
     global _REPLAY_PARSER
     p_was_none = (p is None)

xonsh/shell.py

@@ -9,7 +9,7 @@ from xonsh.xontribs import update_context
 from xonsh.environ import xonshrc_context
 from xonsh.execer import Execer
 from xonsh.platform import (best_shell_type, has_prompt_toolkit,
-                            ptk_version_is_supported, ptk_version_info)
+                            ptk_version_is_supported)
 from xonsh.tools import XonshError, to_bool_or_int
 
 

xonsh/teepty.py

@@ -21,6 +21,7 @@ import threading
 from xonsh.lazyasd import LazyObject, lazyobject
 from xonsh.platform import ON_WINDOWS
 
+
 #
 # Explicit lazy imports for windows
 #
@@ -73,6 +74,7 @@ RE_HIDDEN_BYTES = LazyObject(lambda: re.compile(b'(\001.*?\002)'),
 RE_COLOR = LazyObject(lambda: re.compile(b'\033\[\d+;?\d*m'),
                       globals(), 'RE_COLOR')
 
+
 def _findfirst(s, substrs):
     """Finds whichever of the given substrings occurs first in the given string
     and returns that substring, or returns None if no such strings occur.
@@ -254,7 +256,7 @@ class TeePTY(object):
     def _sanatize_data(self, data):
         i, flag = _findfirst(data, ALTERNATE_MODE_FLAGS)
         if flag is None and self._in_alt_mode:
-            return  b''
+            return b''
         elif flag is not None:
             if flag in START_ALTERNATE_MODE:
                 # This code is executed when the child process switches the terminal into
@@ -362,6 +364,7 @@ class TeePTY(object):
         if 0.0 < delay:
             time.sleep(delay)
 
+
 def _teepty_main():
     tpty = TeePTY()
     tpty.spawn(sys.argv[1:])
@@ -371,4 +374,3 @@ def _teepty_main():
     print(tpty)
     print('-=-'*10)
     print('Returned with status {0}'.format(tpty.wcode))
-

xonsh/timings.py

@@ -17,6 +17,7 @@ import itertools
 
 from xonsh.lazyasd import lazyobject, lazybool
 
+
 @lazybool
 def _HAVE_RESOURCE():
     try: