Merge branch 'master' into rmbash

CHANGELOG.rst

@@ -31,6 +31,8 @@ v0.3.4
   ``activate``/``deactivate`` aliases for the conda environements.
 * Fixed crash resulting from errors other than syntax errors in run control
   file.
+* On Windows if bash is not on the path look in the registry for the defaults
+  install directory for GitForWindows.
 
 
 v0.3.3
@@ -61,8 +63,6 @@ v0.3.3
 * RC files are now executed directly in the appropriate context.
 * ``_`` is now updated by ``![]``, to contain the appropriate
   ``CompletedCommand`` object.
-* On Windows if bash is not on the path look in the registry for the defaults
-  install directory for GitForWindows.
 
 
 

docs/tutorial.rst

@@ -460,11 +460,13 @@ Python Evaluation with ``@()``
 ===============================
 
 The ``@(<expr>)`` operator form works in subprocess mode, and will evaluate
-arbitrary Python code. The result is appended to the subprocess command
-list. If the result is a string, it is appended to the argument list. If the
-result is a list or other non-string sequence, the contents are converted to
-strings and appended to the argument list in order. Otherwise, the result is
-automatically converted to a string. For example,
+arbitrary Python code. The result is appended to the subprocess command list.
+If the result is a string, it is appended to the argument list. If the result
+is a list or other non-string sequence, the contents are converted to strings
+and appended to the argument list in order. If the result in the first position
+is a function, it is treated as an alias (see the section on `Aliases`_ below),
+even if it was not explicitly added to the ``aliases`` mapping.  Otherwise, the
+result is automatically converted to a string. For example,
 
 .. code-block:: xonshcon
 
@@ -476,6 +478,8 @@ automatically converted to a string. For example,
     4
     >>> echo @([42, 'yo'])
     42 yo
+    >>> echo "hello" | @(lambda a, s=None: s.strip + " world")
+    hello world
 
 This syntax can be used inside of a captured or uncaptured subprocess, and can
 be used to generate any of the tokens in the subprocess command list.
@@ -988,9 +992,12 @@ If you were to run ``gco feature-fabulous`` with the above aliases in effect,
 the command would reduce to ``['git', 'checkout', 'feature-fabulous']`` before
 being executed.
 
+
+Callable Aliases
+----------------
 Lastly, if an alias value is a function (or other callable), then this
 function is called *instead* of going to a subprocess command. Such functions
-must have the following signature:
+must have one of the following two signatures
 
 .. code-block:: python
 
@@ -1029,6 +1036,28 @@ must have the following signature:
         # examples the return code would be 0/success.
         return (None, "I failed", 2)
 
+
+.. code-block:: python
+
+    def _mycmd2(args, stdin, stdout, stderr):
+        """args will be a list of strings representing the arguments to this
+        command.  stdin is a read-only file-like object, and stdout and stderr
+        are write-only file-like objects
+        """
+        # This form allows "streaming" data to stdout and stderr
+        import time
+        for i in range(5):
+            time.sleep(i)
+            print(i, file=stdout)
+
+        # In this form, the return value should be a single integer
+        # representing the "return code" of the alias (zero if successful,
+        # non-zero otherwise)
+        return 0
+
+
+Adding and Removing Aliases
+---------------------------
 We can dynamically alter the aliases present simply by modifying the
 built-in mapping.  Here is an example using a function value:
 
@@ -1046,11 +1075,27 @@ built-in mapping.  Here is an example using a function value:
    Otherwise, they may shadow the alias itself, as Python variables take
    precedence over aliases when xonsh executes commands.
 
-Usually, callable alias commands will be run in a separate thread so that
-users may background them interactively. However, some aliases may need to be
-executed on the thread that they were called from. This is mostly useful for debuggers
-and profilers. To make an alias run in the foreground, decorate its function
-with the ``xonsh.proc.foreground`` decorator.
+
+Anonymous Aliases
+-----------------
+As mentioned above, it is also possible to treat functions outside this mapping
+as aliases, by wrapping them in ``@()``.  For example:
+
+.. code-block:: xonshcon
+
+    >>> @(_banana)
+    'My spoon is tooo big!'
+    >>> echo "hello" | @(lambda args, stdin=None: stdin.strip() + args[0]) world
+    hello world
+
+
+Foreground-only Aliases
+-----------------------
+Usually, callable alias commands will be run in a separate thread so that users
+they may be run in the background.  However, some aliases may need to be
+executed on the thread that they were called from. This is mostly useful for
+debuggers and profilers. To make an alias run in the foreground, decorate its
+function with the ``xonsh.proc.foreground`` decorator.
 
 .. code-block:: python
 

news/adqm-anon_aliases.rst

@@ -0,0 +1,15 @@
+**Added:** None
+
+**Changed:**
+
+* ``@()`` now passes through functions as well as strings, which allows for the
+  use of anonymous aliases and aliases not explicitly added to the ``aliases``
+  mapping.
+
+**Deprecated:** None
+
+**Removed:** None
+
+**Fixed:** None
+
+**Security:** None

news/clean-up-premain.rst

@@ -0,0 +1,14 @@
+**Added:** None
+
+**Changed:**
+
+* Cleaned up argument parsing in ``xonsh.main.premain`` by removing the
+  ``undo_args`` hack.
+
+**Deprecated:** None
+
+**Removed:** None
+
+**Fixed:** None
+
+**Security:** None

tests/test_builtins.py

@@ -10,7 +10,7 @@ from nose.tools import assert_equal, assert_true, assert_not_in
 
 from xonsh import built_ins
 from xonsh.built_ins import reglob, regexpath, helper, superhelper, \
-    ensure_list_of_strs
+    ensure_list_of_strs, list_of_strs_or_callables
 from xonsh.environ import Env
 from xonsh.tools import ON_WINDOWS
 
@@ -105,5 +105,13 @@ def test_ensure_list_of_strs():
         obs = ensure_list_of_strs(inp)
         yield assert_equal, exp, obs
 
+def test_list_of_strs_or_callables():
+    f = lambda x: 20
+    cases = [(['yo'], 'yo'), (['yo'], ['yo']), (['42'], 42), (['42'], [42]),
+             ([f], f), ([f], [f])]
+    for exp, inp in cases:
+        obs = list_of_strs_or_callables(inp)
+        yield assert_equal, exp, obs
+
 if __name__ == '__main__':
     nose.runmodule()

tests/test_execer.py

@@ -43,6 +43,17 @@ def test_simple_func():
             "    return '{user}'.format(user='me')\n")
     yield check_parse, code
 
+def test_lookup_alias():
+    code = (
+        'def foo(a,  s=None):\n'
+        '    return "bar"\n'
+        '@(foo)\n')
+    yield check_parse, code
+
+def test_lookup_anon_alias():
+    code = ('echo "hi" | @(lambda a, s=None: a[0]) foo bar baz\n')
+    yield check_parse, code
+
 def test_simple_func_broken():
     code = ('def prompt():\n'
             "    return '{user}'.format(\n"

tests/test_imphooks.py

@@ -25,23 +25,23 @@ def teardown():
         unload_builtins()
 
 def test_import():
-    with mock_xonsh_env({}):
+    with mock_xonsh_env({'PATH': []}):
         import sample
         assert_equal('hello mom jawaka\n', sample.x)
 
 def test_absolute_import():
-    with mock_xonsh_env({}):
+    with mock_xonsh_env({'PATH': []}):
         from xpack import sample
         assert_equal('hello mom jawaka\n', sample.x)
 
 def test_relative_import():
-    with mock_xonsh_env({}):
+    with mock_xonsh_env({'PATH': []}):
         from xpack import relimp
         assert_equal('hello mom jawaka\n', relimp.sample.x)
         assert_equal('hello mom jawaka\ndark chest of wonders', relimp.y)
 
 def test_sub_import():
-    with mock_xonsh_env({}):
+    with mock_xonsh_env({'PATH': []}):
         from xpack.sub import sample
         assert_equal('hello mom jawaka\n', sample.x)
 

tests/test_parser.py

@@ -1553,6 +1553,12 @@ def test_dollar_sub_space():
 def test_ls_dot():
     yield check_xonsh_ast, {}, '$(ls .)', False
 
+def test_lambda_in_atparens():
+    yield check_xonsh_ast, {}, '$(echo hello | @(lambda a, s=None: "hey!") foo bar baz)', False
+
+def test_nested_madness():
+    yield check_xonsh_ast, {}, '$(@$(which echo) ls | @(lambda a, s=None: $(@(s.strip()) @(a[1]))) foo -la baz)', False
+
 def test_ls_dot_nesting():
     yield check_xonsh_ast, {}, '$(ls @(None or "."))', False
 

xonsh/built_ins.py

@@ -401,13 +401,19 @@ def run_subproc(cmds, captured=False):
         elif builtins.__xonsh_stderr_uncaptured__ is not None:
             stderr = builtins.__xonsh_stderr_uncaptured__
         uninew = (ix == last_cmd) and (not _capture_streams)
-        alias = builtins.aliases.get(cmd[0], None)
+
+        if callable(cmd[0]):
+            alias = cmd[0]
+        else:
+            alias = builtins.aliases.get(cmd[0], None)
+            binary_loc = locate_binary(cmd[0])
+
         procinfo['alias'] = alias
         if (alias is None and
                 builtins.__xonsh_env__.get('AUTO_CD') and
                 len(cmd) == 1 and
                 os.path.isdir(cmd[0]) and
-                locate_binary(cmd[0]) is None):
+                binary_loc is None):
             cmd.insert(0, 'cd')
             alias = builtins.aliases.get('cd', None)
 
@@ -416,12 +422,12 @@ def run_subproc(cmds, captured=False):
         else:
             if alias is not None:
                 cmd = alias + cmd[1:]
-            n = locate_binary(cmd[0])
-            if n is None:
+            if binary_loc is None:
                 aliased_cmd = cmd
             else:
                 try:
-                    aliased_cmd = get_script_subproc_command(n, cmd[1:])
+                    aliased_cmd = get_script_subproc_command(binary_loc,
+                                                             cmd[1:])
                 except PermissionError:
                     e = 'xonsh: subprocess mode: permission denied: {0}'
                     raise XonshError(e.format(cmd[0]))
@@ -482,11 +488,6 @@ def run_subproc(cmds, captured=False):
                 raise XonshError(e)
         procs.append(proc)
         prev_proc = proc
-    for proc in procs[:-1]:
-        try:
-            proc.stdout.close()
-        except OSError:
-            pass
     if not prev_is_proxy:
         add_job({
             'cmds': cmds,
@@ -507,6 +508,11 @@ def run_subproc(cmds, captured=False):
     if prev_is_proxy:
         prev_proc.wait()
     wait_for_active_job()
+    for proc in procs[:-1]:
+        try:
+            proc.stdout.close()
+        except OSError:
+            pass
     hist = builtins.__xonsh_history__
     hist.last_cmd_rtn = prev_proc.returncode
     # get output
@@ -626,6 +632,17 @@ def ensure_list_of_strs(x):
     return rtn
 
 
+def list_of_strs_or_callables(x):
+    """Ensures that x is a list of strings or functions"""
+    if isinstance(x, str) or callable(x):
+        rtn = [x]
+    elif isinstance(x, Sequence):
+        rtn = [i if isinstance(i, str) or callable(i) else str(i) for i in x]
+    else:
+        rtn = [str(x)]
+    return rtn
+
+
 def load_builtins(execer=None, config=None, login=False, ctx=None):
     """Loads the xonsh builtins into the Python builtins. Sets the
     BUILTINS_LOADED variable to True.
@@ -657,6 +674,7 @@ def load_builtins(execer=None, config=None, login=False, ctx=None):
     builtins.__xonsh_commands_cache__ = CommandsCache()
     builtins.__xonsh_all_jobs__ = {}
     builtins.__xonsh_ensure_list_of_strs__ = ensure_list_of_strs
+    builtins.__xonsh_list_of_strs_or_callables__ = list_of_strs_or_callables
     # public built-ins
     builtins.XonshError = XonshError
     builtins.XonshBlockError = XonshBlockError
@@ -726,6 +744,7 @@ def unload_builtins():
              'default_aliases',
              '__xonsh_all_jobs__',
              '__xonsh_ensure_list_of_strs__',
+             '__xonsh_list_of_strs_or_callables__',
              '__xonsh_history__',
              ]
     for name in names:

xonsh/main.py

@@ -113,40 +113,6 @@ parser.add_argument('args',
                     default=[])
 
 
-def arg_undoers():
-    au = {
-        '-h': (lambda args: setattr(args, 'help', False)),
-        '-V': (lambda args: setattr(args, 'version', False)),
-        '-c': (lambda args: setattr(args, 'command', None)),
-        '-i': (lambda args: setattr(args, 'force_interactive', False)),
-        '-l': (lambda args: setattr(args, 'login', False)),
-        '--no-script-cache': (lambda args: setattr(args, 'scriptcache', True)),
-        '--cache-everything': (lambda args: setattr(args, 'cacheall', False)),
-        '--config-path': (lambda args: delattr(args, 'config_path')),
-        '--no-rc': (lambda args: setattr(args, 'norc', False)),
-        '-D': (lambda args: setattr(args, 'defines', None)),
-        '--shell-type': (lambda args: setattr(args, 'shell_type', None)),
-        }
-    au['--help'] = au['-h']
-    au['--version'] = au['-V']
-    au['--interactive'] = au['-i']
-    au['--login'] = au['-l']
-
-    return au
-
-
-def undo_args(args):
-    """Undoes missaligned args."""
-    au = arg_undoers()
-    for a in args.args:
-        if a in au:
-            au[a](args)
-        else:
-            for k in au:
-                if a.startswith(k):
-                    au[k](args)
-
-
 def _pprint_displayhook(value):
     if value is None:
         return
@@ -181,10 +147,18 @@ def premain(argv=None):
     builtins.__xonsh_ctx__ = {}
     args, other = parser.parse_known_args(argv)
     if args.file is not None:
-        real_argv = (argv or sys.argv)
-        i = real_argv.index(args.file)
-        args.args = real_argv[i+1:]
-        undo_args(args)
+        arguments = (argv or sys.argv)
+        file_index = arguments.index(args.file)
+        # A script-file was passed and is to be executed. The argument parser
+        # might have parsed switches intended for the script, so reset the
+        # parsed switches to their default values
+        old_args = args
+        args = parser.parse_known_args('')[0]
+        args.file = old_args.file
+        # Save the arguments that are intended for the script-file. Switches
+        # and positional arguments passed before the path to the script-file are
+        # ignored.
+        args.args = arguments[file_index+1:]
     if args.help:
         parser.print_help()
         exit()

xonsh/parsers/base.py

@@ -2182,7 +2182,7 @@ class BaseParser(object):
 
     def p_subproc_atom_pyeval(self, p):
         """subproc_atom : AT_LPAREN test RPAREN"""
-        p0 = xonsh_call('__xonsh_ensure_list_of_strs__', [p[2]],
+        p0 = xonsh_call('__xonsh_list_of_strs_or_callables__', [p[2]],
                         lineno=self.lineno, col=self.col)
         p0._cliarg_action = 'extend'
         p[0] = p0