Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- Stream merge redirects `2>&1` (stderr to stdout's destination) and `1>&2`
(stdout to stderr's destination). Each is a single token with no internal
spaces, and works on command lists, pipeline stages, and quotations.
Unlike POSIX, they are not order-sensitive fd duplication: the merged
stream follows the other stream's *final* destination, so `2>&1 *` captures
both streams interleaved and `[[make] 2>&1 [grep err]] |;` sends stderr
through the pipe, cross-platform.

### Changed

- Each of stdout/stderr now has exactly one destination: combining two
destinations on the same stream (e.g. capture `*` plus file redirect `>`,
`&>` plus `2>`, a second `>`, or a merge plus anything else on that stream)
is now an error, caught both by the static type checker and at runtime.
Previously the extra redirect was silently ignored (capture won over file
redirects) or last-wins. `2>&1` combined with `<>` is also rejected.
- `loop` quotations now honor stderr redirects, append mode, and merges, and
inherit the enclosing quotation's redirected streams (previously a loop
body wrote straight to the terminal even inside a redirected quotation).
- Quotations accept only the redirects that don't change the stack: file
redirects, stdin, and the merges. Captures (`*`, `*b`, `^`, `^b`) on a
quotation now give a clear error at both layers instead of falling into
the multiplication error, and the type checker now rejects `<>` on a
quotation (the runtime always did). Capture individual command lists
instead, e.g. `[[cmd1] [cmd2]] (* !) map`.

### Added

- CLI completions for `cargo`: subcommands (including installed third-party
ones via `cargo --list`), per-subcommand options, and dynamic values for
`--target`, `--features`, `-p`/`--package`, `--bin`/`--example`/`--test`/`--bench`
Expand Down
4 changes: 4 additions & 0 deletions code/syntaxes/mshell.textmate.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@
"name": "constant.language.bool",
"match": "\\b(true|false)\\b"
},
{
"name": "keyword.operator.redirect.merge.mshell",
"match": "2>&1|1>&2"
},
{
"name": "string.quoted.single.mshell",
"begin": "'",
Expand Down
59 changes: 58 additions & 1 deletion doc/execution.inc.html
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,36 @@
</code>
</pre>

<h2 id="stream-merging">Merging one stream into the other</h2>

<p>
Use <code>2>&amp;1</code> to send standard error to wherever standard output ends up,
or <code>1>&amp;2</code> to send standard output to wherever standard error ends up.
These are complete tokens: no spaces are allowed inside them.
Unlike POSIX shells, they are <em>not</em> order-sensitive file descriptor duplication:
a merge means "this stream goes to the other stream's <em>final</em> destination",
so there is no <code>> file 2>&amp;1</code> vs <code>2>&amp;1 > file</code> ordering trap.
Both streams share a single destination, preserving output order, and this works the same on every platform.
</p>

<pre>
<code><span class="mshellLEFT_SQUARE_BRACKET">[</span><span class="mshellLITERAL">yourCommand</span><span class="mshellRIGHT_SQUARE_BRACKET">]</span> <span class="mshellSTDERRTOSTDOUT">2&gt;&amp;1</span> <span class="mshellASTERISK">*</span> <span class="mshellBANG">!</span> <span class="mshellLINECOMMENT"># Captures stdout and stderr interleaved as one string.</span>
<span class="mshellLEFT_SQUARE_BRACKET">[</span><span class="mshellLEFT_SQUARE_BRACKET">[</span><span class="mshellLITERAL">make</span><span class="mshellRIGHT_SQUARE_BRACKET">]</span> <span class="mshellSTDERRTOSTDOUT">2&gt;&amp;1</span> <span class="mshellLEFT_SQUARE_BRACKET">[</span><span class="mshellLITERAL">grep</span> <span class="mshellLITERAL">-i</span> <span class="mshellLITERAL">error</span><span class="mshellRIGHT_SQUARE_BRACKET">]</span><span class="mshellRIGHT_SQUARE_BRACKET">]</span> <span class="mshellPIPE">|</span><span class="mshellEXECUTE">;</span> <span class="mshellLINECOMMENT"># stderr flows through the pipe.</span>
<span class="mshellLEFT_SQUARE_BRACKET">[</span><span class="mshellLITERAL">yourCommand</span><span class="mshellRIGHT_SQUARE_BRACKET">]</span> <span class="mshellSTDOUTTOSTDERR">1&gt;&amp;2</span> <span class="mshellEXECUTE">;</span> <span class="mshellLINECOMMENT"># stdout appears on stderr, e.g. visible even when the surrounding output is captured.</span>
<span class="mshellLEFT_SQUARE_BRACKET">[</span><span class="mshellLITERAL">yourCommand</span><span class="mshellRIGHT_SQUARE_BRACKET">]</span> <span class="mshellPATH">`output.log`</span> <span class="mshellGREATERTHAN">&gt;</span> <span class="mshellSTDERRTOSTDOUT">2&gt;&amp;1</span> <span class="mshellBANG">!</span> <span class="mshellLINECOMMENT"># Both streams into output.log; equivalent to &amp;&gt;.</span>
</code>
</pre>

<h2 id="one-destination-per-stream">Each stream has exactly one destination</h2>

<p>
A stream's destination can be set once: a file redirect, a capture, an in-place redirect, or a merge.
Applying a second destination to the same stream is an error, caught both by the static type checker and at runtime.
For example, <code>[cmd] * `f` ></code> (capture and file redirect on stdout), <code>[cmd] 2>&amp;1 ^</code> (merge and capture on stderr),
and <code>[cmd] 2>&amp;1 1>&amp;2</code> (circular merge) are all rejected.
<code>2>&amp;1</code> also cannot be combined with <code>&lt;&gt;</code>, since stderr text would be written back into the edited file.
</p>

<table>
<caption>Summary of external command operators</caption>
<thead>
Expand All @@ -161,6 +191,8 @@
<tr> <td><code>&amp;>></code></td> <td>Redirect both stdout and stderr to a file.</td> <td>Appends to the file.</td> </tr>
<tr> <td><code>^</code></td> <td>Capture stderr to the stack.</td> <td>As a string.</td> </tr>
<tr> <td><code>^b</code></td> <td>Capture stderr to the stack.</td> <td>As binary.</td> </tr>
<tr> <td><code>2>&amp;1</code></td> <td>Merge stderr into stdout's destination.</td> <td>Single token, no spaces. Not order-sensitive: stderr follows stdout's final destination.</td> </tr>
<tr> <td><code>1>&amp;2</code></td> <td>Merge stdout into stderr's destination.</td> <td>Single token, no spaces.</td> </tr>
<tr> <td><code>&lt;</code></td> <td>Feed stdin from a value.</td> <td>String, path, or binary.</td> </tr>
<tr> <td><code>&lt;&gt;</code></td> <td>In-place file modification.</td> <td>Reads file to stdin, writes stdout back on success.</td> </tr>
<tr> <td><code>&amp;</code></td> <td>Mark the command list to run asynchronously.</td> <td>Marks the list; the trailing <code>;</code>/<code>!</code> starts the subprocess and returns immediately without waiting. Stdout and stderr default to discarded.</td> </tr>
Expand All @@ -170,11 +202,36 @@
<h2 id="quotation-redirection">Redirection on quotations</h2>

<p>
All of the redirection operators above also work on quotations.
The redirection operators that don't change the stack also work on quotations:
file redirects (<code>></code>, <code>>></code>, <code>2></code>, <code>2>></code>, <code>&amp;></code>, <code>&amp;>></code>),
stdin (<code>&lt;</code>), and the merges (<code>2>&amp;1</code>, <code>1>&amp;2</code>).
This is useful when you want to redirect the output of mshell code that uses <code>wl</code>, <code>wle</code>, or other built-in functions that write to stdout or stderr.
It is also useful when you have many commands that you want to run while appending all the outputs to a single file, without having to put the redirection on each command invocation.
</p>

<p>
The captures (<code>*</code>, <code>*b</code>, <code>^</code>, <code>^b</code>) and the in-place redirect (<code>&lt;&gt;</code>) are <em>not</em> allowed on quotations,
because they would change the quotation's stack effect.
Capture the individual command lists inside the quotation instead — for example, run a list of commands and collect each stdout:
</p>

<pre>
<code><span class="mshellLEFT_SQUARE_BRACKET">[</span><span class="mshellLEFT_SQUARE_BRACKET">[</span><span class="mshellLITERAL">cmd1</span><span class="mshellRIGHT_SQUARE_BRACKET">]</span> <span class="mshellLEFT_SQUARE_BRACKET">[</span><span class="mshellLITERAL">cmd2</span><span class="mshellRIGHT_SQUARE_BRACKET">]</span><span class="mshellRIGHT_SQUARE_BRACKET">]</span> <span class="mshellLEFT_PAREN">(</span><span class="mshellASTERISK">*</span> <span class="mshellBANG">!</span><span class="mshellRIGHT_PAREN">)</span> <span class="mshellLITERAL">map</span> <span class="mshellLINECOMMENT"># Run each command, collect stdouts as a list of strings.</span>
</code>
</pre>

<p>
Destination conflicts on quotations (e.g. two <code>></code> redirects, or <code>2>&amp;1</code> plus <code>2></code>) are caught at runtime.
Since redirects never change a quotation's stack effect, they are invisible to the static type checker.
</p>

<p>
A quotation's redirect is resolved each time the quotation executes.
So a <code>></code>-redirected quotation run repeatedly — stored and executed with <code>x</code> several times, or passed to <code>each</code> or <code>map</code> — truncates the file on every execution, leaving only the last run's output.
Use <code>>></code> when a repeatedly-executed quotation should accumulate output.
The exception is <code>loop</code>: the file is opened once when the loop starts, so all iterations write to the same open file.
</p>

<pre>
<code><span class="mshellLEFT_PAREN">(</span>
<span class="mshellSTRING">"Hello from stdout"</span> <span class="mshellLITERAL">wl</span>
Expand Down
43 changes: 42 additions & 1 deletion doc/mshell.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,30 @@ So the following are equivalent:
[yourCommand] ^b *b !
```

#### Merging one stream into the other

Use `2>&1` to send stderr to wherever stdout ends up, or `1>&2` to send stdout to wherever stderr ends up.
These are complete tokens: no spaces are allowed inside them.
Unlike POSIX shells, they are not order-sensitive fd duplication:
a merge means "this stream goes to the other stream's *final* destination",
so there is no `> file 2>&1` vs `2>&1 > file` ordering trap.
Both streams share a single destination, preserving output order, on every platform.

```mshell
[yourCommand] 2>&1 * ! # Captures stdout and stderr interleaved as one string.
[[make] 2>&1 [grep -i error]] |; # stderr flows through the pipe.
[yourCommand] 1>&2 ; # stdout appears on stderr.
[yourCommand] `output.log` > 2>&1 ! # Both streams into output.log; equivalent to &>.
```

#### Each stream has exactly one destination

A stream's destination can be set once: a file redirect, a capture, an in-place redirect, or a merge.
Applying a second destination to the same stream is an error, caught both by the static type checker and at runtime.
For example, ``[cmd] * `f` >`` (capture and file redirect on stdout), `[cmd] 2>&1 ^` (merge and capture on stderr),
and `[cmd] 2>&1 1>&2` (circular merge) are all rejected.
`2>&1` also cannot be combined with `<>`, since stderr text would be written back into the edited file.

Summary of external command operators:

Operator | Effect on external commands | Notes
Expand All @@ -123,13 +147,30 @@ Operator | Effect on external commands | Notes
`&>>` | Redirect both stdout and stderr to a file. | Appends to the file.
`^` | Capture stderr to the stack. | As a string.
`^b` | Capture stderr to the stack. | As binary.
`2>&1` | Merge stderr into stdout's destination. | Single token, no spaces. Not order-sensitive: stderr follows stdout's final destination.
`1>&2` | Merge stdout into stderr's destination. | Single token, no spaces.
`<` | Feed stdin from a value. | String, path, or binary.
`<>` | In-place file modification. | Reads file to stdin, writes stdout back on success.
`&` | Mark the command list to run asynchronously. | Marks the list; the trailing `;`/`!` starts the subprocess and returns immediately without waiting. Stdout and stderr default to discarded.

### Redirection on quotations

All of the redirection operators above also work on quotations. This is useful when you want to redirect the output of mshell code that uses `wl`, `wle`, or other built-in functions that write to stdout or stderr. It is also useful when you have many commands that you want to run while appending all the outputs to a single file, without having to put the redirection on each command invocation.
The redirection operators that don't change the stack also work on quotations:
file redirects (`>`, `>>`, `2>`, `2>>`, `&>`, `&>>`), stdin (`<`), and the merges (`2>&1`, `1>&2`).
This is useful when you want to redirect the output of mshell code that uses `wl`, `wle`, or other built-in functions that write to stdout or stderr.
It is also useful when you have many commands that you want to run while appending all the outputs to a single file, without having to put the redirection on each command invocation.

The captures (`*`, `*b`, `^`, `^b`) and the in-place redirect (`<>`) are *not* allowed on quotations,
because they would change the quotation's stack effect.
Capture the individual command lists inside the quotation instead, e.g. `[[cmd1] [cmd2]] (* !) map` to run each command and collect stdouts.

Destination conflicts on quotations (e.g. two `>` redirects, or `2>&1` plus `2>`) are caught at runtime.
Since redirects never change a quotation's stack effect, they are invisible to the static type checker.

A quotation's redirect is resolved each time the quotation executes.
So a `>`-redirected quotation run repeatedly — stored and executed with `x` several times, or passed to `each` or `map` — truncates the file on every execution, leaving only the last run's output.
Use `>>` when a repeatedly-executed quotation should accumulate output.
The exception is `loop`: the file is opened once when the loop starts, so all iterations write to the same open file.

```mshell
(
Expand Down
Loading