Abridged closures

[memreflect]

Printing cyclical closures, once the code is re-enabled to do so, causes a lot of output, especially when nesting occurs.
To use an example similar to that in #4:

; whatis <={
    @ x a y {
      (a x y) = 0 @{a = $*} {echo $a}
      $&result $x $y
    }
  }
%closure(x='0 $&nestedbinding';a=0;y='%closure(x=''1 $&nestedbinding'';a=0;y=''0 $&nestedbinding'')@ * {echo $x}')@ * {a=$*}
%closure(x='%closure(x=''0 $&nestedbinding'';a=0;y=''1 $&nestedbinding'')@ * {echo $y}';a=0;y='0 $&nestedbinding'){echo $a}

Can you spot the value bound to a in the second line?
To avoid that noise, i have created memreflect/es-shell@abridge-closures that instead produces much cleaner output by default:

%closure(x;a;y)@ * {a=$*}
%closure(x;a;y){echo $a}

My rationale is that there is no way to modify any of the values in the closure binding without using another closure created using the same bindings, meaning there is no reason to know the values.
At the same time, it still helps to know that it's a closure, instead of seeing only @ * {a=$*}, which leads you to believe it sets a dynamic variable a — that's what the Elvish shell does, and while i have accepted that's just how Elvish works, es isn't Elvish.

Do you think this should be added to es?

[jpco]

For some reason it's tricky for me to run your branch without running into segfaults?

My actual objection this proposal, though, is that it breaks the rule that es state needs to be externalizable.

Without abridged closures:

; echo <={let (a = value) {result {echo $a}}}
%closure(a=value){echo $a}
; <={let (a = value) {result {echo $a}}}
value
; ./es -c <={let (a = value) {result {echo $a}}}
value

With abridged closures:

; echo <={let (a = value) {result {echo $a}}}
%closure(a){echo $a}
; <={let (a = value) {result {echo $a}}}
value
; ./es -c <={let (a = value) {result {echo $a}}}
segmentation violation--core dumped

I'm not totally sure what's causing the segfault per se, but even if that were fixed, this last command could never succeed with an abridged closure, because the value of a has been stripped away when stringifying the closure. Given that's a big part of how es gets values from the environment at startup, it seems like a deal-breaker to me.

[memreflect]

I see what you mean about the segfaults after compiling with -O2.
They were a result of using the address of an out of scope variable (dangling pointer), leading to undefined behavior.
I have pushed the fix to my branch, so give things a try.

Your perception of abridged closures isn't quite correct.
It is strictly a matter of human-friendly output.

Internally, they are still closures, whether imported into the environment as strings from a parent shell or as Closure objects created in the current shell.
As a result, your code should work.

sh$ ./es
; ./es -x -c <={let (a = value) {result {echo $a}}}
{%closure(a=value){echo $a}}
value
; whatis <={let (a = value) {result {echo $a}}}
%closure(a){echo $a}
;
; fn-f = <={let (a = value) {result {echo $a}}}
;
; echo $fn-f
%closure(a){echo $a}
; /bin/echo $fn-f
%closure(a=value){echo $a}
; var fn-f
fn-f = '%closure(a=value){echo $a}'

I hope you see the difference now.
The only downside is that you can't use echo to print such an object anymore, which means doing something like this is dangerous:

; g = `{echo $fn-f}
; var g
g = '%closure(a){echo' '$a}'

I'd argue that if you need the full value, you shouldn't be echo'ing it in the first place, but i can understand if that's problematic.
The other option would be to skip the closure bindings entirely and produce only the lambda, block, etc. that gets executed.
That way, it would be executable code at least.

I'm not pushing too hard for this to be included honestly.
I just found the idea useful and thought to share it.

[jpco]

Oh, I see. I made some unfair assumptions about what the segfaults were hiding :). So, if I'm understanding correctly now, this ends up being more of a magic-echo feature than a fundamental change to how closures are formatted. I think that's a lot more palatable.

[memreflect]

No worries, i'm skeptical when newly proposed code causes segfaults too. :)
You're exactly right about the magic-echo feature.
If it actually modified closures so they suddenly didn't work, then it would definitely be more of a bug than a feature.

[jpco]

I'm revisiting this idea now after a while. Previously, my impression of this idea was that it was punching a hole in the nice echo function for little good reason -- in particular, making it diverge from the "externalizable state" behavior of (for example) var.

However, in #229, I got a better sense of where echo already produces output that pretty definitively doesn't work when read back in -- we already have divergence between echo and this "externalization" behavior.

More recently, in #252, I've been dreaming about adding variable-bound file handles, which would allow files to be opened and their handles bound to a variable, allowing nice variable things like lexical scope, rather than requiring users to rely on the (pretty bad) whole-process fd namespace. The major challenge with this idea is externalization -- how would we put a file handle in a %closure such that a child process can read it and end up with an identical handle? It would sidestep this challenge if we could just say that file handles can't be properly externalized, and throw an exception when an attempt is made to do so. However, because there is no promise that echo's output is valid when re-read by the shell, it can simply print something informative for the user (and/or useful for interacting with external programs).

More generally, this could also be useful for an extensible-primitives system that I've been mulling over. Plenty of potential use cases for extensible primitives can be imagined which involve some kind of hard-to-externalize object, or handle to a resource. It could be genuinely useful to allow people to punt on the externalization problem when they're creating a new sort of handle. This could even be used in a possible sci-fi future where continuations are exposed to users. These aren't immediately practical considerations, but I'm trying to think three steps ahead before making design choices.

This is all to say that my mind has opened to abridged closures -- particularly as a special case of a more general decoupling of echo, oriented for other people or other programs, and "externalization", oriented for re-reading back in another shell process, either via the environment, a file, or whatever. This would somewhat parallel Scheme's distinction between display and write, though with some distinctions due to es' different operating environment.

This is largely just daydreaming, but I think it's interesting, and it does enable a number of things that would otherwise be very difficult to make happen.