open file descriptors of the shell aren't inherited by processes launched by the shell, unlike bash or zsh

[adavies42]

compare ksh:

$ echo $KSH_VERSION                                                                                                                                                                                                                                                                                                                                                        
Version AJM 93u+m/1.0.8 2024-01-01
$ exec {fd}<.profile
$ echo $fd
11
$ ksh
$ lsof -ap$$ -d11                                                                                                                                                                                                                                                                                                                                                          
$ 

and bash:

$ echo $BASH_VERSION
5.2.26(1)-release
$ exec {fd}<.profile
$ echo $fd
10
$ bash
$ lsof -ap$$ -d10
COMMAND   PID    USER   FD   TYPE DEVICE SIZE/OFF        NODE NAME
bash    42373 adavies   10r   REG    1,4        0 12988192685 /Users/adavies/.profile
$ 

zsh behaves like bash

this breaks patterns like

$ exec {fd}<.profile                                                                                                                                                                                                                                                                                                                                                       
$ flock $fd
flock: data error: Bad file descriptor
$ 

since flock(1) doesn't see that file descriptor as open

strace shows ksh launches processes with vfork(2), while bash uses clone(2); maybe this is the cause of the difference?

[McDutchie]

This is historic ksh93 behaviour. File descriptors > 2 opened with exec or redirect are opened with the close-on-exec flag. This is documented in the manual page under Built-in Commands → redirect. We cannot change this without breaking backward compatibility.

On ksh 93u+m, the easiest way to work around this is to enable the posix shell option, which disables this behaviour and makes ksh act like other shells in this and a number of other ways. See the manual page under Built-in Commands → set → -o → posix for other things the posix option changes. Those are all historic ksh93 behaviours that are not compliant with the POSIX standard.

Another way to work around this (if you don't want to enable posix mode) is to explicitly redirect the file descriptor to itself as part of the external command invocation, which overrides the close-on-exec flag. But that only works with constant file descriptor numbers < 10, e.g.:

exec 4<.profile
flock 4 4<&4

Perhaps we need a separate shell option to control this behaviour…

[adavies42]

I assume the extent of posix mode could be narrowly scoped, i.e.

set -o posix
exec {fd}<.profile
set +o posix
flock $fd

What exactly does "that only works with constant file descriptor numbers < 10" mean? It doesn't work with the {fd}< syntax? Is that something that can be changed?

[ormaaj]

You can also simply use : {fd}<file to avoid the nasty exec side-effect. Eventually I would like to add a builtin similar to bash's fdflags or zsh's sysopen so that basic fcntl operations are controllable, such as CLOEXEC.

Also most shells with a set -o posix use it for overriding of "undesirable" POSIX behavior, rather than for repairing their own legacy quirks. Combining those objectives into one option seems like a rather blunt instrument, when you'd rather not be posix'd harder.

[adavies42]

In my testing, file descriptors opened with : aren't inherited by child processes, whether or not set -o posix is in effect. Is this expected/intended?

I'm not sure how this is happening, BTW--I don't see CLOEXEC being applied to the fd in the : case.

[ormaaj]

Actually it's just broken. It opens the file and closes it immediately for some reason. Weirdly opens on one FD, DUP's it somewhere else, and closes both.

(cmd)ormaaj 114 (675660) 0 ~ $ strace -DDYYyqqfb execve -e t=%desc -P /dev/null -- ksh -xo posix -c 'if [[ ! -v .sh.pid ]]; then builtin pids; function .sh.pid.get { .sh.value=$(pids -f "%(pid)d") ; }; fi; : {fd}</dev/null; lsfd -p "${.sh.pid}" -Q "FD == ${fd}" -o +flags'
newfstatat(AT_FDCWD</home/ormaaj>, "/dev/null", {st_mode=S_IFCHR|0666, st_rdev=makedev(0x1, 0x3), ...}, 0) = 0
newfstatat(AT_FDCWD</home/ormaaj>, "/dev/null", {st_mode=S_IFCHR|0666, st_rdev=makedev(0x1, 0x3), ...}, 0) = 0
+ [[ ! -v .sh.pid ]]
+ :
openat(AT_FDCWD</home/ormaaj>, "/dev/null", O_RDONLY) = 3</dev/null>
+ {fd}< /dev/null
fcntl(3</dev/null>, F_DUPFD, 10)        = 10</dev/null>
close(3</dev/null>)                     = 0
ioctl(10</dev/null>, TCGETS, 0x7ffca4363f80) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(10</dev/null>, 0, SEEK_CUR)       = 0
fstat(10</dev/null>, {st_mode=S_IFCHR|0666, st_rdev=makedev(0x1, 0x3), ...}) = 0
close(10</dev/null>)                    = 0
+ lsfd -p 3203860 -Q 'FD == 10' -o +flags

{fd} redirects have lots of bugs besides. Some fail to parse, like {fd}<<<foo

ksh -xc 'exec {fd}<<<foo'
+ exec }<<< foo

{fd}<#(()) is broken or unimplemented - you actually have to DUP to FD 0 in order to lseek which usually means swapping fds twice to save stdin. Same with many other redirect types.