Use a lock owned by the open file descriptor on Android

[nekohasekai]

Fixes the file lock on Android. The lock must exclude a second Open of the
same database. At present it does not.

What is wrong today

PR #571 moved Android out of bolt_unix.go and into bolt_android.go, to work
around flock(2) failures on Android. The new file locks with POSIX record
locks (fcntl(F_SETLK)), copied from bolt_solaris.go. A POSIX record lock
belongs to the process, not to the open file descriptor. The lock therefore
loses the two properties that it exists to provide.

Measured on an Android 14 emulator (API 34, kernel 6.1.23), inside a real app
process:

[2] fcntl(F_SETLK) same process, two fds  <- current bolt_android.go
  F_SETLK F_WRLCK on fd_a                        => OK (lock acquired)
  F_SETLK F_WRLCK on fd_b                        => OK (lock acquired)
  F_SETLK F_RDLCK on fd_c (downgrade?)           => OK (lock acquired)
  CHILD PROCESS setlk                            => resource temporarily unavailable
  (closed fd_b -- POSIX says this drops ALL of this process's locks on the file)
  CHILD PROCESS setlk                            => OK (lock acquired)

[5] bbolt.Open same path twice in one process
  first  bolt.Open => err=<nil>
  second bolt.Open => err=<nil>  (want: timeout)

1. A second Open in the same process succeeds. Two DB handles then write
   meta pages and freelist pages to one file. This is the corruption that the
   lock must prevent.
2. A close of any descriptor to the file releases every lock that the process
   holds on that file. Another process can then take the write lock. Exclusion
   between processes is the property that Open documents, and it is lost too.

Why flock(2) failed

Android does not remove or restrict flock. flock fails only on the FUSE
mount that provides emulated external storage. In pf_init(), the MediaProvider
FUSE daemon adds FUSE_CAP_FLOCK_LOCKS to conn->want. But pf_flock and the
.flock entry in the ops table are commented out. libfuse therefore replies
ENOSYS in do_setlk_common. fc->no_flock is 0, so fuse_file_flock()
returns that reply without change. There is no local fallback.

The daemon does not advertise FUSE_CAP_POSIX_LOCKS. fcntl locks therefore
use the posix_lock_file() path and stay in the kernel. This is why fcntl
appeared to work.

Same emulator, same app process, two directories:

Android 14 (API 34)	/data/user/0/<pkg>/files (ext4)	/storage/emulated/0/Android/data/<pkg> (FUSE)
flock(2)	OK	ENOSYS
fcntl(F_SETLK)	OK	OK
fcntl(F_OFD_SETLK)	OK	OK

The failure depends on the filesystem, not on the Android release. It does not
affect a database in the app's own data directory.

This change

Use flock(2). On ENOSYS, use an open file description lock (F_OFD_SETLK)
instead. An OFD lock belongs to the open file description, as a flock lock
does. Both properties above therefore hold. funlock makes the same choice.
The choice is stable for a given file, because it is a property of the
filesystem.

Storage that implements flock(2) continues to use it, and behaves as on every
other unix platform. F_OFD_SETLK requires Linux 3.15. The fallback path always
has it, because the path is reached only on the FUSE mount. Android 11
introduced that mount with a 4.14 or later kernel.

Testing

Added TestOpen_ErrTimeout_SameProcess. It skips on aix and solaris, which
still lock with POSIX record locks.

Verified on an Android 14 emulator (API 34, arm64). A probe and this package's
test binary ran from a real zygote-spawned app process (Seccomp: 2), on both
filesystems:

##########  INTERNAL (ext4)  /data/user/0/io.etcd.lockprobe2/files/lk
  flock(2)            => OK (lock acquired)
  fcntl F_OFD_SETLK   => OK (lock acquired)
  first  bolt.Open    => err=<nil>
  second bolt.Open    => err=timeout   (want: timeout)
  CHILD PROCESS bolt  => timeout
  reopen after Close  => err=<nil>     (want: nil)
--- PASS: TestOpen_ErrTimeout_SameProcess

##########  EXTERNAL (FUSE)  /storage/emulated/0/Android/data/io.etcd.lockprobe2/files/lk
  flock(2)            => function not implemented (errno=38)
  fcntl F_OFD_SETLK   => OK (lock acquired)
  first  bolt.Open    => err=<nil>
  second bolt.Open    => err=timeout   (want: timeout)
  CHILD PROCESS bolt  => timeout
  reopen after Close  => err=<nil>     (want: nil)
--- PASS: TestOpen_ErrTimeout_SameProcess

The full test suite of the package also passes on that emulator, run from the
app's data directory.

No other platform changes: bolt_android.go builds only for GOOS=android.

[kubernetes-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: nekohasekai
Once this PR has been reviewed and has the lgtm label, please assign ahrtr for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment