Re-encoding: behaviour of --no-keep-foreign-metadata and --keep-foreign-metadata

[H2Swine]

I argue that current behaviour is not what users who apply the options would expect ... to the extent any user manually gives foreign metadata commands at all.

Wish for:

• Command flac infile.flac -o outfile.flac --no-keep-foreign-metadata to discard the APPLICATION block with foreign metadata. It stands to reason that users who give such a command, want precisely that. Current behaviour: transfers the block, option means nothing.
• The default should remain to keep any APPLICATION blocks that might be present, but not to give errors in the user's face upon flac infile.flac -o outfile.flac --keep-foreign-metadata-if-present. What else could the user have intended by this command?

Solution:

• When input is a FLAC file, interpret everything about "foreign metadata" not to be the FLAC file-level metadata, but foreign metadata in an APPLICATION block. Error messages will follow suit, but:
• If user gives flac infile.flac -o outfile.flac --keep-foreign-metadata-if-present, return also an INFO that this is indeed default behaviour.

Related:
I think it stands to reason that users who has saved foreign metadata into a .flac file, want those data to be retained upon decoding, as default. Sure it enables injecting privacy-compromising metadata - exactly what is already the behaviour when re-encoding preserves APPLICATION blocks.

[madah81pnz1]

I tried it, but it didn't give an error, only a warning:

flac --no-keep-foreign-metadata -d file_with_foreign_metadata.flac -o decoded.flac

file_with_foreign_metadata.flac: WARNING: found foreign metadata, use --keep-foreign-metadata to restore

Then I tried to check the output .flac file:

$ metaflac --list decoded.flac
decoded.flac: ERROR: reading metadata, status = "FLAC__METADATA_CHAIN_STATUS_NOT_A_FLAC_FILE"

The file does not appear to be a FLAC file.
$ file decoded.flac
decoded.flac: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, stereo 44100 Hz

Having already dealt with similar bugs due to .aifc file extension, this didn't come as a big surprise though.

My error was of course the extra -d, without it it "works", as in: it reproduces the problem,--no-keep-foreign-metadata does nothing:

flac --no-keep-foreign-metadata file_with_foreign_metadata.flac -o decoded.flac

temp_has_metadata.flac: wrote 15370041 bytes, ratio=1,051

$ file decoded.flac
decoded.flac: FLAC audio bitstream data, 16 bit, stereo, 44.1 kHz, 5837184 samples

When an output file is specified with -o, the -d option should really be redundant. Comes back to that flac should really validate that the output file extension (if provided) matches the output format.

The other annoyance is that metaflac --list prints out binary data right to the terminal, --application-data-format=hexdump should be the default option here. An even better default would be a nicer printout for known foreign metadata chunks.

I also agree that --keep-foreign-metadata-if-present should be the default. It atleast shouldn't produce a warning when there is no metadata:

file.flac: WARNING reading foreign metadata: invalid WAVE file: missing "fmt " chunk (024)

This warning is not so clear on what the actual problem is; you might think that there is foreign metadata, but maybe it's invalid? So is the flac file corrupt, truncated or someone made a mistake when encoding it.
Note that this invalid WAVE file: missing "fmt " chunk output happens for any output format, even for AIFF files.

[H2Swine]

This seems to morph into two more issues in addition to the one raised.

FLAC, coming from the *n*x tradition, does only as exception let file extension decide what to do. Related is that operation modes decoding/analysis/test(/help/version) must to be given explicitly while encoding does not.
(A different consequence of that is that if you produce headerless FLAC frames and try to re-encode using -F it will fail, but if you give -dF if will understand that the input is supposed to be flac. Hey here is half an issue there. But who uses that how often?)

As goes metaflac --list does not only list what is there but also output the full content ... yeah, --list --except-block-type=PICTURE is my friend. I guess it is so established that an "overview" option would need a new name.

[madah81pnz1]

This seems to morph into two more issues in addition to the one raised.

Yeah I'm sorry, let's focus on the foreign metadata options.

I think either of these solutions can work:

1. --no-keep-foreign-metadata gives an error for flac-to-flac transcode. This keeps the behaviour consistent, --keep-foreign-metadata and --keep-foreign-metadata-if-present already gives ERROR: --keep-foreign-metadata can only be used with WAVE, Wave64, RF64, or AIFF input
   1.1. As a helpful suggestion to the user trying this, flac could print out a recommendation doing metaflac --remove --block-type=APPLICATION. But as this removes all APPLICATION blocks, I don't know if there a way to only remove the specific APPLICATION blocks that contains foreign meatdata, other than via metaflac --remove --block-number=#. But then it gets complicated; since if this is an error, no output file will be produced, so flac doesn't know the block number, should it then suggest running metaflac on the input source file.
2. --no-keep-foreign-metadata removes all the foreign metadata APPLICATION blocks for flac-to-flac transcode. But then --keep-foreign-metadata and --keep-foreign-metadata-if-present should no longer produce that error, since the default is to always keep all foreign metadata.
   2.1. As there is also no way of removing the VORBIS_COMMENT block, an option for this could be added too. Or what is the use case for removing foreign metadata only when transcoding flac-to-flac? Maybe an option to do a "clean" decode-encode, without any metadata at all, like if you had decoded to raw format and encoded from that.

[H2Swine]

No. 2 I think is aligned with my suggestion, and I think transfer should be the default.

As for use case: sanitizing for privacy, e.g. removing watermarks like ID3's UNIQUEFILEID in ID3.
"Copy prevention, DRM, etc." is an explicit anti-goal for the FLAC project.

Or what is the use case for removing foreign metadata only when transcoding flac-to-flac?

Reasonable question to ask. PICTURE and VORBIS_COMMENT and CUESHEET are (almost) user-accessible, and usually desired - if not, whatever happens is not behind the user's back, but APPLICATION is transferred without any information about what goes on, behind the user's back. Maybe this should go for all APPLICATION ... Of course, if STREAMINFO/SEEKTABLE/PADDING are written anew, there is no use in pretending to not-preserve them.

Besides, if user wants to remove everything, that is much easier by going by way of a decode && encode, than copying tags is. (I see the point in having tag manipulation in flac.exe and not only metaflac, especially as long as metaflac is even pickier about filename.)