Accept CRLF source files and normalize them to LF (#25)#66
Merged
Conversation
CRLF ("Windows") source files historically broke publication with a
misleading "not in W3C format" error, because the old metadata regex
captured the trailing carriage return into attribute values. That regex
was tightened in 920dbdf, which incidentally cured the crash, but the
in-place source rewrite still left CRLF newlines in the body, producing
a source file with mixed line endings.
Normalize CRLF (and bare CR) to LF as each line is read, in both the
metadata and body loops of _process_source_file, so parsing is robust by
construction and the rewritten source comes out uniformly LF.
Adds t/crlf.t covering publication and clean rewriting of a CRLF source.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address review feedback: rather than repeating the normalization regex in both the metadata and body read loops, factor it into a single _normalize_newlines subroutine. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #25.
Diagnosis
Source files with CRLF ("Windows") newlines used to break publication with a misleading
The "time" attribute is not in W3C format.error. The cause was the old metadata regex(.*)$capturing the trailing\rinto attribute values, sotimebecame2018-03-04T12:00:00\rand failed the W3C parse.Commit
920dbdf(2020-05-07) tightened that regex to(.*?)\s*$, which strips trailing whitespace and incidentally cured the crash — the issue just was never closed. Today a CRLF file already publishes successfully. The one remaining wrinkle: the in-place source rewrite kept CRLF in the post body, leaving the source file with mixed line endings (LF metadata + CRLF body).Fix
Normalize
\r\n(and bare\r) to\nas each line is read, in both the metadata and body loops of_process_source_file. Metadata parsing is now robust by construction rather than by regex accident, and the rewritten source comes out uniformly LF.Test
Adds
t/crlf.t: writes a raw-CRLF source file, then asserts it publishes without error, produces its HTML, and that the rewritten source contains no carriage returns. Before the fix only the carriage-return assertion failed (confirming the crash itself was already gone); it passes now. Full suite green (81 tests).🤖 Generated with Claude Code