fix(ci): line endings check fails on symbolic links

[wu21-web]

Description

=== git ls-files --eol ===
  i/lf    w/lf    attr/                         file.txt
  i/      w/      attr/                         link.txt

  === original CI pipeline ===
  git ls-files --eol | grep -v 'i/lf\|i/none\|i/-text'
  i/      w/      attr/                         link.txt
  grep_exit=0

The original step is:

  if git ls-files --eol | grep -v 'i/lf\|i/none\|i/-text'; then
    echo 'Line endings must end with LF. Have you normalized line endings?'
    exit 1
  fi

Git reports symlinks with empty i/ and w/ fields, so the symlink line does not match any of the allowed patterns. grep still finds that line and exits 0, which makes the if branch run and fail the job.

Now the logic is:

1. Make git reformat line endings
2. Compare with git diff --quiet --cached
3. If step 2 exits with non-0, abort and print err msg.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Refactoring (no functional changes)
• Documentation update
• CI / Build / Tooling

How Has This Been Tested?

• make test passes locally
• Manual testing (describe below)
  I tested my fix on the failing pull request feat(pages): standardize web crawler policy #939

Checklist

• My code follows the project's coding style (go fmt, go vet)
• I have performed a self-review of my code
• I have added tests that prove my fix is effective or my feature works
• New and existing unit tests pass locally with my changes
• I have updated the documentation accordingly (if applicable)
• I have signed the CLA

Related Issues

none

[wu21-web]

This is considered urgent for the development of #939

[github-actions]

✅ OpenCodeReview: Review complete: 0 finding(s) across 1 selected item(s).

[wu21-web]

May I request high priority for this one, @lizhengfeng101 @stay-foolish-forever , thank you.

[lizhengfeng101]

Nice catch on the symlink issue — confirmed locally that git ls-files --eol reports empty i//w/ fields for symlinks, which is exactly what breaks the old grep -v pattern. The renormalize + diff approach is a solid fix since it delegates the "is this file normalized" judgment to git's own clean filter instead of parsing ls-files output.

One small suggestion on the failure message. Right now it's a rhetorical question:

echo 'Line endings must end with LF. Have you normalized line endings?'

AGENTS.md already documents the fix (git add --renormalize .), and it's the same command this check uses internally to detect the problem — so it's worth putting directly in the error text instead of making contributors look it up. Also, the other failing steps in this job (Check formatting, go mod tidy) use ::error:: annotations, which highlights the line in the GitHub UI; this step could match that style:

if ! git diff --cached --quiet; then
  echo "::error::Line endings must be LF. Run 'git add --renormalize .' and commit the diff below."
  git diff --cached --stat
  exit 1
fi

Non-blocking either way — just closes the loop between "here's the error" and "here's the exact command to fix it."

[wu21-web]

That is reasonable. Done.

[wu21-web]

@lizhengfeng101 Any updates on this?

[lizhengfeng101]

LGTM