Don't fail the whole request on an unparseable log entry - #532
Closed
MoheyElbaz wants to merge 1 commit into
Closed
Conversation
LaravelLog::parseText() read $matches[1] straight after preg_match without checking whether it matched. A chunk that isn't a Laravel entry, such as a run of null bytes from a partial write, left $matches empty, and Laravel turned the undefined key into an ErrorException. The logs API then returned a 500 for the whole file instead of for the one bad entry. regexPattern() is stricter than the static::$regex the indexer used to accept the entry in the first place, so this also hit ordinary lines: a millisecond precision timestamp, or anything printed before the '[', match $regex but not regexPattern(). So on a failed match, fall back to $regex and keep the datetime, level and environment it finds. Only content that matches neither is marked unparseable, with no severity and the printable part of its first line as the message. The fallback returns into the same code path as a normal match, so the maxLogSize() cap and the first-line message still apply. Handling it with an early return instead left unparseable entries uncapped, which turned one corrupt region into a multi-hundred-MB response. Fixes opcodesio#529
Author
|
Closing this myself — I found three defects in my own patch while re-reviewing it.
Issue #529 is still real and I will open a fresh PR with all three fixed, plus a first-line-length test — my existing size test put the payload on the second line, so it never exercised the unbounded |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #529.
LaravelLog::parseText()reads$matches[1]straight afterpreg_matchwithout checking whether it matched:When the chunk isn't a Laravel entry — the reporter had a run of null bytes from a partial write on a shared volume —
$matchesis empty, Laravel turns the undefined key into anErrorException, and/log-viewer/api/logsreturns a 500 for the whole file rather than for the one bad entry.This isn't limited to corrupt bytes
regexPattern()is stricter than thestatic::$regexthatLog::matches()already used to accept and index the entry. Both of these match$regexbut failregexPattern():So ordinary entries hit this path too.
What this does
On a failed strict match, fall back to
static::$regexand keep the datetime, level and environment it finds. Only content matching neither is marked unparseable, with no severity and the printable part of its first line as the message.The fallback returns into the same code path as a normal match, so the
maxLogSize()cap and the first-line message still apply. I tried an earlyreturnfirst and it was worse: unparseable entries came back uncapped at 600,021 bytes withlog_text_incompleteunset, andmessageheld the entire entry rather than a preview — with 25 entries a page, one corrupt region becomes a multi-hundred-MB response.message[…17.123] local.ERROR: boomTests
11 added to
LaravelLogsTest, covering the corrupt shapes, the loose-pattern recovery, the size cap, the first-line message, and that a valid entry is untouched. All 10 of the new assertions fail withUndefined array key 1against currentmain.Left alone deliberately
IndexedLogReader::next()— itsempty($text)checkreturns null, which ends iteration rather than skipping, so changing it there would silently truncate log lists.$this->textstill holds the original bytes. Stripping control characters from the stored text would destroy legitimate content such as ANSI colour codes, and the size cap already bounds the payload.HorizonLog,HorizonOldLogandRedisLogindex$matches['level']and friends without guards and will 500 the same way on a line they can't parse. That's the same class of bug and arguably belongs inLog::parseText(), but it needs its own tests, so I kept this PR to Undefined array key 1 in LaravelLog when entry is corrupt/null bytes → HTTP 500 #529. Happy to follow up if you want it.