[samsonasik]

Lexer::postprocessTokens() reads Token::$id twice for every token. Token extends the internal PhpToken class, so each property read is comparatively expensive.

This PR reads the token ID once and reuses it.

Tested faster:

Corpus	Tokens scanned	Lexer gain
PHP-Parser vendor	501,000	34.22 → 31.32 ms (~8.5%)
laravel-framework	22,898,098	1586.35 → 1365.66 ms (~13.9%)

[samsonasik]

@staabm @nikic ready for merge 👍

[staabm]

Token extends the internal PhpToken class, so each property read is comparatively expensive.

thats a interessting fact, I did not know about

[staabm]

PHPUnit, when generating code-coverage, also does some PhpToken processing via https://github.com/theseer/tokenizer

maybe you find similar opportunities there

[nikic]

Is it actually true? PhpToken uses normal properties, it does not intercept the property handlers, so I don't see why it should be slow.

[staabm]

@nikic so you say what is faster now is that we have a variable instead of a property-fetch? its unrelated to the fact of userland class vs. native class?

[samsonasik]

Reading $token->id performs an object-property lookup each time.

Subsequent $id reads use a local variable, which PHP accesses more cheaply than an object property.