Hang while highlighting JavaScript using Packages#v4050

When attempting to highlight some JS code we noticed that it would never return and just spin consuming a full core. We're using syntect with a custom pack built from mostly https://github.com/sublimehq/Packages/releases/tag/v4050. The code:

{
    // this is an inline comment

    "foo": "bar" // another inline comment

    /*
       This is a block comment
       that continues on another line
    */
}

This hangs while parsing the /* line. I've pushed a complete testcase, but since this requires a newer version of sublimehq/Packages than currently tested against I expect that this would need a smaller syntax file to be synthesized for a proper testcase.

4050 is already using branch points
https://github.com/sublimehq/Packages/blob/466e8bfa1166fef08282cffdbd0825e12bc8a09f/JavaScript/JavaScript.sublime-syntax#L929-L932
which syntect doesn't support yet.

Hmmm, it would be nice if that gave an error while loading. In any case it seems like an incorrect syntax file shouldn't allow infinite processing to occur, and this code snippet shouldn't be hitting the branch point.

Good point :)

I have narrowed it down to the line_ending_ahead variable in that sublime-syntax file - removing (?:/\*{{block_comment_contents}})? stops the hang from occurring.

@keith-hall

@keith-hall would you mind explaining why bat doesn't suffer from this problem? Delta uses bat's highlighting assets and is suffering from it.

I'm not convinced that bat doesn't suffer from it:

echo -e 'export async function go_to_bar(bar_no) {\n  foo = bar_no\n    /*\n    testing123\n    */\n}\n' | ~/repos/bat/target/release/bat -l JavaScript

hangs, and when I press Ctrl+C, I get the following output:

───────┬────────────────────────────────────────────────────────────────────────
       │ STDIN
───────┼────────────────────────────────────────────────────────────────────────
   1   │ export async function go_to_bar(bar_no) {
   2   │   foo = bar_no

IIRC when trying to determine that too I realised that bat was using JavaScript (Babel) for .js files, not JavaScript, using -l js or -l 'JavaScript (Babel)' in the above command runs successfully. I switched docs.rs over to using this too to avoid the issue in rust-lang/docs.rs#1944.

@keith-hall

Thanks @keith-hall and @Nemo157!

This may be the same defect as #706, which proposes a fix.

#706 diagnoses a loop between two non-consuming rules in the JavaScript syntax: expression-statement-continuation matches empty, because / is in its operator class, and the rule for "blanks and comments, then end of line" also matches empty, so the two hand the position back to each other. The guard in parse_next_token remembers only a non-consuming push; a set leaves the stack depth unchanged, so it never arms.

The reproducer here and the one in #706 share that shape — an expression continuation followed by /*. I hit the same hang in a third project with:

  | { type: "a" }
  /** x */

which hangs identically under fancy-regex and under syntect-onig.

Worth checking whether #706 closes this.

Navigation Menu

Repository navigation

Hang while highlighting JavaScript using Packages#v4050 #460

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Hang while highlighting JavaScript using Packages#v4050 #460

Description

Activity

keith-hall commented on Dec 1, 2022

Nemo157 commented on Dec 1, 2022

keith-hall commented on Dec 1, 2022

dandavison commented on Jun 20, 2023

keith-hall commented on Jun 21, 2023

Nemo157 commented on Jun 21, 2023

dandavison commented on Jun 21, 2023

oetiker commented on Sep 22, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions