Skip to content

automata/nfa/thompson: tweak allocation handling for UTF-8 compiler - #1386

Open
BurntSushi wants to merge 1 commit into
masterfrom
ag/optimize-unicode-regex
Open

automata/nfa/thompson: tweak allocation handling for UTF-8 compiler#1386
BurntSushi wants to merge 1 commit into
masterfrom
ag/optimize-unicode-regex

Conversation

@BurntSushi

Copy link
Copy Markdown
Member

This commit makes the UTF-8 compiler allocation pattern a bit more lazy.
Instead of allocating 10,000 entries up-front, we now allocate via a
smaller chunking pattern. That is, we only allocate N chunks up front,
and each chunk is itself only allocated only when we actually try to put
something inside of it.

This also adds a few unit tests covering existing and new behavior.

This addresses #1316 by making compilation of that regex substantially
faster. Previously, this program

fn main() {
    for _ in 0..100000 {
        let re = regex::bytes::RegexBuilder::new(r"^[^/]+/foo/[^/]+$")
            .unicode(true)
            .build()
            .unwrap();
        assert!(re.is_match(b"bar/foo/baz"));
    }
}

took about 1.86s to run. But with this change, it only takes 1.14s to
run. This still isn't quite as good as the non-Unicode case (which takes
0.64s), but we're closer with this change.

Fixes #1316

This commit makes the UTF-8 compiler allocation pattern a bit more lazy.
Instead of allocating 10,000 entries up-front, we now allocate via a
smaller chunking pattern. That is, we only allocate N chunks up front,
and each chunk is itself only allocated only when we actually try to put
something inside of it.

This also adds a few unit tests covering existing and new behavior.

This addresses #1316 by making compilation of that regex substantially
faster. Previously, this program

```
fn main() {
    for _ in 0..100000 {
        let re = regex::bytes::RegexBuilder::new(r"^[^/]+/foo/[^/]+$")
            .unicode(true)
            .build()
            .unwrap();
        assert!(re.is_match(b"bar/foo/baz"));
    }
}
```

took about 1.86s to run. But with this change, it only takes 1.14s to
run. This still isn't quite as good as the non-Unicode case (which takes
0.64s), but we're closer with this change.

Fixes #1316
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compiling some unicode regexes is 2.5x slower

1 participant