Infinite loop when encountering exclamation mark symbol firstly for BIND9Parser

[egberts]

When encountering the following content:

    allow-transfer {
        trusted_residential_network_acl; // allow XFER from homenet

        external_downstream_nameservers_acl;
        key ddns-sha256-arca-a-key; // TODO tighten this down

        !{ !localhost; any; };
        key master-to-slave-key; // only localhost can use key
        localhost; // not so useful for unsecured RNDC uses
        };

The infinite loop resides at the line:

    !{ !localhost; any; };

Apparently, the exclamation mark is tripping that up when pressing Ctrl-C, as traceback shows:

(Pdb) bt
  /usr/lib/python3.5/runpy.py(193)_run_module_as_main()
-> "__main__", mod_spec)
  /usr/lib/python3.5/runpy.py(85)_run_code()
-> exec(code, run_globals)
  /usr/lib/python3.5/pdb.py(1692)<module>()
-> pdb.main()
  /usr/lib/python3.5/pdb.py(1665)main()
-> pdb._runscript(mainpyfile)
  /usr/lib/python3.5/pdb.py(1546)_runscript()
-> self.run(statement)
  /usr/lib/python3.5/bdb.py(431)run()
-> exec(cmd, globals, locals)
  <string>(1)<module>()
  /home/steve/work/python/reconfigure/test/bind.py(4)<module>()
-> import reconfigure
  /home/steve/.local/lib/python3.5/site-packages/reconfigure-0.1.81-py3.5.egg/reconfigure/includers/auto.py(13)compose()
-> self.compose_rec(origin, origin, tree)
  /home/steve/.local/lib/python3.5/site-packages/reconfigure-0.1.81-py3.5.egg/reconfigure/includers/auto.py(38)compose_rec()
-> self.compose_rec(root, file, subtree)
  /home/steve/.local/lib/python3.5/site-packages/reconfigure-0.1.81-py3.5.egg/reconfigure/includers/auto.py(38)compose_rec()
-> self.compose_rec(root, file, subtree)
  /home/steve/.local/lib/python3.5/site-packages/reconfigure-0.1.81-py3.5.egg/reconfigure/includers/auto.py(20)compose_rec()
-> self.compose_rec(root, origin, child)
  /home/steve/.local/lib/python3.5/site-packages/reconfigure-0.1.81-py3.5.egg/reconfigure/includers/auto.py(36)compose_rec()
-> subtree = self.parser.parse(content)
> /home/steve/.local/lib/python3.5/site-packages/reconfigure-0.1.81-py3.5.egg/reconfigure/parsers/nginx.py(24)parse()
-> tokens, remainder = scanner.scan(' '.join(filter(None, content.split(' '))))
  /usr/lib/python3.5/re.py(367)scan()
-> m = match()

The loop is not stuck in the NginxParser(BaseParser), but in the following function:

tokens, remainder = scanner.scan(' '.join(filter(None, content.split(' '))))

According to the traceback, it is stuck in match():

/usr/lib/python3.5/re.py(367)scan()

Probably a bad regex. I realized that NginxParser is being used. I do think that tokens can be enhanced to allow exclamation mark.

reconfigure/reconfigure/parsers/nginx.py

Line 24 in 2b8729a

tokens, remainder = scanner.scan(' '.join(filter(None, content.split(' '))))

[egberts]

Nor does it match the supplied BIND9Parser regex given in this code snippet as fixed by Ajenti Issue 419:

https://github.com/Eugeny/reconfigure/blame/2b8729a8274deedb5acb1a8319245f74eec863f0/reconfigure/parsers/bind9.py#L15

Regex debugger here.

https://www.debuggex.com/r/smKUP7a7s4SAu3TM

[egberts]

Related issue Ajenti #484 ajenti/ajenti#484
Related issue Ajenti #419 ajenti/ajenti#419

[egberts]

Updated bind9 tokens (which greatly speeds up from 27 minutes to 5 seconds) to as followed:

    tokens = [
        (r"(masters)\s+?([^\s{}]*)\s+?(port|dscp)\s+?([^\s{}]*\s*)\s*(port|dscp)\s*?([^\s{}]*\s*)\s*{", lambda s, t: ('clause5_start', t)), 
        (r"(listen-on-v6|listen-on|masters)\s+?(port|dscp)\s+?([^\s{}]*\s*)\s*{", lambda s, t: ('clause3_start', t)), 
        (r"(channel|masters)\s+?([^\s{}]*\s*)\s*{", lambda s, t: ('clause1_start', t)),
        (r"(acl|key|server)\s+?([^\s{}]*\s*)\s*{", lambda s, t: ('clause1_start', t)),
        (r"(controls|logging|options)\s+?([^\s{}]*\s*)*{", lambda s, t: ('clause0_start', t)),
        (r"(view|zone)\s+?([^\s{}]*)?\s+?([^\s{}]*)\s*{", lambda s, t: ('clause12_start', t)),
        (r"(allow-notify|allow-query-on|allow-query|allow-recursion-on|allow-recursion|allow-transfer|allow-update-forwarding|allow|also-notify|alt-transfer-source-v6|alt-transfer-source|disable-algorithms|dual-stack-servers|forwarders|match-clients|match-destinations|rrset-order|sortlist|update-policy)\s+?([^\s{}]*\s*)*{", lambda s, t: ('statement_start', t)), 
        (r"\#.*?\n", lambda s, t: ('comment', t)),
        (r"//.*?\n", lambda s, t: ('comment', t)),
        (r"/\*.*?\*/", lambda s, t: ('comment', t)),
        (r"((([^\s{};#]+)|({\s*([^\s{};#]+;\s*)*}))\s*?)+;", lambda s, t: ('option', t)),
        (r"\s", lambda s, t: 'whitespace'),
        (r"$^", lambda s, t: 'newline'),
        (r"\};", lambda s, t: 'clause_end'),
    ]

Test passes

:~/work/python/reconfigure/reconfigure/reconfigure/tests$ pytest -v parsers/*_tests.py
============================= test session starts ==============================
platform linux -- Python 3.5.3, pytest-3.7.1, py-1.5.4, pluggy-0.7.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: ~/work/python/reconfigure/reconfigure, inifile:
collected 23 items                                                             

parsers/bind9_tests.py::BIND9ParserTest::test_parse <- reconfigure/tests/parsers/base_test.py PASSED [  4%]
parsers/bind9_tests.py::BIND9ParserTest::test_stringify <- reconfigure/tests/parsers/base_test.py PASSED [  8%]
parsers/bind9_tests.py::BIND9ParserHangTest::test_hang PASSED            [ 13%]
parsers/crontab_tests.py::CrontabParserTest::test_parse <- reconfigure/tests/parsers/base_test.py PASSED [ 17%]
parsers/crontab_tests.py::CrontabParserTest::test_stringify <- reconfigure/tests/parsers/base_test.py PASSED [ 21%]
parsers/exports_tests.py::ExportsParserTest::test_parse <- reconfigure/tests/parsers/base_test.py PASSED [ 26%]
parsers/exports_tests.py::ExportsParserTest::test_stringify <- reconfigure/tests/parsers/base_test.py PASSED [ 30%]
parsers/ini_tests.py::IniParserTest::test_parse <- reconfigure/tests/parsers/base_test.py PASSED [ 34%]
parsers/ini_tests.py::IniParserTest::test_stringify <- reconfigure/tests/parsers/base_test.py PASSED [ 39%]
parsers/iptables_tests.py::IPTablesParserTest::test_parse <- reconfigure/tests/parsers/base_test.py PASSED [ 43%]
parsers/iptables_tests.py::IPTablesParserTest::test_stringify <- reconfigure/tests/parsers/base_test.py PASSED [ 47%]
parsers/jsonparser_tests.py::JsonParserTest::test_parse <- reconfigure/tests/parsers/base_test.py PASSED [ 52%]
parsers/jsonparser_tests.py::JsonParserTest::test_stringify PASSED       [ 56%]
parsers/nginx_tests.py::NginxParserTest::test_parse <- reconfigure/tests/parsers/base_test.py PASSED [ 60%]
parsers/nginx_tests.py::NginxParserTest::test_stringify <- reconfigure/tests/parsers/base_test.py PASSED [ 65%]
parsers/nsd_tests.py::BIND9ParserTest::test_parse <- reconfigure/tests/parsers/base_test.py PASSED [ 69%]
parsers/nsd_tests.py::BIND9ParserTest::test_stringify <- reconfigure/tests/parsers/base_test.py PASSED [ 73%]
parsers/shell_tests.py::ShellParserTest::test_parse <- reconfigure/tests/parsers/base_test.py PASSED [ 78%]
parsers/shell_tests.py::ShellParserTest::test_stringify <- reconfigure/tests/parsers/base_test.py PASSED [ 82%]
parsers/squid_tests.py::SquidParserTest::test_parse <- reconfigure/tests/parsers/base_test.py PASSED [ 86%]
parsers/squid_tests.py::SquidParserTest::test_stringify <- reconfigure/tests/parsers/base_test.py PASSED [ 91%]
parsers/ssv_tests.py::SSVParserTest::test_parse <- reconfigure/tests/parsers/base_test.py PASSED [ 95%]
parsers/ssv_tests.py::SSVParserTest::test_stringify <- reconfigure/tests/parsers/base_test.py PASSED [100%]

========================== 23 passed in 0.83 seconds ===========================
~/work/python/reconfigure/reconfigure/reconfigure/tests$

[egberts]

I'm going to submit a bigger bind9_test to ensure that all bind9 version passes.