Fails to parse classes with unintended comments.

[Erotemic]

If comments do not agree with the indentation level of the current scope, the parser can fail. MWE:

echo '
def foo():
    """
    Example:
        >>> class MyClass:
        >>>     #
        >>>     def method1(self):
        >>>         ...
        >>>     #
        >>>     def method2(self):
        >>>         ...
        >>>     #
        >>> self = MyClass()
    """
' > works.py
xdoctest works.py

echo '

def foo():
    """
    Example:
        >>> class MyClass:
        >>> #
        >>>     def method1(self):
        >>>         ...
        >>> #
        >>>     def method2(self):
        >>>         ...
        >>> #
        >>> self = MyClass()
    """
' > fails.py
xdoctest fails.py

This happens in the parser:

  File "/home/joncrall/code/xdoctest/src/xdoctest/core.py", line 387, in parse_docstr_examples
    for example in parser(docstr, callname=callname, modpath=modpath,
                   ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                          fpath=fpath, lineno=lineno, **parser_kw):
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/joncrall/code/xdoctest/src/xdoctest/core.py", line 308, in parse_auto_docstr_examples
    for example in parse_freeform_docstr_examples(docstr, *args, **kwargs):
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/joncrall/code/xdoctest/src/xdoctest/core.py", line 178, in parse_freeform_docstr_examples
    all_parts = list(parser.DoctestParser().parse(docstr, info))
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/home/joncrall/code/xdoctest/src/xdoctest/parser.py", line 223, in parse
    raise exceptions.DoctestParseError(
        'Failed to parse doctest in {}'.format(failpoint),
        string=string, info=info, orig_ex=orig_ex)
xdoctest.exceptions.DoctestParseError: Failed to parse doctest in _package_groups

With XDOCTEST_DEBUG=1 xdoctest fails.py we get:

Details

------+ DEBUG +------
CALLED doctest_module
exclude = []
argv = []
command = 'all'
module_identifier = 'fails.py'
durations = None
config = {'default_runtime_state': {}, 'offset_linenos': False, 'colored': True, 'reportchoice': 'udiff', 'global_exec': None, 'supress_import_errors': False, 'verbose': 3}
verbose = 3
style = 'auto'
------+ /DEBUG +------
Start doctest_module('fails.py')
Listing tests
Find package calldefs: pkg_identifier = 'fails.py'
About to parse calldefs with do_dynamic=False
Found 1 calldefs
Parsing docstring examples for callname=foo in modpath=fails.py
parser = <function parse_auto_docstr_examples at 0x7cb262ee9b20>
Automatic style is trying google parsing
Next line 0: >>> class MyClass:
state_indent = 0
match = <re.Match object; span=(0, 0), match=''>
line_indent = 0
completing source
Next line 1: >>> #
state_indent = 0
match = <re.Match object; span=(0, 0), match=''>
line_indent = 0
completing source
Next line 2: >>>     def method1(self):
state_indent = 0
match = <re.Match object; span=(0, 0), match=''>
line_indent = 0
completing source
Next line 3: >>>         ...
state_indent = 0
match = <re.Match object; span=(0, 0), match=''>
line_indent = 0
completing source
Next line 4: >>> #
state_indent = 0
match = <re.Match object; span=(0, 0), match=''>
line_indent = 0
completing source
Next line 5: >>>     def method2(self):
state_indent = 0
match = <re.Match object; span=(0, 0), match=''>
line_indent = 0
completing source
Next line 6: >>>         ...
state_indent = 0
match = <re.Match object; span=(0, 0), match=''>
line_indent = 0
completing source
Next line 7: >>> #
state_indent = 0
match = <re.Match object; span=(0, 0), match=''>
line_indent = 0
completing source
Next line 8: >>> self = MyClass()
state_indent = 0
match = <re.Match object; span=(0, 0), match=''>
line_indent = 0
completing source
<FAILPOINT>
!!! FAILED !!!
failpoint = '_package_groups'
    Traceback (most recent call last):
      File "/home/joncrall/code/xdoctest/src/xdoctest/parser.py", line 190, in parse
        all_parts = list(self._package_groups(grouped_lines))
      File "/home/joncrall/code/xdoctest/src/xdoctest/parser.py", line 239, in _package_groups
        for example in self._package_chunk(slines, wlines, lineno):
                       ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/joncrall/code/xdoctest/src/xdoctest/parser.py", line 286, in _package_chunk
        ps1_linenos, mode_hint = self._locate_ps1_linenos(source_lines)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
      File "/home/joncrall/code/xdoctest/src/xdoctest/parser.py", line 588, in _locate_ps1_linenos
        raise syn_ex
      File "/home/joncrall/code/xdoctest/src/xdoctest/parser.py", line 578, in _locate_ps1_linenos
        pt = static.six_axt_parse(source_block)
      File "/home/joncrall/code/xdoctest/src/xdoctest/static_analysis.py", line 1168, in six_axt_parse
        pt = ast.parse(source_block, filename=filename)
      File "/home/joncrall/.pyenv/versions/3.13.2/lib/python3.13/ast.py", line 54, in parse
        return compile(source, filename, mode, flags,
                       _feature_version=feature_version, optimize=optimize)
      File "<source_block>", line 2
        _._ = None
        ^
    IndentationError: expected an indented block after class definition on line 1
    
Failed to parse string = <{[<{[<{[  # xdoc debug
>>> class MyClass:
>>> #
>>>     def method1(self):
>>>         ...
>>> #
>>>     def method2(self):
>>>         ...
>>> #
>>> self = MyClass()

]}>]}>]}>  # xdoc debug end string
info = {
    'callname': 'foo',
    'fpath': 'fails.py',
    'lineno': 6,
    'modpath': 'fails.py',
}
-----
orig_ex = expected an indented block after class definition on line 1 (<source_block>, line 2)
labeled_lines = [
    ('dsrc', '>>> class MyClass:'),
    ('dsrc', '>>> #'),
    ('dsrc', '>>>     def method1(self):'),
    ('dsrc', '>>>         ...'),
    ('dsrc', '>>> #'),
    ('dsrc', '>>>     def method2(self):'),
    ('dsrc', '>>>         ...'),
    ('dsrc', '>>> #'),
    ('dsrc', '>>> self = MyClass()'),
]
grouped_lines = [
    (
        [
            '>>> class MyClass:',
            '>>> #',
            '>>>     def method1(self):',
            '>>>         ...',
            '>>> #',
            '>>>     def method2(self):',
            '>>>         ...',
            '>>> #',
            '>>> self = MyClass()',
        ],
        '',
    ),
]
all_parts = None
</FAILPOINT>
Automatic style is trying freeform parsing
Parsing docstring for callname=foo in modpath=fails.py
Next line 0: 
state_indent = 0
match = None
line_indent = 0
Next line 1: Example:
state_indent = 0
match = <re.Match object; span=(0, 0), match=''>
line_indent = 0
Next line 2:     >>> class MyClass:
state_indent = 0
match = <re.Match object; span=(0, 4), match='    '>
line_indent = 4
completing source
Next line 3:     >>> #
state_indent = 4
match = <re.Match object; span=(0, 4), match='    '>
line_indent = 4
completing source
Next line 4:     >>>     def method1(self):
state_indent = 4
match = <re.Match object; span=(0, 4), match='    '>
line_indent = 4
completing source
Next line 5:     >>>         ...
state_indent = 4
match = <re.Match object; span=(0, 4), match='    '>
line_indent = 4
completing source
Next line 6:     >>> #
state_indent = 4
match = <re.Match object; span=(0, 4), match='    '>
line_indent = 4
completing source
Next line 7:     >>>     def method2(self):
state_indent = 4
match = <re.Match object; span=(0, 4), match='    '>
line_indent = 4
completing source
Next line 8:     >>>         ...
state_indent = 4
match = <re.Match object; span=(0, 4), match='    '>
line_indent = 4
completing source
Next line 9:     >>> #
state_indent = 4
match = <re.Match object; span=(0, 4), match='    '>
line_indent = 4
completing source
Next line 10:     >>> self = MyClass()
state_indent = 4
match = <re.Match object; span=(0, 4), match='    '>
line_indent = 4
completing source
<FAILPOINT>
!!! FAILED !!!
failpoint = '_package_groups'
    Traceback (most recent call last):
      File "/home/joncrall/code/xdoctest/src/xdoctest/parser.py", line 190, in parse
        all_parts = list(self._package_groups(grouped_lines))
      File "/home/joncrall/code/xdoctest/src/xdoctest/parser.py", line 239, in _package_groups
        for example in self._package_chunk(slines, wlines, lineno):
                       ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/joncrall/code/xdoctest/src/xdoctest/parser.py", line 286, in _package_chunk
        ps1_linenos, mode_hint = self._locate_ps1_linenos(source_lines)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
      File "/home/joncrall/code/xdoctest/src/xdoctest/parser.py", line 588, in _locate_ps1_linenos
        raise syn_ex
      File "/home/joncrall/code/xdoctest/src/xdoctest/parser.py", line 578, in _locate_ps1_linenos
        pt = static.six_axt_parse(source_block)
      File "/home/joncrall/code/xdoctest/src/xdoctest/static_analysis.py", line 1168, in six_axt_parse
        pt = ast.parse(source_block, filename=filename)
      File "/home/joncrall/.pyenv/versions/3.13.2/lib/python3.13/ast.py", line 54, in parse
        return compile(source, filename, mode, flags,
                       _feature_version=feature_version, optimize=optimize)
      File "<source_block>", line 2
        _._ = None
        ^
    IndentationError: expected an indented block after class definition on line 1
    
Failed to parse string = <{[<{[<{[  # xdoc debug

Example:
    >>> class MyClass:
    >>> #
    >>>     def method1(self):
    >>>         ...
    >>> #
    >>>     def method2(self):
    >>>         ...
    >>> #
    >>> self = MyClass()

]}>]}>]}>  # xdoc debug end string
info = {
    'callname': 'foo',
    'fpath': None,
    'lineno': 4,
    'modpath': 'fails.py',
}
-----
orig_ex = expected an indented block after class definition on line 1 (<source_block>, line 2)
labeled_lines = [
    ('text', ''),
    ('text', 'Example:'),
    ('dsrc', '    >>> class MyClass:'),
    ('dsrc', '    >>> #'),
    ('dsrc', '    >>>     def method1(self):'),
    ('dsrc', '    >>>         ...'),
    ('dsrc', '    >>> #'),
    ('dsrc', '    >>>     def method2(self):'),
    ('dsrc', '    >>>         ...'),
    ('dsrc', '    >>> #'),
    ('dsrc', '    >>> self = MyClass()'),
]
grouped_lines = [
    [
        '',
        'Example:',
    ],
    (
        [
            '    >>> class MyClass:',
            '    >>> #',
            '    >>>     def method1(self):',
            '    >>>         ...',
            '    >>> #',
            '    >>>     def method2(self):',
            '    >>>         ...',
            '    >>> #',
            '    >>> self = MyClass()',
        ],
        '',
    ),
]
all_parts = None
</FAILPOINT>
Caught an error when parsing

[Erotemic]

Fixed in 1.3.1

[Erotemic]

This hasn't been merged in yet.