Python error "AttributeError: 'NoneType' object has no attribute 'groupdict'" in esbonio.lsp while getting doctree

[davbeek]

When (re)starting the Sphinx server, which (re)starts the esbonio language server in VSCode, I get the following python error. The html preview output in VSCode appears to be correct and complete. No idea how to further investigate this, or what could be the cause.

Esbonio VSCode extension v0.9.2.
Conda esbonio 0.13.0
VSCode 1.68.1

The HTML pages are in _build/html/html.
[client] Build complete {
  "config": {
    "sphinx": {
      "buildDir": "/Users/dvbeek/1-repos/website/exercises/_build/html/html",
      "builderName": "html",
      "confDir": "/Users/dvbeek/1-repos/website/exercises",
      "configOverrides": {},
      "doctreeDir": "/Users/dvbeek/1-repos/website/exercises/_build/html/doctrees",
      "forceFullBuild": false,
      "keepGoing": false,
      "makeMode": true,
      "numJobs": 1,
      "quiet": false,
      "silent": false,
      "srcDir": "/Users/dvbeek/1-repos/website/exercises",
      "tags": [],
      "verbosity": 0,
      "warningIsError": false,
      "command": [
        "sphinx-build",
        "-M",
        "html",
        "/Users/dvbeek/1-repos/website/exercises",
        "/Users/dvbeek/1-repos/website/exercises/_build/html/html",
        "-d",
        "/Users/dvbeek/1-repos/website/exercises/_build/html/doctrees"
      ],
      "version": "4.5.0"
    },
    "server": {
      "logLevel": "debug",
      "logFilter": [],
      "hideSphinxOutput": false
    }
  },
  "error": false,
  "warnings": 0
}
[esbonio.lsp] Getting initial doctree for: '/Users/dvbeek/1-repos/website/exercises/_README.rst'
[esbonio.lsp] Traceback (most recent call last):
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/esbonio/lsp/rst/__init__.py", line 437, in get_initial_doctree
    return read_initial_doctree(filename, self.logger)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/esbonio/lsp/rst/io.py", line 208, in read_initial_doctree
    publisher.publish()
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/core.py", line 217, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/readers/__init__.py", line 72, in read
    self.parse()
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/__init__.py", line 183, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 170, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/statemachine.py", line 239, in run
    context, next_state, result = self.check_line(
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2769, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/statemachine.py", line 239, in run
    context, next_state, result = self.check_line(
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 1270, in bullet
    new_line_offset, blank_finish = self.nested_list_parse(
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 318, in nested_list_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/statemachine.py", line 239, in run
    context, next_state, result = self.check_line(
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2522, in bullet
    listitem, blank_finish = self.list_item(match.end())
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 1289, in list_item
    self.nested_parse(indented, input_offset=line_offset,
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/statemachine.py", line 239, in run
    context, next_state, result = self.check_line(
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2703, in blank
    paragraph, literalnext = self.paragraph(
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 418, in paragraph
    textnodes, messages = self.inline_text(text, lineno)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 427, in inline_text
    nodes, messages = self.inliner.parse(text, lineno,
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 646, in parse
    before, inlines, remaining, sysmessages = method(self, match,
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 789, in interpreted_or_phrase_ref
    nodelist, messages = self.interpreted(rawsource, escaped, role,
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 886, in interpreted
    nodes, messages2 = role_fn(role, rawsource, text, lineno, self)
  File "/Users/dvbeek/opt/miniconda3/lib/python3.9/site-packages/esbonio/lsp/rst/io.py", line 40, in dummy_role
    node.attributes.update(match.groupdict())
AttributeError: 'NoneType' object has no attribute 'groupdict'
[client] Ignoring output:extension-output-swyddfa.esbonio-%231-Esbonio

[alcarney]

The server has the concept of an ‘initial’ doctree which is the doctree you get when all roles and directives are disabled -useful for computing document symbols etc.

I’m assuming that the way the server is disabling roles and directives is causing an issue somewhere… what version of docutils do you have installed?

Would you also be able to share the content of your _README.rst file?

[davbeek]

If you like, I could try to temporarily give you read access to the repo of the project. I tried to temporarily make it public, but somehow that choice is unavailable in our gitlab.tue.nl repo.

docutils 0.17.1

_README.rst:

Exercises
=========

* Each question is placed in a a separate subdirectory, along
  with the required model files. The question is always in the _README.html
  file.

* The folders of essential questions are postfixed with `Essential`.

* The answer to each question can be found in the table of contents bar on the
  left under the link ``Answer`` after each question.

*
  The questions and answers can also be found in the MODULES part of the `4TC00
  website <http://4tc00.wtb.tue.nl/>`_.

.. toctree::
  :caption: Module 1: Single automata
  :numbered: 1

  01-basics/1-single-automata/1-lamp-1-Essential/_README
  01-basics/1-single-automata/2-lamp-2-Essential/_README
  01-basics/1-single-automata/3-traffic-lights-1-Essential/_README
  01-basics/1-single-automata/4-two-place-buffer/_README
  01-basics/1-single-automata/5-dining-philosophers-1/_README
  01-basics/1-single-automata/6-state-space-Essential/_README

.. toctree::
  :caption: Module 1: Networks of automata 1
  :numbered: 1

  01-basics/2-networks-of-automata-1/1-button-lamp-Essential/_README
  01-basics/2-networks-of-automata-1/2-traffic-lights-2-Essential/_README
  01-basics/2-networks-of-automata-1/3-state-space-2-Essential/_README
  01-basics/2-networks-of-automata-1/4-state-space-3-Essential/_README

.. toctree::
  :caption: Module 1: Shorter notations
  :numbered: 1

  01-basics/3-shorter-notation/1-long-model-Essential/_README
  01-basics/3-shorter-notation/2-two-place-buffer-2/_README

.. toctree::
  :caption: Module 1: Networks of automata 2
  :numbered: 1

  01-basics/4-networks-of-automata-2/1-machine-robot-machine-Essential/_README
  01-basics/4-networks-of-automata-2/2-machines-buffer-test-unit/_README
  01-basics/4-networks-of-automata-2/3-dining-philosophers-2/_README


.. toctree::
  :caption: Module 2: Types and values 1
  :numbered: 1

  02-types-values-1/1-operations-Essential/_README
  02-types-values-1/2-integers-and-reals-Essential/_README


.. toctree::
  :caption: Module 2: Data
  :numbered: 1

  03-data/1-referee-1/_README
  03-data/2-parent-and-child/_README
  03-data/3-manufacturing-process-Essential/_README
  03-data/4-vending-machine-Essential/_README
  03-data/5-buffer-black-white-Essential/_README
  03-data/6-three-servers-1/_README

.. toctree::
  :caption: Module 3: Reuse 1
  :numbered: 1

  04-scalable-reuse-1/1-referee-2/_README
  04-scalable-reuse-1/2-height-check-Essential/_README
  04-scalable-reuse-1/3-bouncing-ball-1/_README


.. toctree::
  :caption: Module 3: Time
  :numbered: 1

  05-time/01-coffee-machine-Essential/_README
  05-time/02-inkscape-Essential/_README
  05-time/03-lamp-Essential/_README
  05-time/04-conveyer-Essential/_README
  05-time/05-pusher-Essential/_README
  05-time/06-pusher-with-visualisation-Essential/_README
  05-time/07-pusher-with-product/_README
  05-time/08-bouncing-ball-2/_README
  05-time/09-race/_README
  05-time/10-dancebot/_README
  05-time/11-elevator/_README

.. toctree::
  :caption: Module 4: MBSE
  :numbered: 1

  05a-MBSE/1-double-sliding-doors/_README
  05a-MBSE/3-heater-window-system/_README

.. 05a-MBSE/2-double-sliding-doors-DT/_README

.. toctree::
  :caption: Module 5: Channels
  :numbered: 1

  06-channels/1-classroom-Essential/_README
  06-channels/2-dining-philosophers-3/_README
  06-channels/3-conveyer-pusher-product/_README
  06-channels/4-atm-and-bank-Essential/_README
  06-channels/5-dining-philosophers-4/_README

.. toctree::
  :caption: Module 5: Types and Values 2
  :numbered: 1

  07-types-values-2/1-generalized-fifo-buffer-Essential/_README
  07-types-values-2/2-two-buffers-1-Essential/_README
  07-types-values-2/3-two-buffers-2/_README

.. toctree::
  :caption: Module 5: Reuse 2
  :numbered: 1

  08-scalable-reuse-2/1-manufacturing-process-2-Essential/_README
  08-scalable-reuse-2/2-three-servers-3/_README
  08-scalable-reuse-2/3-frogger/_README

.. toctree::
  :caption: Module 6: Functions and Stochastics
  :numbered: 1

  09-func-stochastics/1-priority-queue-Essential/_README
  09-func-stochastics/2-fortune-wheel/_README

[alcarney]

Thanks, it looks like the server is not handling default roles correctly, should be easy enough to fix :)

[davbeek]

That was quick! Thanks, looking forward for the update to test.

[alcarney]

v0.13.1 is now out! Feel free to re-open this if it doesn't resolve the issue

[davbeek]

I cannot find it on conda yet. Latest version there is 0.13.0. Have you released it on conda (conda-forge)?

I do see it on PyPi, but I would rather not mix conda installed packages with pip installs.

Update 1:
I just noted on my Windows system, that VSCode automatically executed the "Update Language Server Task", which showed in my VSCode terminal: > Executing task: C:\Users|Bert\miniconda3\python -m pip install --upgrade esbonio>=0.12.0 <. It uninstalled Esbonio 0.13.0 and installed 0.13.1. Conda then lists esbonio as 0.13.1 coming from pypi.

My Mac system does not automatically do this. Closing and reopening VSCode and starting esbonio does not help.

Update 2:
I just noticed that after opening a second VSCode window on another repo on my Mac, the exact same task "Update Language Server Task" started and updated esbonio to 0.13.1. Strange ....

Anyway, what counts is that I see no more errors in the esbonio output and the "Go to Definition" lookup also works again!

Update 3 (15:15):
I ran into some problems on Windows and Mac related to pip/conda. I noticed that by now esbonio 0.13.1 was available on conda. There was probably some delay somewhere in updating the conda-forge info. I uninstalled esbonio from pip and installed esbonio (0.13.1) using conda. Everything works like a charm again.

[alcarney]

I cannot find it on conda yet. Latest version there is 0.13.0. Have you released it on conda (conda-forge)?

Yep my bad, I always forget that there’s a lag between PyPi and conda-forge 😅

It uninstalled Esbonio 0.13.0 and installed 0.13.1. Conda then lists esbonio as 0.13.1 coming from pypi.

Hmm not ideal… the VSCode extension currently doesn’t know about conda so it’s going to use the PyPi version everytime. There is an option esbonio.server.updateFrequency though which you can set to never if you want to manage updates yourself and prevent the conda version from being overwritten in the future.

Everything works like a charm again.

I’m glad it all worked out in the end!

[davbeek]

Thanks for the options the tip. Good to know!