Cannot understand range coverage for vim.lsp.buf.code_action()

[avario-cpu]

My issue is about trying to refactor using pylsp with the rope plugin, through a code_action

I'm trying to extract await secondary_windows_spawned.wait() from:

async def run_main_task(slot: int, shop_watcher: ShopWatcher):  
    mute_ssim_prints.set()
    main_task = asyncio.create_task(shop_watcher.scan_for_shop_and_notify())
    await secondary_windows_spawned.wait() # line 33 in nvim (base index 1 for lines and columns)
    await twm.manage_secondary_windows(slot, SECONDARY_WINDOWS)  # line 34
    mute_ssim_prints.clear() 
    await main_task 
    return None

Here is range the lsp is using (from logs):
range = { ["end"] = { character = 43, line = 32 }, start = { character = 1, line = 32 } }, similar = false } }

The attempt to execute a method extraction code action over this range results in a Extracted piece should contain complete statements error.

But, what I was seemingly able to observe as the actual parsed statements in logs are:
await twm.manage_secondary_w (the beginning of line 34)
(ed.wait() (the end of line 33 with some added indentation and a curious ( replacing what would be the normally be the bridging character when concatenating wtih the previous result.)

Despite the range table cleary indicating a span over a single line, it looks like there are two partial lines sent to rope.

Neovim lsp docs mention {row,col} being passed as "tuples" ? but there are no tuples in lua ? Am I missing something there ? There seems to be a curious reversal in the appearing order of the start and end key inside the range table when received by the lsp, but since start and end are, from my understanding, keys, this shouldn't matter ?!

my call basically is:

vim.lsp.buf.code_action(
  range = {
    start = { start_row, start_col },
    ["end"] = { end_row, end_col },
  },
)

The observation of those statments being parsed with the vim.the lsp.buf.code_action() call is what I've assumed from my looking in logs, but I'm not sure. If you wish to find those values, search the below logs for "string>"

full relevant logs:

[DEBUG][2024-08-24 16:11:58] ...m/lsp/client.lua:678	"LSP[pylsp]"	"client.request"	3	"workspace/executeCommand"	{  arguments = { {      document_uri = "file:///C:/Users/ville/MyMegaScript/src/apps/shop_watcher/shop_watcher_main.py",      global_ = false,      range = {        ["end"] = {          character = 43,          line = 32        },        start = {          character = 1,          line = 32        }      },      similar = false    } },  command = "pylsp_rope.refactor.extract.method"}	<function 1>	11
[DEBUG][2024-08-24 16:11:58] .../vim/lsp/rpc.lua:286	"rpc.send"	{  id = 6,  jsonrpc = "2.0",  method = "workspace/executeCommand",  params = {    arguments = { {        document_uri = "file:///C:/Users/ville/MyMegaScript/src/apps/shop_watcher/shop_watcher_main.py",        global_ = false,        range = {          ["end"] = {            character = 43,            line = 32          },          start = {            character = 1,            line = 32          }        },        similar = false      } },    command = "pylsp_rope.refactor.extract.method"  }}
[ERROR][2024-08-24 16:11:58] .../vim/lsp/rpc.lua:770	"rpc"	"C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\bin\\pylsp.CMD"	"stderr"	'2024-08-24 16:11:58,138 W. Europe Daylight Time - ERROR - pylsp_rope.plugin - Exception when doing workspace/executeCommand: Extracted piece should contain complete statements.\r\nTraceback (most recent call last):\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 1157, in _parse_text\r\n    node = ast.parse(body)\r\n           ^^^^^^^^^^^^^^^\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\base\\ast.py", line 33, in parse\r\n    return ast.parse(source, filename=filename, *args, **kwargs)\r\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File "C:\\Users\\ville\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\ast.py", line 52, in parse\r\n    return compile(source, filename, mode, flags,\r\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File "<string>", line 2\r\n    await twm.manage_secondary_w\r\nIndentationError: unexpected indent\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 1161, in _parse_text\r\n    node = ast.parse("(" + body + ")")\r\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\base\\ast.py", line 33, in parse\r\n    return ast.parse(source, filename=filename, *args, **kwargs)\r\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File "C:\\Users\\ville\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\ast.py", line 52, in parse\r\n    return compile(source, filename, mode, flags,\r\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File "<string>", line 1\r\n    (ed.wait()\r\n     ^^^^^^^^^\r\nSyntaxError: invalid syntax. Perhaps you forgot a comma?\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 457, in base_conditions\r\n    if _UnmatchedBreakOrContinueFinder.has_errors(extracted):\r\n       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 1043, in has_errors\r\n    node = _parse_text(code)\r\n           ^^^^^^^^^^^^^^^^^\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 1163, in _parse_text\r\n    node = ast.parse(\r\n           ^^^^^^^^^^\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\base\\ast.py", line 33, in parse\r\n    return ast.parse(source, filename=filename, *args, **kwargs)\r\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File "C:\\Users\\ville\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\ast.py", line 52, in parse\r\n    return compile(source, filename, mode, flags,\r\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File "<string>", line 3\r\n    await twm.manage_secondary_w\r\nIndentationError: unexpected indent\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\pylsp_rope\\plugin.py", line 158, in pylsp_execute_command\r\n    return commands[command](workspace, **arguments[0])(\r\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\pylsp_rope\\refactoring.py", line 44, in __call__\r\n    rope_changeset = self.get_changes()\r\n                     ^^^^^^^^^^^^^^^^^^\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\pylsp_rope\\refactoring.py", line 112, in get_changes\r\n    rope_changeset = refactoring.get_changes(\r\n                     ^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 82, in get_changes\r\n    new_contents = _ExtractPerformer(info).extract()\r\n                   ^^^^^^^^^^^^^^^^^^^^^^^\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 284, in __init__\r\n    _ExceptionalConditionChecker()(self.info)\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 438, in __call__\r\n    self.base_conditions(info)\r\n  File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 462, in base_conditions\r\n    raise RefactoringError(\r\nrope.base.exceptions.RefactoringError: Extracted piece should contain complete statements.\r\n'
[DEBUG][2024-08-24 16:11:58] .../vim/lsp/rpc.lua:408	"rpc.receive"	{  jsonrpc = "2.0",  method = "window/showMessage",  params = {    message = "pylsp-rope: Extracted piece should contain complete statements.",    type = 1  }}

if you prefer the logs wrapped:

Click to expand

[DEBUG][2024-08-24 16:11:58] ...m/lsp/client.lua:678 "LSP[pylsp]" "client.request" 3 "workspace/executeCommand" { arguments = { { document_uri = "file:///C:/Users/ville/MyMegaScript/src/apps/shop_watcher/shop_watcher_main.py", global_ = false, range = { ["end"] = { character = 43, line = 32 }, start = { character = 1, line = 32 } }, similar = false } }, command = "pylsp_rope.refactor.extract.method"} 11 [DEBUG][2024-08-24 16:11:58] .../vim/lsp/rpc.lua:286 "rpc.send" { id = 6, jsonrpc = "2.0", method = "workspace/executeCommand", params = { arguments = { { document_uri = "file:///C:/Users/ville/MyMegaScript/src/apps/shop_watcher/shop_watcher_main.py", global_ = false, range = { ["end"] = { character = 43, line = 32 }, start = { character = 1, line = 32 } }, similar = false } }, command = "pylsp_rope.refactor.extract.method" }} [ERROR][2024-08-24 16:11:58] .../vim/lsp/rpc.lua:770 "rpc" "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\bin\\pylsp.CMD" "stderr" '2024-08-24 16:11:58,138 W. Europe Daylight Time - ERROR - pylsp_rope.plugin - Exception when doing workspace/executeCommand: Extracted piece should contain complete statements.\r\nTraceback (most recent call last):\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 1157, in _parse_text\r\n node = ast.parse(body)\r\n ^^^^^^^^^^^^^^^\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\base\\ast.py", line 33, in parse\r\n return ast.parse(source, filename=filename, *args, **kwargs)\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File "C:\\Users\\ville\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\ast.py", line 52, in parse\r\n return compile(source, filename, mode, flags,\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File "", line 2\r\n await twm.manage_secondary_w\r\nIndentationError: unexpected indent\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 1161, in _parse_text\r\n node = ast.parse("(" + body + ")")\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\base\\ast.py", line 33, in parse\r\n return ast.parse(source, filename=filename, *args, **kwargs)\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File "C:\\Users\\ville\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\ast.py", line 52, in parse\r\n return compile(source, filename, mode, flags,\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File "", line 1\r\n (ed.wait()\r\n ^^^^^^^^^\r\nSyntaxError: invalid syntax. Perhaps you forgot a comma?\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 457, in base_conditions\r\n if _UnmatchedBreakOrContinueFinder.has_errors(extracted):\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 1043, in has_errors\r\n node = _parse_text(code)\r\n ^^^^^^^^^^^^^^^^^\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 1163, in _parse_text\r\n node = ast.parse(\r\n ^^^^^^^^^^\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\base\\ast.py", line 33, in parse\r\n return ast.parse(source, filename=filename, *args, **kwargs)\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File "C:\\Users\\ville\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\ast.py", line 52, in parse\r\n return compile(source, filename, mode, flags,\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File "", line 3\r\n await twm.manage_secondary_w\r\nIndentationError: unexpected indent\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\pylsp_rope\\plugin.py", line 158, in pylsp_execute_command\r\n return commands[command](workspace, **arguments[0])(\r\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\pylsp_rope\\refactoring.py", line 44, in __call__\r\n rope_changeset = self.get_changes()\r\n ^^^^^^^^^^^^^^^^^^\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\pylsp_rope\\refactoring.py", line 112, in get_changes\r\n rope_changeset = refactoring.get_changes(\r\n ^^^^^^^^^^^^^^^^^^^^^^^^\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 82, in get_changes\r\n new_contents = _ExtractPerformer(info).extract()\r\n ^^^^^^^^^^^^^^^^^^^^^^^\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 284, in __init__\r\n _ExceptionalConditionChecker()(self.info)\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 438, in __call__\r\n self.base_conditions(info)\r\n File "C:\\Users\\ville\\AppData\\Local\\nvim-data\\mason\\packages\\python-lsp-server\\venv\\Lib\\site-packages\\rope\\refactor\\extract.py", line 462, in base_conditions\r\n raise RefactoringError(\r\nrope.base.exceptions.RefactoringError: Extracted piece should contain complete statements.\r\n' [DEBUG][2024-08-24 16:11:58] .../vim/lsp/rpc.lua:408 "rpc.receive" { jsonrpc = "2.0", method = "window/showMessage", params = { message = "pylsp-rope: Extracted piece should contain complete statements.", type = 1 }}

[mayank-pahuja]

Heyy, the task of investigating the issue with the vim.lsp.buf.code_action() call not properly handling code ranges for method extraction using pylsp with the rope plugin. The problem seems to be related to the range parameters not covering the complete statements, leading to syntax errors during refactoring.

Example code:-

async def run_main_task(slot: int, shop_watcher: ShopWatcher):
mute_ssim_prints.set()

# Create a new task for scanning and notification
main_task = asyncio.create_task(shop_watcher.scan_for_shop_and_notify())

# Extracted statement
await secondary_windows_spawned.wait()  # Extracted to a new method

await twm.manage_secondary_windows(slot, SECONDARY_WINDOWS)  # line 34
mute_ssim_prints.clear() 
await main_task 
return None