Client Black Formatter: connection to server is erroring.

[Il-Colombo]

I am not able to make the black formatter start! I switched between different versions, I modified the values of the preferred formatters in the settings, I specified the path of the interpreter but nothing seems to work.

I am currently using Python 3.11.64, VS code tells me that Black Formatter cannot format Python files. The Black formatter output console tells me:
2023-07-31 18:07:53.849 [info] Interpreter from setting black-formatter.interpreter: C:\Users\user\AppData\Local\Programs\Python\Python311\python.exe
2023-07-31 18:07:53.849 [info] Interpreter from setting black-formatter.interpreter: C:\Users\user\AppData\Local\Programs\Python\Python311\python.exe
2023-07-31 18:07:53.849 [info] Server run command: C:\Users\user\AppData\Local\Programs\Python\Python311\python.exe c:\Users\user.vscode\extensions\ms-python.black-formatter-2023.4.1\bundled\tool\lsp_server.py
2023-07-31 18:07:53.849 [info] Server: Start requested.
2023-07-31 18:07:53.850 [info] [Error - 6:07:53 PM] Client Black Formatter: connection to server is erroring. Shutting down server.
2023-07-31 18:07:53.850 [info] [Error - 6:07:53 PM] Stopping server failed
2023-07-31 18:07:53.850 [info] Error: Client is not running and can't be stopped. It's current state is: starting

Is there anything else I can do ?

[karthiknadig]

@Il-Colombo Can you try running this from a terminal and see what is happening?
C:\Users\user\AppData\Local\Programs\Python\Python311\python.exe c:\Users\user.vscode\extensions\ms-python.black-formatter-2023.4.1\bundled\tool\lsp_server.py

If there is any issue you should see an error, otherwise it will be blank and waiting. You should be able to terminate it with ctrl+c

[Il-Colombo]

It does not give an error and I can terminate it with ctrl+c

[karthiknadig]

Can you add "black-formatter.trace.server" : "verbose" to your user settings? and also set the logging level for the extension to Trace using the "Developer: Set log level" command from the command pallet and selecting "Black Formatter" from the extension list. Click on the double tick mark icon to make the setting persistent.

Reload, and see if there is any additional information in the logs.

[Il-Colombo]

Nope, nothing new pops up. However the verbose option seems to not be taken into consideration.

[karthiknadig]

Do you see anything in debug console in developer tools Help > Toggle Developer Tools ?

That setting is a hidden setting, so it will look dimmed in the editor. But in the logs Output > Black Formatter, there should be few more details if the server could start up to a point where something could be extracted from the error.

[karthiknadig]

Can you run chcp from a terminal and share the output? Also, can you share your environment variables?

[Il-Colombo]

Here is the output of the console:

2023-07-31 21:26:13.784 [info] Server run command: c:\Users\utente\AppData\Local\Programs\Python\Python311\python.exe c:\Users\utente.vscode\extensions\ms-python.black-formatter-2023.4.1\bundled\tool\lsp_server.py
2023-07-31 21:26:13.784 [info] Server: Start requested.
2023-07-31 21:26:13.784 [debug] Server State: Starting
2023-07-31 21:26:13.784 [info] [Trace - 9:26:13 PM] Sending request 'initialize - (0)'.
2023-07-31 21:26:13.784 [info] Params: {
"processId": 18440,
"clientInfo": {
"name": "Visual Studio Code",
"version": "1.80.2"
},
"locale": "en",
"rootPath": "c:\Users\utente\Git\project",
"rootUri": "file:///c%3A/Users/utente/Git/project",
"capabilities": {
"workspace": {
"applyEdit": true,
"workspaceEdit": {
"documentChanges": true,
"resourceOperations": [
"create",
"rename",
"delete"
],
"failureHandling": "textOnlyTransactional",
"normalizesLineEndings": true,
"changeAnnotationSupport": {
"groupsOnLabel": true
}
},
"configuration": true,
"didChangeWatchedFiles": {
"dynamicRegistration": true,
"relativePatternSupport": true
},
"symbol": {
"dynamicRegistration": true,
"symbolKind": {
"valueSet": [
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26
]
},
"tagSupport": {
"valueSet": [
1
]
},
"resolveSupport": {
"properties": [
"location.range"
]
}
},
"codeLens": {
"refreshSupport": true
},
"executeCommand": {
"dynamicRegistration": true
},
"didChangeConfiguration": {
"dynamicRegistration": true
},
"workspaceFolders": true,
"semanticTokens": {
"refreshSupport": true
},
"fileOperations": {
"dynamicRegistration": true,
"didCreate": true,
"didRename": true,
"didDelete": true,
"willCreate": true,
"willRename": true,
"willDelete": true
},
"inlineValue": {
"refreshSupport": true
},
"inlayHint": {
"refreshSupport": true
},
"diagnostics": {
"refreshSupport": true
}
},
"textDocument": {
"publishDiagnostics": {
"relatedInformation": true,
"versionSupport": false,
"tagSupport": {
"valueSet": [
1,
2
]
},
"codeDescriptionSupport": true,
"dataSupport": true
},
"synchronization": {
"dynamicRegistration": true,
"willSave": true,
"willSaveWaitUntil": true,
"didSave": true
},
"completion": {
"dynamicRegistration": true,
"contextSupport": true,
"completionItem": {
"snippetSupport": true,
"commitCharactersSupport": true,
"documentationFormat": [
"markdown",
"plaintext"
],
"deprecatedSupport": true,
"preselectSupport": true,
"tagSupport": {
"valueSet": [
1
]
},
"insertReplaceSupport": true,
"resolveSupport": {
"properties": [
"documentation",
"detail",
"additionalTextEdits"
]
},
"insertTextModeSupport": {
"valueSet": [
1,
2
]
},
"labelDetailsSupport": true
},
"insertTextMode": 2,
"completionItemKind": {
"valueSet": [
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25
]
},
"completionList": {
"itemDefaults": [
"commitCharacters",
"editRange",
"insertTextFormat",
"insertTextMode"
]
}
},
"hover": {
"dynamicRegistration": true,
"contentFormat": [
"markdown",
"plaintext"
]
},
"signatureHelp": {
"dynamicRegistration": true,
"signatureInformation": {
"documentationFormat": [
"markdown",
"plaintext"
],
"parameterInformation": {
"labelOffsetSupport": true
},
"activeParameterSupport": true
},
"contextSupport": true
},
"definition": {
"dynamicRegistration": true,
"linkSupport": true
},
"references": {
"dynamicRegistration": true
},
"documentHighlight": {
"dynamicRegistration": true
},
"documentSymbol": {
"dynamicRegistration": true,
"symbolKind": {
"valueSet": [
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26
]
},
"hierarchicalDocumentSymbolSupport": true,
"tagSupport": {
"valueSet": [
1
]
},
"labelSupport": true
},
"codeAction": {
"dynamicRegistration": true,
"isPreferredSupport": true,
"disabledSupport": true,
"dataSupport": true,
"resolveSupport": {
"properties": [
"edit"
]
},
"codeActionLiteralSupport": {
"codeActionKind": {
"valueSet": [
"",
"quickfix",
"refactor",
"refactor.extract",
"refactor.inline",
"refactor.rewrite",
"source",
"source.organizeImports"
]
}
},
"honorsChangeAnnotations": false
},
"codeLens": {
"dynamicRegistration": true
},
"formatting": {
"dynamicRegistration": true
},
"rangeFormatting": {
"dynamicRegistration": true
},
"onTypeFormatting": {
"dynamicRegistration": true
},
"rename": {
"dynamicRegistration": true,
"prepareSupport": true,
"prepareSupportDefaultBehavior": 1,
"honorsChangeAnnotations": true
},
"documentLink": {
"dynamicRegistration": true,
"tooltipSupport": true
},
"typeDefinition": {
"dynamicRegistration": true,
"linkSupport": true
},
"implementation": {
"dynamicRegistration": true,
"linkSupport": true
},
"colorProvider": {
"dynamicRegistration": true
},
"foldingRange": {
"dynamicRegistration": true,
"rangeLimit": 5000,
"lineFoldingOnly": true,
"foldingRangeKind": {
"valueSet": [
"comment",
"imports",
"region"
]
},
"foldingRange": {
"collapsedText": false
}
},
"declaration": {
"dynamicRegistration": true,
"linkSupport": true
},
"selectionRange": {
"dynamicRegistration": true
},
"callHierarchy": {
"dynamicRegistration": true
},
"semanticTokens": {
"dynamicRegistration": true,
"tokenTypes": [
"namespace",
"type",
"class",
"enum",
"interface",
"struct",
"typeParameter",
"parameter",
"variable",
"property",
"enumMember",
"event",
"function",
"method",
"macro",
"keyword",
"modifier",
"comment",
"string",
"number",
"regexp",
"operator",
"decorator"
],
"tokenModifiers": [
"declaration",
"definition",
"readonly",
"static",
"deprecated",
"abstract",
"async",
"modification",
"documentation",
"defaultLibrary"
],
"formats": [
"relative"
],
"requests": {
"range": true,
"full": {
"delta": true
}
},
"multilineTokenSupport": false,
"overlappingTokenSupport": false,
"serverCancelSupport": true,
"augmentsSyntaxTokens": true
},
"linkedEditingRange": {
"dynamicRegistration": true
},
"typeHierarchy": {
"dynamicRegistration": true
},
"inlineValue": {
"dynamicRegistration": true
},
"inlayHint": {
"dynamicRegistration": true,
"resolveSupport": {
"properties": [
"tooltip",
"textEdits",
"label.tooltip",
"label.location",
"label.command"
]
}
},
"diagnostic": {
"dynamicRegistration": true,
"relatedDocumentSupport": false
}
},
"window": {
"showMessage": {
"messageActionItem": {
"additionalPropertiesSupport": true
}
},
"showDocument": {
"support": true
},
"workDoneProgress": true
},
"general": {
"staleRequestSupport": {
"cancel": true,
"retryOnContentModified": [
"textDocument/semanticTokens/full",
"textDocument/semanticTokens/range",
"textDocument/semanticTokens/full/delta"
]
},
"regularExpressions": {
"engine": "ECMAScript",
"version": "ES2020"
},
"markdown": {
"parser": "marked",
"version": "1.1.0"
},
"positionEncodings": [
"utf-16"
]
},
"notebookDocument": {
"synchronization": {
"dynamicRegistration": true,
"executionSummarySupport": true
}
}
},
"initializationOptions": {
"settings": [
{
"cwd": "c:\Users\utente\Git\project",
"workspace": "file:///c%3A/Users/utente/Git/project",
"args": [],
"path": [],
"interpreter": [
"c:\Users\utente\AppData\Local\Programs\Python\Python311\python.exe"
],
"importStrategy": "fromEnvironment",
"showNotifications": "onWarning"
}
],
"globalSettings": {
"cwd": "C:\Users\utente\AppData\Local\Programs\Microsoft VS Code",
"workspace": "C:\Users\utente\AppData\Local\Programs\Microsoft VS Code",
"args": [],
"path": [],
"interpreter": [],
"importStrategy": "fromEnvironment",
"showNotifications": "onWarning"
}
},
"trace": "verbose",
"workspaceFolders": [
{
"uri": "file:///c%3A/Users/utente/Git/project",
"name": "project"
}
]
}

2023-07-31 21:26:13.784 [info] [Error - 9:26:13 PM] Client Black Formatter: connection to server is erroring. Shutting down server.
2023-07-31 21:26:13.784 [info] [Error - 9:26:13 PM] Stopping server failed

[karthiknadig]

@Il-Colombo Can you try "black-formatter.importStrategy": "useBundled"?

[karthiknadig]

My suspicion is that it is trying to load something from the environment that is incompatible, and is causing a failure that is getting not captured correctly. I am hoping running the server with this setting "black-formatter.importStrategy": "useBundled" might get it to work right.

[Il-Colombo]

nope, still the same problem

[karthiknadig]

This is unfortunate, I don't have enough information here to diagnose.

Can you share your environment details?

C:\Users\user\AppData\Local\Programs\Python\Python311\python.exe --version
C:\Users\user\AppData\Local\Programs\Python\Python311\python.exe -m pip list

/cc @dbaeumer Is there any way to know why this error occurs connection to server is erroring. Shutting down server.

[Il-Colombo]

Version 3.11.64
black 23.7.0
certifi 2023.7.22
charset-normalizer 3.2.0
click 8.1.6
colorama 0.4.6
coverage 6.5.0
coveralls 3.3.1
docopt 0.6.2
flake8 6.0.0
idna 3.4
iniconfig 2.0.0
mccabe 0.7.0
mypy-extensions 1.0.0
numpy 1.25.2
packaging 23.1
pandas 2.0.3
pathspec 0.11.2
pip 23.1.2
platformdirs 3.10.0
pluggy 1.2.0
pycodestyle 2.10.0
pyflakes 3.0.1
pytest 7.4.0
python-dateutil 2.8.2
pytz 2023.3
requests 2.31.0
setuptools 65.5.0
six 1.16.0
tzdata 2023.3
urllib3 2.0.4

[karthiknadig]

@Il-Colombo Where did you install the python from? The version seems weird, the latest version from python.org is 3.11.4: https://www.python.org/downloads/release/python-3114/

[Il-Colombo]

sorry mine is 3.11.4 (64 bit)

[karthiknadig]

Do you see anything under this?

Can you run chcp from a terminal and share the output?

Also your environment variables?

[Il-Colombo]

I don't see any eager

chcp -> 437

Name                           Value
----                           -----
__PSLockDownPolicy             0
ALLUSERSPROFILE                C:\ProgramData
APPDATA                        C:\Users\Utente\AppData\Roaming
CHROME_CRASHPAD_PIPE_NAME      \\.\pipe\LOCAL\crashpad_34016_YSGJWTVEUARINKNJ
CLASSPATH                      .;C:\Program Files\Java\jdk-13.0.2\lib;C:\Program Files\Java\jdk-13.0.2\jre\lib
COLORTERM                      truecolor
CommonProgramFiles             C:\Program Files\Common Files
CommonProgramFiles(x86)        C:\Program Files (x86)\Common Files
CommonProgramW6432             C:\Program Files\Common Files
COMPUTERNAME                   NIDOMSI
ComSpec                        C:\WINDOWS\system32\cmd.exe
configsetroot                  C:\WINDOWS\ConfigSetRoot
DriverData                     C:\Windows\System32\Drivers\DriverData
GIT_ASKPASS                    c:\Users\Utente\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\git\dist\askpass.sh
HOMEDRIVE                      C:
HOMEPATH                       \Users\Utente
IntelliJ IDEA                  C:\Program Files\JetBrains\IntelliJ IDEA 2020.1.2\bin;
JAVA_HOME                      C:\Program Files\Java\jdk-13.0.2
JAVA_TOOL_OPTIONS              -Dfile.encoding="UTF-8"
JDK_HOME                       C:\Program Files\Java\jdk-13.0.2
LANG                           en_US.UTF-8
LOCALAPPDATA                   C:\Users\Utente\AppData\Local
NUMBER_OF_PROCESSORS           12
OneDrive                       C:\Users\Utente\OneDrive
OneDriveConsumer               C:\Users\Utente\OneDrive
ORIGINAL_XDG_CURRENT_DESKTOP   undefined
OS                             Windows_NT
Path                           C:\Program Files (x86)\VMware\VMware Player\bin\;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\C... 
PATHEXT                        .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.CPL
PROCESSOR_ARCHITECTURE         AMD64
PROCESSOR_IDENTIFIER           Intel64 Family 6 Model 158 Stepping 10, GenuineIntel
PROCESSOR_LEVEL                6
PROCESSOR_REVISION             9e0a
ProgramData                    C:\ProgramData
ProgramFiles                   C:\Program Files
ProgramFiles(x86)              C:\Program Files (x86)
ProgramW6432                   C:\Program Files
PSModulePath                   C:\Program Files (x86)\Wiechecki.it\PowerShell\Modules\;C:\Program Files\WindowsPowerShell\Modules;C:\WINDOWS\system32\Windo...
PUBLIC                         C:\Users\Public
PyCharm                        C:\Program Files\JetBrains\PyCharm 2020.1.2\bin;
SESSIONNAME                    Console
SystemDrive                    C:
SystemRoot                     C:\WINDOWS
TEMP                           C:\Users\Utente\AppData\Local\Temp
TERM_PROGRAM                   vscode
TERM_PROGRAM_VERSION           1.80.2
TMP                            C:\Users\Utente\AppData\Local\Temp
USERDOMAIN                     NIDOMSI
USERDOMAIN_ROAMINGPROFILE      NIDOMSI
USERNAME                       Utente
USERPROFILE                    C:\Users\Utente
VBOX_MSI_INSTALL_PATH          C:\Program Files\Oracle\VirtualBox\
VSCODE_GIT_ASKPASS_EXTRA_ARGS  --ms-enable-electron-run-as-node
VSCODE_GIT_ASKPASS_MAIN        c:\Users\Utente\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\git\dist\askpass-main.js
VSCODE_GIT_ASKPASS_NODE        C:\Users\Utente\AppData\Local\Programs\Microsoft VS Code\Code.exe
VSCODE_GIT_IPC_HANDLE          \\.\pipe\vscode-git-cf0da6c614-sock
VSCODE_INJECTION               1
windir                         C:\WINDOWS
ZES_ENABLE_SYSMAN              1

[karthiknadig]

Do you see anything under this?

@Il-Colombo I needed the output from Extension Host. From Output > Extension Host.

[Il-Colombo]

[karthiknadig]

@dbaeumer I have a seen a few reports of this type of issue. In some cases, for whatever reason, spawning the process manually and handing the stdio streams to LSP client works better than having LSP client launch it. Looking at the LSP client code I don't see what is going wrong, it seems to launch it normally as one would using node spawn. Is there any way we can extract details on why the stream is erroring? Like did the process exit, if so what was the exit code, or something was written to the stream that it should not have, etc.

[dbaeumer]

@karthiknadig this most of the time happens if the process can be started and then an exception occurs in the server process before it reached the code reading messages (in node for example a require statement fails).

I currently don't print this to the LOG but I should. I opened microsoft/vscode-languageserver-node#1313

Feel free to provide a PR. Code will go into https:/microsoft/vscode-languageserver-node/blob/f2ff7d55464a1f58f978cb6635bd8865f050553c/client/src/node/main.ts#L1

[lyndseyjw]

I am still experiencing this issue & am receiving error code 32097 .. is there currently a known solution?

[karthiknadig]

@lyndseyjw A solution you can try is to point the black-formatter.interpreter to run using python that is not the one in your environment. This error occurs when some package in the environment loads on python load and writes to stdout. We are working on solution for this that avoids working over stdout.

Here is what I recommend you try:
Set black-formatter.importStrategy as useBundled. Set black-formatter.interpreter to python from your system. On windows you can set it to a python.exe or ["py", "-3"]. On mac/linux you can set it to /usr/bin/python3. See if this works, if not we'll need to dig further on what is causing a write to stdout.

[lyndseyjw]

hi @karthiknadig ! Adding both "black-formatter.importStrategy": "useBundled" & "black-formatter.interpreter": ["/usr/bin/python3"] to my settings.json file worked for me, thank you so much!

[dcgudeman]

I have tried everything in this thread and nothing seems to work. This has really disrupted my workflow.

[karthiknadig]

@dcgudeman You can pin the ms-python.python extension to 2023.18.0 or older to use the old mechanism old formatting. You can do that by selecting "Install another version" option from the extensions view.

As for using this extension, If you have tried the suggestion here #280 (comment) then you need to check if any of the packages installed in the interpreter chosen to run the black server are writing to stdout. Avoid installing packages into the system python, as it can break normal operation of not only some of your projects but can break some system processes as well.

[dcgudeman]

That did fix my formatting issue but isort is still not working. Overall I am concerned with how this extension is proceeding. I don't think I have an unusual setup. Just using poetry to manage dependencies and activate the python sell to run code locally. I think these changes require a lot more QA. I spent multiple hours trying to figure out what broke. At this point I don't know how I will be productive in a python environment while using vscode. These changes have made the experience completely unbearable.

[karthiknadig]

@dcgudeman The issue stems from how VSCode and python communicate. We are working on an alternate mode of communication that does not rely on stdout. Some packages do the wrong thing of writing to stdout on load, this is not common. Typically, nothing should write to stdout on import.

You can apply the same fix to isort extension as well. isort.interpreter and isort.importStrategy should address this.

There is another alternative. You can use Ruff extension, apply this fix to it once, and ruff can do import sorting similar to isort, formatting similar to black, and it implements multiple linters (pylint, flake8, pycodestyle, pydocstyle, etc). It is written in rust so it is incredibly fast.

[dcgudeman]

@karthiknadig what is the status of this issue? Has a version been released with a fix?

[lhmrlzh]

A possible reason is that, you choose an environment that is not compatible with black formatter, such as pydoc3.10 or so