Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Critical AttributeError: 'bool' object has no attribute 'lower' when request_id_header is set to a boolean #14482

Closed
wmwsagara opened this issue Nov 18, 2022 · 9 comments
Closed

Critical AttributeError: 'bool' object has no attribute 'lower' when request_id_header is set to a boolean #14482

wmwsagara opened this issue Nov 18, 2022 · 9 comments
Labels
X-Needs-Info This issue is blocked awaiting information from the reporter

Comments

@wmwsagara
Copy link

Description

I installed Synapse 1.71.0 on Ubuntu 22.04.1 LTS.

When try to connect using curl for test, develops TLS errors as follows:
curl -ivkL -X POST -d '{"type":"m.login.password", "user":"sagara", "password":"mypass"}' "https://192.168.1.110:8448/_matrix/client/r0/login"
curl: (56) OpenSSL SSL_read: error:0A000126:SSL routines::unexpected eof while reading, errno 0

The openssl on Ubuntu 22.04.1 LTS:
apt list --installed | grep openssl
openssl/jammy-updates,jammy-security,now 3.0.2-0ubuntu1.7 amd64 [installed]

I installed Synapse 1.71.0 on FreeBSD 13.1.
Above curl statement works without an issue.

$ openssl version
OpenSSL 1.1.1o-freebsd 3 May 2022

The issue seems to be very much that Synapse 1.71.0 is not compatible with openssl 3.0.2. Probably require openssl 3 specific settings on Synapse.

Appreciate this issue on Ubuntu could be fixed as early as possible.

Steps to reproduce

  • list the steps
  • that reproduce the bug
  • using hyphens as bullet points

When try to connect using curl for test, develops TLS errors as follows:
curl -ivkL -X POST -d '{"type":"m.login.password", "user":"sagara", "password":"mypass"}' "https://192.168.1.110:8448/_matrix/client/r0/login"

  • Connected to 192.168.1.110 (192.168.1.110) port 8448 (#0)

  • SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384

  • TLSv1.2 (OUT), TLS header, Supplemental data (23):

  • TLSv1.3 (OUT), TLS alert, decode error (562):

  • OpenSSL SSL_read: error:0A000126:SSL routines::unexpected eof while reading, errno 0

  • Closing connection 0
    curl: (56) OpenSSL SSL_read: error:0A000126:SSL routines::unexpected eof while reading, errno 0

Homeserver

On our locally hosted homeserver.

Synapse Version

Synapse 1.71.0

Installation Method

Debian packages from packages.matrix.org

Platform

Ubuntu 22.04.1 LTS. Linux 5.15.0-52-generic. x86_64 GNU/Linux.

Relevant log output

* OpenSSL SSL_read: error:0A000126:SSL routines::unexpected eof while reading, errno 0

Anything else that would be useful to know?

Synapse 1.71.0 works well on FreeBSD with openssl 1.1.1o.
@DMRobertson
Copy link
Contributor

Please provide Synapse logs to help us debug your problem.

@DMRobertson DMRobertson added the X-Needs-Info This issue is blocked awaiting information from the reporter label Nov 18, 2022
@wmwsagara
Copy link
Author

wmwsagara commented Nov 18, 2022 •
edited by clokep
Loading 

2022-11-11 19:13:40,380 - twisted - 274 - CRITICAL - sentinel -
Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/server.py", line 227, in process
    self.render(resrc)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/http/site.py", line 255, in render
    request_id = self.get_request_id()
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/http/site.py", line 180, in get_request_id
    request_id_value = self.getHeader(self.request_id_header)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/http.py", line 1095, in getHeader
    value = self.requestHeaders.getRawHeaders(key)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/http_headers.py", line 251, in getRawHeaders
    encodedName = self._encodeName(name)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/http_headers.py", line 125, in _encodeName
    return name.lower()
AttributeError: 'bool' object has no attribute 'lower'


2022-11-11 19:13:40,385 - twisted - 274 - CRITICAL - sentinel - Unhandled Error
Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/python/log.py", line 96, in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/python/log.py", line 80, in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/python/context.py", line 117, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/python/context.py", line 82, in callWithContext
    return func(*args, **kw)
--- <exception caught here> ---
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/internet/posixbase.py", line 683, in _doReadOrWrite
    why = selectable.doRead()
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/internet/tcp.py", line 248, in doRead
    return self._dataReceived(data)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/internet/tcp.py", line 253, in _dataReceived
    rval = self.protocol.dataReceived(data)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/protocols/tls.py", line 329, in dataReceived
    self._flushReceiveBIO()
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/protocols/tls.py", line 295, in _flushReceiveBIO
    ProtocolWrapper.dataReceived(self, bytes)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/protocols/policies.py", line 110, in dataReceived
    self.wrappedProtocol.dataReceived(data)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/http.py", line 3151, in dataReceived
    return self._channel.dataReceived(data)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/http.py", line 2465, in dataReceived
    return basic.LineReceiver.dataReceived(self, data)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/protocols/basic.py", line 555, in dataReceived
    why = self.rawDataReceived(data)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/http.py", line 2471, in rawDataReceived
    self._transferDecoder.dataReceived(data)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/http.py", line 1797, in dataReceived
    finishCallback(data[contentLength:])
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/http.py", line 2334, in _finishRequestBody
    self.allContentReceived()
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/http.py", line 2447, in allContentReceived
    req.requestReceived(command, path, version)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/http/site.py", line 521, in requestReceived
    return super().requestReceived(command, path, version)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/http.py", line 1032, in requestReceived
    self.process()
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/server.py", line 229, in process
    self.processingFailed(failure.Failure())
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/server.py", line 414, in processingFailed
    self.finish()
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/http/site.py", line 341, in finish
    assert self.logcontext is not None
builtins.AssertionError:

@squahtx
Copy link
Contributor

squahtx commented Nov 18, 2022 

  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/http/site.py", line 180, in get_request_id
    request_id_value = self.getHeader(self.request_id_header)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/http.py", line 1095, in getHeader
    value = self.requestHeaders.getRawHeaders(key)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/http_headers.py", line 251, in getRawHeaders
    encodedName = self._encodeName(name)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/twisted/web/http_headers.py", line 125, in _encodeName
    return name.lower()
AttributeError: 'bool' object has no attribute 'lower'

Could you post the listeners section of the homeserver config? In particular, the value of request_id_header seems to be causing the error seen here.

@wmwsagara
Copy link
Author

wmwsagara commented Nov 19, 2022 •
edited
Loading

In Ubuntu:
listeners:

  • port: 8448
    tls: true
    type: http
    x_forwarded: true
    request_id_header: true
    bind_addresses: ['0.0.0.0']
    resources:
    • names: [client, federation]
      compress: false

I commented out the "request_id_header: true" line and restarted the Synapse server, now TLS works!

In FreeBSD config, the request_id_header is missing.

Btw, how to use the request_id_header? Is it true/false? Documentation is silent on it.

Doc: https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html

Sean, many thanks for the reply.

@DMRobertson
Copy link
Contributor

Btw, how to use the request_id_header? Is it true/false? Documentation is silent on it.

See https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html#listeners:

request_id_header: The header extracted from each incoming request that is used as the basis for the request ID. The request ID is used in logs and tracing to correlate and match up requests. When unset, Synapse will automatically generate sequential request IDs. This option is useful when Synapse is behind a reverse-proxy.

Added in Synapse 1.68.0.

@DMRobertson
Copy link
Contributor

In particular it is a header, i.e. a string, not a boolean.

We could do with better validation of the config here (#12651).

@DMRobertson DMRobertson changed the title Synapse 1.71.0 not compatible with Ubuntu 22.04 Critical AttributeError: 'bool' object has no attribute 'lower' when request_id_header is set to a boolean Nov 21, 2022
@wmwsagara
Copy link
Author

@DMRobertson Thank you for the comment on this. It best if could take an initiative to update the documentation with examples.

@nigini
Copy link

nigini commented Dec 7, 2022

@DMRobertson Could you give us an example of the recommended configuration for a reverse proxy situation (as it seems we all here having this issue are coming from that particular recommendation.)?

@DMRobertson
Copy link
Contributor

This field

  • is completely optional,
  • is for debugging purposes only, and
  • can be safely omitted.

We'd graciously welcome PRs which improve the wording to clarify this.

On matrix.org we have the config set to request_id_header: "cf-ray" so that the request IDs in logs are expressed in terms of Cloudflare's Ray IDs. Cloudflare even has examples for doing this at the reverse-proxy level rathe rthan the application level.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
X-Needs-Info This issue is blocked awaiting information from the reporter
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@nigini @squahtx @DMRobertson @wmwsagara