Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Add GCC and GNU Make to the Nix flake development environment so that ruff can be compiled. #16090

Merged
merged 3 commits into from
Sep 6, 2023

Conversation

reivilibre
Copy link
Contributor

I found that compiling/installing ruff needed cc as a linker and make.

Adding these got me away and running.

Base: develop

Original commit schedule, with full messages:

  1. Add gcc and GNU make to the Nix flake

@reivilibre
Copy link
Contributor Author

I found this is also needed for Complement, otherwise Go silently disables CGO and then silently ignores files with import "C" like the bindings to libolm.

@reivilibre
Copy link
Contributor Author

Andrew seems to have some problem with enabling GCC in the flake breaking his external binaries from the system. Not clear how we fix this...

@sumnerevans
Copy link
Contributor

I needed gcc for pyicu to install correctly. This is the error that I was getting before:

direnv: loading ~/projects/github.com/sumnerevans/beeper/github.com/beeper/synapse/.envrc
direnv: using flake . --impure
direnv: nix-direnv: renewed cache
Installing dependencies from lock file
direnv: ([/nix/store/4c3k16mbpv8idz418pwcai5in552qr4v-direnv-2.32.3/bin/direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.

Package operations: 1 install, 0 updates, 0 removals

  • Installing pyicu (2.11): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke build_wheel

  (running 'icu-config --version')

  Building PyICU 2.11 for ICU 73.2 (max ICU major version supported: 73)

  (running 'icu-config --cxxflags --cppflags')
  Adding CFLAGS="-std=c++11 -I/nix/store/db79xky20dvsvs5546ffpv5db70yjlxq-icu4c-73.2-dev/include" from /nix/store/59ly20xxpsii9yl9g2xq13by7kymkzw1-devenv-profile/bin/icu-config
  (running 'icu-config --ldflags')
  Adding LFLAGS="-L/nix/store/6i4vwgsiqp8jd8pb8cgf99sx8081rmsj-icu4c-73.2/lib -licui18n -licuuc -licudata" from /nix/store/59ly20xxpsii9yl9g2xq13by7kymkzw1-devenv-profile/bin/icu-config
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-cpython-310
  creating build/lib.linux-x86_64-cpython-310/icu
  copying py/icu/__init__.py -> build/lib.linux-x86_64-cpython-310/icu
  running build_ext
  building 'icu._icu_' extension
  creating build/temp.linux-x86_64-cpython-310
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -I/nix/store/nhnxylvmaisx7pw8pyyx3a8cq93xanpx-libxcrypt-4.4.36/include -fPIC -I/tmp/tmppbia3ezg/.venv/include -I/nix/store/jhflvwr40xbb0xr6jx4311icp9cym1fp-python3-3.10.12/include/python3.10 -c _icu_.cpp -o build/temp.linux-x86_64-cpython-310/_icu_.o -std=c++11 -std=c++11 -I/nix/store/db79xky20dvsvs5546ffpv5db70yjlxq-icu4c-73.2-dev/include -DPYICU_VER=\"2.11\" -DPYICU_ICU_MAX_VER=\"73\"
  error: command 'gcc' failed: No such file or directory


  at /nix/store/2l4y0jspbrs0ii3wh038lfffnllx3r0w-python3.10-poetry-1.5.1/lib/python3.10/site-packages/poetry/installation/chef.py:147 in _prepare
      143│
      144│                 error = ChefBuildError("\n\n".join(message_parts))
      145│
      146│             if error is not None:
    → 147│                 raise error from None
      148│
      149│             return path
      150│
      151│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with pyicu (2.11) not supporting PEP 517 builds. You can verify this by running 'pip wheel --use-pep517 "pyicu (==2.11)"'.

Poetry install failed. Run 'poetry install' manually.
direnv: export +C_INCLUDE_PATH +DEVENV_DOTFILE +DEVENV_PROFILE +DEVENV_ROOT +DEVENV_STATE +GOPATH +GOROOT +IN_NIX_SHELL +LD_LIBRARY_PATH +LIBRARY_PATH +PERL5LIB +PGDATA +PGHOST +PGPORT +PKG_CONFIG_PATH +POETRY_INSTALLER_NO_BINARY +POETRY_VIRTUALENVS_CREATE +POETRY_VIRTUALENVS_IN_PROJECT +POETRY_VIRTUALENVS_PATH +PYTHONPATH +REDISDATA +STARSHIP_SESSION_KEY +STARSHIP_SHELL +VIRTUAL_ENV +name ~PATH ~XDG_CONFIG_DIRS ~XDG_DATA_DIRS

I didn't have any errors installing ruff.

@anoadragon453
Copy link
Member

anoadragon453 commented Sep 6, 2023

Andrew seems to have some problem with enabling GCC in the flake breaking his external binaries from the system. Not clear how we fix this...

This appears to only occur if your system packages are compiled with a different glibc version what the development environment pulls in by including gcc. This is likely for the vast majority of distros, but unlikely for @reivilibre and I who are running NixOS unstable on our dev machines, which the development environment also uses. Testing in an old Ubuntu 20.04 VM makes the problem clear however.

synapse on  rei/flake_gcc_etc is 📦 v1.90.0rc1 via 🐍 v3.10.12 (matrix-synapse-py3.10) via 🦀 v1.70.0 via ❄️  impure (devenv-shell-env) 
❯ sudo chmod a+r /etc/apt/keyrings/docker.gpg ^C

 -o /etc/apt/keyrings/docker.gpgcker.com/linux/ubuntu/gpg | sudo gpg --dearmor 
curl: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/gry70n80hgvpfsgr2zihw69ffg6knnd8-devenv-profile/lib/libsqlite3.so.0)
curl: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /nix/store/gry70n80hgvpfsgr2zihw69ffg6knnd8-devenv-profile/lib/libsqlite3.so.0)
curl: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_ABI_DT_RELR' not found (required by /nix/store/1x4ijm9r1a88qk7zcmbbfza324gx1aac-glibc-2.37-8/lib/libm.so.6)
File '/etc/apt/keyrings/docker.gpg' exists. Overwrite? (y/N) y
gpg: no valid OpenPGP data found.

@reivilibre discovered that if we unset LD_LIBRARY_PATH that devenv sets, then we seem to fix the GLIBC errors with no adverse effects. For reference, devenv sets my LD_LIBRARY_PATH env var to the absolute path of .devenv/profile/lib:. That directory contains the following libraries installed by devenv:

Click to show contents of `.devenv/profile/lib`
user@pc:~/synapse$ ls .devenv/profile/lib
adminpack.so                         libssl.so.3
amcheck.so                           libstd-0d91c78a7710ed2e.so
auth_delay.so                        libtest-deccc6ea001455e8.so
auto_explain.so                      libturbojpeg.so
autoinc.so                           libturbojpeg.so.0
bloom.so                             libturbojpeg.so.0.2.0
btree_gin.so                         libwebpdecoder.la
btree_gist.so                        libwebpdecoder.so
citext.so                            libwebpdecoder.so.3
cmake                                libwebpdecoder.so.3.1.7
cube.so                              libwebpdemux.la
cyrillic_and_mic.so                  libwebpdemux.so
dblink.so                            libwebpdemux.so.2
debug                                libwebpdemux.so.2.0.13
dict_int.so                          libwebp.la
dict_snowball.so                     libwebpmux.la
dict_xsyn.so                         libwebpmux.so
earthdistance.so                     libwebpmux.so.3
engines-3                            libwebpmux.so.3.0.12
euc2004_sjis2004.so                  libwebp.so
euc_cn_and_mic.so                    libwebp.so.7
euc_jp_and_sjis.so                   libwebp.so.7.1.7
euc_kr_and_mic.so                    libxml2.la
euc_tw_and_big5.so                   libxml2.so
file_fdw.so                          libxml2.so.2
fuzzystrmatch.so                     libxml2.so.2.11.4
girepository-1.0                     libxmlsec1-gcrypt.la
hstore.so                            libxmlsec1-gcrypt.so
icu                                  libxmlsec1-gcrypt.so.1
insert_username.so                   libxmlsec1-gcrypt.so.1.2.34
_int.so                              libxmlsec1-gnutls.la
isn.so                               libxmlsec1-gnutls.so
latin2_and_win1250.so                libxmlsec1-gnutls.so.1
latin_and_mic.so                     libxmlsec1-gnutls.so.1.2.34
libcrypto.so                         libxmlsec1.la
libcrypto.so.3                       libxmlsec1-nss.la
libecpg_compat.so                    libxmlsec1-nss.so
libecpg_compat.so.3                  libxmlsec1-nss.so.1
libecpg_compat.so.3.14               libxmlsec1-nss.so.1.2.34
libecpg.so                           libxmlsec1-openssl.la
libecpg.so.6                         libxmlsec1-openssl.so
libecpg.so.6.14                      libxmlsec1-openssl.so.1
libexslt.la                          libxmlsec1-openssl.so.1.2.34
libexslt.so                          libxmlsec1.so
libexslt.so.0                        libxmlsec1.so.1
libexslt.so.0.8.21                   libxmlsec1.so.1.2.34
libffi.la                            libxslt.la
libffi.so                            libxslt-plugins
libffi.so.8                          libxslt.so
libffi.so.8.1.2                      libxslt.so.1
libicudata.so                        libxslt.so.1.1.38
libicudata.so.73                     lo.so
libicudata.so.73.2                   ltree.so
libicui18n.so                        moddatetime.so
libicui18n.so.73                     old_snapshot.so
libicui18n.so.73.2                   ossl-modules
libicuio.so                          pageinspect.so
libicuio.so.73                       passwordcheck.so
libicuio.so.73.2                     perl5
libicutest.so                        pg_buffercache.so
libicutest.so.73                     pgcrypto.so
libicutest.so.73.2                   pg_freespacemap.so
libicutu.so                          pgoutput.so
libicutu.so.73                       pg_prewarm.so
libicutu.so.73.2                     pgrowlocks.so
libicuuc.so                          pg_stat_statements.so
libicuuc.so.73                       pgstattuple.so
libicuuc.so.73.2                     pg_surgery.so
libjpeg.so                           pg_trgm.so
libjpeg.so.62                        pg_visibility.so
libjpeg.so.62.3.0                    pgxml.so
libLLVM-16-rust-1.71.0-stable.so     pgxs
libnotify.so                         pkgconfig
libnotify.so.4                       plpgsql.so
libnotify.so.4.0.0                   postgres_fdw.so
libolm.so                            python3.10
libolm.so.3                          refint.so
libolm.so.3.2.15                     rustlib
libpgcommon.a                        seg.so
libpgcommon_shlib.a                  sslinfo.so
libpgfeutils.a                       tablefunc.so
libpgport.a                          tcn.so
libpgport_shlib.a                    test_decoding.so
libpgtypes.so                        tsm_system_rows.so
libpgtypes.so.3                      tsm_system_time.so
libpgtypes.so.3.14                   unaccent.so
libpq.so                             utf8_and_big5.so
libpq.so.5                           utf8_and_cyrillic.so
libpq.so.5.14                        utf8_and_euc2004.so
libpqwalreceiver.so                  utf8_and_euc_cn.so
libpqxx-7.7.so                       utf8_and_euc_jp.so
libpqxx.la                           utf8_and_euc_kr.so
libpqxx.so                           utf8_and_euc_tw.so
libpython3.10.so                     utf8_and_gb18030.so
libpython3.10.so.1.0                 utf8_and_gbk.so
libpython3.so                        utf8_and_iso8859_1.so
librustc_driver-8196f181ccc4f9f6.so  utf8_and_iso8859.so
libsharpyuv.la                       utf8_and_johab.so
libsharpyuv.so                       utf8_and_sjis2004.so
libsharpyuv.so.0                     utf8_and_sjis.so
libsharpyuv.so.0.0.1                 utf8_and_uhc.so
libsqlite3.la                        utf8_and_win.so
libsqlite3.so                        uuid-ossp.so
libsqlite3.so.0                      xmlsec1Conf.sh
libsqlite3.so.0.8.6                  xsltConf.sh
libssl.so

The theory is that if LD_LIBRARY_PATH points to libraries that are built with the pulled in gcc, such as libsqlite3.so.0, then we're effectively injecting separate and unexpected versions of libraries when running system tools. Those tools were compiled with a different glibc version, so we see version 'GLIBC_X.YY' not found errors. Also see this upstream devenv issue: cachix/devenv#555

As a workaround, we can add the following enterShell directive to flake.nix:

# Clear the LD_LIBRARY_PATH environment variable on shell init.
#
# By default, devenv will set LD_LIBRARY_PATH to point to .devenv/profile/lib. This causes
# issues when we include `gcc` as a dependency to build C libraries, as the version of glibc
# that the development environment's cc compiler uses may differ from that of the system.
#
# When LD_LIBRARY_PATH is set, system tools will attempt to use the development environment's
# libraries. Which, when built against an different glibc version lead, to "version 'GLIBC_X.YY' not
# found" errors.
enterShell = ''
    unset LD_LIBRARY_PATH
'';

which stops the injection. Surprisingly, the tools that do need these libraries (sqlite3) still work fine in the development environment. I assume they're finding the built libraries using a different method (pkg-config perhaps).

@reivilibre
Copy link
Contributor Author

@reivilibre discovered that if we unset LD_LIBRARY_PATH that devenv sets

(it's because I only run NixOS stable and the flake is usually based on some random commit from master that I had to do this, fwiw. It hasn't been causing me any trouble yet as far as I've seen!)

@reivilibre
Copy link
Contributor Author

aahhhh, seems there's something called an RPATH (runtime library search path) embedded in a binary and that the Nix-built binaries usually point to their libraries in there, but LD_LIBRARY_PATH overrides that and that's why unsetting it fixes it.

NixOS/nixpkgs#327854

@reivilibre reivilibre marked this pull request as ready for review September 6, 2023 10:29
@reivilibre reivilibre requested a review from a team as a code owner September 6, 2023 10:29
Copy link
Member

@anoadragon453 anoadragon453 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm other than a couple tiny nits. Thank you for doing this :)

@@ -89,6 +89,10 @@
})
# The rust-analyzer language server implementation.
rust-analyzer
# GCC includes a linker; needed for building `ruff`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# GCC includes a linker; needed for building `ruff`
# GCC includes a linker; needed for building `ruff`

I've been trying to separate distinct sections with a space :)

Comment on lines +251 to +252
# libraries. Which, when built against an different glibc version lead, to "version 'GLIBC_X.YY' not
# found" errors.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correcting past me.

Suggested change
# libraries. Which, when built against an different glibc version lead, to "version 'GLIBC_X.YY' not
# found" errors.
# libraries. Which, when built against a different glibc version lead, to "version 'GLIBC_X.YY'
# not found" errors.

@reivilibre reivilibre merged commit 35934b0 into develop Sep 6, 2023
37 checks passed
@reivilibre reivilibre deleted the rei/flake_gcc_etc branch September 6, 2023 13:35
@anoadragon453
Copy link
Member

@reivilibre did you mean to skip the suggestions?

@reivilibre
Copy link
Contributor Author

nope, that was a mistake :S

@reivilibre reivilibre restored the rei/flake_gcc_etc branch September 6, 2023 14:05
reivilibre added a commit that referenced this pull request Sep 6, 2023
* Suggestions from PR

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <[email protected]>

---------

Signed-off-by: Olivier Wilkinson (reivilibre) <[email protected]>
yingziwu added a commit to yingziwu/synapse that referenced this pull request Oct 3, 2023
No significant changes since 1.93.0rc1.

The following issues are fixed in 1.93.0 (and RCs).

- [GHSA-4f74-84v3-j9q5](GHSA-4f74-84v3-j9q5) / [CVE-2023-41335](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-41335) — Low Severity

  Temporary storage of plaintext passwords during password changes.

- [GHSA-7565-cq32-vx2x](GHSA-7565-cq32-vx2x) / [CVE-2023-42453](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-42453) — Low Severity

  Improper validation of receipts allows forged read receipts.

See the advisories for more details. If you have any questions, email [email protected].

- Add automatic purge after all users have forgotten a room. ([\matrix-org#15488](matrix-org#15488))
- Restore room purge/shutdown after a Synapse restart. ([\matrix-org#15488](matrix-org#15488))
- Support resolving homeservers using `matrix-fed` DNS SRV records from [MSC4040](matrix-org/matrix-spec-proposals#4040). ([\matrix-org#16137](matrix-org#16137))
- Add the ability to use `G` (GiB) and `T` (TiB) suffixes in configuration options that refer to numbers of bytes. ([\matrix-org#16219](matrix-org#16219))
- Add span information to requests sent to appservices. Contributed by MTRNord. ([\matrix-org#16227](matrix-org#16227))
- Add the ability to enable/disable registrations when using CAS. Contributed by Aurélien Grimpard. ([\matrix-org#16262](matrix-org#16262))
- Allow the `/notifications` endpoint to be routed to workers. ([\matrix-org#16265](matrix-org#16265))
- Enable users to easily unsubscribe to notifications emails via the `List-Unsubscribe` header. ([\matrix-org#16274](matrix-org#16274))
- Report whether a user is `locked` in the [List Accounts admin API](https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#list-accounts), and exclude locked users by default. ([\matrix-org#16328](matrix-org#16328))

- Fix a long-standing bug where multi-device accounts could cause high load due to presence. ([\matrix-org#16066](matrix-org#16066), [\matrix-org#16170](matrix-org#16170), [\matrix-org#16171](matrix-org#16171), [\matrix-org#16172](matrix-org#16172), [\matrix-org#16174](matrix-org#16174))
- Fix a long-standing bug where appservices using [MSC2409](matrix-org/matrix-spec-proposals#2409) to receive `to_device` messages would only get messages for one user. ([\matrix-org#16251](matrix-org#16251))
- Fix bug when using workers where Synapse could end up re-requesting the same remote device repeatedly. ([\matrix-org#16252](matrix-org#16252))
- Fix long-standing bug where we kept re-requesting a remote server's key repeatedly, potentially causing delays in receiving events over federation. ([\matrix-org#16257](matrix-org#16257))
- Avoid temporary storage of sensitive information. ([\matrix-org#16272](matrix-org#16272))
- Fix bug introduced in Synapse 1.49.0 when using dehydrated devices ([MSC2697](matrix-org/matrix-spec-proposals#2697)) and refresh tokens. Contributed by Hanadi. ([\matrix-org#16288](matrix-org#16288))
- Fix a long-standing bug where invalid receipts would be accepted. ([\matrix-org#16327](matrix-org#16327))
- Use standard name for UTF-8 charset in emails. ([\matrix-org#16329](matrix-org#16329))
- Don't try refetching device lists for users on remote hosts that are marked as "down". ([\matrix-org#16298](matrix-org#16298))

- Fix typos in the documentation. ([\matrix-org#16282](matrix-org#16282))
- Link to the Alpine Linux community package for Synapse. ([\matrix-org#16304](matrix-org#16304))
- Use string for `federation_client_minimum_tls_version` documentation examples. Contributed by @jcgruenhage. ([\matrix-org#16353](matrix-org#16353))

- Allow modules to delete rooms. ([\matrix-org#15997](matrix-org#15997))
- Add GCC and GNU Make to the Nix flake development environment so that `ruff` can be compiled. ([\matrix-org#16090](matrix-org#16090), [\matrix-org#16263](matrix-org#16263))
- Fix type checking when using the new version of Twisted. ([\matrix-org#16235](matrix-org#16235))
- Delete device messages asynchronously and in staged batches using the task scheduler. ([\matrix-org#16240](matrix-org#16240), [\matrix-org#16311](matrix-org#16311), [\matrix-org#16312](matrix-org#16312), [\matrix-org#16313](matrix-org#16313))
- Bump minimum supported Rust version to 1.61.0. ([\matrix-org#16248](matrix-org#16248))
- Update rust to version 1.71.1 in the nix development environment. ([\matrix-org#16260](matrix-org#16260))
- Simplify server key storage. ([\matrix-org#16261](matrix-org#16261))
- Reduce CPU overhead of change password endpoint. ([\matrix-org#16264](matrix-org#16264))
- Stop purging from tables slated for removal. ([\matrix-org#16273](matrix-org#16273))
- Improve type hints. ([\matrix-org#16276](matrix-org#16276), [\matrix-org#16301](matrix-org#16301), [\matrix-org#16325](matrix-org#16325), [\matrix-org#16326](matrix-org#16326))
- Raise `setuptools_rust` version cap to 1.7.0. ([\matrix-org#16277](matrix-org#16277))
- Fix using the new task scheduler causing lots of CPU to be used. ([\matrix-org#16278](matrix-org#16278))
- Upgrade CI run of Python 3.12 from rc1 to rc2. ([\matrix-org#16280](matrix-org#16280))
- Include values in SQL debug when using `execute_values` with Postgres. ([\matrix-org#16281](matrix-org#16281))
- Enable additional linting checks. ([\matrix-org#16283](matrix-org#16283))
- Refactor `receipts_graph` Postgres transactions to stop error messages. ([\matrix-org#16299](matrix-org#16299))
- Small improvements to logging in replication code. ([\matrix-org#16309](matrix-org#16309))
- Remove a reference cycle in background processes. ([\matrix-org#16314](matrix-org#16314))
- Only use literal strings for background process names. ([\matrix-org#16315](matrix-org#16315))
- Refactor `get_user_by_id`. ([\matrix-org#16316](matrix-org#16316))
- Speed up task to delete to-device messages. ([\matrix-org#16318](matrix-org#16318))
- Avoid patching code in tests. ([\matrix-org#16349](matrix-org#16349))
- Test against PostgreSQL 16. ([\matrix-org#16351](matrix-org#16351))

* Bump mypy from 1.4.1 to 1.5.1. ([\matrix-org#16300](matrix-org#16300))
* Bump black from 23.7.0 to 23.9.1. ([\matrix-org#16295](matrix-org#16295))
* Bump docker/build-push-action from 4 to 5. ([\matrix-org#16336](matrix-org#16336))
* Bump docker/login-action from 2 to 3. ([\matrix-org#16339](matrix-org#16339))
* Bump docker/metadata-action from 4 to 5. ([\matrix-org#16337](matrix-org#16337))
* Bump docker/setup-qemu-action from 2 to 3. ([\matrix-org#16338](matrix-org#16338))
* Bump furo from 2023.8.19 to 2023.9.10. ([\matrix-org#16340](matrix-org#16340))
* Bump gitpython from 3.1.32 to 3.1.35. ([\matrix-org#16267](matrix-org#16267), [\matrix-org#16279](matrix-org#16279))
* Bump mypy-zope from 1.0.0 to 1.0.1. ([\matrix-org#16291](matrix-org#16291))
* Bump pillow from 10.0.0 to 10.0.1. ([\matrix-org#16344](matrix-org#16344))
* Bump regex from 1.9.4 to 1.9.5. ([\matrix-org#16233](matrix-org#16233))
* Bump ruff from 0.0.286 to 0.0.290. ([\matrix-org#16342](matrix-org#16342))
* Bump serde_json from 1.0.105 to 1.0.107. ([\matrix-org#16296](matrix-org#16296), [\matrix-org#16345](matrix-org#16345))
* Bump twisted from 22.10.0 to 23.8.0. ([\matrix-org#16235](matrix-org#16235))
* Bump types-pillow from 10.0.0.2 to 10.0.0.3. ([\matrix-org#16293](matrix-org#16293))
* Bump types-setuptools from 68.0.0.3 to 68.2.0.0. ([\matrix-org#16292](matrix-org#16292))
* Bump typing-extensions from 4.7.1 to 4.8.0. ([\matrix-org#16341](matrix-org#16341))
Fizzadar added a commit to beeper/synapse-legacy-fork that referenced this pull request Oct 27, 2023
No significant changes since 1.93.0rc1.

The following issues are fixed in 1.93.0 (and RCs).

- [GHSA-4f74-84v3-j9q5](GHSA-4f74-84v3-j9q5) / [CVE-2023-41335](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-41335) — Low Severity

  Temporary storage of plaintext passwords during password changes.

- [GHSA-7565-cq32-vx2x](GHSA-7565-cq32-vx2x) / [CVE-2023-42453](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-42453) — Low Severity

  Improper validation of receipts allows forged read receipts.

See the advisories for more details. If you have any questions, email [email protected].

- Add automatic purge after all users have forgotten a room. ([\matrix-org#15488](matrix-org#15488))
- Restore room purge/shutdown after a Synapse restart. ([\matrix-org#15488](matrix-org#15488))
- Support resolving homeservers using `matrix-fed` DNS SRV records from [MSC4040](matrix-org/matrix-spec-proposals#4040). ([\matrix-org#16137](matrix-org#16137))
- Add the ability to use `G` (GiB) and `T` (TiB) suffixes in configuration options that refer to numbers of bytes. ([\matrix-org#16219](matrix-org#16219))
- Add span information to requests sent to appservices. Contributed by MTRNord. ([\matrix-org#16227](matrix-org#16227))
- Add the ability to enable/disable registrations when using CAS. Contributed by Aurélien Grimpard. ([\matrix-org#16262](matrix-org#16262))
- Allow the `/notifications` endpoint to be routed to workers. ([\matrix-org#16265](matrix-org#16265))
- Enable users to easily unsubscribe to notifications emails via the `List-Unsubscribe` header. ([\matrix-org#16274](matrix-org#16274))
- Report whether a user is `locked` in the [List Accounts admin API](https://matrix-org.github.io/synapse/latest/admin_api/user_admin_api.html#list-accounts), and exclude locked users by default. ([\matrix-org#16328](matrix-org#16328))

- Fix a long-standing bug where multi-device accounts could cause high load due to presence. ([\matrix-org#16066](matrix-org#16066), [\matrix-org#16170](matrix-org#16170), [\matrix-org#16171](matrix-org#16171), [\matrix-org#16172](matrix-org#16172), [\matrix-org#16174](matrix-org#16174))
- Fix a long-standing bug where appservices using [MSC2409](matrix-org/matrix-spec-proposals#2409) to receive `to_device` messages would only get messages for one user. ([\matrix-org#16251](matrix-org#16251))
- Fix bug when using workers where Synapse could end up re-requesting the same remote device repeatedly. ([\matrix-org#16252](matrix-org#16252))
- Fix long-standing bug where we kept re-requesting a remote server's key repeatedly, potentially causing delays in receiving events over federation. ([\matrix-org#16257](matrix-org#16257))
- Avoid temporary storage of sensitive information. ([\matrix-org#16272](matrix-org#16272))
- Fix bug introduced in Synapse 1.49.0 when using dehydrated devices ([MSC2697](matrix-org/matrix-spec-proposals#2697)) and refresh tokens. Contributed by Hanadi. ([\matrix-org#16288](matrix-org#16288))
- Fix a long-standing bug where invalid receipts would be accepted. ([\matrix-org#16327](matrix-org#16327))
- Use standard name for UTF-8 charset in emails. ([\matrix-org#16329](matrix-org#16329))
- Don't try refetching device lists for users on remote hosts that are marked as "down". ([\matrix-org#16298](matrix-org#16298))

- Fix typos in the documentation. ([\matrix-org#16282](matrix-org#16282))
- Link to the Alpine Linux community package for Synapse. ([\matrix-org#16304](matrix-org#16304))
- Use string for `federation_client_minimum_tls_version` documentation examples. Contributed by @jcgruenhage. ([\matrix-org#16353](matrix-org#16353))

- Allow modules to delete rooms. ([\matrix-org#15997](matrix-org#15997))
- Add GCC and GNU Make to the Nix flake development environment so that `ruff` can be compiled. ([\matrix-org#16090](matrix-org#16090), [\matrix-org#16263](matrix-org#16263))
- Fix type checking when using the new version of Twisted. ([\matrix-org#16235](matrix-org#16235))
- Delete device messages asynchronously and in staged batches using the task scheduler. ([\matrix-org#16240](matrix-org#16240), [\matrix-org#16311](matrix-org#16311), [\matrix-org#16312](matrix-org#16312), [\matrix-org#16313](matrix-org#16313))
- Bump minimum supported Rust version to 1.61.0. ([\matrix-org#16248](matrix-org#16248))
- Update rust to version 1.71.1 in the nix development environment. ([\matrix-org#16260](matrix-org#16260))
- Simplify server key storage. ([\matrix-org#16261](matrix-org#16261))
- Reduce CPU overhead of change password endpoint. ([\matrix-org#16264](matrix-org#16264))
- Stop purging from tables slated for removal. ([\matrix-org#16273](matrix-org#16273))
- Improve type hints. ([\matrix-org#16276](matrix-org#16276), [\matrix-org#16301](matrix-org#16301), [\matrix-org#16325](matrix-org#16325), [\matrix-org#16326](matrix-org#16326))
- Raise `setuptools_rust` version cap to 1.7.0. ([\matrix-org#16277](matrix-org#16277))
- Fix using the new task scheduler causing lots of CPU to be used. ([\matrix-org#16278](matrix-org#16278))
- Upgrade CI run of Python 3.12 from rc1 to rc2. ([\matrix-org#16280](matrix-org#16280))
- Include values in SQL debug when using `execute_values` with Postgres. ([\matrix-org#16281](matrix-org#16281))
- Enable additional linting checks. ([\matrix-org#16283](matrix-org#16283))
- Refactor `receipts_graph` Postgres transactions to stop error messages. ([\matrix-org#16299](matrix-org#16299))
- Small improvements to logging in replication code. ([\matrix-org#16309](matrix-org#16309))
- Remove a reference cycle in background processes. ([\matrix-org#16314](matrix-org#16314))
- Only use literal strings for background process names. ([\matrix-org#16315](matrix-org#16315))
- Refactor `get_user_by_id`. ([\matrix-org#16316](matrix-org#16316))
- Speed up task to delete to-device messages. ([\matrix-org#16318](matrix-org#16318))
- Avoid patching code in tests. ([\matrix-org#16349](matrix-org#16349))
- Test against PostgreSQL 16. ([\matrix-org#16351](matrix-org#16351))

* Bump mypy from 1.4.1 to 1.5.1. ([\matrix-org#16300](matrix-org#16300))
* Bump black from 23.7.0 to 23.9.1. ([\matrix-org#16295](matrix-org#16295))
* Bump docker/build-push-action from 4 to 5. ([\matrix-org#16336](matrix-org#16336))
* Bump docker/login-action from 2 to 3. ([\matrix-org#16339](matrix-org#16339))
* Bump docker/metadata-action from 4 to 5. ([\matrix-org#16337](matrix-org#16337))
* Bump docker/setup-qemu-action from 2 to 3. ([\matrix-org#16338](matrix-org#16338))
* Bump furo from 2023.8.19 to 2023.9.10. ([\matrix-org#16340](matrix-org#16340))
* Bump gitpython from 3.1.32 to 3.1.35. ([\matrix-org#16267](matrix-org#16267), [\matrix-org#16279](matrix-org#16279))
* Bump mypy-zope from 1.0.0 to 1.0.1. ([\matrix-org#16291](matrix-org#16291))
* Bump pillow from 10.0.0 to 10.0.1. ([\matrix-org#16344](matrix-org#16344))
* Bump regex from 1.9.4 to 1.9.5. ([\matrix-org#16233](matrix-org#16233))
* Bump ruff from 0.0.286 to 0.0.290. ([\matrix-org#16342](matrix-org#16342))
* Bump serde_json from 1.0.105 to 1.0.107. ([\matrix-org#16296](matrix-org#16296), [\matrix-org#16345](matrix-org#16345))
* Bump twisted from 22.10.0 to 23.8.0. ([\matrix-org#16235](matrix-org#16235))
* Bump types-pillow from 10.0.0.2 to 10.0.0.3. ([\matrix-org#16293](matrix-org#16293))
* Bump types-setuptools from 68.0.0.3 to 68.2.0.0. ([\matrix-org#16292](matrix-org#16292))
* Bump typing-extensions from 4.7.1 to 4.8.0. ([\matrix-org#16341](matrix-org#16341))

# -----BEGIN PGP SIGNATURE-----
#
# iQFEBAABCgAuFiEEBTGR3/RnAzBGUif3pULk7RsPrAkFAmUS8iEQHGVyaWtAbWF0
# cml4Lm9yZwAKCRClQuTtGw+sCXFgB/912+T+BydS290UECCXp9kpRB5xo3aWe8mX
# NCx9Oor1TRLBpLhlQWk786gP1Q9JAQpmA4z6kovjKaLG1b4oLbZNjbPG4hEYc8ow
# /rVzGor52pfyS7uS5GW+rRmapcw4AYND6hA9XGELupf2joC8LXioSCEVG4cxwD8E
# IgIbLc87C7KpaUkNbDEz3jzZ3/BVRGcIYyhF3zTK2ZApvH2qsegq8wKYx4EYJnfh
# 87DXtTCNwA+bW6XZYPtUwPKjZ+TGB11IizxmQySGLbAxvH+GUan8X8TizGyxaqaA
# FDk3yMBbUo0R7ljDgL5YsZXT6qsZz+IBz/bsMzSbZ39f/yEUqHak
# =1/pL
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue Sep 26 16:00:49 2023 BST
# gpg:                using RSA key 053191DFF4670330465227F7A542E4ED1B0FAC09
# gpg:                issuer "[email protected]"
# gpg: Can't check signature: No public key

# Conflicts:
#	.github/workflows/docker.yml
#	.github/workflows/push_complement_image.yml
#	.github/workflows/release-artifacts.yml
#	.github/workflows/tests.yml
#	poetry.lock
#	synapse/appservice/scheduler.py
#	synapse/handlers/pagination.py
#	synapse/handlers/room.py
#	synapse/rest/client/account_data.py
#	tests/rest/client/test_receipts.py
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants