Datasets conflicts with fsspec 2024.9

[cw-igormorgado]

Describe the bug

Installing both in latest versions are not possible

pip install "datasets==3.0.1" "fsspec==2024.9.0"

But using older version of datasets is ok

pip install "datasets==1.24.4" "fsspec==2024.9.0"

Steps to reproduce the bug

pip install "datasets==3.0.1" "fsspec==2024.9.0"

Expected behavior

install both versions.

Environment info

debian 11.
python 3.10.15

[geekodour]

Yes, I need to use the latest version of fsspec and datasets for my usecase.
fsspec/s3fs#888 (comment)
apache/arrow#34363 (comment)

last version where things install without conflict is: 2.14.4

So this issue starts from:
https:/huggingface/datasets/releases/tag/2.14.5