Check dependencies for pip's new dep resolver

[rgaudin]

In October 2020, the new dependency resolver for pip will replace the current one (in pip 20.3). We'll still have the ability to use the old one until pip 21.1.

We should ensure that all our Python scrapers can still be installed with the new dependencies resolver ; especially the ones we are not actively working on as this resolver-change will be long-forgotten when we'll come back to those in a late future.

See list of scrapers.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[brainwane]

I found this issue via a GitHub search and figured I ought to mention some useful troubleshooting and workaround tips from the documentation, especially now that pip 20.3 is out and the new resolver is the default:

• If pip is taking longer to install packages, read Dependency resolution backtracking for ways to reduce the time pip spends backtracking due to dependency conflicts.

• If you don’t want pip to actually resolve dependencies, use the --no-deps option. This is useful when you have a set of package versions that work together in reality, even though their metadata says that they conflict. For guidance on a long-term fix, read Fixing conflicting dependencies.

• If you run into resolution errors and need a workaround while you’re fixing their root causes, you can choose the old resolver behavior using the flag --use-deprecated=legacy-resolver. This will work until we release pip 21.0 (see Deprecation timeline).

[kelson42]

@brainwane Hi Sumana, so nice to see you there :) Thank you for the comment, I guess @rgaudin will carry on with this ASAP.

[brainwane]

Hi @kelson42 - nice to see you again! Hope all goes well with the dependency improvements.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[kelson42]

@rgaudin We should probably move on this if this has not been done so far. Should we open similar tickets in other Python repos?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[kelson42]

Wonder how newly created https:/openzim/_python-bootstrap impacts this ticket.

[rgaudin]

In itself, it doesn't but as we switch to it, it will become obsolete as we update scrapers.

Goal behind this overview ticket was to proactively test (on all scrapers) the new resolver and update dependencies accordingly at a time pip allowed both resolver to be used.
Given we already have overview ticket to update python versions, I am closing this as duplicate.