'pip freeze PKG' outputs the installed version of PKG and its dependencies #1946

benoitbryon · 2014-07-23T18:30:54Z

This pull-request is a proposal around "recursive dependencies listing":

pip freeze PKG outputs only version of PKG (a subset of pip freeze)
pip freeze --recursive PKG outputs versions of PKG and its (recursive) dependencies
pip show PKG also outputs "Require recursive: " which contains PKG's recursive dependencies (not PKG itself)

My motivation (i.e. my use case) is to get a known good set of versions for a project.
As a developer, in order to get the known good set of versions of a project,
Given a Python environment (typically a virtualenv) where my project is installed,
Given I made sure project works in this environment (i.e. validated the "known good" set of versions)
Given the environment contains other projects and developer tools (such as bpython, pytest, tox, ...)
I want to get the versions of PKG and its dependencies only, i.e. ignore everything in the environment not required by my project.

An example... Let's install bpython, Sphinx and Fabric:

$ pip install bpython Sphinx Fabric

pip freeze works as always:

$ pip freeze
bpython==0.13.1
docutils==0.12
ecdsa==0.11
Fabric==1.9.0
Jinja2==2.7.3
MarkupSafe==0.23
paramiko==1.14.0
pycrypto==2.6.1
Pygments==1.6
Sphinx==1.2.2

pip freeze bpython shows only versions of bpython:

$ pip freeze bpython
bpython==0.13.1

pip freeze --recursive bpython and its recursive dependencies, a.k.a. versions of everything related to bpython in the current Python environment:

$ pip freeze --recursive bpython
bpython==0.13.1
Pygments==1.6

Same idea for pip freeze Sphinx and pip freeze --recursive Sphinx:

$ pip freeze Sphinx
Sphinx==1.2.2

$ pip freeze --recursive Sphinx
docutils==0.12
Jinja2==2.7.3
MarkupSafe==0.23
Pygments==1.6
Sphinx==1.2.2

pip freeze bpython Sphinx displays frozen versions for both bpython and Sphinx:

$ pip freeze bpython Sphinx
bpython==0.13.1
Sphinx==1.2.2

$ pip freeze --recursive bpython Sphinx
bpython==0.13.1
docutils==0.12
Jinja2==2.7.3
MarkupSafe==0.23
Pygments==1.6
Sphinx==1.2.2

pip show bpython Sphinx has additional "Requires recursive: " items:

$ pip show bpython Sphinx

---
Name: bpython
Version: 0.13.1
Location: /mnt/data/web/django-generic-filters/env/pip/lib/python2.7/site-packages
Requires: pygments
Requires recursive: pygments

---
Name: Sphinx
Version: 1.2.2
Location: /mnt/data/web/django-generic-filters/env/pip/lib/python2.7/site-packages
Requires: Jinja2, Pygments, docutils
Requires recursive: markupsafe, Pygments, Jinja2, docutils

…and its recursive dependencies.

…ckage '{NAME}'.

benoitbryon · 2014-07-23T18:34:00Z

I created this pull request as a proof-of-concept. I didn't took too much care of the tests or the code style. Because I guess the feature/usage has to be discussed/accepted first. And then obviously I would do my best to make this pull-request mergeable.

benoitbryon · 2014-07-23T18:37:06Z

pip/util.py

+ for dep in dist.requires():
+ dependencies.add(dep.project_name)
+ dependencies.update(recursive_dependencies([dep.project_name]))
+ except KeyError:


This except could be moved at line 412 to not catch exception from anywhere else.

…endencies()' to 'pip.util.get_recursive_dependencies()'. This function now takes a list of positional arguments (was a list as first positional argument).

…ve:' line.

benoitbryon · 2014-07-25T06:22:37Z

The Travis build looks nearly good, except some timeout error: https://travis-ci.org/pypa/pip/builds/30765608
I guess this timeout error is not related to this pull-request, but I do not have the permissions to restart the failed py27 build... unless I add some commits :(

Ivoz · 2014-07-25T06:40:53Z

Restarted

benoitbryon · 2014-07-25T07:33:57Z

Restarted

Thanks @Ivoz :)

The latest py27 build failed with errors (including some timeout errors). Still, I think errors are not related to the changes I introduced... or I cannot figure out how they are related. Any idea?

benoitbryon · 2014-07-25T08:47:46Z

\o/ the build passed! Looks like this pull-request does not break current features.
That said, what do you think about the feature proposal?

If it is accepted, I guess the docs (and perhaps more tests) have to be updated too! I am okay to do it, but I'd like some feedback about the concept before I put more efforts in the docs.

benoitbryon · 2014-08-14T08:37:24Z

Any comments about this feature? Should I ask the question on another channel?

thedrow · 2014-08-26T10:45:54Z

Look pretty good. It should probably include more tests though.

benoitbryon · 2014-08-26T12:13:33Z

It should probably include more tests though

@thedrow: do you have examples about these tests? What should they check?

xavfernandez · 2014-10-09T21:34:29Z

#1308 is similar but simpler.

Maybe an option --with-dependencies could reconciliate the two propositions...

The addition of "Requires recursive" to pip show is a nice idea though.

xavfernandez · 2014-10-09T21:45:07Z

#858 is an other one

…lop.

…ake it recursive. Refs pypa#1308.

benoitbryon · 2014-10-26T14:57:38Z

@thedrow: Added functional tests in tests/functional/test_freeze.py

@xavfernandez: I changed the feature to look like #1308:

pip freeze shows all installed packages
pip freeze PKG shows only PKG
pip freeze --recursive PKG shows PKG and its (recursive) dependencies
I performed a simple case insensitive match: pip freeze SamPLE shows sample but not anothersample. I think the match rules (seen in Freeze with args #1308 and in arguments to freeze are used to filter installed packages #858) can be added later, as another pull request.
I did not merge Freeze with args #1308 because it is a very old pull request and I was unable to update it easily and resolve conflicts

benoitbryon · 2014-10-26T15:06:20Z

:( I updated the pull-request on top of recent "develop" branch... perhaps it was not a good idea.
I mean, I do not understand why the tests fail. There is some error in functional/test_install.py, which should not be changed by this pull-request. See https://travis-ci.org/pypa/pip/jobs/39079327#L234

…es recursive' in 'pip show'.

benoitbryon · 2014-10-27T08:33:55Z

@megies: does this pull-request match yours?

benoitbryon · 2014-10-27T08:35:12Z

Note: just discovered pipdeptree by @naiquevin.

megies · 2014-10-27T13:58:09Z

@megies: does this pull-request match yours?

Mine can just be dropped, this looks way more advanced.. ;)

benoitbryon · 2014-10-27T14:10:37Z

Note: updated top-level description of ticket, to reflect pull-request's latest changes.

benoitbryon · 2014-10-28T15:45:10Z

I do not understand the build error :(
Any idea there?

On my computer, I have other errors, including errors related to pypa/pip-test-package... I do not know why the results are not the same as told by Travis. That said, I do not understand the relationship with this PR.

…elop.

benoitbryon · 2014-11-13T14:18:00Z

Just updated the branch on top of "develop" and the tests are ok!
Is there something else I can do for this pull-request to be merged?

msabramo · 2014-12-04T07:31:57Z

I would want to compare this to pipdeptree (https://pypi.python.org/pypi/pipdeptree), which I've used in the past.

This PR shows a flat view. pipdeptree shows a tree. But this PR has some interesting ways of invoking it. Maybe combine the two?

benoitbryon · 2014-12-16T08:37:34Z

Yes! ready for another round of review :)

msabramo · 2014-12-20T17:01:07Z

This has merge conflicts because my PR #2173 was merged. I can take a stab at fixing it.

…edux Conflicts: pip/commands/freeze.py pip/commands/show.py tests/functional/test_show.py

benoitbryon · 2015-01-22T08:24:58Z

Thanks @msabramo for the update!

msabramo · 2015-01-22T21:31:56Z

I like this. @dstufft, @pfmoore ?

pfmoore · 2015-01-22T22:09:08Z

Seems reasonable. I'm happy for it to be merged. Any objections @dstufft @carljm @jezdez ?

msabramo · 2015-02-07T01:07:28Z

This one needs another look from a code contributor

msabramo · 2015-02-21T02:31:21Z

Any objections?

msabramo · 2015-02-28T21:24:14Z

I think the silence means that @pfmoore should merge this. 😄

pfmoore · 2015-03-02T08:33:51Z

As it's a behaviour change, I'd like to see a bit more than silence from the other devs... :-) Also, my comment above was meant to imply that I have no objections to someone doing the work to merge it (i.e., review the code, check the corner cases, ensure test coverage is OK, etc) not that I have done that work myself. I'll try to get some time to look at this a bit more deeply, but I don't have a need for it myself (I don't use pip freeze at all) so it may be a while before I get the chance.

msabramo · 2015-03-05T16:44:22Z

Hmmm. Well I'd feel bad about making you take time to review something that you have no vested interest in. Maybe someone else is more interested in this feature?

msabramo · 2015-03-14T04:02:31Z

Maybe a core contributor (other than @pfmoore who already looked a bit) look at what's proposed, if not the code? If the idea is not sound, then it can be closed and if it is, it can hopefully get reviewed and merged before it becomes outdated.

dstufft · 2016-05-18T11:30:23Z

Accidentally closed this, reopening. Sorry!

BrownTruck · 2016-05-19T12:02:24Z

Hello!

As part of an effort to ease the contribution process and adopt a more standard workflow pip has switched to doing development on the master branch. However, this Pull Request was made against the develop branch so it will need to be resubmitted against master. Unfortunately, this pull request does not cleanly merge against the current master branch.

If you do nothing, this Pull Request will be automatically closed by @BrownTruck since it cannot be merged.

If this pull request is still valid, please rebase it against master (or merge master into it) and resubmit it against the master branch, closing and referencing the original Pull Request.

If you choose to rebase/merge and resubmit this Pull Request, here is an example message that you can copy and paste:

This pull-request is a proposal around "recursive dependencies listing":



pip freeze PKG outputs only version of PKG (a subset of pip freeze)

pip freeze --recursive PKG outputs versions of PKG and its (recursive) dependencies

pip show PKG also outputs "Require recursive: " which contains PKG's recursive dependencies (not PKG itself)


My motivation (i.e. my use case) is to get a known good set of versions for a project.
As a developer, in order to get the known good set of versions of a project,
Given a Python environment (typically a virtualenv) where my project is installed,
Given I made sure project works in this environment (i.e. validated the "known good" set of versions)
Given the environment contains other projects and developer tools (such as bpython, pytest, tox, ...)
I want to get the versions of PKG and its dependencies only, i.e. ignore everything in the environment not required by my project.

An example... Let's install bpython, Sphinx and Fabric:

$ pip install bpython Sphinx Fabric

pip freeze works as always:

$ pip freeze
bpython==0.13.1
docutils==0.12
ecdsa==0.11
Fabric==1.9.0
Jinja2==2.7.3
MarkupSafe==0.23
paramiko==1.14.0
pycrypto==2.6.1
Pygments==1.6
Sphinx==1.2.2

pip freeze bpython shows only versions of bpython:

$ pip freeze bpython
bpython==0.13.1

pip freeze --recursive bpython and its recursive dependencies, a.k.a. versions of everything related to bpython in the current Python environment:

$ pip freeze --recursive bpython
bpython==0.13.1
Pygments==1.6

Same idea for pip freeze Sphinx and pip freeze --recursive Sphinx:

$ pip freeze Sphinx
Sphinx==1.2.2

$ pip freeze --recursive Sphinx
docutils==0.12
Jinja2==2.7.3
MarkupSafe==0.23
Pygments==1.6
Sphinx==1.2.2


pip freeze bpython Sphinx displays frozen versions for both bpython and Sphinx:

$ pip freeze bpython Sphinx
bpython==0.13.1
Sphinx==1.2.2

$ pip freeze --recursive bpython Sphinx
bpython==0.13.1
docutils==0.12
Jinja2==2.7.3
MarkupSafe==0.23
Pygments==1.6
Sphinx==1.2.2

pip show bpython Sphinx has additional "Requires recursive: " items:

$ pip show bpython Sphinx
---
Name: bpython
Version: 0.13.1
Location: /mnt/data/web/django-generic-filters/env/pip/lib/python2.7/site-packages
Requires: pygments
Requires recursive: pygments
---
Name: Sphinx
Version: 1.2.2
Location: /mnt/data/web/django-generic-filters/env/pip/lib/python2.7/site-packages
Requires: Jinja2, Pygments, docutils
Requires recursive: markupsafe, Pygments, Jinja2, docutils

---

*This was migrated from pypa/pip#1946 to reparent it to the ``master`` branch. Please see original pull request for any previous discussion.*

BrownTruck · 2016-05-26T10:58:44Z

This Pull Request was closed because it cannot be automatically reparented to the master branch and it appears to have bit rotted.

Please feel free to re-open it or re-submit it if it is still valid and you have rebased it onto master or merged master into it.

msabramo · 2016-06-02T01:12:40Z

Too bad. This seems useful.

benoitbryon added 2 commits July 23, 2014 19:56

'pip freeze {NAME}' outputs the installed versions of package {NAME} …

982a891

…and its recursive dependencies.

'pip show {NAME}' also shows the list of recursive dependencies of pa…

cc94c3a

…ckage '{NAME}'.

benoitbryon reviewed Jul 23, 2014
View reviewed changes

benoitbryon added 2 commits July 24, 2014 18:04

Fixed line count in 'pip show' tests. Renamed 'pip.util.recursive_dep…

24a2dda

…endencies()' to 'pip.util.get_recursive_dependencies()'. This function now takes a list of positional arguments (was a list as first positional argument).

In 'pip show' tests, shifted output lines to insert 'Requires recursi…

d84fb4a

…ve:' line.

benoitbryon added 2 commits October 26, 2014 12:19

Psyko-rebased branch feature/freeze-subset-recursively on top of deve…

ffa2694

…lop.

pip freeze is not recursive by default. Added --recursive option to m…

4396efa

…ake it recursive. Refs pypa#1308.

Updated examples in documentation around 'pip freeze PKG' and 'Requir…

28d098c

…es recursive' in 'pip show'.

Psycho-rebased branch feature/freeze-subset-recursively on top of dev…

4b43c91

…elop.

benoitbryon mentioned this pull request Nov 13, 2014

Add the --dependents option to the show command. #811

Closed

naiquevin mentioned this pull request Nov 25, 2014

Great Tool - Get it into pip as pip deptree :) tox-dev/pipdeptree#1

Closed

msabramo mentioned this pull request Dec 12, 2014

Proposal: pip freeze --no-auto-installs to list all the packages I installed, but not their dependencies #2100

Closed

This was referenced Dec 20, 2014

Feature/freeze subset recursively (rebase) #2234

Closed

Freeze with args #1308

Closed

msabramo added a commit to msabramo/pip that referenced this pull request Jan 16, 2015

Finish merging @benoitbryon's pypa#1946 with develop

970ff8f

Merge remote-tracking branch 'origin/develop' into freeze_recursive_r…

186ddf2

…edux Conflicts: pip/commands/freeze.py pip/commands/show.py tests/functional/test_show.py

msabramo mentioned this pull request Mar 6, 2015

Add a pip check command. #2492

Closed

dstufft closed this May 18, 2016

dstufft reopened this May 18, 2016

BrownTruck added asked to reparent auto-bitrotten PRs that died because they weren't updated labels May 19, 2016

BrownTruck removed the asked to reparent label May 26, 2016

BrownTruck closed this May 26, 2016

lock bot added the auto-locked Outdated issues that have been locked by automation label Jun 3, 2019

lock bot locked as resolved and limited conversation to collaborators Jun 3, 2019

'pip freeze PKG' outputs the installed version of PKG and its dependencies #1946

'pip freeze PKG' outputs the installed version of PKG and its dependencies #1946

Conversation

benoitbryon commented Jul 23, 2014 • edited by BrownTruck Loading

benoitbryon commented Jul 23, 2014

benoitbryon Jul 23, 2014

Choose a reason for hiding this comment

benoitbryon commented Jul 25, 2014

Ivoz commented Jul 25, 2014

benoitbryon commented Jul 25, 2014

benoitbryon commented Jul 25, 2014

benoitbryon commented Aug 14, 2014

thedrow commented Aug 26, 2014

benoitbryon commented Aug 26, 2014

xavfernandez commented Oct 9, 2014

xavfernandez commented Oct 9, 2014

benoitbryon commented Oct 26, 2014

benoitbryon commented Oct 26, 2014

benoitbryon commented Oct 27, 2014

benoitbryon commented Oct 27, 2014

megies commented Oct 27, 2014

benoitbryon commented Oct 27, 2014

benoitbryon commented Oct 28, 2014

benoitbryon commented Nov 13, 2014

msabramo commented Dec 4, 2014

benoitbryon commented Dec 16, 2014

msabramo commented Dec 20, 2014

benoitbryon commented Jan 22, 2015

msabramo commented Jan 22, 2015

pfmoore commented Jan 22, 2015

msabramo commented Feb 7, 2015

msabramo commented Feb 21, 2015

msabramo commented Feb 28, 2015

pfmoore commented Mar 2, 2015

msabramo commented Mar 5, 2015

msabramo commented Mar 14, 2015

dstufft commented May 18, 2016

BrownTruck commented May 19, 2016

BrownTruck commented May 26, 2016

msabramo commented Jun 2, 2016

benoitbryon commented Jul 23, 2014 •

edited by BrownTruck

Loading