Change the hashes in the installation report to be a mapping

[pradyunsg]

What's the problem this feature will solve?

Currently, pip install --report generates hash information that looks like:

        "archive_info": {
          "hash": "sha256=18f3e912f9ad1bdec27fb06b8198a2ccc32f201e24174cec1b3424dda605a310"
        }

This means that we can't show hash information with multiple algorithms easily.

Describe the solution you'd like

Change archive information into being a mapping of hashes, similar to PEP 691:

      "hashes": {"sha256": "...", "blake2b": "..."},

We can put this key in archive_info, and replace the existing hash key with it.

Alternative Solutions

Not really. The other option is doing something like adding & as a separator between hashes, but that sounds like a horrible thing to do given the alternatives we have and that we're allowed to make backwards incompatible changes right now.

Additional context

N/A

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pradyunsg]

/cc @sbidoul

[uranusjr]

For an alternative solution (not suggesting we should do it instead), we can make hashes a list of algo=hashval strings.

[pradyunsg]

It's a mapping -- let's store it as one. I expect that users will compose one anyway, regardless of what format we spew it out in. ([s.split("=") for s in li] vs di.items(), di.get(algo) vs {alternative too difficult to code-golf})

[sbidoul]

Yeah, I've had that in mind too.

I think that should be done as an evolution of PEP 610.

I'm under the impression that PyPI only returns one hash type, though ?

[uranusjr]

The JSON API returns multiple hashes.

[sbidoul]

The JSON API returns multiple hashes.

It is foreseen in the PEP 691 API yes, but in practice pypi.org returns sha256 only.

Anyway I agree this should be done, I'm not sure about the urgency.

[pradyunsg]

Let's do this before we stabilise the format.

[pradyunsg]

I've removed the needs standard thing, because it's already technically possible, through existing standards, to have multiple hashes from multiple algorithms.

[sbidoul]

I've removed the needs standard thing, because it's already technically possible, through existing standards, to have multiple hashes from multiple algorithms.

Yes but no :) The standard used for the download_info field is PEP 610 and it is important to keep it so because

• in some case, they are direct URLs,
• and sometimes it comes from origin.json in the wheel cache for which we use the direct URL format too,
• and because the closer we are to documented interoperability standards, the easier it will be to consume the format.

So let's do the standard update work first, please. It should be relatively easy: add the hashes field you propose and write up the SHOULD/MUST language to address backward compatibility concerns for consumers and producers.

[pradyunsg]

I'm very confused. The pip installation report format is explicitly noted to be a pip-only non-interoperability standard format.

Why is a PEP/standard process involved in changing it?

[pradyunsg]

I can understand that if we wanted to extend the download info key in the standard, to allow storing the metadata in that format there -- but we're not blocked on that. Like, literally, we can add a dict([s.split("=")]) in the pip-specific format, and close this issue. And, come around to dealing with multiple hashes later.

Sure, that's not the best way to do things, but my point is that we don't have to wait on a standardisation process and this is firmly a detail we can change today.

[pfmoore]

I agree. I think we should future-proof the pip-only format now, before people start relying on it, and then we're prepared should PEP 610 get extended later. And if PEP 610 never gets extended, we've lost nothing and we've gained a small amount of user friendliness (we do the .split("=") so you don't have to 🙂)

[sbidoul]

Of course we could do without changing PEP 610.

Yet let me add these arguments:

• the installation report is not an interoperability standard but it composes interoperability standards to ease consumption (as well as production)
• download_info is explicitly documented as being PEP 610 compliant
• diverging from it will make the implementation and documentation much more complex
• it will make it harder to consume for our users
• the fact that PEP 610 supports a single hash only is clearly an oversight

I'll try to prepare an amendment PEP 610 and I'll see how well it fits.

[pradyunsg]

I'll try to prepare an amendment PEP 610 and I'll see how well it fits.

A very gentle nudge on this. :)

[sbidoul]

Hi @pradyunsg, this is still on my radar, but I won't have time to tackle this before November/December.

[pradyunsg]

That's perfectly reasonable. We should keep the warning about the format until we address this though.

[sbidoul]

I think this change will be backward compatible, though, as I plan to propose new a hashes field to the PEP 610 data structure, while keeping hash for backward compatibility.

[sbidoul]

@pradyunsg As promised, here is the draft PEP to amend the direct_url.json data structure.

[pradyunsg]

Thank you for doing this @sbidoul! ^.^

[sbidoul]

@pradyunsg @pfmoore how can we progress with this? I think this is the only issue blocking us marking the installation report format as stable.

Quite understandably, this being a niche issue with little or no practical interest right now, my little draft PEP did not raise any comment.

There are a few options:

• move forward with the draft PEP (I'll need your help for that)
• keep the implementation as is (with one hash) and declare the install report format as stable, knowing a backward compatible path forward exists, and that we can dig out the draft PEP if/when practical needs arise
• keep the install report format in experimental status
• implement a ad-hoc deviation from the PEP 610 format in the installation report (which would complicate the implementation and documentation quite a bit, and I'm really not keen to do it)

[pfmoore]

I've posted on the draft PEP thread on Discourse, suggesting that in the absence of any objections, this change can be made as a text-only change to the spec. If no-one has any problem with that over the next week, let's just make the change.

Either way, I say we declare the report format as stable. So basically either your first or second option, with the first being my preference for "just get it done with" reasons 🙂

[sbidoul]

Perfect. Thank you, Paul.

[pradyunsg]

+1 to all that Paul said!