Calculate a hash for CRL with x509_crl_info

[JuddTracy-DAS]

SUMMARY

I would like x509_crl_info to return a hash that can be used for CRL checking.

ISSUE TYPE

• Feature Idea

COMPONENT NAME

x509_crl_info

ADDITIONAL INFORMATION

Using the option SSLCARevocationPath in apache expects each crl to have a link with the hash as the name pointing to the crl in the directory. The hash can be generated in openssl using:

openssl crl -hash -noout -in crl_file.pem

[felixfontein]

I assume this is basically what the CertificateRevocationList.fingerprint() method computes (https://cryptography.io/en/latest/x509/reference.html#cryptography.x509.CertificateRevocationList.fingerprint). I guess we could return a dictionary with fingerprints in all available hashes, then users can use whichever fingerprint they prefer.

This is blocked by #150 though, since it would introduce serious conflicts. I'll start moving the first few commits into separate PRs, then these can be reviewed and merged individually. Once the PR for 3cd0056 is merged, this can be implemented without causing great pain for #150 :)

[felixfontein]

I went ahead and moved the commit to #203. Once that's reviewed and merged, I'll start working on this.

[JuddTracy-DAS]

I assume this is basically what the CertificateRevocationList.fingerprint() method computes (https://cryptography.io/en/latest/x509/reference.html#cryptography.x509.CertificateRevocationList.fingerprint). I guess we could return a dictionary with fingerprints in all available hashes, then users can use whichever fingerprint they prefer.

This is blocked by #150 though, since it would introduce serious conflicts. I'll start moving the first few commits into separate PRs, then these can be reviewed and merged individually. Once the PR for 3cd0056 is merged, this can be implemented without causing great pain for #150 :)

I am not sure that is the same thing. Looking at the openssl crl commands there is the option to calculate the hash and the fingerprint and they are different results here is an example for one of my crls

openssl crl -in files/pki/crl/tmp/crl.pem -noout -hash

b01b3c03


openssl crl -in files/pki/crl/tmp/crl.pem -noout -fingerprint

SHA1 Fingerprint=4A:17:C7:2F:11:B2:92:34:20:D6:D7:1A:7F:E0:95:DA:AC:F9:88:7C

[felixfontein]

Hmm, so -hash basically prints a hash of the CRL issuer (https:/openssl/openssl/blob/master/apps/crl.c#L308). The implementation of X509_NAME_hash_ex is here: https:/openssl/openssl/blob/master/crypto/x509/x509_cmp.c#L282 It looks to me (without trying it out) that it encodes the issuer name as DER (https://cryptography.io/en/latest/x509/reference.html#cryptography.x509.Name.public_bytes), computes its SHA-1, and combines the first four bytes of the result to a 32bit unsigned int, which is printed with 8 hex digits. I wonder whether this comes from some RFC...

[MarkusTeufelberger]

Sounds like something OpenSSL specific (https://stackoverflow.com/questions/34095440/hash-algorithm-for-certificate-crl-directory)

Aaaand of course they also changed it over the years (there's also an -hash_old option... https://www.openssl.org/docs/manmaster/man1/openssl-crl.html)

[felixfontein]

@MarkusTeufelberger the old one is using MD5 instead of SHA1. Let's see when SHA1 will be replaced by SHA2, and we end up having -hash_old_old, -hash_old and -hash ;)

[felixfontein]

I played around with this a bit. Generating the old hash is pretty simple. Generating the new (current) hash is more complicated: OpenSSL uses the 'canonical' encoding of the X.509 Name, which seems to be something OpenSSL internal: https:/openssl/openssl/blob/master/crypto/x509/x_name.c#L293-L379 Unsurprisingly, cryptography does not provide any functionality to retrieve that.

[JuddTracy-DAS]

I appreciate you looking at it. I think think the functionally that I was requesting would be useful for others, but I have had some performance issues with x509_crl_info and large crls that have made me re-evaluate using the module. Some of the CRLs would take 30 minutes to process and that would not work in the long run for what I would need this functionality. Since cryptography doesn't support it, I am not opposed to closing this request and just keep using the scripts for the time being.

[felixfontein]

@JuddTracy-DAS would you mind sharing a CRL which needs 30 minutes to process with me? I'm curious why it is so slow. I would guess the problem is exporting all entries, which has to be tunnelled through JSON to ansible. If that's the only problem, we could add a switch to disable exporting all entries.

(You can find my email address in my commits.)

[JuddTracy-DAS]

I am running this on an older system and it took around 15 minutes to process not 30, but I wouldn't expect it to cause too much of a speed difference. There a many DISA DoD CRLs that are slow to process. I don't know what it is behind the scenes that is cause it but there are a massive amount of entries in the CRL. Here are a couple you can download DODEMAILCA_49.crl DODEMAILCA_51.crl and a complete list of them can be found unclass-dod_eca_crldps_nipr_20190529

[JuddTracy-DAS]

Also when I would process them all in a loop things would slow down even more. Not sure if memory was an issue or what, but it would take a very long time to process them all.

[felixfontein]

I ran time ansible localhost -m community.crypto.x509_crl_info -a path=DODEMAILCA_49.crl:

1. With listing of revoked_certificates removed: 3 seconds
2. With listing of revoked_certificates active, but the list is thrown away after computing it: 1 minute
3. Without modification (i.e. everything is dumped): I interrupted at 3 minutes while there were screenfulls of output scrolling by

So I guess having an option to not return the list of revoked certificates would speed it up dramatically for you.

[felixfontein]

I created #232 to allow to not return the list of revoked certificates.

[felixfontein]

Here a similar question has been asked about subject and issuer hash for certificates: https://groups.google.com/g/ansible-project/c/KnonsMNLxeY/m/TrNO2ghcAgAJ

[felixfontein]

openssl x509 -issuer_hash uses X509_issuer_name_hash, which calls X509_NAME_hash_ex as well - i.e. the same code as for openssl crl -hash is used.