The ACME modules does not seem to support ACME directories that do not contain a "meta" key in their directory response, resulting in module crash

[ConsciaDBA]

SUMMARY

The ACME modules does not seem to support ACME directories that do not contain a "meta" key in their directory response, resulting in module crash. According to the RFC for acme v2 its also only a optional field that can be set (https://tools.ietf.org/html/rfc8555#section-7.1.1) and not required. The ACME server in step-ca does for example not respond with a meta object.

ISSUE TYPE

• Bug Report

COMPONENT NAME

community.crypto.acme_*

ANSIBLE VERSION

ansible 2.9.18
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.8.5 (default, Jan 27 2021, 15:41:15) [GCC 9.3.0]

STEPS TO REPRODUCE

Run any ACME_* module against a ACME server that does not respond with a "meta" object in its directory response, for example step-ca ACME server.

Supported (letsencrypt acme server):

{
  "keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",
  "kjozKDlyNww": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
    "website": "https://letsencrypt.org"
  },
  "newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",
  "revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"
}

Not supported (step-ca self-hosted acme server):

{
    "newNonce": "https://192.168.1.40/acme/acme/new-nonce",
    "newAccount": "https://192.168.1.40/acme/acme/new-account",
    "newOrder": "https://192.168.1.40/acme/acme/new-order",
    "revokeCert": "https://192.168.1.40/acme/acme/revoke-cert",
    "keyChange": "https://192.168.1.40/acme/acme/key-change"
}

I think it may be related to this line due to the meta key not existing at all

community.crypto/plugins/module_utils/acme/account.py

Line 59 in 201920d

if (external_account_binding is not None or self.client.directory['meta'].get('externalAccountRequired')) and allow_creation:

EXPECTED RESULTS

Run without issue

ACTUAL RESULTS

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'meta'
fatal: [SERVERNAME]: FAILED! => {
    "changed": false,
    "rc": 1
}

MSG:

MODULE FAILURE
See stdout/stderr for the exact error


MODULE_STDERR:

Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1619519110.5056522-9014-83116195850232/AnsiballZ_acme_account.py", line 102, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1619519110.5056522-9014-83116195850232/AnsiballZ_acme_account.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1619519110.5056522-9014-83116195850232/AnsiballZ_acme_account.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible_collections.community.crypto.plugins.modules.acme_account', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/lib/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_community.crypto.acme_account_payload_8l5sqjz4/ansible_community.crypto.acme_account_payload.zip/ansible_collections/community/crypto/plugins/modules/acme_account.py", line 338, in <module>
  File "/tmp/ansible_community.crypto.acme_account_payload_8l5sqjz4/ansible_community.crypto.acme_account_payload.zip/ansible_collections/community/crypto/plugins/modules/acme_account.py", line 258, in main
  File "/tmp/ansible_community.crypto.acme_account_payload_8l5sqjz4/ansible_community.crypto.acme_account_payload.zip/ansible_collections/community/crypto/plugins/module_utils/acme/account.py", line 207, in setup_account
  File "/tmp/ansible_community.crypto.acme_account_payload_8l5sqjz4/ansible_community.crypto.acme_account_payload.zip/ansible_collections/community/crypto/plugins/module_utils/acme/account.py", line 59, in _new_reg
  File "/tmp/ansible_community.crypto.acme_account_payload_8l5sqjz4/ansible_community.crypto.acme_account_payload.zip/ansible_collections/community/crypto/plugins/module_utils/acme/acme.py", line 87, in __getitem__
KeyError: 'meta'

[felixfontein]

You are right, currently the code assumes that meta is always present. I'll take a look later today how to fix that.

[felixfontein]

I created a PR to fix this (#221). Please test it. I'd be happy to hear whether the modules will work with step-ca (after this is fixed); I already wanted to test that once, but never managed to find time to set step-ca up and try it out yet...

[ConsciaDBA]

That did not work, but what worked was changing your new line to this:

if 'meta' not in self.directory:
    self.directory["meta"] = {}

Thanks a lot for your help

[felixfontein]

Ok, that was stupid :) I've updated the PR...

So were you able to create an account and obtain a certificate once you applied your fix of my fix? Or how far did you got with it?

[ConsciaDBA]

Yep, i was able to run the acme_certficiate module directly (which from what i can read automatically creates account, given a account_key), request a certificate, validate the challenge (DNS-01) and get the signed certificate.
So it seems to be working :)

[felixfontein]

@ConsciaDBA awesome! In that case I'll create a new release once the fix is merged, instead of waiting for more bug reports ;)