[WIP] sys/net/*coap: Make APIs (more) transport agnostic

[maribu]

Contribution description

This changes the API of nanocoap with the goal to reduce the expose of UDP specifics in the API. The plan is to eventually support transports such as CoAP over TCP and CoAP over WebSocket directly in nanocoap while sharing most of the code, as e.g. the CoAP Option processing remains identical. Specifically, the plan is to unlock a transport with modules and introduce overhead for dispatching to specific transport only when multiple transports are actually in use.

Support for OSCORE directly in nanocoap is probably not sensible, as the serialization is very much unlike the other transports. A unified CoAP API for multiple transports including OSCORE is probably best implemented on top. But when limited to the boring set of CoAP transports, we probably can support them well with nanocoap with less overhead.

API Changes:

Breaking Changes without Deprecation

• coap_pkt_t now contains a uint8_t *buf pointer instead of a
  coap_hdr_t * pointer to the beginning of the buffer
  • This will also work when the buffer is used by non-UDP
    transports

Application developers often have little need to touch this, so
hopefully this will not cause to many breakage. The breakage it causes
is easy to fix and spotted at compile time.

Breaking Changes with Deprecation

• coap_hdr_t has been renamed to coap_udp_hdr_t
  • A deprecated alias was created for deprecation
• coap_hdr*() functions have been deprecated
  • Equivalent coap_pkt*() functions have been created that work on coap_pkt_t * instead of coap_hdr_t *
• coap_build_hdr() has been renamed to coap_build_udp_hdr() and that works on an uint8_t * buffer with a given length, rather than on a coap_hdr_t * with a figers crossed length
  • a deprecated coap_build_hdr() function was added that calls to coap_build_udp_hdr() and has the same signature, so that users have time to update

Internal users have been updated, except for tests.

The deprecated functions will probably exposed only when nanocoap is used with only UDP as transport, as those will wreak havoc on non-UDP CoAP packets. That should be fine, as existing apps hardly will make use of other transports anyway.

Testing procedure

Existing apps and tests should work as before. I will do some limited testing soon and do more rigorous testing once an implementation of nanocoap using TCP as transport is ready. Otherwise there are likely still UDP specifics in the API that I have overlooked.

Issues/PRs references

See also #20792

[maribu]

This PR is not ready for review yet, I just want to make the work more transparent.

[mguetschow]

Cross-referencing @carl-tud's ongoing work with a superset of the goals of this one: #20792

[riot-ci]

Murdock results

✔️ PASSED

84cbedc examples/gcoap: upgrade users of deprecated nanocoap APIs

Success	Failures	Total	Runtime
10214	0	10215	13m:15s

Artifacts

• Documentation preview

[chrysn]

Could the .hdr pointer stay there unused and deprecated if none of the non-UDP gcoap modules are enabled?

Then users who just upgrade their applications would see a deprecation rather than breakage, and would just need to follow that deprecation's pointers ("use the setters and getters") before either they go to the next release or they enable any of the new modules. The Rust wrappers (affected in RIOT-OS/rust-riot-wrappers#129) could then do the same: use the old ways unless a new module is enabled, and after the next release they can go all-in with the new accessor, and thus always work with the current release and with the mai^Wmaster branch.

[chrysn]

To clarify: If it's only the Rust wrappers affected, workarounds are possible (but necessary) to make things work there by detecting whether any code getter/setter is present. But if making it easier for riot-wrappers also eases other users' transitions, maybe we should do that. I can work with either outcome.

[maribu]

Could the .hdr pointer stay there unused and deprecated if none of the non-UDP gcoap modules are enabled?

I think adding an unnamed union containing .hdr and .buf if - and only if - only UDP is used would be a compromise. Having a .hdr when there is (possibly) no UDP header present is quite a footgun, as .hdr is tied towards a UDP header. We should also not have the union there when the CI is run, so that users of coap_pkt_t::hdr don't sneak back in.

[chrysn]

Yes, an unnamed union .buf ∪ .hdr that only has the old item if that is legal (ie. UDP is the only transport) would be good.

CI should probably not interfere here because that'd also impede the Rust side -- but after the next release, the old item can go away anyway, and then we're enforcing the deprecation. Until then I think we'll have to rely on people following the deprecation.

[maribu]

@chrysn: The trick with the unnamed union to make hdr and buf aliases does not work for the Rust side:

error[E0609]: no field `hdr` on type `riot_sys::coap_pkt_t`
   --> /data/riotbuild/.cargo/git/checkouts/rust-riot-wrappers-155be8faf0b21fa1/d7858f5/src/gcoap.rs:383:32
    |
383 |         unsafe { (*(*self.pkt).hdr).code = code };
    |                                ^^^ unknown field
    |
help: one of the expressions' fields has a field of the same name
    |
383 |         unsafe { (*(*self.pkt).__bindgen_anon_1.hdr).code = code };
    |                                +++++++++++++++++

For more information about this error, try `rustc --explain E0609`.

[maribu]

A full CI run witj tests enabled just passed. So regressions that our tests would catch are no longer present.

Remaining ToDos:

• find out how to move forward on the rust side
• prove, that this change actually allows other transports to be implemented

[chrysn]

On the Rust side, there is no immediate action needed while the .hdr field is unconditional. Before that becomes conditional, #129 can be adjusted to only do its changes under the same conditions (because the modules are only available in the next release).

Didn't notice #20900 (comment), revisiting…

[maribu]

I rebased on top of #20917 and changed the latest commit to use RIOT-OS/rust-riot-wrappers#130 for the RIOT wrappers.

[maribu]

Rebased on master. If I still fine issues with the API when implementing CoAP over TCP as proof of concept, I may still adapt this PR. Once confirmed that this is a sane base for adding other transports, I'll mark this PR as ready.

I think the PR will now be more stable and reviewers could take an early look now.