Release 2020.04 - RC2

[leandrolanzieri]

This issue lists the status of all tests for the Release Candidate 2 of the 2020.04 release.

Specs tested:

• 01-ci
  • Task #01 - Compile test
  • Task #02 - Unittests on native
  • Task #03 - Unittests on native separated
  • Task #04 - Unittests on iotlab-m3
• 02-tests
  • Task #01 - All tests on native
  • Task #02 - Subset of tests on iotlab-m3 @leandrolanzieri
  • Task #03 - Subset of tests on samr21-xpro @leandrolanzieri
• 03-single-hop-ipv6-icmp
  • Task #01 - ICMPv6 multicast echo on native
  • Task #02 - ICMPv6 link-local echo on native
  • Task #03 - ICMPv6 link-local echo on native (1 hour)
  • Task #04 - ICMPv6 stress test on native (1 hour)
  • Task #05 - ICMPv6 stress test on native (neighbor cache stress)
  • Task #06 - ICMPv6 link-local echo on native (IPv6 fragmentation)
• 04-single-hop-6lowpan-icmp
  • Task #01 - ICMPv6 link-local echo with iotlab-m3
  • Task #02 - ICMPv6 multicast echo with iotlab-m3/samr21-xpro
  • Task #03 - ICMPv6 echo with large payload
  • Task #04 - ICMPv6 echo with iotlab-m3/samr21-xpro 15 minutes
  • Task #05 (Experimental) - ICMPv6 multicast echo with samr21-xpro/remote
  • Task #06 (Experimental) - ICMPv6 link-local echo with samr21-xpro/remote
  • Task #07 (Experimental) - ICMPv6 multicast echo with samr21-xpro/zero + xbee
  • Task #08 (Experimental) - ICMPv6 echo with samr21-xpro/zero + xbee
  • Task #09 - ICMPv6 stress test on iotlab-m3
  • Task #10 (Exprimental) - ICMPv6 echo with large payload (IPv6 fragmentation)
• 05-single-hop-route
  • Task #01 - ICMPv6 echo unicast addresess on native (default route)
  • Task #02 - ICMPv6 echo unicast addresess on iotlab-m3 (default route)
  • Task #03 - ICMPv6 echo unicast addresess on native (specific route)
  • Task #04 - ICMPv6 echo unicast addresess on iotlab-m3 (static route)
• 06-single-hop-udp
  • Task #01 - UDP on iotlab-m3
  • Task #02 - UDP on iotlab-m3 (UDP port compression)
  • Task #03 - UDP on native (non-existent neighbor)
  • Task #04 - UDP on iotlab-m3 (non-existent neighbor)
  • Task #05 - Empty UDP on native
  • Task #06 - Empty UDP on iotlab-m3
• 07-multi-hop
  • Task #01 - ICMPv6 echo on iotlab-m3 with three hops (static route)
  • Task #02 - UDP on iotlab-m3 with three hops (static route)
  • Task #03 - ICMPv6 echo on iotlab-m3 with three hops (RPL route)
  • Task #04 - UDP on iotlab-m3 with three hops (RPL route)
  • Task #05 (Experimental) - UDP with large payload on iotlab-m3 with three hops (RPL route)
• 08-interop
  • Task #01 - ICMPv6 echo between native and Linux
  • Task #02 - ICMPv6 echo between iotlab-m3 and Linux with 6LowPAN
  • Task #03 - UDP exchange between iotlab-m3 and Contiki
  • Task #04 - ICMPv6 echo between iotlab-m3 and Internet host through Linux with 6LowPAN
  • Task #05 - ICMPv6 echo between iotlab-m3 and Internet host through RIOT border router
  • Task #06 - UDP between iotlab-m3 and Internet host through RIOT border router
  • Task #07 - UDP between iotlab-m3 and Internet host through RIOT border router (200b payload)
  • Task #08 - UDP between GNRC and lwIP on iotlab-m3
  • Task #09 - UDP between GNRC and emb6 on iotlab-m3
  • Task #10 - UDP between lwIP and emb6 on iotlab-m3
  • Task #11 - UDP exchange between iotlab-m3 and Zephyr
• 09-coap
  • Task #01 - CORD Endpoint
  • Task #02 - Confirmable retries
  • Task #03 - Block1
  • Task #04 - Block2
  • Task #05 - Observe registration and notification
• 10-icmpv6-error
  • Task #01 - Destination unreachable - no route to destination
  • Task #02 - Destination unreachable - Beyond scope of source address
  • Task #03 - Destination unreachable - address unreachable (target node address)
  • Task #04 - Destination unreachable - address unreachable (neighbor cache miss)
  • Task #05 - Destination unreachable - port unreachable
  • Task #06 - Destination unreachable - port unreachable (large payload)
  • Task #07 - Packet too big
  • Task #08 - Time exceeded - hop limit exceeded in transit
  • Task #09 - Parameter problem - erroneous header field encountered
  • Task #10 - IPv6-in-IPv6 encapsulation
• 11-lorawan
  • Task #01 - LoRaWAN example
  • Task #02 - OTAA join procedure
  • Task #03 - ABP join procedure
  • Task #04 - LoRaWAN device parameters persistence
• 99-compile-and-test-one-board
  • Task #01 - Run tests on different hardwares

[leandrolanzieri]

PRs backported between RC1 and RC2

These PRs were merged into 2020.04-branch after 2020.04-RC1 tag was created

PR	Description
#13854	xtimer: set now pointer correctly in _update_short_timers() loop
#13883	tests/emcute: add small inter-packet gap between server replies
#13887	drivers/at25xxx: return read/written bytes to match MTD API
#13890	examples/gcoap: fix variable scope
#13904	sys/crypto/modes/ccm: handle input_len = 0
#13905	tests/xtimer_now32_overflow: simplify
#13921	boards/common/msba2: Add MINITERMFLAGS to avoid reset
#13924	cpu/lpc2387: fix RTC leap year calculation
#13938	ztimer: periph_rtt: ensure RTT_MAX_VALUE is honored
#13940	drivers/xbee: disable CTS/RTS pin unless periph_uart_hw_fc is used

[leandrolanzieri]

Summary of tests on RC1

Tests which did not pass and should be rerun

• [02-tests] Task 02 - Subset of tests on iotlab-m3
  • tests/emcute has been fixed
• [02-tests] Task 03 - Subset of tests on samr21-xpro
  • tests/emcute has been fixed

Tests which did not pass but were expected to fail

• [03-single-hop-ipv6-icmp] Task 05 - ICMPv6 stress test on native (neighbor cache stress)

  • Expected because of #12565, see results here.

• [07-multihop] Task 05 (Experimental) - UDP with large payload on iotlab-m3 with three hops (RPL route)

  • The behavior is the same since 2019.10 release: when there is fragmentation in two layers packets are lost. This is passing when using 1232 bytes of payload, but not 2048 as the spec mentions.
  • See full description here.

• [99-compile-and-test-one-board] Task 01 - Run tests on different hardware

  • There were failures on some boards. Some got fixed already. No show-stoppers here.
  • Results here.

[leandrolanzieri]

[02-tests] Task 02 - Subset of tests on iotlab-m3: PASS
[02-tests] Task 03 - Subset of tests on samr21-xpro: PASS

I confirm that tests/emcute has been fixed. The rest of the results stay the same as in RC1.

[leandrolanzieri]

There are some PRs which were backported. Particularly RIOT-OS/RIOT#13945 I will create RC3 after RIOT-OS/RIOT#13961 is merged and backported.

[leandrolanzieri]

Closing in favor of #154