Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Release 2022.07 - RC1 #253

Closed
64 of 88 tasks
chrysn opened this issue Jul 18, 2022 · 16 comments
Closed
64 of 88 tasks

Release 2022.07 - RC1 #253

chrysn opened this issue Jul 18, 2022 · 16 comments
Labels
help wanted

Comments

@chrysn
Copy link
Member

chrysn commented Jul 18, 2022 •
edited by jia200x
Loading

This issue lists the status of all tests for the Release Candidate 1 of the 2022.07 release.

Specs tested:
@riot-ci
Copy link

riot-ci commented Jul 18, 2022 •
edited
Loading

Test Report

Task Outcome
✔ 01. Task 02 - Unittests on native PASSED
✔ 01. Task 03 - Unittests on native separated [tests-analog_util] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-base64] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bcd] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bitfield] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bloom] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-checksum] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-chunked_ringbuffer] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-clif] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-color] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-core] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-credman] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-div] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ecc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fib] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fib_sr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fmt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-frac] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gcoap] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6_hdr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6_nib] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_mac_internal] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_netif_pktq] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_sixlowpan_frag_vrb] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_udp] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-hashes] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ieee802154] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-inet_csum] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv4_addr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv6_addr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv6_hdr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-kernel_defines] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-luid] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-matstat] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-mtd] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-nanocoap] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-nanocoap_cache] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-netopt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-netreg] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-phydat] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pkt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pktbuf] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pktqueue] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-printf_float] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-priority_pktqueue] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-rtc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-rtt_rtc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-saul_reg] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-scanf_float] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-seq] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sht1x] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan_ctx] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan_sfr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sock_dns_cache] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sock_util] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-timex] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-tsrb] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-turo] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-uri_parser] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ut_process] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-uuid] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-vfs] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-zptr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ztimer64] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ztimer] PASSED
✔ 01. Task 04 - Unittests on iotlab-m3 PASSED
✔ 03. Task 01 - ICMPv6 multicast echo on native PASSED
✔ 03. Task 02 - ICMPv6 link-local echo on native PASSED
✔ 03. Task 03 - ICMPv6 link-local echo on native (1 hour) PASSED
✔ 03. Task 04 - ICMPv6 stress test on native (1 hour) PASSED
✔ 03. Task 05 - ICMPv6 stress test on native (neighbor cache stress) PASSED
✔ 03. Task 06 - ICMPv6 link-local echo on native (IPv6 fragmentation) PASSED
✔ 04. Task 02 - ICMPv6 multicast echo with iotlab-m3/samr21-xpro PASSED
✔ 04. Task 03 - ICMPv6 echo with large payload PASSED
✔ 04. Task 04 - ICMPv6 echo with iotlab-m3/samr21-xpro 15 minutes PASSED
🟡 04. Task 05 (Experimental) - ICMPv6 multicast echo with samr21-xpro/cc2538 [lille] SKIPPED
🟡 04. Task 06 (Experimental)- ICMPv6 link-local echo with samr21-xpro/cc2538 [lille] SKIPPED
🟡 04. Task 07 (Experimental) - ICMPv6 multicast echo with samr21-xpro/zero + xbee SKIPPED
🟡 04. Task 08 (Experimental) - ICMPv6 echo with samr21-xpro/zero + xbee SKIPPED
✔ 04. Task 09 - ICMPv6 stress test on iotlab-m3 PASSED
✖ 04. Task 10 (Experimental) - ICMPv6 echo with large payload (IPv6 fragmentation) FAILED
✔ 04. Task 11 (Experimental) - ICMPv6 stress test on nrf802154 PASSED
✔ 04. Task 12 (Experimental) - ICMPv6 multicast echo with iotlab-m3/nrf802154 PASSED
✔ 04. Task 13 (Experimental) - ICMPv6 link-local echo with iotlab-m3/nrf802154 PASSED
✔ 05. Task 01 - ICMPv6 echo unicast addresess on native (default route) PASSED
✔ 05. Task 02 - ICMPv6 echo unicast addresess on iotlab-m3 (default route) PASSED
✔ 05. Task 04 - ICMPv6 echo unicast addresess on iotlab-m3 (static route) PASSED
✔ 06. Task 01 - UDP on iotlab-m3 PASSED
✔ 06. Task 02 - UDP on iotlab-m3 (UDP port compression) PASSED
✔ 06. Task 03 - UDP on native (non-existent neighbor) PASSED
✔ 06. Task 04 - UDP on iotlab-m3 (non-existent neighbor) PASSED
✔ 06. Task 05 - Empty UDP on native PASSED
✔ 06. Task 06 - Empty UDP on iotlab-m3 PASSED
✔ 07. Task 01 - ICMPv6 echo on iotlab-m3 with three hops (static route) PASSED
✔ 07. Task 02 - UDP on iotlab-m3 with three hops (static route) PASSED
✔ 07. Task 03 - ICMPv6 echo on iotlab-m3 with three hops (RPL route) PASSED
✔ 07. Task 04 - UDP on iotlab-m3 with three hops (RPL route) PASSED
✔ 08. Task 01 - ICMPv6 echo between native and Linux PASSED
✔ 08. Task 08 - UDP between GNRC and lwIP on iotlab-m3 PASSED
✔ 09. Task 01 - CORD Endpoint PASSED
✔ 09. Task 02 - Confirmable retries [False-0] PASSED
✔ 09. Task 02 - Confirmable retries [True-1] PASSED
✔ 09. Task 03 - Block1 PASSED
✔ 09. Task 04 - Block2 PASSED
✔ 09. Task 05 - Observe registration and notification PASSED
✔ 10. Task 01 - Destination unreachable - no route to destination PASSED
✔ 10. Task 02 - Destination unreachable - Beyond scope of source address PASSED
✔ 10. Task 03 - Destination unreachable - address unreachable (target node address) PASSED
✔ 10. Task 04 - Destination unreachable - address unreachable (neighbor cache miss) PASSED
✔ 10. Task 05 - Destination unreachable - port unreachable PASSED
✔ 10. Task 06 - Destination unreachable - port unreachable (large payload) PASSED
✔ 10. Task 07 - Packet too big PASSED
✔ 10. Task 08 - Time exceeded - hop limit exceeded in transit PASSED
✔ 10. Task 09 - Parameter problem - erroneous header field encountered PASSED
✔ 10. Task 10 - IPv6-in-IPv6 encapsulation PASSED
✔ 11. Task 01 - LoRaWAN example PASSED
✔ 11. Task 02 - OTAA join procedure PASSED
✖ 11. Task 03 - ABP join procedure FAILED
✔ 11. Task 04 - LoRaWAN device parameters persistence PASSED
✖ 11. Task 05 - GNRC LoRaWAN (Over The Air Activation) [otaa] FAILED

@MrKevinWeiss
Copy link
Contributor

Normally @jia200x runs the LoRa tests (task 11). I think there are some issues with the required network for the autotests to work.

@MrKevinWeiss
Copy link
Contributor

Maybe we can ask @miri64 to take a look a04. Task 10 (Experimental) - ICMPv6 echo with large payload (IPv6 fragmentation) as it seems like it is failing now but was passing the last release.

@chrysn
Copy link
Member Author

chrysn commented Jul 19, 2022

The fragmentation one I'm just looking into (although admittedly more to understand better how they are run; when it comes to the actual issue it's likely I'd indeed need her help).

@MrKevinWeiss
Copy link
Contributor

I will try to run (or kindly ask HAW people to run) the interop tests.

@miri64
Copy link
Member

miri64 commented Jul 19, 2022

Maybe we can ask @miri64 to take a look a04. Task 10 (Experimental) - ICMPv6 echo with large payload (IPv6 fragmentation) as it seems like it is failing now but was passing the last release.

From the test report it looks like, that the test already failed in the flashing stage. So probably just a communication error with the testbed. Should pass in the next run (we can also just restart the workflow to see if it works this time).

@MrKevinWeiss
Copy link
Contributor

MrKevinWeiss commented Jul 19, 2022 •
edited
Loading

08. Task 03 - UDP exchange between iotlab-m3 and Contiki - PASS

iotlab-m3


udp send fe80::0212:4b00:0422:9db8 3000 1
2022-07-19 15:20:05,354 # udp send fe80::0212:4b00:0422:9db8 3000 1
2022-07-19 15:20:05,368 # Success: send 1 byte to [fe80::0212:4b00:0422:9db8]:3000
> udp send fe80::0212:4b00:0422:9db8 3000 128
2022-07-19 15:20:11,065 # udp send fe80::0212:4b00:0422:9db8 3000 128
2022-07-19 15:20:11,082 # Success: send 128 byte to [fe80::0212:4b00:0422:9db8]:3000

cc2538dk

2022-07-19 15:20:05,371 # 1 bytes from [fe80::a884:9937:4f86:7b4e]:3000
2022-07-19 15:20:11,085 # 128 bytes from [fe80::a884:9937:4f86:7b4e]:3000

@miri64
Copy link
Member

miri64 commented Jul 19, 2022 •
edited
Loading

Test Report

Task Outcome
✔ 04. Task 10 (Experimental) - ICMPv6 echo with large payload (IPv6 fragmentation) PASSED

@miri64
Copy link
Member

miri64 commented Jul 19, 2022

Passed locally :-).

@miri64
Copy link
Member

miri64 commented Jul 19, 2022

(I used tox -e test -- -k "spec04 and task10" to locally test)

@MrKevinWeiss
Copy link
Contributor

arduino-mega2560 failures

@MrKevinWeiss
Copy link
Contributor

MrKevinWeiss commented Jul 20, 2022 •
edited
Loading

pba-d-01-kw2x failures

@jia200x
Copy link
Member

jia200x commented Jul 22, 2022

11. Task 03 - ABP join procedure - PASS 
> loramac set dr 5
loramac set dr 5
> loramac set rx2_dr 3
loramac set rx2_dr 3
> loramac join abp
loramac join abp
Join procedure succeeded!
> loramac tx RIOT hola uncnf 123
loramac tx RIOT hola uncnf 123
Usage: loramac tx <payload> [<cnf|uncnf>] [port]
> loramac tx RIOT uncnf 123
loramac tx RIOT uncnf 123
Message sent with success
> loramac tx RIOT uncnf 123
loramac tx RIOT uncnf 123
The Network Server has pending data
Message sent with success
> loramac tx RIOT cnf 123
loramac tx RIOT cnf 123
Received ACK from network
Message sent with success
> loramac set dr 4   
loramac set dr 4
> loramac tx RIOT uncnf 123
loramac tx RIOT uncnf 123
Message sent with success
> loramac set dr 0          
loramac set dr 0
> loramac tx RIOT cnf 123
loramac tx RIOT cnf 123
Received ACK from network
Message sent with success
>

@jia200x
Copy link
Member

jia200x commented Jul 22, 2022

Task 05 - GNRC LoRaWAN (Over The Air Activation) - PASS 
> ifconfig
ifconfig
Iface  3  HWaddr: 26:0B:69:B5  Frequency: 868500000Hz  RSSI: -157  BW: 125kHz  SF: 12  CR: 4/5  Link: up 
           TX-Power: 14dBm  State: SLEEP  Demod margin.: 0  Num gateways.: 0 
          IQ_INVERT  
          RX_SINGLE  OTAA  L2-PDU:767  
          
> help
help
Command              Description
---------------------------------------
ifconfig             Configure network interfaces
pm                   interact with layered PM subsystem
reboot               Reboot the node
txtsnd               Sends a custom string as is over the link layer
version              Prints current RIOT_VERSION
> help        
help
Command              Description
---------------------------------------
ifconfig             Configure network interfaces
pm                   interact with layered PM subsystem
reboot               Reboot the node
txtsnd               Sends a custom string as is over the link layer
version              Prints current RIOT_VERSION
> ifconfig 3 -ack_req
ifconfig 3 -ack_req
success: unset option
> txtsnd 3 01 RIOT
txtsnd 3 01 RIOT
> PKTDUMP: data received:
~~ SNIP  0 - size:   4 byte, type: NETTYPE_UNDEF (0)
00000000  DE  AD  BE  EF
~~ SNIP  1 - size:   9 byte, type: NETTYPE_NETIF (-1)
if_pid: 3  rssi: -32768  lqi: 0
flags: 0x0
src_l2addr: (nil)
dst_l2addr: 02
~~ PKT    -  2 snips, total size:  13 byte
ifconfig 3 set dr 5
ifconfig 3 set dr 5
success: set datarate on interface 3 to 5
> ifconfig 3 ack_req
ifconfig 3 ack_req
success: set option
> txtsnd 3 01 RIOT
txtsnd 3 01 RIOT
> txtsnd 01 RIOT
txtsnd 01 RIOT
usage: txtsnd <if> [<L2 addr>|bcast] <data>
> txtsnd 3 01 RIOT
txtsnd 3 01 RIOT
>

@jia200x
Copy link
Member

jia200x commented Jul 22, 2022

All (available) LoRaWAN test passed!

@chrysn
Copy link
Member Author

chrysn commented Aug 1, 2022

Closing in favor of #254

@chrysn chrysn closed this as completed Aug 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@chrysn @miri64 @jia200x @riot-ci @MrKevinWeiss