sliptty: introduce a new SLIP to TUN tool #10477

miri64 · 2018-11-27T13:50:28Z

Contribution description

sliptty is a new tool for SLIP-to-TUN dispatching. In comparison to tunslip6 it is made to be very stripped down (592 vs 1294 loc). However, in addition to that it supports stdio packet formats according to SLIPMUX (however not the CoAP part - yet).

Some stuff was taken from ethos to ensure IoT-LAB support and some stuff from tunslip6 and #6718 to ensure MacOSX support.

Testing procedure

The an application using slipdev should be able to communicate with Linux via the interface created by
dist/tools/sliptty/start_network (without stdio of course). Also try pinging the node (or the linux host) with payload sizes 184 and 211. They will cause the length field of the IPv6 header to be escaped according to the slip protocol, so if it works it works correctly.

Please also test on MacOSX.

Issues/PRs references

Some code migrated from #6718. There will be a follow-up to adapt slipdev for the stdio multiplexing.

jnohlgard

Some comments below. Will test this later but not much time right now.

dist/tools/sliptty/start_network

dist/tools/sliptty/Makefile

dist/tools/sliptty/start_network

jnohlgard

Some new comments

dist/tools/sliptty/start_network

jnohlgard · 2018-12-17T08:30:15Z

The ethos script has a hack to make the Linux host use a proper source address when the host has no IPv6 global addresses configured (any address not beginning in fe80::).

ip a a fd00:dead:beef::1/128 dev lo

Without this hack and without any other global IPv6 addresses configured on the Linux machine, the outgoing packets directed to global addresses on the RIOT nodes will have the correct destination address, but in lack of global addresses, the source address will be the automatic link local address of the tun0 interface (not fe80::1, but the fe80::random:numbers address). The RIOT node messes up because of missing neighbor entry for the tun0 LL addr and sends replies over the wireless instead.

I don't like the hack, but it is unrelated to this PR to fix the neighbor configuration for the point-to-point tunnel interface.

miri64 · 2018-12-17T09:56:47Z

The RIOT node messes up because of missing neighbor entry for the tun0 LL addr and sends replies over the wireless instead.

Sounds more like the RIOT node somewhere loses the interface context :-/. Really hard for me to debug for me, since I don't have access to a machine that has no global IPv6 address. But I try to find one :D (or set-up a VM for that).

miri64 · 2018-12-17T10:57:53Z

The culprit is this netif rewrite

RIOT/sys/net/gnrc/network_layer/ipv6/gnrc_ipv6.c

Lines 619 to 623 in 469c31c

 else if ((!ipv6_addr_is_link_local(&hdr->dst)) || 

 (*netif == NULL)) { 

 *netif = gnrc_netif_get_by_ipv6_addr(&hdr->dst); 

 return (*netif == NULL); 

 }

Now I'm unsure what the correct behavior would be. IMHO, since the global address is on the other interface, but the source is link-local address I'd say sending a ICMPv6 error destination unreachable (beyond scope of source address) back to the originator and dropping the packet.

miri64 · 2018-12-17T11:00:15Z

(which it actually does in

RIOT/sys/net/gnrc/network_layer/ipv6/gnrc_ipv6.c

Lines 758 to 761 in 469c31c

 if (ipv6_addr_is_link_local(&(hdr->src)) && 

 !ipv6_addr_is_link_local(&(hdr->dst))) { 

 gnrc_icmpv6_error_dst_unr_send(ICMPV6_ERROR_DST_UNR_SCOPE, pkt); 

 }

but since the packet is not technically forwarded and the interface is rewriten this is never happening).

miri64 · 2018-12-17T11:14:16Z

@gebart regardless, I'd rather merge without the hack and find a satisfactory fix for this. Do you agree?

miri64 · 2018-12-17T11:14:48Z

Because we apparently never fixed that bug, because the hack was shadowing it ;-).

jnohlgard · 2018-12-20T13:32:54Z

@miri64 I agree, let's fix it properly in a separate PR.

jnohlgard

Does the RIOT side handle empty SLIP frames?
draft-bormann-t2trg-slipmux-02 recommends using SLIP_END both at the start and the end of the frame, currently we only use SLIP_END at the end of the frame.

https://tools.ietf.org/html/draft-bormann-t2trg-slipmux-02#section-6

To make SLIP framing robust, it is important to send SLIP frame delimiters (0xc0) before and after each SLIP frame (maybe unless frames follow each other back to back). This means that empty frames need to be silently ignored by a receiver.

miri64 · 2019-01-24T08:32:39Z

Does the RIOT side handle empty SLIP frames?

Have to check.

miri64 · 2019-03-27T17:38:13Z

Code-wise it should be able to. I don't have the ability to test at the moment...

miri64 · 2019-04-02T13:17:08Z

Code-wise it should be able to. I don't have the ability to test at the moment...

@gebart I compiled examples/gnrc_border_router with this patch and started dist/tools/sliptty/sliptty (after compiling it) with the according port /dev/ttyUSB1 in my case. The then initialized TUN interface I configured as follows (taken from dist/tools/sliptty/start_network):

sudo ip link set up dev sl0
sudo ip address add fe80::1/64 dev sl0
sudo ip neigh add fe80::2 dev sl0

I then tried to ping the node which worked:

ping6 -c 3 fe80::2%sl0
PING fe80::2%sl0(fe80::2) 56 data bytes
64 bytes from fe80::2: icmp_seq=1 ttl=64 time=93.5 ms
64 bytes from fe80::2: icmp_seq=2 ttl=64 time=93.5 ms
64 bytes from fe80::2: icmp_seq=3 ttl=64 time=93.5 ms

--- fe80::2%sl0 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 93.536/93.538/93.541/0.249 ms

I then terminated slipdev and used xxd to inject empty SLIP frames and too short IPv6 packets:

echo "c0c0" | xxd -r -p > /dev/ttyUSB0
echo "c060c0" | xxd -r -p > /dev/ttyUSB0
echo "c06000c0" | xxd -r -p > /dev/ttyUSB0

I see nothing wrong with the output:

2019-04-02 15:14:01,700 - INFO # gnrc_netif: GNRC_NETDEV_MSG_TYPE_EVENT received
2019-04-02 15:14:01,703 - INFO # gnrc_netif: event triggered -> 2
2019-04-02 15:14:01,707 - INFO # gnrc_netif: waiting for incoming messages
2019-04-02 15:14:01,712 - INFO # gnrc_netif: GNRC_NETDEV_MSG_TYPE_EVENT received
2019-04-02 15:14:01,714 - INFO # gnrc_netif: event triggered -> 2
2019-04-02 15:14:01,718 - INFO # gnrc_netif: waiting for incoming messages
[…]
2019-04-02 15:15:15,942 - INFO # gnrc_netif: GNRC_NETDEV_MSG_TYPE_EVENT received
2019-04-02 15:15:15,945 - INFO # gnrc_netif: event triggered -> 2
2019-04-02 15:15:15,948 - INFO # gnrc_netif: waiting for incoming messages
2019-04-02 15:15:15,953 - INFO # gnrc_netif: GNRC_NETDEV_MSG_TYPE_EVENT received
2019-04-02 15:15:15,956 - INFO # gnrc_netif: event triggered -> 2
2019-04-02 15:15:15,959 - INFO # gnrc_netif: waiting for incoming messages
[…]
2019-04-02 15:15:53,590 - INFO # gnrc_netif: GNRC_NETDEV_MSG_TYPE_EVENT received
2019-04-02 15:15:53,593 - INFO # gnrc_netif: event triggered -> 2
2019-04-02 15:15:53,597 - INFO # gnrc_netif: waiting for incoming messages
2019-04-02 15:15:53,601 - INFO # gnrc_netif: GNRC_NETDEV_MSG_TYPE_EVENT received
2019-04-02 15:15:53,604 - INFO # gnrc_netif: event triggered -> 2
2019-04-02 15:15:53,608 - INFO # gnrc_netif: waiting for incoming messages
2019-04-02 15:15:53,612 - INFO # ipv6: GNRC_NETAPI_MSG_TYPE_RCV received
2019-04-02 15:15:53,616 - INFO # ipv6: error marking IPv6 header, dropping packet
2019-04-02 15:15:53,619 - INFO # ipv6: waiting for incoming message.

So in conclusion: The RIOT side is able to handle empty SLIP frames ;-)

miri64 · 2019-04-02T13:20:38Z

BTW the one-byte packet is declined in gnrc_netif_raw.c:

RIOT/sys/net/gnrc/netif/gnrc_netif_raw.c

Line 68 in d6cad3c

if (nread <= 1) { /* we need at least 1 byte to identify IP version */

This doesn't seem right to me in combination with the comment, but not harmful either. ~~I will piggy-back a fix in #10480~~ (that PR doesn't touch gnrc_netif_raw so this would be weird).

stale · 2019-10-04T13:34:49Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

benpicco · 2020-02-19T12:32:50Z

What's the advantage of SLIP over ethos?
We are having trouble with packet loss at moderately light data rates with ethos, would SLIP likely be a better option?

miri64 · 2020-02-19T13:01:19Z

What's the advantage of SLIP over ethos?

It is simpler (no intermediate MAC layer) and a ~~standardized~~ specified protocol (so there are tools that support SLIP, other than our self-rolled ones, see e.g. this wikibook on how to use with Linux, though IPv6 support for those is rare)

We are having trouble with packet loss at moderately light data rates with ethos, would SLIP likely be a better option?

As SLIP only uses byte stuffing, while ethos also adds Ethernet logic, there might be a performance benefit, but IIRC from my bachelor thesis there is also a performance cap depending on the baudrate (duh).

miri64 · 2020-03-04T12:49:06Z

I merged just the current master and used an FTDI adapter with a samr21-xpro for the SLIP connection via the second UART and I did not have those problems using

[TERM1]$ USE_SLIP=1 make -C examples/gnrc_border_router/ flash -j16
[TERM2]$ make -C dist/tools/sliptty
[TERM2]$ dist/tools/sliptty/start_network 2001:db8::/64 /dev/ttyUSB0
[TERM1]$ USE_SLIP=1 make -C examples/gnrc_border_router/ term
> ping6 fe80::1%7
2020-03-04 13:09:35,726 #  ping6 fe80::1%7
2020-03-04 13:09:35,743 # 12 bytes from fe80::1: icmp_seq=0 ttl=64 time=11.950 ms
2020-03-04 13:09:36,743 # 12 bytes from fe80::1: icmp_seq=1 ttl=64 time=11.898 ms
2020-03-04 13:09:37,742 # 12 bytes from fe80::1: icmp_seq=2 ttl=64 time=12.067 ms
2020-03-04 13:09:37,742 # 
2020-03-04 13:09:37,745 # --- fe80::1 PING statistics ---
2020-03-04 13:09:37,750 # 3 packets transmitted, 3 packets received, 0% packet loss
2020-03-04 13:09:37,755 # round-trip min/avg/max = 11.898/11.971/12.067 ms
> ping6 -s 100 fe80::1%7
2020-03-04 13:10:10,330 # ping6 -s 100 fe80::1%7
2020-03-04 13:10:10,365 # 108 bytes from fe80::1: icmp_seq=0 ttl=64 time=29.046 ms
2020-03-04 13:10:11,364 # 108 bytes from fe80::1: icmp_seq=1 ttl=64 time=29.260 ms
2020-03-04 13:10:12,363 # 108 bytes from fe80::1: icmp_seq=2 ttl=64 time=29.132 ms
2020-03-04 13:10:12,364 # 
2020-03-04 13:10:12,366 # --- fe80::1 PING statistics ---
2020-03-04 13:10:12,371 # 3 packets transmitted, 3 packets received, 0% packet loss
2020-03-04 13:10:12,376 # round-trip min/avg/max = 29.046/29.146/29.260 ms
> ping6 -s 184 fe80::1%7
2020-03-04 13:11:22,211 # ping6 -s 184 fe80::1%7
2020-03-04 13:11:22,260 # 192 bytes from fe80::1: icmp_seq=0 ttl=64 time=43.995 ms
2020-03-04 13:11:23,259 # 192 bytes from fe80::1: icmp_seq=1 ttl=64 time=43.941 ms
2020-03-04 13:11:24,258 # 192 bytes from fe80::1: icmp_seq=2 ttl=64 time=44.173 ms
2020-03-04 13:11:24,259 # 
2020-03-04 13:11:24,261 # --- fe80::1 PING statistics ---
2020-03-04 13:11:24,267 # 3 packets transmitted, 3 packets received, 0% packet loss
2020-03-04 13:11:24,271 # round-trip min/avg/max = 43.941/44.036/44.173 ms
> ping6 -s 211 fe80::1%7
2020-03-04 13:12:15,067 # ping6 -s 211 fe80::1%7
2020-03-04 13:12:15,121 # 219 bytes from fe80::1: icmp_seq=0 ttl=64 time=48.642 ms
2020-03-04 13:12:16,120 # 219 bytes from fe80::1: icmp_seq=1 ttl=64 time=48.866 ms
2020-03-04 13:12:17,119 # 219 bytes from fe80::1: icmp_seq=2 ttl=64 time=48.570 ms
2020-03-04 13:12:17,119 # 
2020-03-04 13:12:17,122 # --- fe80::1 PING statistics ---
2020-03-04 13:12:17,127 # 3 packets transmitted, 3 packets received, 0% packet loss
2020-03-04 13:12:17,132 # round-trip min/avg/max = 48.570/48.692/48.866 ms

Same from the host side:

[mlenders@sarajevo RIOT]<3 ping6 fe80::2%sl0
PING fe80::2%sl0(fe80::2%sl0) 56 data bytes
64 bytes from fe80::2%sl0: icmp_seq=2 ttl=64 time=21.2 ms
64 bytes from fe80::2%sl0: icmp_seq=3 ttl=64 time=21.3 ms
64 bytes from fe80::2%sl0: icmp_seq=4 ttl=64 time=21.4 ms
64 bytes from fe80::2%sl0: icmp_seq=5 ttl=64 time=21.4 ms
^C
--- fe80::2%sl0 ping statistics ---
5 packets transmitted, 4 received, 20% packet loss, time 4031ms
rtt min/avg/max/mdev = 21.182/21.325/21.440/0.094 ms
[mlenders@sarajevo RIOT]<3 ping6 -s 184 fe80::2%sl0
PING fe80::2%sl0(fe80::2%sl0) 184 data bytes
192 bytes from fe80::2%sl0: icmp_seq=1 ttl=64 time=44.0 ms
192 bytes from fe80::2%sl0: icmp_seq=2 ttl=64 time=44.4 ms
192 bytes from fe80::2%sl0: icmp_seq=3 ttl=64 time=44.3 ms
192 bytes from fe80::2%sl0: icmp_seq=4 ttl=64 time=43.9 ms
192 bytes from fe80::2%sl0: icmp_seq=5 ttl=64 time=44.3 ms
192 bytes from fe80::2%sl0: icmp_seq=6 ttl=64 time=44.3 ms
192 bytes from fe80::2%sl0: icmp_seq=7 ttl=64 time=44.0 ms
192 bytes from fe80::2%sl0: icmp_seq=8 ttl=64 time=44.1 ms
192 bytes from fe80::2%sl0: icmp_seq=9 ttl=64 time=44.2 ms
^C
--- fe80::2%sl0 ping statistics ---
9 packets transmitted, 9 received, 0% packet loss, time 8011ms
rtt min/avg/max/mdev = 43.942/44.166/44.399/0.157 ms
[mlenders@sarajevo RIOT]<3 ping6 -s 211 fe80::2%sl0
PING fe80::2%sl0(fe80::2%sl0) 211 data bytes
219 bytes from fe80::2%sl0: icmp_seq=1 ttl=64 time=49.1 ms
219 bytes from fe80::2%sl0: icmp_seq=2 ttl=64 time=49.1 ms
219 bytes from fe80::2%sl0: icmp_seq=3 ttl=64 time=49.1 ms
219 bytes from fe80::2%sl0: icmp_seq=4 ttl=64 time=49.3 ms
219 bytes from fe80::2%sl0: icmp_seq=5 ttl=64 time=49.0 ms
^C
--- fe80::2%sl0 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4005ms
rtt min/avg/max/mdev = 48.950/49.141/49.342/0.124 ms

I also double checked that the length field contains in fact the bytes 0xc0 (SLIP END byte) and 0xdb (SLIP ESC byte) respectively with wireshark.

Maybe the bug is in #10480, so can you please retry without it?

benpicco · 2020-03-04T23:06:55Z

Yup, works like a charm when used standalone!
It would do 2 MBaud with a FT232RL without any hickups.

Feel free to squash - want to integrate it into examples/gnrc_border_router too?
It's just two lines.

dist/tools/sliptty/start_network

miri64 · 2020-03-05T11:41:04Z

Feel free to squash - want to integrate it into examples/gnrc_border_router too?
It's just two lines.

It is already. The change of TERMPROG / TERMFLAG does not work without the multiplexing that is introduced in #10480.

benpicco · 2020-03-05T13:36:37Z

The change of TERMPROG / TERMFLAG does not work without the multiplexing that is introduced in #10480.

It's still pretty handy as it takes all the prefix/baudrate/port parameters straight from the Makefile 🤷

miri64 · 2020-03-05T13:56:54Z

The change of TERMPROG / TERMFLAG does not work without the multiplexing that is introduced in #10480.

It's still pretty handy as it takes all the prefix/baudrate/port parameters straight from the Makefile 🤷

But then again: this disables make term to provide you a terminal, but just starts the (non-interactable) host endpoint for the SLIP connection. So IMHO, completely breaks expected behavior.

benpicco · 2020-03-05T14:11:05Z

Fair enough …

How about

# TODO uncomment after #10480 is merged
# TERMPROG ?= sudo sh $(RIOTTOOLS)/sliptty/start_network.sh
# TERMFLAGS ?= $(IPV6_PREFIX) $(PORT) $(SLIP_BAUDRATE)

So people who want to use SLIP already (and don't care so much about stdout) don't have to memorize/figure out the right start parameters themself.

miri64 · 2020-03-05T14:45:59Z

They can also still use tunslip. I would prefer to add this in #10480 only.

benpicco · 2020-03-05T14:53:14Z

But I'd still have to search for the configuration of tunslip as well…

Well then, guess I have to forego this bit of comfort for now - please squash.

miri64 · 2020-03-05T14:54:47Z

Well then, guess I have to forego this bit of comfort for now - please squash.

IMHO the comfort you seek is a trap ;-).

miri64 · 2020-03-05T14:55:15Z

I will rebase and adapt #10480 to your wishes as soon as this PR is merged :-)

miri64 · 2020-03-05T14:55:37Z

Squashed

benpicco

Tested on Linux with and without #10480
stdio output over SLIPMUX works, but connection errors occur.
stable connection without SLIPMUX, 2000000 Baud over FT232L worked without issues.

miri64 · 2020-03-05T16:10:10Z

stdio output over SLIPMUX works, but connection errors occur.
stable connection without SLIPMUX, 2000000 Baud over FT232L worked without issues.

Note: #10480 does not really implement SLIPMUX, as SLIPMUX includes a CoAP parser ;-)

benpicco · 2020-03-05T17:08:32Z

SLIPMUX includes a CoAP parser

Well of course, why wouldn't it! 😆

miri64 · 2020-03-05T17:10:50Z

Have fun ;-) https://tools.ietf.org/html/draft-bormann-t2trg-slipmux-03

benpicco · 2020-03-05T17:16:41Z

The present report is somewhat radical in that it does not provide a
common staple of interactive computer access: A command line
interface (CLI), or "shell".

This would be easy to add, but distracts from the use of the platform
as a "thing" - it should not have to carry an (even primitive) user
interface; instead it should provide what would have been "shell
commands" as CoAP resources.

As a transition aid, existing shell commands can first be converted
to just accept their parameters via CoAP but continue to provide
their output as ASCII text over the diagnostic channel.

However, in order to aid script-driven use of the platform, the next
step should then be to also provide the response to the command in a
CoAP response, possibly structured for better use by the script.
Often code that was designed to format the data for human consumption
can be simplified to just ship the raw data, e.g. in a CBOR data item
[RFC7049].

How do they say: When you've invented a hammer, everything starts to look like a nail.

miri64 · 2020-03-05T17:37:34Z

How do they say: When you've invented a hammer, everything starts to look like a nail.

But that's basically it. The assumption of SLIPMUX is: you already have an IoT node there that is talked to by CBOR via CoAP, so why not re-use those parsers instead of requiring new ones.

miri64 · 2020-03-05T17:38:52Z

(+ if you don't support the full feature set a CoAP message parser could be as minimal as a simple whitespace tokenizer, maybe even more minimal).

benpicco · 2020-03-05T17:41:17Z

Yea but as a debugging tool, you don't want the added complexity.
When sending CoAP messages, you might just as well use the IP link that you've just established.

miri64 · 2020-03-05T17:57:34Z

Yea but as a debugging tool, you don't want the added complexity.
When sending CoAP messages, you might just as well use the IP link that you've just established.

That's why you still have the diagnostic channel (which in an extended way [due to its bidirectionality] #10480 basically implemented)

miri64 added Area: network Area: Networking Type: new feature The issue requests / The PR implemements a new feature for RIOT Area: tools Area: Supplementary tools labels Nov 27, 2018

miri64 requested review from aabadie, kaspar030 and smlng November 27, 2018 13:50

miri64 force-pushed the sliptty/feat/initial branch from ca5d846 to 66aeb28 Compare November 27, 2018 14:48

miri64 mentioned this pull request Nov 27, 2018

slipdev: provide stdio multiplexing over SLIP #10480

Merged

jnohlgard reviewed Dec 13, 2018

View reviewed changes

jnohlgard reviewed Dec 16, 2018

View reviewed changes

dist/tools/sliptty/start_network Outdated Show resolved Hide resolved

dist/tools/sliptty/start_network Outdated Show resolved Hide resolved

dist/tools/sliptty/start_network Outdated Show resolved Hide resolved

dist/tools/sliptty/start_network Outdated Show resolved Hide resolved

miri64 mentioned this pull request Dec 18, 2018

ethos: Add OS X compatibility #6718

Closed

jnohlgard reviewed Dec 20, 2018

View reviewed changes

miri64 mentioned this pull request Apr 9, 2019

tools/tunslip: correct typos and refine readme #11363

Merged

miri64 mentioned this pull request Sep 25, 2019

ethos: fails to respond to first message. #11988

Open

stale bot added the State: stale State: The issue / PR has no activity for >185 days label Oct 4, 2019

miri64 removed the State: stale State: The issue / PR has no activity for >185 days label Oct 4, 2019

benpicco reviewed Mar 4, 2020

View reviewed changes

dist/tools/sliptty/start_network Outdated Show resolved Hide resolved

sliptty: introduce a new SLIP to TUN tool

917ba19

miri64 force-pushed the sliptty/feat/initial branch from fc69a75 to 917ba19 Compare March 5, 2020 14:55

benpicco added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Mar 5, 2020

benpicco approved these changes Mar 5, 2020

View reviewed changes

benpicco merged commit 0b1dce3 into RIOT-OS:master Mar 5, 2020

miri64 deleted the sliptty/feat/initial branch March 5, 2020 17:10

This was referenced Mar 6, 2020

gnrc_ipv6_nib: don't ignore PL for route #10627

Merged

sliptty/start_network.sh: configure global address for SLIP interface #13572

Merged

leandrolanzieri added this to the Release 2020.04 milestone Mar 13, 2020

miri64 mentioned this pull request Aug 4, 2020

dist/tools/sliptty/start_network.sh: IPv6 connectivity is broken on PC #14689

Open

sliptty: introduce a new SLIP to TUN tool #10477

sliptty: introduce a new SLIP to TUN tool #10477

Conversation

miri64 commented Nov 27, 2018 • edited Loading

Contribution description

Testing procedure

Issues/PRs references

jnohlgard left a comment

Choose a reason for hiding this comment

jnohlgard left a comment

Choose a reason for hiding this comment

jnohlgard commented Dec 17, 2018

miri64 commented Dec 17, 2018 • edited Loading

miri64 commented Dec 17, 2018

miri64 commented Dec 17, 2018

miri64 commented Dec 17, 2018

miri64 commented Dec 17, 2018

jnohlgard commented Dec 20, 2018

jnohlgard left a comment

Choose a reason for hiding this comment

miri64 commented Jan 24, 2019

miri64 commented Mar 27, 2019

miri64 commented Apr 2, 2019

miri64 commented Apr 2, 2019 • edited Loading

stale bot commented Oct 4, 2019

benpicco commented Feb 19, 2020

miri64 commented Feb 19, 2020 • edited Loading

miri64 commented Mar 4, 2020

benpicco commented Mar 4, 2020 • edited Loading

miri64 commented Mar 5, 2020

benpicco commented Mar 5, 2020

miri64 commented Mar 5, 2020 • edited Loading

benpicco commented Mar 5, 2020

miri64 commented Mar 5, 2020

benpicco commented Mar 5, 2020

miri64 commented Mar 5, 2020

miri64 commented Mar 5, 2020

miri64 commented Mar 5, 2020 • edited Loading

benpicco left a comment

Choose a reason for hiding this comment

miri64 commented Mar 5, 2020

benpicco commented Mar 5, 2020

miri64 commented Mar 5, 2020

benpicco commented Mar 5, 2020 • edited Loading

miri64 commented Mar 5, 2020

miri64 commented Mar 5, 2020

benpicco commented Mar 5, 2020

miri64 commented Mar 5, 2020

miri64 commented Nov 27, 2018 •

edited

Loading

miri64 commented Dec 17, 2018 •

edited

Loading

miri64 commented Apr 2, 2019 •

edited

Loading

miri64 commented Feb 19, 2020 •

edited

Loading

benpicco commented Mar 4, 2020 •

edited

Loading

miri64 commented Mar 5, 2020 •

edited

Loading

miri64 commented Mar 5, 2020 •

edited

Loading

benpicco commented Mar 5, 2020 •

edited

Loading