[WASI] sockets

[pavelsavara]

Limitations

permanent design of WASI ?

• WASIp2 doesn't support SO_LINGER -> PlatformNotSupportedException
• WASIp2 doesn't support DONTFRAGMENT->IP_MTU_DISCOVER -> PlatformNotSupportedException
• WASIp2 doesn't support Unix domain socket -> PlatformNotSupportedException
• WASIp2 doesn't support Raw socket -> PlatformNotSupportedException
• WASIp2 doesn't support DualMode IPv6+IPv4
  • explicit set to true using the API-> PlatformNotSupportedException
  • no opt-in by default when IPv6
  • default Socket constructor without addressFamily parameter uses IPv4 only!
• WASIp2 doesn't support multicast or broadcast -> PlatformNotSupportedException
• WASIp2 is always providing PacketInformation
• WASIp2 is always SO_REUSEADDR

temporary limitation

• WASIp2 doesn't support most of IOControl -> PlatformNotSupportedException until
• WASIp2 doesn't support SO_RCVTIMEO and SO_SNDTIMEO -> PlatformNotSupportedException for now

until multi-threading WASI

• WASIp2 doesn't have threads and we don't want to block the main thread on network operations
  • So only async and non-blocking APIs are supported
  • So all synchronous APIs which could block -> PlatformNotSupportedException
    • Connect, Send, Receive, Poll, Accept, SendFile, SendTo, ReceiveFrom
    • when the socket is non-blocking, those APIs only make sense together with Select, Poll
      • which can't be implemented without blocking all other C# tasks
    • and so it doesn't make sense to consider non-blocking option for any of them
  • also all Begin* End* methods -> PlatformNotSupportedException. Because the *End methods are blocking.
  • we force all sockets to be non-blocking via SOCK_NONBLOCK on libc level
  • Socket.Blocking is default false as opposed to all other dotnet platforms.
  • as a consequence all blocking APIs of NetworkStream are also -> PlatformNotSupportedException

Changes in this PR

• use libc: sendto and recvfrom for WASI instead of missing recvmsg and sendmsg.
  • therefore no I/O vector
• SystemNative_GetWasiSocketDescriptor -> wasi-libc internal descriptor_table_get_ref
  • so that we could poll for socket pollables together with clock and http pollables
• new WasiEventLoop.RegisterWasiPollHook for the same reason
• I re-used SafeSocketHandle.Unix.cs, Socket.Unix.cs, SocketAsyncContext.Unix.cs
• I didn't re-use SocketAsyncEngine.Unix.cs and it's threads
• I implemented new SocketAsyncEngine.Wasi.cs and integrated it with pollables and main event loop
• in the future, when there is multi-threading, sharing unix/posix PAL will make even more sense
• integrating with libc allows C-code linked with the same libc to PInvoke SafeHandle and share it

Alternative design

• we could eventually implement the same functionality without wasi-libc emulation, directly on top of wasi-sockets API.
• the feature set would be the same, but code re-use would be much smaller

Unit tests

• lot of the tests are using blocking APIs, they are annotated with IsThreadingSupported now
• I updated few tests which were blocking but didn't have to stay blocking
• few tests will need further investigation, [ActiveIssue]
• I marked some tests with [SkipOnPlatform], for the APIs which I believe would not be fixed in the future

TODO

• TODO revisit HAVE_SOCKADDR_UN_SUN_PATH and UnixDomainSocketEndPoint

Related issues

Filled wasi-libc issues

• fcntl doesn't implement O_NONBLOCK for wasi:socket handles WebAssembly/wasi-libc#536
• fstat doesn't return S_IFSOCK for wasi:socket handles WebAssembly/wasi-libc#537
• ioctl doesn't implement FIONREAD for wasi:socket handles WebAssembly/wasi-libc#538
• getsockopt, setsockopt doesn't implement SO_RCVTIMEO,SO_SNDTIMEO for wasi:socket handles WebAssembly/wasi-libc#539
• __wasi_sockets_utils__map_error is missing conversion for NETWORK_ERROR_CODE_INVALID_STATE for connect wasi:socket  WebAssembly/wasi-libc#540
• polling on mix of libc-posix socket/file descriptors with non-libc WASI pollable handles WebAssembly/wasi-libc#542

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[dicej]

LGTM. Thanks for doing all this!

I think we should eventually consider supporting blocking operations even without multithreading. Some single-threaded command line apps are content to use blocking operations since they have nothing else to do besides wait for the network. And although it's clearly an antipattern, some popular libraries (e.g. Microsoft.Data.SqlClient) do sync-on-async operations. Clearly we want to fix those libraries anyway (and the SqlClient folks are already planning to do so), but in the meantime I'd be inclined to maximize compatibility by supporting the existing patterns.

Anyway, I'm fine with the PNEs for blocking ops in this PR -- it's a reasonable default.

[pavelsavara]

I think we should eventually consider supporting blocking operations even without multithreading.

I was thinking about it. Filesystem and DNS requests are blocking after all.
But I think that it would do more harm than good.

• Consider that even timers/timeout are implemented as Pollable in WASI. They would not work!
• I think that most dotnet developers and existing codebases assume that scheduling jobs on "thread pool" or starting other async tasks, would make progress independently from I/O. This would not work if we block on network I/O.
• We could not run nested thread-pool loop from inside of synchronous method. It would break state of whatever is on the stack.
• It's much better to "fail fast" at compile time, instead of having users debugging it and trying to "fix it".
• If customers can't switch the code to async, it's probably not going to work on single thread anyway.
• "Pit of Success" in dotnet means we should not create traps like that.

to maximize compatibility by supporting the existing patterns.

• We don't have any other dotnet target which would behave like that.
• We have ST in browser, which behaves consistently with what I implemented here.

[dicej]

Filesystem and DNS requests are blocking after all.

Not necessarily -- WASI provides pollables for DNS and (most?) filesystem ops. Granted, the host implementation may require blocking + multithreading due to OS limitations, but from the guest's perspective they're async.

Consider that even timers/timeout are implemented as Pollable in WASI. They would not work!

Timeouts could still work; the blocking API would still use pollables in the implementation.

I think that most dotnet developers and existing codebases assume that scheduling jobs on "thread pool" or starting other async tasks, would make progress independently from I/O. This would not work if we block on network I/O.

That's fair; I'm new enough to .NET that I don't know how tightly bound the concepts of blocking I/O and multithreading are. My only real-world data point so far is SqlClient, which worked fine when we added blocking I/O to the prototype.

We could not run nested thread-pool loop from inside of synchronous method. It would break state of whatever is on the stack.

We can call wasi:io/poll/poll (or wasi:io/poll/pollable.block) directly, though, i.e. without running the whole thread pool loop recursively.

Your other points seem valid, though.

[pavelsavara]

Not necessarily -- WASI provides pollables for DNS and (most?) filesystem ops.

But we consume that via wasi-libc which is blocking. We are not going to implement WASI-only filesystem PAL any time soon.

Timeouts could still work; the blocking API would still use pollables in the implementation.
We can call wasi:io/poll/poll (or wasi:io/poll/pollable.block) directly, though, i.e. without running the whole thread pool loop recursively.

We would learn that something got unblocked, but we can't call the continuations from inside blocking call.
And we can't return from it without it being finished either.

[dicej]

Not necessarily -- WASI provides pollables for DNS and (most?) filesystem ops.

But we consume that via wasi-libc which is blocking. We are not going to implement WASI-only filesystem PAL any time soon.

👍

Timeouts could still work; the blocking API would still use pollables in the implementation.
We can call wasi:io/poll/poll (or wasi:io/poll/pollable.block) directly, though, i.e. without running the whole thread pool loop recursively.

We would learn that something got unblocked, but we can't call the continuations from inside blocking call. And we can't return from it without it being finished either.

There wouldn't be any Tasks or continuations, though, right? We're just doing a blocking call and returning the result without using any of the Task infrastructure. E.g. call wasi:io/streams/input-stream.read; if that returns an empty buffer, call subscribe and pollable.block, then call read again, then we're done.

[pavelsavara]

We would learn that something got unblocked, but we can't call the continuations from inside blocking call. And we can't return from it without it being finished either.

There wouldn't be any Tasks or continuations, though, right?

We are getting very off-topic on this PR. But I think that it would go wasi:clock->pollable event->C# timer-> CancellationTokenSource.Cancel -> any C# code with side-effects. All of it synchronously from that pollable event. And that's breaking state invariants of the code calling the pending blocking synchronous sendto.
Even if we returned EWOULDBLOCK the while loop around that sendTo would be very probably in synchronous code. And so TP loop would not get a chance to propagate that cancellation (out of the synchronous code). We could try untangle that, but let's not do it not here.

[jsturtevant]

A question that came up last week during the BCA plummers summit was if this will automatically pull in the wasi-socket imports for all wasm components after implemented or only when System.net.Sockets api's are used?

[wfurt]

I'm not sure we need to throw if the platform does more than asked for. If we have concerns we can can strip to info internally IMHO.

[pavelsavara]

this is only throwing when user asks to disable PacketInformation

[wfurt]

right. We can still simply throw it out. And not receiving the info is default AFAIK. mostly perf improvement IMHo but that probably does not matter for your use case.

[wfurt]

I don;t think you need this. This is related to thread pool optimization and there are no threads on WASI, right?

[pavelsavara]

There is InlineSocketCompletionsEnabled -> SafeSocketHandle.PreferInlineCompletions -> context.PreferInlineCompletions which needs it.

I don't understand why this is not a static member on SafeSocketHandle. Do we expect the env variable to change mid-flight ?

For WASI I made it constant on SocketAsyncEngine

[wfurt]

generally looks ok to me. There many check IsWasi. It may be worth to come up with variable describing the limitation - ... like SupportsSync ... but I guess that is just style and personal preference.

[pavelsavara]

There many check IsWasi. It may be worth to come up with variable describing the limitation - ... like SupportsSync ...

When the WASI public surface stabilizes bit more, we will add UnsupportedOSPlatformAttribute which will prevent user code to compile with those APIs. And most of the conditions in the socket code will disappear.

But for now, you are right that expressing reason is better. I added Socket.OSSupportsThreads for that.

[pavelsavara]

A question that came up last week during the BCA plummers summit was if this will automatically pull in the wasi-socket imports for all wasm components after implemented or only when System.net.Sockets api's are used?

It will create that dependency, I think via wasi:cli imports.
I hope that dependency on descriptor_table_get_ref will not keep the linker from trimming it.

[liveans]

Well, I had couple of questions but already got answer to them from the discussions/messages under this PR, everything looks good now to me.