feat: IPC transport (UNIX domain socket)

[merklefruit]

Warning: this is a breaking API change. -> No it's not after #78!

Added the IPC transport type, backed by tokio::net::UnixStream. This required an abstraction over the Transport address type, which is now handled by a trait Address:

/// A trait for address types that can be used by any transport.
pub trait Address: Clone + Debug + Send + Sync + Unpin + Hash + Eq + 'static {}

/// IP address types, used for TCP and QUIC transports.
impl Address for SocketAddr {}

/// File system path, used for IPC transport.
impl Address for PathBuf {}

// trait with generic address
pub trait Transport<A: Address> { ... }

This is because IPC sockets bind to actual filesystem paths rather than IP addresses.
For this reason, now most sockets have a generic Transport<A: Address> impl block, plus an impl with the Transport<SocketAddr> concrete type and same for Transport<PathBuf>.

Benchmarks

Added benchmarks to our criterion suite. Here are a few interesting results:

# TCP BYTES
reqrep_single_thread_tcp_bytes/512
                        time:   [26.503 ms 26.561 ms 26.666 ms]
                        thrpt:  [183.11 MiB/s 183.83 MiB/s 184.24 MiB/s]

reqrep_single_thread_tcp_bytes/65536
                        time:   [324.38 ms 335.76 ms 349.26 ms]
                        thrpt:  [1.7475 GiB/s 1.8178 GiB/s 1.8816 GiB/s]

# TCP RPS
reqrep_single_thread_tcp_rps/512
                        time:   [26.382 ms 26.641 ms 26.918 ms]
                        thrpt:  [371.50 Kelem/s 375.36 Kelem/s 379.04 Kelem/s]

reqrep_single_thread_tcp_rps/65536
                        time:   [316.30 ms 321.33 ms 326.84 ms]
                        thrpt:  [30.596 Kelem/s 31.120 Kelem/s 31.615 Kelem/s]

# IPC BYTES
reqrep_single_thread_ipc_bytes/512
                        time:   [11.088 ms 11.109 ms 11.141 ms]
                        thrpt:  [438.28 MiB/s 439.55 MiB/s 440.35 MiB/s]

reqrep_single_thread_ipc_bytes/65536
                        time:   [391.24 ms 397.22 ms 406.57 ms]
                        thrpt:  [1.5012 GiB/s 1.5365 GiB/s 1.5600 GiB/s]

# IPC RPS
reqrep_single_thread_ipc_rps/512
                        time:   [10.966 ms 11.024 ms 11.101 ms]
                        thrpt:  [900.83 Kelem/s 907.13 Kelem/s 911.93 Kelem/s]

reqrep_single_thread_ipc_rps/65536
                        time:   [390.30 ms 392.84 ms 396.88 ms]
                        thrpt:  [25.196 Kelem/s 25.456 Kelem/s 25.621 Kelem/s]

Interestingly, with the current implementation IPC has a much higher throughput with smaller messages, while TCP is a bit faster with bigger messages.

• 512 bytes throughput: TCP: 183.83 MiB/s vs ICP: 439 MiB/s
• 512 bytes RPS: TCP: 375.36 Kelem/s vs ICP: 907.13 Kelem/s

This is a 2.4x increase for small messages!

Related links:

• Unix domain sockets slower than TCP for larger messages MagicStack/uvloop#345
• https://lists.freebsd.org/pipermail/freebsd-performance/2005-February/001143.html

[mempirate]

Looks great 🚀

[esemeniuc]

Linux results show IPC faster

reqrep_single_thread_tcp_bytes/512
                        time:   [58.944 ms 59.215 ms 59.385 ms]
                        thrpt:  [82.224 MiB/s 82.458 MiB/s 82.838 MiB/s]
reqrep_single_thread_tcp_bytes/65536
                        time:   [394.10 ms 397.37 ms 400.86 ms]
                        thrpt:  [1.5226 GiB/s 1.5360 GiB/s 1.5487 GiB/s]
Found 2 outliers among 10 measurements (20.00%)
  2 (20.00%) high mild

reqrep_single_thread_tcp_rps/512
                        time:   [59.426 ms 60.302 ms 60.940 ms]
                        thrpt:  [164.10 Kelem/s 165.83 Kelem/s 168.28 Kelem/s]
Found 3 outliers among 10 measurements (30.00%)
  1 (10.00%) low mild
  2 (20.00%) high severe
reqrep_single_thread_tcp_rps/65536
                        time:   [407.11 ms 412.95 ms 419.19 ms]
                        thrpt:  [23.855 Kelem/s 24.216 Kelem/s 24.563 Kelem/s]

Benchmarking reqrep_single_thread_ipc_bytes/512: Warming up for 3.0000 s2024-08-30T20:54:58.576416Z  INFO msg_socket::rep::driver: New connection from "/tmp/msg-bench-reqrep-ipc.sock"
reqrep_single_thread_ipc_bytes/512
                        time:   [44.413 ms 44.576 ms 44.733 ms]
                        thrpt:  [109.15 MiB/s 109.54 MiB/s 109.94 MiB/s]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe
reqrep_single_thread_ipc_bytes/65536
                        time:   [326.62 ms 331.29 ms 336.15 ms]
                        thrpt:  [1.8157 GiB/s 1.8424 GiB/s 1.8687 GiB/s]

reqrep_single_thread_ipc_rps/512
                        time:   [43.902 ms 44.189 ms 44.412 ms]
                        thrpt:  [225.17 Kelem/s 226.30 Kelem/s 227.78 Kelem/s]
reqrep_single_thread_ipc_rps/65536
                        time:   [329.07 ms 330.47 ms 331.82 ms]
                        thrpt:  [30.137 Kelem/s 30.260 Kelem/s 30.389 Kelem/s]