Skip to content
This repository has been archived by the owner on Sep 27, 2021. It is now read-only.

Commit

Permalink
feat(*): rewrite
Browse files Browse the repository at this point in the history
  • Loading branch information
thetutlage committed Feb 25, 2018
1 parent 4e13356 commit a8b7d15
Show file tree
Hide file tree
Showing 30 changed files with 8,445 additions and 0 deletions.
13 changes: 13 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# editorconfig.org
root = true

[*]
indent_size = 2
indent_style = space
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true

[*.md]
trim_trailing_whitespace = false
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
coverage
node_modules
.DS_Store
npm-debug.log
.idea
out
.nyc_output
12 changes: 12 additions & 0 deletions .npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
coverage
node_modules
.DS_Store
npm-debug.log
test
.travis.yml
.editorconfig
benchmarks
.idea
bin
out
.nyc_output
Empty file added README.md
Empty file.
18 changes: 18 additions & 0 deletions doc/external.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
= Websocket

=== Always rely on websocket connection and do not make use of long-polling etc.
This makes it easier to scale using Node.js cluster module and distribute load on multiple servers, without implimenting sticky sessions.

Whereas, with long-polling, we need to make sure right request goes to right node when upgrading to Websocket connection, which is impossible with cluster module.

=== Middleware on handshake
All of the middleware will be executed on handshake, which keeps sockets layer clean, since once you are connected, you are safe.

=== Open packet
Server will emit open packet, sharing important info like `pingInterval`, `pingTimeout` etc.

=== Ping/Pong
Server and client both needs to play ping/pong. If no ping received from client during defined `pingInterval`, the connection will be closed from the server.

=== Client closing connection
Client can close connection by sending `close` event, optionally with `code` and `status` message. Client will receive the `close` event back in return.
38 changes: 38 additions & 0 deletions doc/internal.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
= Websocket

=== Detecting memory leaks
Ensure there are no memory leaks.

1. Always drop sockets when they have been closed.
2. Ping/pong to drop inactive sockets.
3. Do not store too much data.

=== Handle chrome disconnect bug
Chrome disconnect is not graceful so handle the error code and ignore it.

=== Add proper debugging
Proper debugging is super useful

=== Data encoders
Allow encoders to be configurable. Useful when someone is not using one of the official client libraries.
All of the messages are packed using link:https://www.npmjs.com/package/msgpack-lite[Msgpack lite].

=== Packet specs
----
{
t: 'event name',
d: 'data associated with it'
}
----

one underlying connection
one underlying request

=> channels are routes
=> connection has it's own id
=> socket id is `name#connect-id`
=> multiple sockets one for each channels
=> socket disconnects using events
=> socket connects using events
=> packet reference https:/socketio/socket.io-protocol#packet
=>
174 changes: 174 additions & 0 deletions doc/protocol.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
# Adonis WebSocket
Adonis WebSocket library works on top of [WebSocket protocol](https://tools.ietf.org/html/rfc6455) and uses [ws](https:/websockets/ws) library as the base to build upon.

This document describes the philosophy behind the library and shares the custom vocabulary added to the process.

## Terms Used
#### Packet
The packet sent from `client -> server` and `server -> client`. Each packet must have a type.

#### Channels
Channels makes it possible to separate the application concerns without creating a new TCP connection.

#### Topics
Topics are subscribed on a given channel. If channel name is static then the topic name will be the same as the channel name.

For example:

```js
Ws.channel('chat', function ({ socket }) {
console.log(socket.topic)
// will always be `chat`
})
```

If channel name has a wildcard, then multiple matching topics can be subscribed.

```js
Ws.channel('chat:*', function ({ socket }) {
})
```

This time topic name can be anything after `chat:`. It can be `chat:watercooler` or `chat:design` and so on.

The dynamic channel names makes it even simple to have a dynamic topics and a user can subscribe to any topic they want.

## Pure WebSockets
Adonis WebSocket uses pure WebSocket connection and never relies on pooling. All of the browsers have support for WebSockets and there is no point in adding fallback layers.

By creating a pure WebSocket connection, we make it easier to scale apps horizontally, without relying on sticky sessions. Whereas with solutions like `socket.io` you need sticky sessions and it's even harder to use Node.js cluster module.

## Multiplexing
If you have worked with WebSockets earlier ( without any library ), you would have realized, there is no simple way to separate application concerns with in a single TCP connection.

If you want to have multiple channels like `chat` and `news`, the client have to open 2 separate TCP connections, which is waste of resources and overhead on server and client both.

Adonis introduces a layer of Channels, which uses a single TCP connection and uses messages a means of communicating within the channels.

If you are a consumer of the Adonis WebSocket library, you will get a clean abstraction to make use of channels.

If you are a developer creating a client library, then you will have to understand the concept of Packets and what they mean.

## Packets
Packets are a way to communicate between client and server using WebSocket messages.

For example: A packet to join a channel looks as follows.

```js
{
t: 1,
d: { topic: 'chat' }
}
```

1. The `t` is the packet code.
2. And `d` is the packet data. Each packet type has it's own data requirements.

Actions like `JOIN` and `LEAVE` are always acknowledged from the server with a successful acknowledgement or with an error.

Following is an example of `JOIN_ERROR`.

```js
{
t: 4,
d: {
topic: 'chat',
message: 'Topic has already been joined'
}
}
```

Here's the list of packet types and their codes.

```js
{
OPEN: 0,
JOIN: 1,
LEAVE: 2,
JOIN_ACK: 3,
JOIN_ERROR: 4,
LEAVE_ACK: 5,
LEAVE_ERROR: 6,
EVENT: 7,
PING: 8,
PONG: 9
}
```

**Why numbers?** : Because it's less data to transfer.

A simple example of using Packets to recognize the type of message. The following code is supposed to be executed on browser.

```js
// assuming Adonis encoders library is pulled from CDN.

const ws = new WebSocket('ws://localhost:3333')
const subscriptions = new Map()

function makeJoinPacket (topic) {
return { t: 1, d: { topic } }
}

ws.onopen = function () {
// storing we initiated for the subscription
subscriptions.set('chat', false)

const payload = msgpack.encode(makeJoinPacket('chat'))
ws.send(payload)
}

ws.onmessage = function (payload) {
const packet = msgpack.decode(payload)

if (packet.t && packet.t === 3) {
// join acknowledgement from server
}

if (packet.t && packet.t === 4) {
// join error from server
}
}
```

## Contracts
By now as you know the messaging packets are used to build the channels and topics flow, below is the list of contracts **client and server** has to follow.

1. **client**: `JOIN` packet must have a topic.
2. **server**: Server acknowledges the `JOIN` packet with `JOIN_ERROR` or `JOIN_ACK` packet. Both the packets will have the topic name in them.
3. **server**: Ensure a single TCP connection can join a topic only for one time.
4. **client**: Optionally can enforce a single topic subscription, since server will enforce it anyway.
5. **client**: `EVENT` packet must have a topic inside the message body, otherwise packet will be dropped by the server.
6. **server**: `EVENT` packet must have a topic inside the message body, otherwise packet will be dropped by the client too.

The `LEAVE` flow works same as the `JOIN` flow.

## Ping/Pong
Wish networks would have been reliable, but till then always be prepared for ungraceful disconnections. Ping/Pong is a standard way for client to know that a server is alive and vice-versa.

In order to distribute load, AdonisJs never pings clients to find if they are alive or not, instead clients are expected to ping the server after given interval.

If a client fails to ping the server, their connection will be dropped after defined number of retries. Also for every `ping`, client will receive a `pong` from the server, which tells the client that the server is alive.

AdonisJs supports standard [ping/pong frames](https://tools.ietf.org/html/rfc6455#section-5.5.2) and if your client doesn't support sending these frames, then you can send a message with the `Packet type = PING`.

1. A single ping/pong game is played for a single TCP connection, there is no need to ping for each channel subscription.
2. When a connection is established, server sends an `OPEN` packet to the client, which contains the data to determine the ping interval.

```js
{
t: 0,
d: {
serverInterval: 30000,
serverAttempts: 3,
clientInterval: 25000,
clientAttempts: 3,
connId: 'connection unique id'
}
}
```

All of the times are in milliseconds and `clientAttempts` is the number of attempts to be made by the client before declaring server as dead and same is true for server using the `serverAttempts` property.

## Browser Support
WebSockets are supported on all major browsers, so there is no point of adding weird fallbacks.
https://caniuse.com/#feat=websockets
69 changes: 69 additions & 0 deletions example/decoders.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
const parser = require('socket.io-parser')
const msgpack = require('msgpack-lite')

const encoder = new parser.Encoder()
const decoder = new parser.Decoder()

function encode (payload) {
return new Promise((resolve) => {
let msgpackEncoded
let messagePackDecoded

msgpack.Encoder().on('data', (data) => {
msgpackEncoded = data
encoder.encode(payload, function (encoded) {
decode(msgpackEncoded, 'msgpack')
.then((result) => {
messagePackDecoded = result
return decode(encoded, 'socketio')
})
.then((result) => {
resolve({
msgpack: {
size: Buffer.byteLength(msgpackEncoded),
decoded: messagePackDecoded
},
socketio: {
size: Buffer.byteLength(encoded[0]),
decoded: result
}
})
})
})
}).encode(payload)
})
}

function decode (payload, service) {
return new Promise((resolve) => {
if (service === 'msgpack') {
return resolve(msgpack.decode(payload))
}
decoder.on('decoded', function (decoded) {
resolve(decoded)
})
for (var i = 0; i < payload.length; i++) {
decoder.add(payload[i])
}
})
}

encode({type: parser.EVENT, data: { username: 'foo', age: 22 }})
.then((result) => {
console.log('json', JSON.stringify(result, null, 2))
return encode({type: parser.BINARY_EVENT, data: Buffer.from('1234')})
})
.then((result) => {
console.log('buffer', JSON.stringify(result, null, 2))
return encode({type: parser.BINARY_EVENT, data: new Int8Array(10)})
})
.then((result) => {
console.log('int8array', JSON.stringify(result, null, 2))
return encode({type: parser.BINARY_EVENT, data: new ArrayBuffer([2])})
})
.then((result) => {
console.log('arraybuffer', JSON.stringify(result, null, 2))
})
.catch((error) => {
console.log('error', error)
})
37 changes: 37 additions & 0 deletions example/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
<!DOCTYPE html>
<html>
<head>
<title></title>
</head>
<body>
<script type="text/javascript" src="https://rawgit.com/kawanet/msgpack-lite/master/dist/msgpack.min.js"></script>
<script type="text/javascript">
const client = new window.WebSocket('ws://localhost:3000/adonis-ws')

client.onerror = function (event) {
console.log(event)
}

client.onmessage = function (packet) {
var arrayBuffer
var fileReader = new window.FileReader()

fileReader.onload = function () {
arrayBuffer = this.result
}

fileReader.onloadend = function () {
console.log(window.msgpack.decode(new Uint8Array(arrayBuffer)))
setTimeout(() => {
client.send(window.msgpack.encode({ event: 'close' }))
}, 2000)
}
fileReader.readAsArrayBuffer(packet.data)
}

client.onopen = function (event) {
console.log('open')
}
</script>
</body>
</html>
19 changes: 19 additions & 0 deletions example/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
'use strict'

const http = require('http')
const { Config } = require('@adonisjs/sink')
const fs = require('fs')
const path = require('path')

const Ws = require('../src/Ws')
const ws = new Ws(new Config())

const server = http.createServer((req, res) => {
const html = fs.readFileSync(path.join(__dirname, './index.html'))
res.writeHead(200, { 'content-type': 'text/html' })
res.write(html)
res.end()
})

ws.listen(server)
server.listen(3000)
Loading

0 comments on commit a8b7d15

Please sign in to comment.