The Fastest Way to Fix %eyre

~dachus-tiprel - 2.22.23

This is a followup post to my WebSockets proposal. While I still intend to implement WS in the future, that is a large project that will take months; we need something that will work now. This plan is a robust, simple, and easy way to get great performance out of %eyre without much effort. The idea is to build a separate system within eyre that doesn't use subscriptions or SSE - just pokes and scries. Here is a rough outline of the interactions:

1. %eyre <> Client Interaction

1. Client begins with no data, and must scry out the current state of the %gall app (this must be implemented by the %gall app)
2. Client tells %eyre that it wants to establish a realtime connection with that app
3. %eyre acknowledges that the connection has been made
4. Client sends GETs to %eyre every 1s for new updates on that connection.
   • This is essentially a channel subscription, but where the client is responsible for requesting updates, rather than %eyre being responsible for sending them down.

2. %eyre Vane Logic

1. Client asks %eyre to establish a connection
2. If correct, %eyre creates an outbox of new messages for that connection
3. The outbox will get filled by the gall app with new updates
4. The client will GET the latest contents of the outbox
   • To prevent bloat, we have to clear the outbox from time to time. There are a bunch of options here - easiest is probably just to use a %behn timer once we have confirmation that certain messages have been received.
5. If there are no pings for a long time, eyre will eliminate this connection
   • Eyre’s current parameters will work, though we can tune them. They say "if there hasn’t been a client ping in 30s, move it to a waiting state. If a connection is in the waiting state and it has 50 messages, remove it. If a connection in the waiting state hasn’t been reconnected in 12h, kill it"

3. %eyre <> %gall Communication

When writing a gall app using this new system, there are no subscriptions involved. Intead of %facts, we will add a new $task to %eyre called %push. %pushing fills all outboxes that need this data from your app. The card will look something like [%pass /my-wire %arvo %e %push %my-mark !>(data)]. This is really similar to a %fact, except we are manually pushing everything to %eyre, rather than relying on the subscription system to do it for us.

Conclusion

This should solve the big problem with SSE; reconnection is extremely simple and cannot be terminated accidentally by forces outside of our control. Furthermore, I can build this quickly (~1mo), and will keep backwards compatibility.

~dachus-tiprel

Credit to ~timluc-miptev and ~rovnys-ricfer for this idea

powered by %blog, download at ~hanrut-sillet-dachus-tiprel