Socket.io Pools

Pools are a common architecture pattern to tackle requesting, caching, and syncing data entities within the Architus ecosystem between the web frontend and API server. These include Discord entities, such as Guilds, Users, and Roles, as well as Architus entities, such as Auto Responses.

Each entity pool is a dictionary of IDs to corresponding data object that grows as the frontend needs to display more values.

The reason to introduce the pool pattern is to standardize the way the web dashboard handles data entities. This design includes three main parts:

1. Requesting - The pool pattern is designed to request relevant entities/pools on-demand to reduce unnecessary network traffic and memory footprint. Note that this doesn’t mean that entire pools can’t be requested at once: it’s just that these won’t be requested until needed.
2. Caching - By caching previously requested entities, the perceived speed of the application is improved. For example, as the remainder of the entities can be requested and loaded in the background while the cached ones are displayed instantly.
3. Updating - Each entity can change throughout its lifecycle (and more importantly, a browser session), so receiving and propagating updates to “spectating” views is critical for the web dashboard to appear responsive. The backend server sends out server push events and the pool updates the corresponding entities in its local cache.

In the settings module, a variety of different setting types require a knowledge of the various relevant data entities to the current guild for validation and autocompletion. In this case, entity pools should be requested when they are needed, such as when they are needed for autocompletion for the first time. In addition, entities that are immediately displaed should be requested upon load accordingly, in order to have aspects of them like color or avatar immediately displayed.

On the left side of the main app view, relevant guilds where the Architus bot is a member are displayed, using a handful of attributes of each guild such as avatar, name, and admin status. In order for the app to seem responsive, these should reflect changes as they are made, and as such, can be included in the pool pattern to receive entity update events.

In general, the different entity pools can be divided into three distinct pool types based on the entities they contain:

This category includes Discord entities that are specific to a certain guild and are only necessary to request, cache, and update in the context of the current guild. The specific entities in this category are:

1. Members (corresponding Discord API entity) - Guild member objects, which include information like nickname and roles

   Note that these will need to be synchronized with User entities

2. Channels (corresponding Discord API entity) - Guild channels (text/voice) objects, which include information such as name, type, and permission settings

3. Roles (corresponding Discord API entity) - Guild role objects, which include information such as name and color

This category includes Discord entities that generalize beyond the context of a given guild, and as such, can be shared between contexts. By caching previously seen entities (even from another server), the perceived speed of the frontend can be faster as the remainder of the entity pool loads. The relevant entities are:

1. Users (corresponding Discord API entity) - Discord user objects, which include information such as username, avatar, and discriminator
2. Emojis (corresponding Discord API entity) - Discord emoji objects, which include information such as name/shortcode and image
3. Guilds (corresponding Discord API entity) - Discord guild (server) objects, which contain information such as name, picture, and owner

This category includes data entities that are specific to the Architus ecosystem. This includes:

1. AutoResponses (corresponding docs page) - Auto response objects

2. SettingValues (corresponding docs page) - Setting value objects

   Note that this does not include the *presentational schema* of settings, which are not considered a pool

The system is designed to integrate with Redux and the Socket.io middleware on the frontend to keep a local store of the entities.

The sub-categories are divided into whether they are guild-specific or guild-agnostic and stored accordingly. For guild-specific entities, the pools are first stored by the guild id as the top level category, and then by pool type. On the other hand, for guild-agnostic entities, each pool type is the top level category and entities are shared between contexts.

This event potentially requires guild id for context and results in an entire list of data entities returned

This event also potentially requires guild id for context and results in a single data entity as a result.

This event is a special case for member fuzzy search.

Consider generalizing to all entities? Are there sufficient use cases?

Client receive all updates for entities in the guild they are currently spectating, whether they have seen the entity before or not. Because of this, newly added or edited entities can be added to an otherwise empty pool, and deleted entity updates can be safely dropped.

More information about the spectating protocol can be found on the socket.io implementation docs.

Updating events are server push events and their payloads are incorporated into the local cache when received. With this, there are three possible update types:

1. add - New entity added. Payload contains ID and newly created entity. This also cascades to linked entities (see Foreign Keys).
2. update - Entity information updated. Payload contains ID and newly updated entity. This does not need to cascade, though the entire entity must be included to support the scenario where the client sees the entity for the first time.
3. remove - Entity removed. Payload contains only the ID, though this change needs to cascade to linked entities.

Many entities reference other data entities, such as guilds containing references to members. To prevent data duplication, entity IDs are used to perform this referencing in place of entire entities. This causes a cascading effect to be necessary upon entity updates (see Update Scenarios).

Consider using some type of dependency graph to encode this information for frontend/backend processing of dependency cascades

Last modified 2 years ago Edit this page on GitHub