1 Message Store DRAFT
Dan Ballard edited this page 2019-09-24 13:56:22 -07:00
This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

1. Intro

Cwtch needs a robust message store that can handle different types and sizes of messages, maintain ordering even when messages arrive out of order, and provides a consentful system allowing fine grained control of when messages are stored or not, and when to purge them. It should also handle other fine grained data like deliver-receipts. The message store should be the source of truth for the system on message related questions (order, data, time, delivery status). It also needs to be durable and not lose data in the case of application crashes or faults, which is especially important in a mobile setting.

1.1. Requirements

Cwtch currently supports two fundemenally different streams where by messages can come from: group chats on cwtch servers, and direct p2p messaging. Group messages by necesity of meta data resistance need to be the same size in transit, and that also caps out how large they can be, but currently p2p messages can be any size and are planed to be used to large data such as images and files. Messages need to be stored in order and support out of order delivery. The message store needs to be durable, it should be file backed.

  • Durability: Should not lose mesages, even if not succesfuly sent, when app exits
  • Disappearing Messages: Meesages should be able to be purged, or have purging policies attached to them
  • Message Events: sent, recevied receipt, received new message
  • Ordering: always present ordered list, even if receiving out of order

1.2 Questions

  • How to handle sending message sent receipts? Is the message store the source of truth so this can be resumed after a quit and restart? Should the timeline be polled at start for unsent messages to resume trying?
  • Event notifications of new messages: who is responsible for notifying UI of new messages, especially when they arrive out of order, and also notifying hte ui of delivery receipt updates?

2. Potential Solutions

2.1 Local Timelines vs Central Message Store

We could start with creating a Timeline object and each Group and contact in our contact list would be given one to track group and p2p chats. Alternatively, we could create a central message store, that is handed all new messages and message events, and internally stores them in a way to make reading a timeline by Place in order easy for the using application.

2.2 Journal Fronted Message Store

Regardless of the central store or timeline per Place/contact, the best solution for durability will be to front the in memory store with an ondisk journal, as that as events happen (message sending, message in a received) we can record them at the time they happen, in a linear format to make replay in case of app exit reliable. This starts to introduce concerns in cases we dont want long term message storeage (disappearing messages).

2.3 Retention and Purge Policy

Each timeline will need to be have a retention and purge policy, initially copied from a system default. This can control the length of time messages should be stored for, trigger purges, and in cases of absolute zero retention, bypass journaling. This more than other requirments starts to favor a centralized message store approach, so that it can create a schedule of deletion, and set one thread and timer to manage it.

2.3 Pruning/Triming and Dissappearing Messages

Messages can always be deleted client side. Global pruning policies can be set and per timelines ones can override it, only in order to shorten the pruning period. Messages can have disappearing policies attached to them, set by the sender, and it will be respected, overriding any local pruning policy. Diappearing messages will never be journaled or stored, they opt out of the storage system for safety, they simply exist in memory storage. This should inform UI decisions on max lifespace probably keeping it set to "hours". This more than other requirments starts to favor a centralized message store approach, so that it can create a schedule of deletion, and set one thread and timer to manage it.

2.4 Ordering

We record received timestamps in messages, and the come with sent timestamps. We will order timelines by sent time.

2.5 Scheduler

There will be a scheduler where deletion and pruning tasks are registered with

2.6 Deep storage

in some cases, users will still be storing a lot of messages and we may not want to load them all at start up. We should be able to load just the latest N messages from storage. For the system work work fully, we would need the scheduler to store its own file related deletion schedules in a recoverable log so that it could still know about ares scheduled for deletion without them being loaded. Aside from recommending the deletion schedule logs be saved for file based activity, this suggests we store histories in chunks, along sizes N, so that as pruning policies dictate, we can just perform a file delete operation to keep faithful.

3. Detailed Design

There will be a centralized message store. It will support:

  • Init(): loads timelines from filesystem, loads journals, if any, and plays back last message history, also restarting sending of messages if some found unsent, starts a pruning thread that watches messages set to disappear and following message triming policies
  • SetDefaultTrimingPolicy( policy )
  • SetTimelineTrimingPolicy ( timelineID, policy )
  • AddMessage( timelineId, message )
  • UpdateMessage( timelineId, messageID, status )
  • GetTimeline( timelineId, max )
  • GetUnsentMessages( ?timelineID? )

3.1 Saving

The default flow of a message coming into the store will be to hit the journal, a log/append style file for the timeline, if active. Then it will enter the in memory storage of the timeline. At preset times, the in memory timelines will be saved to the filesystem and the journals purged. We prefer journal writes as they are linear and in order. The in memory store is capable of changing its order and so needs its entierty written each save, so we want to minimize that somewhat, and not for instane trigger that every single time a message is received in a high volumn discussion.

3.2 Triming and Disappearing messages

There will be a cental thread and system to calculate the schedule of times to drop messages. As messages are dropped from the store, a save event will be triggered. Additionally we will need a mechanism to update the UI.

4. Work to do

TBD pending review of plan