We use RaftActor’s persistence in a rather twisty way to store two pieces of state:
snapshots, in our custom snapshot store
journal entries (and book-keeping to go with them) in our custom persistence provider
Furthermore, we exchange state in terms of AppendEntries and InstallSnapshot, both of which offer a message slicing mechanism for state which ends up applied to state (and to user state, i.e. ShardDataTree and scenery).
Design the interface to access state entries, EntryStore, which will allow:
each journal entry has RaftEntryMeta
there is a provision to read the specified entry
Internally, the entry store needs work like a deque, tracking commitIndex, which is the used to supported:
with oldest-entry trimming (i.e. when covered by a snapshot), for index<=commitIndex
selective invalidate (i.e. when purging previous term), for index > commitIndex
For the initial cut we want to use RaftJournal, which stores Payloads, with bare minimum lifted from atomix-storage, etc. if need be. Note we need to fragment entries, so EntryReader would return a RaftEntry fragment – index,term,<last:boolean>,data[]. That means commitIndex → EntryReader.nextIndex() needs to be tracked.
Activity
Show:
Robert Varga January 28, 2025 at 4:46 PM
The bare minimum we need is glue between PersistenceProvider and ReplicatedLogImpl. One implementation talks to PersistenceProvider, the other to SegmentedJournal.
The second part of this is dealing with which ReplicatedLogEntries are in-memory and which are in cold storage.
Medium
We use RaftActor’s persistence in a rather twisty way to store two pieces of state:
snapshots, in our custom snapshot store
journal entries (and book-keeping to go with them) in our custom persistence provider
Furthermore, we exchange state in terms of AppendEntries and InstallSnapshot, both of which offer a message slicing mechanism for state which ends up applied to state (and to user state, i.e. ShardDataTree and scenery).
Design the interface to access state entries, EntryStore, which will allow:
each journal entry has RaftEntryMeta
there is a provision to read the specified entry
Internally, the entry store needs work like a deque, tracking commitIndex, which is the used to supported:
with oldest-entry trimming (i.e. when covered by a snapshot), for index<=commitIndex
selective invalidate (i.e. when purging previous term), for index > commitIndex
For the initial cut we want to use RaftJournal, which stores Payloads, with bare minimum lifted from atomix-storage, etc. if need be. Note we need to fragment entries, so EntryReader would return a RaftEntry fragment – index,term,<last:boolean>,data[]. That means commitIndex → EntryReader.nextIndex() needs to be tracked.