We use RaftActor’s persistence in a rather twisty way to store two pieces of state:
snapshots, in our custom snapshot store
journal entries (and book-keeping to go with them) in our custom persistence provider
Furthermore, we exchange state in terms of AppendEntries and InstallSnapshot, both of which offer a message slicing mechanism for state which ends up applied to state (and to user state, i.e. ShardDataTree and scenery).
Design the interface to access state snapshots, SnapshotStore, which will allow:
to start receiving a snapshot in Follower role, where each snapshot is identified as “<lastIndex>-<term>-<leader>.receive”, driven by InstallSnapshot and encapsulating SnapshotTracker logic
perform an asynchronous ‘apply state’ from this handle, during which the snapshot is identified as “<lastIndex>-<term>-<leader>.apply”
snapshots are stored in a directory
there is an archive of applied snapshots, identified as “<lastIndex>-<term>.archived”
the snapshot with the greatest lastIndex is considered “last snapshot”, from which a set of Payloads are stored (i.e. journal entries)
Activity
Show:
Robert Varga 14 hours ago
This ends up creating quite a number of patches, as the interface between sal-akka-raft and its users (i.e. sal-distributed-datastore) is quite arcane. The overall theme is improved scalability, though.
Highest
We use RaftActor’s persistence in a rather twisty way to store two pieces of state:
snapshots, in our custom snapshot store
journal entries (and book-keeping to go with them) in our custom persistence provider
Furthermore, we exchange state in terms of AppendEntries and InstallSnapshot, both of which offer a message slicing mechanism for state which ends up applied to state (and to user state, i.e. ShardDataTree and scenery).
Design the interface to access state snapshots, SnapshotStore, which will allow:
to start receiving a snapshot in Follower role, where each snapshot is identified as “<lastIndex>-<term>-<leader>.receive”, driven by InstallSnapshot and encapsulating SnapshotTracker logic
perform an asynchronous ‘apply state’ from this handle, during which the snapshot is identified as “<lastIndex>-<term>-<leader>.apply”
snapshots are stored in a directory
there is an archive of applied snapshots, identified as “<lastIndex>-<term>.archived”
the snapshot with the greatest lastIndex is considered “last snapshot”, from which a set of Payloads are stored (i.e. journal entries)