Improve FileChannelJournalSegmentWriter's buffer management

Description

As noted in https://lf-opendaylight.atlassian.net/browse/CONTROLLER-2043#icft=CONTROLLER-2043, entry indexing does a rather poor job in handling its memory buffer. During indexing it specifically does this at the end of the loop iteration:

// Read more bytes from the segment if necessary. if (memory.remaining() < maxEntrySize) { memory.clear(); channel.position(position); channel.read(memory, position); memory.flip(); } memory.mark(); length = memory.getInt();

What this means is that it discards the buffer if there is even a slight chance that it holds an incomplete entry. The above condition holds true for all entries stored in the last maxEntrySize bytes of a file – which in our case can be thousands of entries!

The logic here needs to be improved so that:

  1. we check to see if there are at least 4 bytes available (to satisfy reading of the header)

  2. check and validate the entry size to see if the entry checksum and body is in the memory

  3. if it turns out we need more data, first run memory.compact() and then read up to the buffer's capacity

The end result should be that we never throw away bytes we have read and always fill the buffer its limit, if possible.

Activity

Show:

Robert Varga March 9, 2024 at 4:26 PM

Done

Details

Assignee

Reporter

Labels

Components

Fix versions

Priority

Created March 9, 2024 at 4:24 PM
Updated February 6, 2025 at 2:13 PM
Resolved March 15, 2024 at 3:28 PM