Improve FileChannelJournalSegmentWriter's buffer management
Description
is blocked by
split from
Activity
Show:
Robert Varga March 9, 2024 at 4:26 PM
There is a code overlap between https://lf-opendaylight.atlassian.net/browse/CONTROLLER-2043#icft=CONTROLLER-2043 and https://lf-opendaylight.atlassian.net/browse/CONTROLLER-2095#icft=CONTROLLER-2095. Those need to be addressed before this issue is tackled.
Done
Details
Details
Assignee
Oleksandr Zharov
Oleksandr ZharovReporter
Robert Varga
Robert VargaLabels
Components
Priority
Created March 9, 2024 at 4:24 PM
Updated February 6, 2025 at 2:13 PM
Resolved March 15, 2024 at 3:28 PM
As noted in https://lf-opendaylight.atlassian.net/browse/CONTROLLER-2043#icft=CONTROLLER-2043, entry indexing does a rather poor job in handling its memory buffer. During indexing it specifically does this at the end of the loop iteration:
// Read more bytes from the segment if necessary. if (memory.remaining() < maxEntrySize) { memory.clear(); channel.position(position); channel.read(memory, position); memory.flip(); } memory.mark(); length = memory.getInt();
What this means is that it discards the buffer if there is even a slight chance that it holds an incomplete entry. The above condition holds true for all entries stored in the last maxEntrySize bytes of a file – which in our case can be thousands of entries!
The logic here needs to be improved so that:
we check to see if there are at least 4 bytes available (to satisfy reading of the header)
check and validate the entry size to see if the entry checksum and body is in the memory
if it turns out we need more data, first run memory.compact() and then read up to the buffer's capacity
The end result should be that we never throw away bytes we have read and always fill the buffer its limit, if possible.