Add support for multiple inflight files and one shot hash small files

The IO Ring now supports bashing multiple submissions and can handle
multiple files at the same time.

Hashing small files using XXH3_128bits() instead of the streaming
pipeline(XXH3_128bits_reset(), XXH3_128bits_update(),
XXH3_128bits_digest()), this reduses the overhead of creating a state
and digest, coupled with the IO Ring it improves the hashing of small
files whose size is inferior to the size of IO Ring buffers
This commit is contained in:
2026-04-02 12:08:47 +01:00
parent 41ac164881
commit 0294498538
3 changed files with 354 additions and 251 deletions

View File

@@ -87,7 +87,7 @@ int main(int argc, char **argv) {
printf(" Selected instruction set: %s\n", get_xxhash_instruction_set());
// Align IO Ring block size to the system page size
u64 g_ioring_read_block = ALIGN_UP_POW2(IORING_READ_BLOCK, g_pagesize);
g_ioring_buffer_size = ALIGN_UP_POW2(IORING_BUFFER_SIZE, g_pagesize);
// -------------------------------
// Scanning and hashing
// -------------------------------
@@ -120,6 +120,7 @@ int main(int argc, char **argv) {
// Starting hash threads
size_t num_hash_threads = num_threads;
// size_t num_hash_threads = 1;
WorkerContext workers[num_hash_threads];
Thread *hash_threads =