amir/filehasher
1
0
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Reworking process_completion() function

Browse Source
This commit is contained in:
amir
2026-05-02 16:17:17 +01:00
parent fb83c3114f
commit 73aa4808f2
5 changed files with 269 additions and 164 deletions
Download Patch File Download Diff File Expand all files Collapse all files

53
README.md
View File

@@ -26,13 +26,38 @@ It is a high performance cross platform Windows and Linux compatible program, it
UCRT64 uses the modern Universal C Runtime (ucrtbase.dll), which supports the newest APIs,
the standard MSYS2 uses the legacy msvcrt.dll and does not support IO Ring.
To install:
pacman -S mingw-w64-ucrt-x86_64-gcc
pacman -S mingw-w64-ucrt-x86_64-clang
pacman -S mingw-w64-ucrt-x86_64-cmake
or:
pacman -S mingw-w64-ucrt-x86_64-gcc
pacman -Syu
And add to path:
C:\msys64\ucrt64\bin
Additionally, to use clang-cl install the latest version of Windows SDK and MSVC, or at least select these in Visual Studio Installer:
* MSVC Build tools fo x64/86.
* C++ Build tools core features.
* MSBuild support for LLVM (clang-cl) toolset.
* Windows Universal C runtime.
* Windows Universal CRT SDK.
* Windows 11 SDK.
And use the MSVC command prompt or run a script to add MSVC environment variables to current session.
Ex: for PowerShell Terminal save as .ps1 (not persistent):
```ps1
# Add MS visual studio environment variables
cmd /c '"C:\Program Files (x86)\Microsoft Visual Studio\18\BuildTools\VC\Auxiliary\Build\vcvarsall.bat" x64 && set' |
ForEach-Object {
if ($_ -match "^(.*?)=(.*)$") {
Set-Item -Path "Env:$($matches[1])" -Value $matches[2]
}
}
```
Optional: to use the build system
pacman -S mingw-w64-ucrt-x86_64-cmake
The build system uses Ninja and fallsback to make, in Windows it prefers clang-cl > gcc > clang, and in Linux gcc > clang.
### Using a build system
| Command | Description|
| :--- | :--- |
@@ -52,7 +77,7 @@ clang -g -O0 file_hasher.c xxhash.c xxh_x86dispatch.c -o filehasher
clang-cl /Zi /Od file_hasher.c xxhash.c xxh_x86dispatch.c
## Linux
**Requirements**: GCC, CMake and Ninja
**Requirements**: GCC or clang, optional CMake, Ninja or make.
### Using a build system
| Command | Description|
@@ -109,13 +134,16 @@ While both systems share the same core concept, their APIs and management styles
| **Partial Updates** | Supports `IORING_REGISTER_FILES_UPDATE` to swap specific indices. | No partial updates; a new registration replaces the entire table. |
| **Scope of Operations** | Extremely broad (files, sockets, timers, signals, etc.). | Primarily focused on file storage (read, write, flush). |
### Completion Wait count
### Completion Wait count and peek
To avoid busy waiting when receiving CQEs, we can use io_uring_submit_and_wait() in Linux by entering a wait count,
the threads sleeps until the count of CQEs are received, in windows the wait_count is present in SubmitIoRing()
the threads sleep until the count of CQEs are received, in windows the wait_count is present in SubmitIoRing()
but is not implemented yet, so we wait with a completion event for a single completion. Another limitation on the completion
event is that the kernel will waik up the thread only when receiving the first CQE, after that we need to drain the completion
queue completely before sleeping again, or we enter an eternal slumber. And my config, each time the thread wakes up
it receives rarely more than 3 to 5 CQEs and most of the time only one CQE.
queue completely before sleeping again, or we enter an eternal slumber.
In the other hand, in Linux we can batch pop completions with io_uring_peek_batch_cqe() + io_uring_cq_advance(),
in Windows we can only pop one completion at a time with PopIoRingCompletion() (equivalent to io_uring_peek_cqe() + io_uring_cqe_seen()).
To simulate the same behavior as the Linux functions we use a double loop, an outer loop to control how much we wait
and in inner loop to drain all the available completions.
### Filtering CQEs
@@ -149,12 +177,15 @@ IO Ring implementation.
"Increase the limit to solve this warning.\n");
```
The Memlock limit in Linux restricts the amount of memory a process can
"lock" into physical RAM using the mlock() family of system calls. This
The Memlock limit in Linux restricts the amount of memory that can be
"locked" into physical RAM using the mlock() family of system calls. This
prevents the operating system from swapping that memory out to disk.
And registering buffers will lock the buffers memory so the hardware
can access it directly without kernel intervention and prevents the kernel from
swapping it to the SSD or HDD. Increase the limit to be able to register the buffers.
swapping it to the SSD or HDD.
This limit does not apply to a single process, but it applies to what all the runnig processes can lock, so in order
to be able to register the buffers, we need to set it to unlimited or increase it to at least:
num_hash_threads * NUM_BUFFERS_PER_THREAD * IORING_BUFFER_SIZE + extra memory reserved for other processes.
#### *Modifying the Limit*
The method for changing the memlock limit depends on whether you are
@@ -166,7 +197,7 @@ the /etc/security/limits.conf file. Add the following lines:
```conf
# Example for a specific user (replace 'username'), unlimited or a custom value in KB
username soft memlock unlimited
username hard memlock unlimited
username hard memlock unlimitedhttps://wiki.postgresql.org/wiki/AIO
```
```conf
# Example for all users

96
build.bat
View File

@@ -2,14 +2,23 @@
setlocal enabledelayedexpansion
:: ============================================================================
:: build.bat - Build script with compiler preference: clang-cl > gcc > clang
:: Usage: build [Release|Debug] [clean]
:: build.bat
:: ============================================================================
:: Get script directory (project root)
set SCRIPT_DIR=%~dp0
set SCRIPT_DIR=%SCRIPT_DIR:~0,-1%
:: ---------------------------------------------------------------------------
:: Default values
:: ---------------------------------------------------------------------------
set BUILD_TYPE=Release
set CLEAN_BUILD=0
:: --------------------------------------------------------------------------
:: Parse arguments
:: --------------------------------------------------------------------------
:parse_args
if "%~1"=="" goto :main
@@ -29,27 +38,34 @@ if /i "%~1"=="clean" (
goto :parse_args
)
:: Unknown argument fallback (the *)
echo Unknown argument: %~1
echo Usage: .\%~nx0 [Release^|Debug] [clean]
echo Usage: build [Release^|Debug] [clean]
exit /b 1
:main
set BUILD_DIR=build\windows\%BUILD_TYPE%
set BUILD_DIR=%SCRIPT_DIR%\build\windows\%BUILD_TYPE%
echo === Building filehasher (%BUILD_TYPE%) ===
:: --------------------------------------------------------------------------
:: Clean if requested
:: --------------------------------------------------------------------------
if %CLEAN_BUILD%==1 (
echo Cleaning...
if exist "%BUILD_DIR%" rmdir /s /q "%BUILD_DIR%" 2>nul
)
:: Create build dir
:: --------------------------------------------------------------------------
:: Create build directory
:: --------------------------------------------------------------------------
if not exist "%BUILD_DIR%" mkdir "%BUILD_DIR%"
pushd "%BUILD_DIR%"
:: Find compiler in preferred order
:: --------------------------------------------------------------------------
:: Compiler selection
:: --------------------------------------------------------------------------
set CC=
where clang-cl >nul 2>&1
if !ERRORLEVEL! equ 0 (
echo Compiler: clang-cl ^(preferred^)
@@ -71,12 +87,14 @@ if !ERRORLEVEL! equ 0 (
goto :find_generator
)
echo ERROR: No suitable compiler found! (clang-cl, gcc, or clang required)
echo ERROR: No suitable compiler found!
popd
exit /b 1
:: --------------------------------------------------------------------------
:: Generator selection (prefer ninja)
:: --------------------------------------------------------------------------
:find_generator
:: Find Ninja for build system
set GEN=
where ninja >nul 2>&1
if !ERRORLEVEL! equ 0 (
@@ -86,35 +104,67 @@ if !ERRORLEVEL! equ 0 (
echo Generator: Default
)
:: --------------------------------------------------------------------------
:: Configure
:: --------------------------------------------------------------------------
echo.
echo Configuring CMake...
set CMD=cmake ../../.. %GEN% %CC% -DCMAKE_BUILD_TYPE=%BUILD_TYPE% -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
:: --------------------------------------------------------------------------
:: compile_commands.json logic
:: --------------------------------------------------------------------------
set EXPORT_COMPILE_COMMANDS=OFF
if /i "%BUILD_TYPE%"=="Release" (
if exist "%SCRIPT_DIR%\compile_commands.json" (
echo compile_commands.json already exists - skipping generation
) else (
echo compile_commands.json will be generated
set EXPORT_COMPILE_COMMANDS=ON
)
)
set CMD=cmake "%SCRIPT_DIR%" %GEN% %CC% -DCMAKE_BUILD_TYPE=%BUILD_TYPE% -DCMAKE_EXPORT_COMPILE_COMMANDS=%EXPORT_COMPILE_COMMANDS%
echo !CMD!
!CMD!
if !ERRORLEVEL! neq 0 (echo ERROR: Configuration failed & popd & exit /b 1)
if !ERRORLEVEL! neq 0 (
echo ERROR: Configuration failed
popd
exit /b 1
)
:: --------------------------------------------------------------------------
:: Build
:: --------------------------------------------------------------------------
echo.
echo Building...
cmake --build . --config %BUILD_TYPE%
if !ERRORLEVEL! neq 0 (echo ERROR: Build failed & popd & exit /b 1)
if !ERRORLEVEL! neq 0 (
echo ERROR: Build failed
popd
exit /b 1
)
:: Check if compile_commands.json exists in the current build directory
if exist "compile_commands.json" (
echo.
echo clangd: compile_commands.json generated
:: Copy from current build dir up two levels to the project root
copy /Y "compile_commands.json" "..\..\..\compile_commands.json" >nul 2>&1
if !ERRORLEVEL! equ 0 (
echo clangd: Copied to project root
) else (
echo clangd: Could not copy to project root
:: --------------------------------------------------------------------------
:: Copy compile_commands.json (only if generated)
:: --------------------------------------------------------------------------
if /i "%EXPORT_COMPILE_COMMANDS%"=="ON" (
if exist "compile_commands.json" (
echo.
echo clangd: compile_commands.json generated
copy /Y "compile_commands.json" "%SCRIPT_DIR%\compile_commands.json" >nul 2>&1
if !ERRORLEVEL! equ 0 (
echo clangd: Copied to project root
) else (
echo clangd: Copy failed
)
)
)
popd
echo.
echo === Build Complete ===
echo Executable: %BUILD_DIR%\filehasher.exe

35
build.sh
View File

@@ -74,7 +74,7 @@ mkdir -p "${BUILD_DIR}"
cd "${BUILD_DIR}"
# ---------------------------------------------------------------------------
# Find compiler (prefer gcc, fallback to clang)
# Compiler selection (prefer gcc, fallback to clang)
# ---------------------------------------------------------------------------
echo -e "${YELLOW}Detecting compiler...${NC}"
@@ -135,7 +135,7 @@ fi
echo
# ---------------------------------------------------------------------------
# Find build system (prefer ninja)
# Generator selection (prefer ninja)
# ---------------------------------------------------------------------------
echo -e "${YELLOW}Selecting build system...${NC}"
@@ -165,6 +165,22 @@ echo
# Configure
# ---------------------------------------------------------------------------
echo -e "${YELLOW}Configuring CMake...${NC}"
# --------------------------------------------------------------------------
# compile_commands.json logic
# --------------------------------------------------------------------------
EXPORT_COMPILE_COMMANDS=OFF
if [[ "$BUILD_TYPE" == "Release" ]]; then
if [[ -f "${SCRIPT_DIR}/compile_commands.json" ]]; then
echo -e " compile_commands.json already exists - skipping generation"
else
echo -e " compile_commands.json will be generated"
EXPORT_COMPILE_COMMANDS=ON
fi
fi
echo -e " Build type: ${BUILD_TYPE}"
echo -e " Compiler: ${CC_NAME}"
echo -e " Generator: ${GENERATOR_NAME}"
@@ -173,7 +189,7 @@ cmake "${SCRIPT_DIR}" \
-G "${GENERATOR}" \
-DCMAKE_BUILD_TYPE="${BUILD_TYPE}" \
-DCMAKE_C_COMPILER="${CC_BINARY}" \
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON
-DCMAKE_EXPORT_COMPILE_COMMANDS=${EXPORT_COMPILE_COMMANDS}
if [[ $? -ne 0 ]]; then
echo -e "${RED}CMake configuration failed!${NC}"
@@ -243,12 +259,13 @@ echo -e " Output: ${BUILD_DIR}/"
# ---------------------------------------------------------------------------
# Copy compile_commands.json for clangd
# ---------------------------------------------------------------------------
if [[ -f "${BUILD_DIR}/compile_commands.json" ]]; then
echo -e " clangd: compile_commands.json generated"
# Always copy the latest version
cp "${BUILD_DIR}/compile_commands.json" "${SCRIPT_DIR}/compile_commands.json"
echo -e " clangd: Copied to project root"
if [[ "${EXPORT_COMPILE_COMMANDS}" == "ON" ]]; then
if [[ -f "${BUILD_DIR}/compile_commands.json" ]]; then
echo -e " clangd: compile_commands.json generated"
cp "${BUILD_DIR}/compile_commands.json" "${SCRIPT_DIR}/compile_commands.json"
echo -e " clangd: Copied to project root"
fi
fi
echo

1
config.h
View File

@@ -15,6 +15,7 @@
#define IORING_BUFFER_SIZE KiB(256)
#define NUM_BUFFERS_PER_THREAD 32
#define MAX_ACTIVE_FILES 16
#define MAX_WAIT_COUNT (NUM_BUFFERS_PER_THREAD / 2)
#define SUBMIT_TIMEOUT_MS 10000
#define IORING_DEBUG_PRINTS 0

248
platform.c
View File

@@ -5,6 +5,7 @@
#include "lf_mpmc.h"
#include "arena.c"
#include <stdint.h>
#include <stdio.h>
// xxhash include
@@ -1149,12 +1150,6 @@ typedef struct {
uint32_t MaxVersion;
} IoRingCapabilities;
typedef struct {
BUILD_READ_RETURN_VALUE ResultCode;
uint32_t Information;
uintptr_t UserData;
} IoRingCQE;
// ------------------------ IO Ring Abstraction -------------------------
#if defined(_WIN32) || defined(_WIN64)
@@ -1192,13 +1187,20 @@ static int close_ioring(ThreadIoContext *thread_ctx) {
#define MAKE_BUF_INFO(a, l) \
(IORING_BUFFER_INFO) { .Address = (a), .Length = (uint32_t)(l) }
static int ioring_submit(ThreadIoContext *thread_ctx, uint32_t wait_count,
uint32_t *submitted) {
HRESULT hr = SubmitIoRing(thread_ctx->ring, 0, SUBMIT_TIMEOUT_MS, submitted);
// HRESULT hr = SubmitIoRing(ring, wait_count, SUBMIT_TIMEOUT_MS, submitted);
static int ioring_submit(ThreadIoContext *thread_ctx, uint32_t *submitted) {
// The wait_count in windows is not implemented yet, so we wait with a
// completion event for a single completion
// uint32_t wait_count = MIN(thread_ctx->num_submissions, MAX_WAIT_COUNT);
// The wait_count in windows is not implemented yet, so we wait in
// ioring_pop_completion()
HRESULT hr;
// if (wait_count > 0) {
// hr = SubmitIoRing(ring, wait_count, SUBMIT_TIMEOUT_MS, submitted);
// } else {
hr = SubmitIoRing(thread_ctx->ring, 0, SUBMIT_TIMEOUT_MS, submitted);
// }
if (thread_ctx->num_submissions > 0) {
WaitForSingleObject(thread_ctx->completion_event, SUBMIT_TIMEOUT_MS);
}
@@ -1222,7 +1224,7 @@ static void ioring_register_buffers(ThreadIoContext *thread_ctx,
error_msg, (unsigned int)hr);
}
// Submit registration
ioring_submit(thread_ctx, 0, NULL);
ioring_submit(thread_ctx, NULL);
}
#if USE_REGISTERED_FILES
@@ -1293,52 +1295,69 @@ static BUILD_READ_RETURN_VALUE ioring_build_read(ThreadIoContext *thread_ctx,
return hr;
}
static int ioring_pop_completion(IoRingHandle ring, IoRingCQE *cqe) {
IORING_CQE win_cqe;
static void ioring_process_completions(ThreadIoContext *thread_ctx) {
uint32_t waited = 0;
uint32_t target = MIN(thread_ctx->num_submissions, MAX_WAIT_COUNT);
while (1) {
HRESULT hr = PopIoRingCompletion(ring, &win_cqe);
while (waited < target) {
if (hr == S_FALSE)
// No CQE available
return 0;
// ---- Drain all available CQEs (non-blocking) ----
while (1) {
IORING_CQE win_cqe;
if (FAILED(hr))
return -1;
HRESULT hr = PopIoRingCompletion(thread_ctx->ring, &win_cqe);
// Unlike linux, The Windows implementation treats buffer and file
// registration as an asynchronous operation that we submit to the ring,
// similar to a read or write. Those operations produce CQEs (completion
// queue entries) that we filter here using
// cqe.UserData == USERDATA_REGISTER
if (win_cqe.UserData == USERDATA_REGISTER)
continue;
cqe->ResultCode = win_cqe.ResultCode;
cqe->Information = win_cqe.Information;
cqe->UserData = win_cqe.UserData;
// Check for error and print warning
if (FAILED(win_cqe.ResultCode)) {
char error_msg[256];
FormatMessageA(FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS,
NULL, win_cqe.ResultCode, 0, error_msg, sizeof(error_msg),
NULL);
// Try to get the file path from the buffer
IoBuffer *buf = (IoBuffer *)win_cqe.UserData;
const char *file_path = "unknown";
if (buf && buf->file && buf->file->fe) {
file_path = buf->file->fe->path;
if (hr == S_FALSE) {
// No more CQEs available right now
break;
}
fprintf(stderr,
"WARNING: I/O completion error for file '%s' - Error: %s (Code: "
"0x%lx)\n",
file_path, error_msg, win_cqe.ResultCode);
if (FAILED(hr)) {
fprintf(stderr, "WARNING: PopIoRingCompletion failed (0x%lx)\n", hr);
return;
}
// Skip internal registration completions
if (win_cqe.UserData == USERDATA_REGISTER) {
continue;
}
IoBuffer *buf = (IoBuffer *)win_cqe.UserData;
FileReadContext *file = buf->file;
if (SUCCEEDED(win_cqe.ResultCode)) {
buf->result = 0;
buf->bytes_read = win_cqe.Information;
} else {
buf->result = win_cqe.ResultCode;
buf->bytes_read = 0;
char error_msg[256];
FormatMessageA(
FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, NULL,
win_cqe.ResultCode, 0, error_msg, sizeof(error_msg), NULL);
fprintf(stderr,
"WARNING: I/O completion error for file '%s' - Error: %s "
"(Code: 0x%lx)\n",
buf->file->fe->path, error_msg, win_cqe.ResultCode);
}
file->active_reads--;
file->reads_completed++;
thread_ctx->num_submissions--;
// Count only "real" completions toward wait budget
waited++;
}
return 1;
// ---- If we already waited enough, exit ----
if (waited >= target) {
break;
}
// ---- Otherwise wait for more completions ----
WaitForSingleObject(thread_ctx->completion_event, SUBMIT_TIMEOUT_MS);
}
}
@@ -1431,10 +1450,11 @@ static void ioring_register_buffers(ThreadIoContext *thread_ctx,
struct rlimit limit;
getrlimit(RLIMIT_MEMLOCK, &limit);
fprintf(stderr,
"WARNING: Buffer registration failed due to memlock limits "
"(ENOMEM).\n"
"See README for more informations.\n");
fprintf(
stderr,
"WARNING: Buffer registration failed due to Memlock limit, Error: "
"Cannot allocate memory (code: -12, ENOMEM).\n"
"See README for more informations.\n");
} else {
// For any other error (e.g., EFAULT, EBUSY, EINVAL)
@@ -1530,10 +1550,11 @@ static int ioring_build_read(ThreadIoContext *thread_ctx,
return 0;
}
static int ioring_submit(ThreadIoContext *thread_ctx, uint32_t wait_count,
uint32_t *submitted) {
static int ioring_submit(ThreadIoContext *thread_ctx, uint32_t *submitted) {
int ret;
uint32_t wait_count = MIN(thread_ctx->num_submissions, MAX_WAIT_COUNT);
if (wait_count > 0) {
ret = io_uring_submit_and_wait(&((IoUring *)thread_ctx->ring)->ring,
wait_count);
@@ -1553,58 +1574,46 @@ static int ioring_submit(ThreadIoContext *thread_ctx, uint32_t wait_count,
return 0;
}
static int ioring_pop_completion(IoRingHandle ring, IoRingCQE *cqe) {
static void ioring_process_completions(ThreadIoContext *thread_ctx) {
IoUring *impl = (IoUring *)thread_ctx->ring;
struct io_uring_cqe *cqe_ptr = NULL;
struct io_uring_cqe *cqes[NUM_BUFFERS_PER_THREAD];
int ret = io_uring_peek_cqe(&((IoUring *)ring)->ring, &cqe_ptr);
unsigned count =
io_uring_peek_batch_cqe(&impl->ring, cqes, NUM_BUFFERS_PER_THREAD);
if (ret == -EAGAIN) {
// No CQE available
return 0;
if (count == 0) {
return;
}
if (ret < 0) {
// Error
fprintf(stderr, "WARNING: io_uring_peek_cqe error - Error: %s (Code: %d)\n",
strerror(-ret), ret);
return -1;
}
for (unsigned i = 0; i < count; i++) {
struct io_uring_cqe *cqe = cqes[i];
if (!cqe_ptr) {
return 0;
}
int res = cqe->res;
int res = cqe_ptr->res;
IoBuffer *buf = (IoBuffer *)cqe->user_data;
FileReadContext *file = buf->file;
if (res >= 0) {
cqe->ResultCode = 0;
cqe->Information = (uint32_t)res;
} else {
cqe->ResultCode = res;
cqe->Information = 0;
}
if (res >= 0) {
buf->result = 0;
buf->bytes_read = (uint32_t)res;
} else {
buf->result = res;
buf->bytes_read = 0;
cqe->UserData = (uintptr_t)cqe_ptr->user_data;
io_uring_cqe_seen(&((IoUring *)ring)->ring, cqe_ptr);
// Check for error and print warning
if (res < 0) {
// Try to get the file path from the buffer
IoBuffer *buf = (IoBuffer *)cqe->UserData;
const char *file_path = "unknown";
if (buf && buf->file && buf->file->fe) {
file_path = buf->file->fe->path;
fprintf(stderr,
"WARNING: I/O completion error for file '%s' - Error: %s (Code: "
"%d)\n",
buf->file->fe->path, strerror(-res), res);
}
fprintf(
stderr,
"WARNING: I/O completion error for file '%s' - Error: %s (Code: %d)\n",
file_path, strerror(-res), res);
file->active_reads--;
file->reads_completed++;
thread_ctx->num_submissions--;
}
return 1;
// Mark CQE as seen, equivalent to io_uring_cqe_seen() but marks multiple CQEs
io_uring_cq_advance(&impl->ring, count);
}
FileHandle ioring_open_file(FileEntry *fe) {
@@ -1790,22 +1799,22 @@ static void return_buffer(ThreadIoContext *ctx, IoBuffer *buf) {
}
// -------------------------- Process completions ---------------------------
static void process_completions(ThreadIoContext *thread_ctx) {
IoRingCQE cqe;
while (ioring_pop_completion(thread_ctx->ring, &cqe) == 1) {
IoBuffer *buf = (IoBuffer *)cqe.UserData;
FileReadContext *file = buf->file;
buf->result = cqe.ResultCode;
buf->bytes_read = cqe.Information;
file->active_reads--;
file->reads_completed++;
thread_ctx->num_submissions--;
}
}
// static void process_completions(ThreadIoContext *thread_ctx) {
// IoRingCQE cqe;
//
// while (ioring_pop_completion(thread_ctx->ring, &cqe) == 1) {
//
// IoBuffer *buf = (IoBuffer *)cqe.UserData;
// FileReadContext *file = buf->file;
//
// buf->result = cqe.ResultCode;
// buf->bytes_read = cqe.Information;
//
// file->active_reads--;
// file->reads_completed++;
// thread_ctx->num_submissions--;
// }
// }
// -------------------- File operations -----------------------
static int init_file(ThreadIoContext *thread_ctx, FileReadContext *file,
@@ -2074,8 +2083,7 @@ static THREAD_RETURN hash_worker_ioring(void *arg) {
FileQueue fq;
memset(&fq, 0, sizeof(fq));
uint32_t submitted = 0;
uint32_t wait_count;
uint32_t submitted;
// Main pipeline loop
for (;;) {
@@ -2083,13 +2091,14 @@ static THREAD_RETURN hash_worker_ioring(void *arg) {
// Submit new reads
build_pending_reads(thread_ctx, &fq, worker_ctx);
wait_count = MIN(thread_ctx->num_submissions, NUM_BUFFERS_PER_THREAD - 6);
submitted = 0;
ioring_submit(thread_ctx, wait_count, &submitted);
ioring_submit(thread_ctx, &submitted);
// Process completions
process_completions(thread_ctx);
ioring_process_completions(thread_ctx);
// Hash files
hash_ready_files(thread_ctx, &fq, worker_ctx);
#if IORING_DEBUG_STATS
printf(
@@ -2098,9 +2107,6 @@ static THREAD_RETURN hash_worker_ioring(void *arg) {
thread_ctx->active_files, fq.count);
#endif
// Hash files
hash_ready_files(thread_ctx, &fq, worker_ctx);
// Exit condition
if (!thread_ctx->submitting && thread_ctx->active_files == 0 &&
thread_ctx->num_submissions == 0) {