Finalizing the implementation of file registration

Adding the file system check in Linux(can be enabled from the config
file)
Adding a more options to the config file
Writing the README

.gitignore

@@ -10,3 +10,4 @@ temp_code.c
 /file_hasher
 /io_uring_test
 /file_hasher
+/io_uring_test

README.md

@@ -1,26 +1,221 @@
 # filehasher
 
-Collects some metadata and hashes files.
+## Presentation  
+Collects some metadata and hashes files. It outputs the path, hash, size, creation and 
+last modification dates and the author in file_hasher.txt.  
+Creation and modification dates and author can be disabled in the config file.  
 
-## Building:  
-### Windows:  
-#### Release:  
-clang-cl /O3 file_hasher.c xxh_x86dispatch.c
+It is a high performance cross platform Windows and Linux compatible program, it uses:  
+  * Multiple threads for scanning and hashing (multi-threading can be disabled in the config file).  
+  * Stores the generated data in thread local configurable arenas that support growing 
+  by committing more memory and chaining blocks.  
+  * Two Multi Producer Multi Consumer queues, one for the scanners and one between the scanners and hashers.  
+  * xxh3_128bits algorithm from xxhash, that supports SIMD instruction sets (SSE2, AVX2, AVX512)
+  and uses a runtime dispatcher to select the best available instruction set.  
+  * IO Ring for asynchronous I/O in Windows and the equivalent io_uring in Linux. 
+  The implementation is event driven, thread local, uses DMA and direct disk I/O, 
+  bypassing the OS cache completely, registered buffers (and registered files in io_uring), 
+  it supports bashing multiple submissions and can handle multiple files at the same time.  
+  It can be disabled in the config file.
+  * Fallback to buffered I/O if there is errors in the IO Ring path.  
 
-Note: MinGW does not provide IO Ring headers yet, to fix that include ioringapi.c, this will dynamically load all the functions and define all the symbols necessary to replace the official header.  
-clang -O3 file_hasher.c xxh_x86dispatch.c -o file_hasher  
-gcc -O3 file_hasher.c xxh_x86dispatch.c -o file_hasher  
+## Building  
+### Windows  
+#### Release  
 
-#### Debug:  
-clang-cl /Zi /Od file_hasher.c xxh_x86dispatch.c  
-clang -g -O0 file_hasher.c xxh_x86dispatch.c -o file_hasher  
-gcc -g -O0 file_hasher.c xxh_x86dispatch.c -o file_hasher  
+**Note**: Make sur to use UCRT64 environment from MSYS2 instead of the standard MinGW environment. 
+UCRT64 uses the modern Universal C Runtime (ucrtbase.dll), which supports the newest APIs, 
+the standard MSYS2 uses the legacy msvcrt.dll and does not support IO Ring.  
+To install:  
+pacman -S mingw-w64-ucrt-x86_64-gcc  
+pacman -S mingw-w64-ucrt-x86_64-clang
+pacman -Syu
+And add to path:  
+C:\msys64\ucrt64\bin  
 
-### Linux:  
-#### Release:  
-clang -O3 file_hasher.c xxhash.c xxh_x86dispatch.c -pthread -luring -o file_hasher  
+gcc -O3 file_hasher.c xxhash.c xxh_x86dispatch.c -o file_hasher  
+clang -O3 file_hasher.c xxhash.c xxh_x86dispatch.c -o file_hasher  
+clang-cl /O2 file_hasher.c xxhash.c xxh_x86dispatch.c  
+
+#### Debug  
+gcc -g -O0 file_hasher.c xxhash.c xxh_x86dispatch.c -o file_hasher  
+clang -g -O0 file_hasher.c xxhash.c xxh_x86dispatch.c -o file_hasher  
+clang-cl /Zi /Od file_hasher.c xxhash.c xxh_x86dispatch.c  
+
+### Linux  
+#### Release  
 gcc -O3 file_hasher.c xxhash.c xxh_x86dispatch.c -pthread -luring -o file_hasher  
+clang -O3 file_hasher.c xxhash.c xxh_x86dispatch.c -pthread -luring -o file_hasher  
 
-#### Debug:  
-clang -g -O0 file_hasher.c xxhash.c xxh_x86dispatch.c -pthread -luring -o file_hasher  
+#### Debug  
 gcc -g -O0 file_hasher.c xxhash.c xxh_x86dispatch.c -pthread -luring -o file_hasher  
+clang -g -O0 file_hasher.c xxhash.c xxh_x86dispatch.c -pthread -luring -o file_hasher  
+
+## Notes about the IO Ring implementations  
+### IO Ring  
+
+#### File registration  
+Registering files is a performance optimization that allows the kernel to allocate an array 
+of descriptors/handles to pre-validate and maintain long-term references to file handles. 
+Instead of passing a standard file descriptor/handle with every I/O request, you pass a simple integer 
+index into a pre-registered table.  
+
+The Linux implementation has io_uring_register_files_scarse() to create an empty array of descriptors 
+(initialized with -1) without having to create and initialize it in the user space, and we can 
+use io_uring_register_files_update() to update one or more entries. Windows on the other hand 
+is limited to BuildIoRingRegisterFileHandles() only, so we need to re register the entire array of handles 
+each time. This is why there is a provided macro in config.h to disable or enable it.  
+
+##### Why Register Files? (The Benefits)  
+When you use a standard file descriptor in a high-frequency I/O loop, 
+the kernel must perform several "hidden" tasks for every single operation:
+  * Permission Checks: Validating that the process still has the right to read/write 
+  that specific file.  
+  * Reference Counting: Incrementing the file's internal reference count at the start of 
+  the I/O and decrementing it at the end to ensure the file isn't closed while in use.  
+  * Object Lookup: Traversing the internal "file descriptor table" to find the actual 
+  kernel object associated with your integer ID.  
+
+Registering the files performs these checks once at registration time. Subsequent 
+I/O operations skip these steps, significantly reducing CPU overhead and latency, 
+especially when handling thousands of small I/O operations per second.  
+
+##### Comparison: Linux vs. Windows Implementation  
+While both systems share the same core concept, their APIs and management styles differ significantly. 
+Feature 	Linux (io_uring)	Windows (IoRing)  
+API Call	io_uring_register	BuildIoRingRegisterFileHandles  
+Registration Method	Synchronous system call that blocks until the table is set up.	Asynchronous request submitted to the ring just like a read/write operation.
+Partial Updates	Supports IORING_REGISTER_FILES_UPDATE to swap specific indices without a full reset.	Does not support partial updates; a new registration call replaces the entire existing table.
+Memory Mapping	User must manually mmap() the queues into their address space.	The kernel handles memory mapping automatically when the ring is created.
+Scope of Operations	Extremely broad (files, sockets, timers, signals, even other rings).	Primarily focused on file storage (read, write, flush).
+
+#### Completion Wait count  
+To avoid busy waiting when receiving CQEs, we can use io_uring_submit_and_wait() in Linux by entering a wait count, 
+the threads sleeps until the count of CQEs are received, in windows the wait_count is present in SubmitIoRing() 
+but is not implemented yet, so we wait with a completion event for a single completion. Another limitation on the completion 
+event is that the kernel will waik up the thread only when receiving the first CQE, after that we need to drain the completion
+queue completely before sleeping again, or we enter an eternal slumber. And my config, each time the thread wakes up 
+it receives rarely more than 3 to 5 CQEs and most of the time only one CQE.
+
+#### Filtering CQEs  
+
+Unlike Linux, The Windows implementation treats buffer and file registration 
+as an asynchronous operation that we submit to the ring, similar to a read or write. 
+Those operations produce CQEs (completion queue entries) that we filter here using 
+cqe.UserData == USERDATA_REGISTER  
+```c
+    if (win_cqe.UserData == USERDATA_REGISTER)
+      continue;
+```
+
+### io_uring  
+
+#### Creation flags  
+io_uring provides a lot of configuration flags compared to IO Ring, some 
+of them are at the creation and others during the operations, here what 
+we use in this implementation at creation time and is lacking in the 
+IO Ring implementation.  
+
+  * IORING_SETUP_SINGLE_ISSUER: Since we are using a thread local io_uring, we can 
+  set this flag to remove the atomic operations.  
+  * IORING_SETUP_DEFER_TASKRUN: By default, the kernel sends an interrupts when a CQE 
+  is ready, we use this flag to disable this syscall and wait for a specific number of 
+  CQEs to be ready to group them, this reduces the number of syscall.  
+
+#### Memlock limit warning  
+
+```c
+     "WARNING: Buffer registration failed due to memlock limits (ENOMEM).\n"
+     "Increase the limit to solve this warning.\n");
+```
+
+The Memlock limit in Linux restricts the amount of memory a process can
+"lock" into physical RAM using the mlock() family of system calls. This
+prevents the operating system from swapping that memory out to disk.  
+And registering buffers will lock the buffers memory so the hardware
+can access it directly without kernel intervention and prevents the kernel from 
+swapping it to the SSD or HDD. Increase the limit to be able to register the buffers.  
+
+##### Modifying the Limit:  
+The method for changing the memlock limit depends on whether you are
+managing a user session or a system service.  
+1. For Users and Interactive Sessions  
+To permanently increase the limit for a specific user or group, modify
+the /etc/security/limits.conf file. Add the following lines:  
+
+```conf
+ # Example for a specific user (replace 'username'), unlimited or a custom value in KB
+ username   soft    memlock    unlimited
+ username   hard    memlock    unlimited
+
+ # Example for all users
+ *          soft    memlock    unlimited
+ *          hard    memlock    unlimited
+```
+
+Soft Limit: The value the user starts with; can be raised up to the
+hard limit.  
+
+Hard Limit: The absolute maximum; only a privileged user
+(root) can increase this. Values: Can be set in Kilobytes (KB) or as
+unlimited.  
+
+2. For Systemd Services
+Settings in limits.conf do not affect background services managed by
+systemd. To increase the limit for a service, edit its service file
+(e.g., /etc/systemd/system/myservice.service) and add:  
+```conf
+  [Service]
+  LimitMEMLOCK=infinity
+```
+
+##### Why Register Buffers?  
+In a standard "unregistered" I/O operation, the kernel must perform several 
+expensive steps for every single read or write:  
+  * Virtual-to-Physical Mapping: The kernel has to translate your application's 
+  virtual memory addresses into physical RAM addresses.  
+  * Page Pinning: The kernel must "pin" the memory pages (using get_user_pages) 
+  to prevent them from being swapped to disk or moved while the hardware 
+  (like your SSD) is writing to them.  
+  * TLB Overhead: Constant mapping and unmapping put pressure on the Translation 
+  Lookaside Buffer (TLB), which can slow down the CPU.  
+
+Registering the buffers performs all of this "pinning" and "mapping" once.  
+
+#### Direct I/O: O_DIRECT (Linux) and FILE_FLAG_NO_BUFFERING (Windows)
+
+Modern operating systems normally use a page cache when reading files. This means file 
+data is first loaded into kernel memory and then copied to user space. While this improves 
+performance for many workloads, it introduces extra memory usage and copy overhead.  
+
+Both Linux and Windows provide a way to bypass this cache and perform direct I/O:  
+
+Linux: O_DIRECT
+Windows: FILE_FLAG_NO_BUFFERING
+
+These flags instruct the OS to transfer data directly between disk and user-provided buffers, avoiding the page cache.
+
+##### Benefits
+1. Reduced memory overhead
+Avoids polluting the OS page cache
+Especially useful for large sequential reads (e.g. hashing, backups)
+2. Lower CPU usage
+Eliminates extra memory copies between kernel and user space
+3. Predictable performance
+No interference from cache eviction or readahead heuristics
+More consistent throughput for streaming workloads
+4. Better scalability
+Ideal for high-throughput, multi-threaded I/O pipelines
+Prevents cache contention between threads
+5. Avoids double caching
+Important when the application already manages its own buffering
+
+##### File system compatibility
+Not all file systems are compatible with O_DIRECT, if we try to open files residing in an NTFS partition, 
+most of the time it will fail, and some times it opens but the CQEs return with an error code bad 
+descriptor, and it causes some lags.
+
+To address this issue the program falls back to sequential read when the open fails, and falls back to
+buffered sequential hashing if we receive an error in the CQEs. There is also a file system detection 
+that we can enable in the config file, it will enable the collection of the file system in scan_folder() 
+and the file will be opened accordingly, but it costs one additional syscall / directory.

base.h

@@ -33,6 +33,7 @@
 #include <sys/eventfd.h>
 #include <sys/resource.h>
 #include <sys/stat.h>
+#include <sys/vfs.h>
 #include <unistd.h>
 #endif
 

binaries/changelog.txt

@@ -14,7 +14,7 @@ v3.2: Making the lock free MPMC queue growable
 Add padding to avoir false sharing
 Add sleep() and SwitchToThread() to limit spinning
 
-v3.3: Fix bug slots used before initialization,compare and swap is protecting updating committed, but it is not protecting the memory initialization. Adding atomic_flag commit_lock to protect against that
+v3.3: Fix bug slots used before initialization, compare and swap is protecting updating committed, but it is not protecting the memory initialization. Adding atomic_flag commit_lock to protect against that
 Fix bug multiple threads committing at the same time, fixed by using atomic_flag commit_lock and re-checking committed after acquiring the lock
 Reorder helper functions
 
@@ -55,4 +55,4 @@ Hashing small files using XXH3_128bits() instead of the streaming pipeline(XXH3_
 fixing the xxh_x86dispatch warnings
 Updating the progress printing function
 Implementing a config file
-
+Writing the README file

compile_commands.json

@@ -1,7 +0,0 @@
-[
-  {
-    "directory": "D:/Code/c/filehasher",
-    "command": "clang-cl /O2 file_hasher.c xxh_x86dispatch.c",
-    "file": "file_hasher.c"
-  }
-]

config.h

@@ -1,27 +1,31 @@
 
 #define FILE_HASHES_TXT "file_hashes.txt"
-#define HASH_STRLEN 33 // 128-bit hex (32 chars) + null
-#define MAX_PATHLEN 4096
-#define READ_BLOCK (KiB(64))
 
-#define MULTI_THREADED true
+// Metadata selection
+#define FILE_TIMES 1 // created and modified time
+#define FILE_OWNER 1
+
+#define MULTI_THREADING 1
+#define READ_BLOCK KiB(64)
 
 // -------------------- IO Ring Configuration ----------------------
 #define USE_IORING 1
 
 #if USE_IORING
-#define IORING_BUFFER_SIZE (KiB(256))
+#define IORING_BUFFER_SIZE KiB(256)
 #define NUM_BUFFERS_PER_THREAD 32
-#define MAX_ACTIVE_FILES 32
-#define SUBMIT_TIMEOUT_MS 30000
+#define MAX_ACTIVE_FILES 16
 
-#define IORING_DEBUG_PRINTS false
-#define IORING_DEBUG_STATS false
+#define SUBMIT_TIMEOUT_MS 10000
+#define IORING_DEBUG_PRINTS 0
+#define IORING_DEBUG_STATS 0
 
 #if defined(_WIN32) || defined(_WIN64)
-#define USE_REGISTERED_FILES false
+#define USE_REGISTERED_FILES 1
+
 #elif defined(__linux__)
-#define USE_REGISTERED_FILES true
-#endif
+#define USE_REGISTERED_FILES 1
+#define CHECK_FILE_SYSTEM 0
 
 #endif
+#endif

file_hasher.c

@@ -82,7 +82,7 @@ int main(int argc, char **argv) {
   // Logical threads = CPU cores * 2
   uint32_t cpu_threads = cpu_cores * 2;
 
-#if MULTI_THREADED
+#if MULTI_THREADING
   uint32_t num_scan_threads = cpu_threads;
   uint32_t num_hash_threads = cpu_threads;
 

io_uring_test.c

@@ -17,9 +17,9 @@ gcc -o io_uring_test io_uring_test.c -luring
 #include <sys/stat.h>
 #include <unistd.h>
 
-#define TEST_FILE "test_io_uring.txt"
 #define BUFFER_SIZE 4096
 #define NUM_BUFFERS 4
+#define NUM_REGISTERED_FILES 3 // Test with 3 files
 
 // Colors for output
 #define COLOR_GREEN "\033[0;32m"
@@ -50,25 +50,17 @@ static void print_step(const char *step) {
   printf(COLOR_YELLOW "\n>>> Testing: %s" COLOR_RESET "\n", step);
 }
 
-// Create a test file with known content
-static int create_test_file(void) {
-  const char *test_content =
-      "Hello, io_uring! This is a test file for async I/O operations.\n"
-      "Line 2: Testing reads with registered buffers.\n"
-      "Line 3: The quick brown fox jumps over the lazy dog.\n"
-      "Line 4: ABCDEFGHIJKLMNOPQRSTUVWXYZ\n"
-      "Line 5: 0123456789\n";
-
-  FILE *f = fopen(TEST_FILE, "w");
+static int create_test_file(const char *filename, const char *content) {
+  FILE *f = fopen(filename, "w");
   if (!f) {
     perror("Failed to create test file");
     return -1;
   }
 
-  fprintf(f, "%s", test_content);
+  fprintf(f, "%s", content);
   fclose(f);
 
-  print_info("Test file created successfully");
+  printf("  Created test file: %s\n", filename);
   return 0;
 }
 
@@ -93,16 +85,14 @@ static int test_register_buffers(struct io_uring *ring, void **buffers,
                                  struct iovec *iovs, TestResults *results) {
   print_step("Buffer registration");
 
-  // Allocate and prepare buffers
   size_t total_size = BUFFER_SIZE * NUM_BUFFERS;
-  *buffers = aligned_alloc(4096, total_size); // Page-aligned for O_DIRECT
+  *buffers = aligned_alloc(4096, total_size);
   if (!*buffers) {
     print_failure("Buffer allocation", strerror(errno));
     results->failed++;
     return -1;
   }
 
-  // Initialize iovecs
   for (int i = 0; i < NUM_BUFFERS; i++) {
     iovs[i].iov_base = (char *)*buffers + (i * BUFFER_SIZE);
     iovs[i].iov_len = BUFFER_SIZE;
@@ -121,334 +111,287 @@ static int test_register_buffers(struct io_uring *ring, void **buffers,
   return 0;
 }
 
-// Test 3: Open file
-// Modified test_open_file function
-static int test_open_file(int *fd, TestResults *results) {
-  print_step("File opening");
+// Test 3: Register files sparse (empty table)
+static int test_register_files_sparse(struct io_uring *ring, unsigned nr_files,
+                                      TestResults *results) {
+  print_step("Sparse file registration (empty table)");
 
-  // Get file size
-  struct stat st;
-  if (stat(TEST_FILE, &st) != 0) {
-    print_failure("stat", strerror(errno));
-    results->failed++;
-    return -1;
-  }
+  int ret = io_uring_register_files_sparse(ring, nr_files);
+  if (ret < 0) {
+    if (ret == -EINVAL) {
+      print_info(
+          "io_uring_register_files_sparse not supported (kernel < 5.19)");
+      print_info("Trying regular file registration with invalid fds...");
 
-  // Check if file size is page-aligned
-  int page_size = plat_get_pagesize();
-  size_t file_size = st.st_size;
+      // Fallback: register with invalid fds
+      int *invalid_fds = calloc(nr_files, sizeof(int));
+      if (!invalid_fds) {
+        print_failure("Allocating invalid fds array", "Out of memory");
+        results->failed++;
+        return -1;
+      }
 
-  printf("  File size: %zu bytes\n", file_size);
-  printf("  Page size: %d bytes\n", page_size);
+      for (int i = 0; i < nr_files; i++) {
+        invalid_fds[i] = -1; // Mark all as invalid
+      }
 
-  if (file_size % page_size != 0) {
-    printf("  Extending read size from %zu to %zu bytes\n", file_size,
-           ALIGN_UP_POW2(file_size, page_size));
-  }
+      ret = io_uring_register_files(ring, invalid_fds, nr_files);
+      free(invalid_fds);
 
-  // Try O_DIRECT first
-  *fd = open(TEST_FILE, O_RDONLY | O_DIRECT);
-  if (*fd < 0) {
-    print_info("O_DIRECT failed, trying without it");
-    *fd = open(TEST_FILE, O_RDONLY);
-    if (*fd < 0) {
-      print_failure("open", strerror(errno));
+      if (ret < 0) {
+        print_failure("Regular file registration also failed", strerror(-ret));
+        results->failed++;
+        return -1;
+      }
+      print_success("File table registered (regular, with invalid fds)");
+    } else {
+      print_failure("io_uring_register_files_sparse", strerror(-ret));
       results->failed++;
       return -1;
     }
-    print_info("Using buffered I/O (O_DIRECT not available)");
   } else {
-    print_success("File opened with O_DIRECT");
+    printf("  Registered empty file table with %u slots\n", nr_files);
+    print_success("Sparse file table created");
   }
 
   results->passed++;
   return 0;
 }
 
-// Test 4: Build and submit read operation
-static int test_submit_read(struct io_uring *ring, int fd, struct iovec *iovs,
-                            int buffer_id, uint64_t user_data,
-                            TestResults *results) {
-  print_step("Building and submitting read operation");
+// Test 4: Update file slot and read from it
+static int test_file_read_loop(struct io_uring *ring, struct iovec *iovs,
+                               const char **filenames,
+                               const char **expected_contents, int num_files,
+                               TestResults *results) {
+  print_step("File slot update and read loop");
 
-  // Get file size for proper alignment
-  struct stat st;
-  if (fstat(fd, &st) != 0) {
-    print_failure("fstat", strerror(errno));
+  int *fds = calloc(num_files, sizeof(int));
+  if (!fds) {
+    print_failure("Allocating fd array", "Out of memory");
     results->failed++;
     return -1;
   }
 
-  u32 page_size = plat_get_pagesize();
-  size_t file_size = st.st_size;
-  size_t read_size = BUFFER_SIZE;
-
-  // For O_DIRECT, ensure read size is sector-aligned
-  if (read_size > file_size) {
-    read_size = ALIGN_UP_POW2(file_size, page_size);
-    printf("  Adjusted read size to %zu bytes for O_DIRECT alignment\n",
-           read_size);
+  // Open all files first
+  for (int i = 0; i < num_files; i++) {
+    fds[i] = open(filenames[i], O_RDONLY);
+    if (fds[i] < 0) {
+      print_failure("Opening file", filenames[i]);
+      results->failed++;
+      // Close already opened files
+      for (int j = 0; j < i; j++)
+        close(fds[j]);
+      free(fds);
+      return -1;
+    }
+    printf("  Opened %s (fd=%d)\n", filenames[i], fds[i]);
   }
 
-  struct io_uring_sqe *sqe = io_uring_get_sqe(ring);
-  if (!sqe) {
-    print_failure("io_uring_get_sqe", "No available SQE");
-    results->failed++;
-    return -1;
-  }
+  // Test loop: update slot, submit read, verify
+  for (int slot = 0; slot < num_files; slot++) {
+    printf("\n  --- Testing slot %d with file '%s' ---\n", slot,
+           filenames[slot]);
 
-  // Prepare read operation using registered buffer
-  io_uring_prep_read_fixed(sqe, fd, iovs[buffer_id].iov_base, read_size, 0,
-                           buffer_id);
-  io_uring_sqe_set_data64(sqe, user_data);
+    // Update the file registration for this slot
+    printf("  Updating slot %d with fd %d...\n", slot, fds[slot]);
+    int ret = io_uring_register_files_update(ring, slot, &fds[slot], 1);
 
-  int ret = io_uring_submit(ring);
-  if (ret < 0) {
-    print_failure("io_uring_submit", strerror(-ret));
-    results->failed++;
-    return -1;
-  }
+    if (ret < 0) {
+      print_failure("File registration update", strerror(-ret));
+      results->failed++;
+      continue;
+    }
+    printf("  Slot update result: %d (expected 1)\n", ret);
 
-  print_success("Read operation submitted successfully");
-  results->passed++;
-  return 0;
-}
+    // Get file size for read size calculation
+    struct stat st;
+    if (fstat(fds[slot], &st) != 0) {
+      print_failure("fstat", strerror(errno));
+      results->failed++;
+      continue;
+    }
 
-// Test 5: Wait for completion
-static int test_wait_completion(struct io_uring *ring,
-                                struct io_uring_cqe **cqe,
-                                TestResults *results) {
-  print_step("Waiting for completion");
+    size_t file_size = st.st_size;
+    size_t read_size = BUFFER_SIZE;
 
-  int ret = io_uring_wait_cqe(ring, cqe);
-  if (ret < 0) {
-    print_failure("io_uring_wait_cqe", strerror(-ret));
-    results->failed++;
-    return -1;
-  }
+    // Adjust read size for O_DIRECT if needed
+    int page_size = plat_get_pagesize();
+    if (read_size > file_size) {
+      read_size = ALIGN_UP_POW2(file_size, page_size);
+    }
 
-  print_success("Completion received");
-  results->passed++;
-  return 0;
-}
+    printf("  File size: %zu, read size: %zu\n", file_size, read_size);
 
-// Test 6: Process completion
-static int test_process_completion(struct io_uring_cqe *cqe,
-                                   uint64_t expected_user_data,
-                                   TestResults *results) {
-  print_step("Processing completion");
+    // Clear buffer for this test
+    memset(iovs[0].iov_base, 0, BUFFER_SIZE);
 
-  uint64_t user_data = io_uring_cqe_get_data64(cqe);
-  int res = cqe->res;
-
-  printf("  Completion data:\n");
-  printf("    User data: %lu (expected: %lu)\n", user_data, expected_user_data);
-  printf("    Result: %d bytes read\n", res);
-
-  if (user_data != expected_user_data) {
-    print_failure("User data mismatch",
-                  "User data doesn't match expected value");
-    results->failed++;
-    return -1;
-  }
-
-  if (res < 0) {
-    print_failure("Read operation", strerror(-res));
-    results->failed++;
-    return -1;
-  }
-
-  print_success("Completion processed successfully");
-  results->passed++;
-  return res; // Return number of bytes read
-}
-
-// Test 7: Verify read data
-static int test_verify_data(struct iovec *iovs, int buffer_id, int bytes_read,
-                            TestResults *results) {
-  print_step("Data verification");
-
-  char *data = (char *)iovs[buffer_id].iov_base;
-
-  printf("  Read data (first 200 chars):\n");
-  printf("  ---\n");
-  for (int i = 0; i < bytes_read && i < 200; i++) {
-    putchar(data[i]);
-  }
-  if (bytes_read > 200)
-    printf("...");
-  printf("\n  ---\n");
-
-  // Check if data is not empty
-  if (bytes_read == 0) {
-    print_failure("Data verification", "No data read");
-    results->failed++;
-    return -1;
-  }
-
-  // Check if data contains expected content
-  if (strstr(data, "io_uring") == NULL) {
-    print_failure("Data verification", "Expected content not found");
-    results->failed++;
-    return -1;
-  }
-
-  print_success("Data verified successfully");
-  results->passed++;
-  return 0;
-}
-
-// Test 8: Test multiple concurrent reads
-static int test_concurrent_reads(struct io_uring *ring, int fd,
-                                 struct iovec *iovs, TestResults *results) {
-  print_step("Concurrent reads test");
-
-  int num_reads = 3;
-  int submitted = 0;
-
-  // Submit multiple reads
-  for (int i = 0; i < num_reads; i++) {
+    // Submit read using registered file
     struct io_uring_sqe *sqe = io_uring_get_sqe(ring);
     if (!sqe) {
-      print_failure("Getting SQE for concurrent read", "No available SQE");
+      print_failure("Getting SQE", "No available SQE");
       results->failed++;
-      return -1;
+      continue;
     }
 
-    off_t offset = i * 100; // Read from different offsets
-    io_uring_prep_read_fixed(sqe, fd, iovs[i].iov_base, BUFFER_SIZE, offset, i);
-    io_uring_sqe_set_data64(sqe, i);
-    submitted++;
-  }
+    // Use slot index with fixed file flag
+    io_uring_prep_read_fixed(sqe, slot, iovs[0].iov_base, read_size, 0, 0);
+    sqe->flags |= IOSQE_FIXED_FILE;
+    io_uring_sqe_set_data64(sqe, 100 + slot); // Unique user_data per slot
 
-  int ret = io_uring_submit(ring);
-  if (ret != submitted) {
-    char msg[64];
-    snprintf(msg, sizeof(msg), "Expected %d, got %d", submitted, ret);
+    ret = io_uring_submit(ring);
+    if (ret < 0) {
+      print_failure("Submitting read", strerror(-ret));
+      results->failed++;
+      continue;
+    }
+    printf("  Submitted read (1 SQE)\n");
 
-    print_failure("Submitting concurrent reads", msg);
-    results->failed++;
-    return -1;
-  }
-
-  print_success("Concurrent reads submitted");
-
-  // Wait for and process completions
-  for (int i = 0; i < submitted; i++) {
+    // Wait for completion
     struct io_uring_cqe *cqe;
     ret = io_uring_wait_cqe(ring, &cqe);
     if (ret < 0) {
-      print_failure("Waiting for concurrent read completion", strerror(-ret));
+      print_failure("Waiting for completion", strerror(-ret));
       results->failed++;
-      return -1;
+      continue;
     }
 
+    // Process completion
     uint64_t user_data = io_uring_cqe_get_data64(cqe);
-    int res = cqe->res;
+    int bytes_read = cqe->res;
+
+    printf("  Completion: user_data=%lu, result=%d\n", (unsigned long)user_data,
+           bytes_read);
+
+    if (bytes_read < 0) {
+      print_failure("Read operation", strerror(-bytes_read));
+      results->failed++;
+      io_uring_cqe_seen(ring, cqe);
+      continue;
+    }
+
+    if (user_data != 100 + slot) {
+      print_failure("User data mismatch", "Wrong user_data value");
+      results->failed++;
+      io_uring_cqe_seen(ring, cqe);
+      continue;
+    }
+
+    // Verify the data
+    char *data = (char *)iovs[0].iov_base;
+    printf("  Data read (%d bytes): %.*s\n", bytes_read,
+           bytes_read < 100 ? bytes_read : 100, data);
+
+    if (strstr(data, expected_contents[slot]) == NULL) {
+      print_failure("Data verification",
+                    "Expected content not found in read data");
+      results->failed++;
+    } else {
+      print_success("Data verified successfully");
+      results->passed++;
+    }
 
-    printf("  Concurrent read %lu completed: %d bytes read\n", user_data, res);
     io_uring_cqe_seen(ring, cqe);
+
+    // Invalidate the slot after use (mark as -1)
+    int invalid_fd = -1;
+    ret = io_uring_register_files_update(ring, slot, &invalid_fd, 1);
+    if (ret < 0) {
+      printf("  Warning: Could not invalidate slot %d: %s\n", slot,
+             strerror(-ret));
+    }
   }
 
-  print_success("Concurrent reads completed successfully");
-  results->passed++;
+  // Close all files
+  for (int i = 0; i < num_files; i++) {
+    if (fds[i] >= 0)
+      close(fds[i]);
+  }
+  free(fds);
+
   return 0;
 }
 
-// Cleanup function
-static void cleanup(struct io_uring *ring, int fd, void *buffers) {
-  if (fd >= 0)
-    close(fd);
-  if (buffers) {
-    io_uring_unregister_buffers(ring);
-    free(buffers);
-  }
-  io_uring_queue_exit(ring);
-  remove(TEST_FILE);
-}
-
 int main() {
   TestResults results = {0, 0};
   struct io_uring ring;
-  int fd = -1;
   void *buffers = NULL;
   struct iovec iovs[NUM_BUFFERS];
 
   printf(COLOR_BLUE "\n========================================\n");
-  printf("    io_uring Test Suite\n");
+  printf("    io_uring Sparse File Registration Test\n");
   printf("========================================\n" COLOR_RESET);
 
-  // Create test file
-  if (create_test_file() != 0) {
-    return 1;
+  // Define test files and their content
+  const char *filenames[] = {"test_file_0.txt", "test_file_1.txt",
+                             "test_file_2.txt"};
+
+  const char *contents[] = {
+      "This is file 0: Hello World! The quick brown fox jumps over the lazy "
+      "dog.",
+      "This is file 1: io_uring is awesome for async I/O operations!",
+      "This is file 2: Testing sparse file registration with multiple files."};
+
+  const char *expected_substrings[] = {"Hello World", "io_uring is awesome",
+                                       "sparse file registration"};
+
+  int num_files = 3;
+
+  // Create all test files
+  print_info("Creating test files...");
+  for (int i = 0; i < num_files; i++) {
+    if (create_test_file(filenames[i], contents[i]) != 0) {
+      return 1;
+    }
   }
 
   // Test 1: Create io_uring
   if (test_io_uring_create(&ring, &results) != 0) {
-    cleanup(&ring, fd, buffers);
-    return 1;
+    goto cleanup_files;
   }
 
   // Test 2: Register buffers
   if (test_register_buffers(&ring, &buffers, iovs, &results) != 0) {
-    cleanup(&ring, fd, buffers);
-    return 1;
+    io_uring_queue_exit(&ring);
+    goto cleanup_files;
   }
 
-  // Test 3: Open file
-  if (test_open_file(&fd, &results) != 0) {
-    cleanup(&ring, fd, buffers);
-    return 1;
+  // Test 3: Register empty file table (sparse)
+  if (test_register_files_sparse(&ring, num_files, &results) != 0) {
+    io_uring_unregister_buffers(&ring);
+    free(buffers);
+    io_uring_queue_exit(&ring);
+    goto cleanup_files;
   }
 
-  // Test 4: Submit read
-  uint64_t test_user_data = 12345;
-  if (test_submit_read(&ring, fd, iovs, 0, test_user_data, &results) != 0) {
-    cleanup(&ring, fd, buffers);
-    return 1;
-  }
+  // Test 4: Loop through files, update slots, read and verify
+  test_file_read_loop(&ring, iovs, filenames, expected_substrings, num_files,
+                      &results);
 
-  // Test 5: Wait for completion
-  struct io_uring_cqe *cqe;
-  if (test_wait_completion(&ring, &cqe, &results) != 0) {
-    cleanup(&ring, fd, buffers);
-    return 1;
-  }
+  // Cleanup
+  io_uring_unregister_files(&ring);
+  io_uring_unregister_buffers(&ring);
+  free(buffers);
+  io_uring_queue_exit(&ring);
 
-  // Test 6: Process completion
-  int bytes_read = test_process_completion(cqe, test_user_data, &results);
-  if (bytes_read < 0) {
-    cleanup(&ring, fd, buffers);
-    return 1;
-  }
-  io_uring_cqe_seen(&ring, cqe);
-
-  // Test 7: Verify data
-  if (test_verify_data(iovs, 0, bytes_read, &results) != 0) {
-    cleanup(&ring, fd, buffers);
-    return 1;
-  }
-
-  // Test 8: Concurrent reads
-  if (test_concurrent_reads(&ring, fd, iovs, &results) != 0) {
-    cleanup(&ring, fd, buffers);
-    return 1;
+cleanup_files:
+  // Remove test files
+  for (int i = 0; i < num_files; i++) {
+    remove(filenames[i]);
   }
 
   // Print summary
+  int total = results.passed + results.failed;
   printf(COLOR_BLUE "\n========================================\n");
   printf("           TEST SUMMARY\n");
   printf("========================================\n" COLOR_RESET);
-  printf("  Total tests: %d\n", results.passed + results.failed);
+  printf("  Total tests: %d\n", total);
   printf(COLOR_GREEN "  Passed: %d\n" COLOR_RESET, results.passed);
   if (results.failed > 0) {
     printf(COLOR_RED "  Failed: %d\n" COLOR_RESET, results.failed);
+    printf(COLOR_RED "\n  ✗ SOME TESTS FAILED!\n" COLOR_RESET);
   } else {
-    printf(COLOR_GREEN "  ✓ ALL TESTS PASSED!\n" COLOR_RESET);
+    printf(COLOR_GREEN "\n  ✓ ALL TESTS PASSED!\n" COLOR_RESET);
   }
 
-  // Cleanup
-  cleanup(&ring, fd, buffers);
-
   return results.failed > 0 ? 1 : 0;
 }