EntryHandle: Zero-Copy Data Access

Loading…

EntryHandle: Zero-Copy Data Access

Relevant source files

• simd-r-drive-entry-handle/README.md
• simd-r-drive-entry-handle/src/constants.rs
• simd-r-drive-entry-handle/src/entry_handle.rs
• simd-r-drive-entry-handle/src/lib.rs

EntryHandle is the primary data structure for accessing entry payloads in rust-simd-r-drive. It acts as a zero-copy owner of a sub-slice within a memory-mapped file (Mmap), ensuring that data is never copied from the disk buffer into application memory during read operations.

Overview and Lifecycle

An EntryHandle binds an Arc<Mmap> with a specific Range<usize> and EntryMetadata simd-r-drive-entry-handle/src/entry_handle.rs:9-19 By using an Arc, the underlying memory mapping is kept alive as long as at least one handle exists, even if the DataStore that created it is closed or the file is remapped simd-r-drive-entry-handle/src/entry_handle.rs:123-125

Data Flow: From Disk to Application

The following diagram illustrates how EntryHandle bridges the gap between the raw memory map and the user’s byte-slice access.

Diagram: EntryHandle Memory Mapping Architecture

graph TD
    subgraph "Disk Storage"
        FILE["test_storage.bin"]
end

    subgraph "Process Address Space"
        MMAP["memmap2::Mmap (Memory Mapping)"]
ARC["Arc<Mmap> (Shared Ownership)"]
end

    subgraph "Code Entities (EntryHandle)"
        EH["EntryHandle"]
META["EntryMetadata"]
RANGE["Range<usize> (Payload Bounds)"]
end

 
   FILE -.->|mmap| MMAP
    MMAP --- ARC
    ARC --- EH
 
   EH --> META
 
   EH --> RANGE
    
 
   EH -.->|as_slice| SLICE["&[u8] (Zero-Copy View)"]
RANGE -.->|indexes| MMAP

Sources: simd-r-drive-entry-handle/src/entry_handle.rs:7-19 simd-r-drive-entry-handle/src/entry_handle.rs:151-155

Core API Methods

Construction

• from_arc_mmap(mmap_arc, range, metadata) : The standard constructor used by the DataStore reader and EntryIterator. It wraps an existing shared mapping without allocations simd-r-drive-entry-handle/src/entry_handle.rs:129-139
• from_owned_bytes_anon(bytes, key_hash) : Creates an in-memory entry backed by an anonymous MmapMut. This involves exactly one copy of the input bytes into the mapping, which is then sealed as read-only simd-r-drive-entry-handle/src/entry_handle.rs:87-113 It computes a CRC32C checksum during construction to maintain consistency with file-backed entries simd-r-drive-entry-handle/src/entry_handle.rs:95-106

Data Access

• as_slice() : Returns a &[u8] pointing directly into the memory-mapped region. This is the core of the zero-copy guarantee simd-r-drive-entry-handle/src/entry_handle.rs:151-155
• Deref Implementation: EntryHandle implements std::ops::Deref, allowing it to be used interchangeably with &[u8] (e.g., *entry_handle) simd-r-drive-entry-handle/src/entry_handle.rs:36-42
• clone_arc() : Instead of a standard Clone (which might imply deep copying), this method explicitly increments the Arc reference count, creating a new handle to the same data simd-r-drive-entry-handle/src/entry_handle.rs:179-185

Metadata and Validation

• key_hash() : Returns the 64-bit XXH3 hash of the key associated with this entry simd-r-drive-entry-handle/src/entry_handle.rs:200-202
• is_valid_checksum() : Re-computes the CRC32C of the payload in the as_slice() range and compares it against the stored metadata.checksum simd-r-drive-entry-handle/src/entry_handle.rs:215-219
• size() : Returns the length of the payload in bytes simd-r-drive-entry-handle/src/entry_handle.rs:232-234
• file_size() : Returns the total size of the underlying memory-mapped file simd-r-drive-entry-handle/src/entry_handle.rs:242-244

Sources: simd-r-drive-entry-handle/src/entry_handle.rs:21-244 simd-r-drive-entry-handle/src/entry_handle.rs:87-113

Internal API and Feature Gates

When the expose-internal-api feature is enabled (or during tests), additional methods are available to inspect the physical layout and memory addresses:

• offset_range() : Returns the Range<u64> representing the absolute byte offsets within the file where the payload is located simd-r-drive-entry-handle/src/entry_handle.rs:260-262
• address_range() : Returns the Range<usize> of virtual memory addresses (pointers) where the entry is currently mapped simd-r-drive-entry-handle/src/entry_handle.rs:271-274
• arc_ptr() : (Test-only) Returns the raw pointer to the Mmap struct to verify that multiple handles share the same underlying allocation simd-r-drive-entry-handle/src/entry_handle.rs:30-32

Sources: simd-r-drive-entry-handle/src/entry_handle.rs:22-33 simd-r-drive-entry-handle/src/entry_handle.rs:250-275

Alignment and SIMD Compatibility

The EntryHandle is designed to support SIMD operations and zero-copy typed views. Because the DataStore enforces PAYLOAD_ALIGNMENT (64 bytes) simd-r-drive-entry-handle/src/constants.rs18 the as_slice() pointer is often suitable for direct casting to SIMD types or aligned buffers simd-r-drive-entry-handle/src/debug_assert_aligned.rs:26-35

Diagram: Zero-Copy Memory Access Path

Sources: simd-r-drive-entry-handle/src/constants.rs:13-18 simd-r-drive-entry-handle/src/debug_assert_aligned.rs:66-81

Integration with Storage Operations

The DataStore utilizes EntryHandle in its read operations and iteration. When an entry is read, the DataStore constructs an EntryHandle using the current file mapping. This allows the system to verify integrity via is_valid_checksum() before passing the handle to the caller. The debug_assert_aligned_offset function ensures that file offsets used during handle construction adhere to the 64-byte boundary required for SIMD-optimized access simd-r-drive-entry-handle/src/debug_assert_aligned.rs:66-81

Sources: simd-r-drive-entry-handle/src/entry_handle.rs:129-139 simd-r-drive-entry-handle/src/debug_assert_aligned.rs:66-81