SIMD Copy Implementation

Loading…

SIMD Copy Implementation

Relevant source files

• extensions/Cargo.toml
• src/storage_engine/simd_copy.rs
• src/utils.rs
• src/utils/align_or_copy.rs
• tests/align_or_copy_tests.rs

The simd_copy utility provides a high-performance memory copy abstraction that leverages hardware-specific vector instructions. By utilizing SIMD (Single Instruction, Multiple Data) , the system can process multiple bytes in a single CPU cycle, which is critical for the high-throughput requirements of the append-only storage engine.

Overview

The implementation targets specific architectures (x86_64 and AArch64) with specialized intrinsic functions while maintaining a safe scalar fallback for unsupported hardware src/storage_engine/simd_copy.rs:111-138

Architecture Support Matrix

Sources: src/storage_engine/simd_copy.rs:10-15 src/storage_engine/simd_copy.rs:16-108

Implementation Details

x86_64 Path: AVX2

On x86_64 systems, the implementation uses Advanced Vector Extensions 2 (AVX2) src/storage_engine/simd_copy.rs:18-19

• Intrinsics : It uses _mm256_loadu_si256 to load 256 bits (32 bytes) of unaligned data from the source and _mm256_storeu_si256 to write to the destination src/storage_engine/simd_copy.rs:47-55
• Loop Unrolling : The main loop processes data in 32-byte increments src/storage_engine/simd_copy.rs:40-58
• Tail Handling : Any remaining bytes (less than 32) are copied using the standard copy_from_slice scalar method src/storage_engine/simd_copy.rs61

AArch64 Path: NEON

For ARM64/AArch64 architectures, the implementation utilizes NEON (Advanced SIMD) src/storage_engine/simd_copy.rs:64-67

• Intrinsics : It uses vld1q_u8 for 128-bit (16-byte) loads and vst1q_u8 for 16-byte stores src/storage_engine/simd_copy.rs:94-101
• Loop Unrolling : The loop processes data in 16-byte increments src/storage_engine/simd_copy.rs:88-104
• Tail Handling : Remaining bytes are handled via copy_from_slice src/storage_engine/simd_copy.rs107

Runtime Feature Detection and Logging

The simd_copy function acts as a dispatcher. On x86_64, it uses the std::is_x86_feature_detected!("avx2") macro to check CPU capabilities at runtime src/storage_engine/simd_copy.rs114

If a compatible SIMD unit is not found (common in virtualized environments like Windows 11 Arm in UTM on Apple Silicon), the system logs a warning src/storage_engine/simd_copy.rs:119-122 To prevent log spam in high-frequency copy operations, the LOG_ONCE mechanism is employed using std::sync::Once src/storage_engine/simd_copy.rs:8-9 This ensures the “AVX2 not detected” warning is emitted exactly once per process lifetime src/storage_engine/simd_copy.rs:121-123

Logic Flow: SIMD Dispatcher

The following diagram illustrates how the simd_copy function selects the appropriate implementation path based on the target architecture and runtime features.

SIMD Dispatcher Logic

graph TD
    "Entry[simd_copy]" --> "ArchCheck{target_arch?}"
    
    "ArchCheck" -- "x86_64" --> "AVX2Check{is_x86_feature_detected!('avx2')}"
    "AVX2Check" -- "Yes" --> "simd_copy_x86"
    "AVX2Check" -- "No" --> "WarnOnce[LOG_ONCE: warn!]"
    "WarnOnce" --> "ScalarFallback[dst.copy_from_slice]"
    
    "ArchCheck" -- "aarch64" --> "simd_copy_arm"
    
    "ArchCheck" -- "other" --> "ScalarFallback"
    
    subgraph "x86_64 Implementation"
        "simd_copy_x86" --> "AVX_Loop[while i < chunks * 32]"
        "AVX_Loop" --> "AVX_Intrinsics[_mm256_loadu/storeu]"
        "AVX_Intrinsics" --> "AVX_Tail[copy_from_slice tail]"
    end
    
    subgraph "ARM Implementation"
        "simd_copy_arm" --> "NEON_Loop[while i < chunks * 16]"
        "NEON_Loop" --> "NEON_Intrinsics[vld1q/vst1q_u8]"
        "NEON_Intrinsics" --> "NEON_Tail[copy_from_slice tail]"
    end

Sources: src/storage_engine/simd_copy.rs:8-9 src/storage_engine/simd_copy.rs:35-138

Data Flow and Memory Safety

The SIMD functions are marked unsafe because they perform raw pointer arithmetic and bypass certain slice bounds checks for performance src/storage_engine/simd_copy.rs:35-83

1. Bounds Calculation : The length is determined by the minimum of the destination and source slice lengths to prevent buffer overflows src/storage_engine/simd_copy.rs36 src/storage_engine/simd_copy.rs84
2. Pointer Casting : Slices are converted to raw pointers (as_ptr() / as_mut_ptr()) and cast to the appropriate SIMD vector types (e.g., *const __m256i for AVX2) src/storage_engine/simd_copy.rs:47-55
3. Unaligned Access : The implementation specifically uses “unaligned” load/store instructions (_mm256_loadu_si256, vld1q_u8), which allows the functions to work on any byte-aligned slice without requiring strict 32-byte or 16-byte memory alignment src/storage_engine/simd_copy.rs:47-55 src/storage_engine/simd_copy.rs:94-101

Entity Mapping: Implementation to Intrinsics

Sources: src/storage_engine/simd_copy.rs:8-15 src/storage_engine/simd_copy.rs:47-55 src/storage_engine/simd_copy.rs:94-101

Related Utilities: align_or_copy

The simd_copy utility is closely related to align_or_copy, which provides zero-copy parsing of binary data into typed slices src/utils/align_or_copy.rs:3-6

align_or_copy uses slice::align_to::<T>() to attempt a zero-copy borrow src/utils/align_or_copy.rs57 If the memory is misaligned or the size is not a multiple of the element size, it falls back to an owned Vec<T> src/utils/align_or_copy.rs:60-74 This is verified in integration tests where misaligned buffers trigger the Cow::Owned fallback path tests/align_or_copy_tests.rs:23-29

Sources: src/utils/align_or_copy.rs:1-75 tests/align_or_copy_tests.rs:23-29