Benchmarks

Loading…

Benchmarks

Relevant source files

• benches/contention_benchmark.rs
• benches/storage_benchmark.rs

This page details the performance measurement suite for the SIMD R Drive. The benchmarks are designed to validate the efficiency of the append-only storage engine, particularly its SIMD-optimized paths, zero-copy reads, and concurrent access patterns.

Storage Benchmark (storage_benchmark.rs)

The storage_benchmark is a single-process micro-benchmark that evaluates the core DataStore operations under high load. It processes 1,000,000 entries to measure throughput for writes and various read patterns benches/storage_benchmark.rs:1-3

Implementation Details

The benchmark operates on a NamedTempFile to ensure a clean state and automatic cleanup benches/storage_benchmark.rs:33-46 It uses a fixed entry size of 8 bytes (storing u64 values in little-endian) to simplify validation benches/storage_benchmark.rs:20-24

Benchmark Phases:

1. Append Entries : Writes 1M entries using batch_write with a batch size of 1024. It formats keys as bench-key-{i} and flushes batches to the store benches/storage_benchmark.rs:52-83
2. Sequential Reads : Uses the DataStore iterator (via into_iter()) to traverse entries from newest to oldest, verifying data integrity via u64::from_le_bytes benches/storage_benchmark.rs:98-118
3. Random Reads : Performs 1M random lookups using read() to test KeyIndexer performance and XXH3 hashing speed benches/storage_benchmark.rs:124-149
4. Batch Reads : Uses batch_read() to verify 1M entries in vectorized chunks of 1024, testing the efficiency of bulk key lookups benches/storage_benchmark.rs:155-181

Data Flow: Storage Benchmark

The following diagram illustrates how the benchmark interacts with the DataStore and its internal components.

Storage Benchmark Execution Flow

graph TD
    subgraph "Benchmark_Process"
        [Main] --> [benchmark_append_entries]
        [Main] --> [benchmark_sequential_reads]
        [Main] --> [benchmark_random_reads]
        [Main] --> [benchmark_batch_reads]
    end

    subgraph "DataStore_API"
        [benchmark_append_entries] -- "batch_write()" --> DS_Writer["DataStoreWriter"]
        [benchmark_sequential_reads] -- "into_iter()" --> DS_Iter["EntryIterator"]
        [benchmark_random_reads] -- "read()" --> DS_Reader["DataStoreReader"]
        [benchmark_batch_reads] -- "batch_read()" --> DS_Reader
    end

    subgraph "Internal_Storage_Engine"
 
       DS_Writer --> KI["KeyIndexer"]
DS_Writer --> AL["Append-Only Log"]
DS_Reader --> KI
 
       DS_Reader --> MM["Mmap (Zero-Copy)"]
DS_Iter --> MM
    end

Sources: benches/storage_benchmark.rs:32-41 benches/storage_benchmark.rs:85-92 benches/storage_benchmark.rs:104-109 benches/storage_benchmark.rs:133-136 benches/storage_benchmark.rs:185-186

Contention Benchmark (contention_benchmark.rs)

The contention_benchmark uses the criterion framework to measure lock-contention throughput when many concurrent writers are active benches/contention_benchmark.rs:1-8

Implementation Details

The benchmark evaluates the performance of the engine under heavy concurrent write pressure using a Tokio runtime to manage asynchronous writer tasks benches/contention_benchmark.rs:33-35

Parameters:

• Payload Sizes : Tests with 128 B, 4 KiB, and 64 KiB payloads to measure how data volume affects lock hold times and write throughput benches/contention_benchmark.rs20
• Concurrency : Spawns 8 independent writer tasks using tokio::spawn benches/contention_benchmark.rs:21-49
• Workload : Each task performs 1,000 writes of random payloads (generated via rng.random()) to prevent easy compression or filesystem-level optimizations benches/contention_benchmark.rs:22-56

Code Entity Mapping: Contention

This diagram maps the benchmark entities to the synchronization primitives and runtime components.

Contention Benchmark Mapping

graph LR
    subgraph "Benchmark_Harness"
        CRIT["Criterion::BenchmarkGroup"]
RT["Tokio::Runtime"]
end

    subgraph "Task_Execution"
        RT -- "block_on" --> ITER["Iteration"]
ITER -- "spawn" --> T["Writer Tasks (x8)"]
T -- "loop" --> DS_WRITE["DataStore::write"]
end

    subgraph "Storage_Engine"
        DS_WRITE -- "Acquires" --> INDEX_LOCK["RwLock<KeyIndexer>"]
DS_WRITE -- "Acquires" --> FILE_LOCK["RwLock<File>"]
DS_WRITE -- "Writes" --> MMAP["Mmap (Atomic Tail)"]
end

Sources: benches/contention_benchmark.rs:28-35 benches/contention_benchmark.rs:45-59 benches/contention_benchmark.rs63

Execution and Interpretation

Running Benchmarks

Benchmarks are executed via Cargo. To ensure they are optimized, always run with the --release flag.

CI Integration

The GitHub Actions workflow includes a specific step to verify that benchmarks compile across all supported platforms and feature flag combinations. This is done using the cargo bench --workspace --no-run flag to prevent long-running benchmarks from stalling the CI pipeline while still ensuring API compatibility and code correctness across features like parallel or expose-internal-api.

Interpreting Results

The storage_benchmark outputs human-readable rates using the thousands crate for digit separation benches/storage_benchmark.rs14

The contention_benchmark provides detailed timing statistics per payload size via Criterion, helping identify if lock contention becomes a bottleneck as payload sizes increase benches/contention_benchmark.rs:36-40

Sources: benches/storage_benchmark.rs:77-82 benches/storage_benchmark.rs:112-117 benches/storage_benchmark.rs:143-148 benches/storage_benchmark.rs:175-180 benches/contention_benchmark.rs:36-40