Testing and CI/CD

Loading…

Testing and CI/CD

Relevant source files

• .github/workflows/rust-tests.yml
• .gitignore
• tests/README.md

The rust-simd-r-drive project employs a multi-layered testing strategy and an extensive CI/CD pipeline to ensure the correctness of its append-only storage engine across different operating systems, hardware architectures, and feature flag configurations.

The test suite is designed to validate core storage invariants—such as CRC32C checksum integrity, payload alignment, and thread safety—while the CI/CD pipelines automate linting, cross-platform builds, documentation generation, and release workflows for both Rust and Python components.

Test Suite Organization

The testing infrastructure is primarily located in the tests/ directory of the core crate, supplemented by unit tests within the source files. The suite relies heavily on conditional compilation to test various performance optimizations.

Rust Integration Tests

Integration tests are organized by functional area to isolate behaviors like concurrency, persistence, and SIMD alignment. Many tests require specific Cargo features, such as expose-internal-api to inspect the underlying file structure or parallel to exercise Rayon-based iterators.

Test Execution Logic

Sources: tests/README.md:1-26 src/storage_engine/traits/reader.rs:4-21 src/storage_engine/traits/writer.rs:4-65

For a detailed breakdown of test files and feature-gated testing, see Rust Test Suite.

Performance Benchmarking

The project includes a benchmarking suite that is checked for compilation during every CI run via cargo bench --workspace --no-run .github/workflows/rust-tests.yml:65-66 These benchmarks cover:

• Storage Benchmarks : Sequential vs. random I/O performance.
• Contention Benchmarks : Measuring lock overhead in the DataStore.

Sources: .github/workflows/rust-tests.yml:64-66 .gitignore:1-12

CI/CD Pipelines

The project uses GitHub Actions to orchestrate a comprehensive suite of workflows. These pipelines ensure that every pull request and tag follows the project’s quality standards.

Pipeline Architecture

The following diagram illustrates how the CI/CD workflows interact with the codebase and external package registries:

CI/CD Workflow Mapping

graph TD
    subgraph "GitHub_Actions"
        RT["rust-tests.yml"]
RL["rust-lint.yml"]
BD["build-docs.yml"]
RR["rust-release.yml"]
PB["python-build.yml"]
PNR["python-net-release.yml"]
end

    subgraph "Codebase_Entities"
        WS["Cargo_Workspace"]
DS["DataStore_(src/storage_engine)"]
CLI["CLI_(src/main.rs)"]
PY["Python_Bindings_(bindings/python)"]
end

    subgraph "Registries"
        CR["crates.io"]
PYPI["PyPI"]
end

 
   RT --> WS
 
   RL --> WS
 
   BD --> DS
 
   RR --> CR
 
   PB --> PY
 
   PNR --> PYPI

Sources: .github/workflows/rust-tests.yml:1-7 .github/workflows/rust-tests.yml:15-35

Cross-Platform Matrix

The rust-tests.yml workflow utilizes a build matrix to guarantee compatibility across major operating systems and feature combinations. It runs on ubuntu-latest, macos-latest, and windows-latest .github/workflows/rust-tests.yml22

Feature Flag Test Matrix

Automation and Releases

Beyond testing, the CI/CD system handles:

• Concurrency Management : Automatically cancels older jobs when new commits are pushed .github/workflows/rust-tests.yml:9-13
• Caching : Speeds up builds by caching ~/.cargo/registry and target/ .github/workflows/rust-tests.yml:45-57
• Rust Toolchain : Uses dtolnay/rust-toolchain@stable for consistent build environments .github/workflows/rust-tests.yml:41-42
• Python Integration : Validating the WebSocket-based Python client against a live Rust server instance.
• Release Orchestration : Automated publishing to Crates.io and PyPI when version tags are pushed.

For details on specific workflow configurations and release triggers, see CI/CD Pipelines.

Sources: .github/workflows/rust-tests.yml:9-66