Getting Started

Loading…

Getting Started

Relevant source files

• .cargo/config.toml.example
• .github/workflows/python-net-release.yml
• .github/workflows/rust-tests.yml
• .gitignore
• Cargo.toml

This page provides the technical details required to build, configure, and run the SIMD R Drive project locally. It covers the workspace architecture, the role of various Cargo features, and a quick-start guide for the Command Line Interface (CLI).

Workspace Structure

The project is organized as a Cargo workspace to separate the core storage engine from its various extensions, handles, and experimental network layers. The workspace uses resolver = "2" for modern dependency resolution Cargo.toml27

Workspace Members

The workspace is defined in the root Cargo.toml and includes the following crates:

The workspace also explicitly excludes Python bindings from the Cargo resolver to allow them to be managed by maturin and uv Cargo.toml:23-26

Sources: Cargo.toml:14-27 Cargo.toml:60-63

Logical Component Diagram

The following diagram illustrates the relationship between the workspace members and how they interact to form the system.

Workspace Entity Map

graph TD
    subgraph "Core_Workspace"
        A["simd-r-drive (Core & CLI)"]
B["simd-r-drive-entry-handle"]
end

    subgraph "Extensions_Crate"
        C["simd-r-drive-extensions"]
end

    subgraph "Network_Experiments"
        D["simd-r-drive-ws-server"]
E["simd-r-drive-ws-client"]
F["simd-r-drive-muxio-service-definition"]
end

 
   A --> B
 
   C --> A
 
   D --> A
 
   D --> F
 
   E --> F
 
   B -.->|Zero-copy view| A

Sources: Cargo.toml:14-22 Cargo.toml88

Cargo Features

The project uses feature gates to manage dependencies and performance characteristics. These can be toggled during compilation using --features.

Sources: Cargo.toml:72-78 Cargo.toml87

Local Configuration

.cargo/config.toml.example

The project provides a .cargo/config.toml.example file. This is particularly useful for developers working on the experimental network layers who may need to patch dependencies like muxio-rpc-service-caller or muxio-tokio-rpc-client to local paths for simultaneous development across repositories .cargo/config.toml.example:1-5

The .gitignore is configured to ignore the actual .cargo/config.toml, allowing developers to maintain local overrides without committing them .gitignore12

Sources: .cargo/config.toml.example:1-5 .gitignore12

CLI Quick-Start

The simd-r-drive CLI is the primary entry point for interacting with the storage engine from the terminal. It uses clap for argument parsing Cargo.toml82 and supports streaming data via stdin/stdout.

Core Commands

The CLI supports various operations including metadata inspection and data manipulation.

Sources: Cargo.toml82

sequenceDiagram
    participant User
    participant CLI as "Cli (clap::Parser)"
    participant EXEC as "execute_command (cli_parser.rs)"
    participant DS as "DataStore (storage_engine)"

    User->>CLI: simd-r-drive data.bin write my_key 'my_value'
    CLI->>EXEC: execute_command(&cli)
    EXEC->>DS: DataStore::open(path)
    alt "Direct Value"
        EXEC->>DS: DataStore::write(key, value)
    else "Piped Stdin"
        EXEC->>DS: DataStore::write_stream(key, &mut stdin)
    end
    DS-->>User: Stored 'my_key'

Data Flow: CLI Execution

The following diagram shows the data flow when executing a write command via the CLI, mapping CLI entities to core storage logic.

CLI Write Data Flow

Sources: Cargo.toml82

Common Usage Examples

1. Writing a value:

2. Piping a file into storage:

3. Reading with a specific buffer size:

4. Compacting storage:

Building and Testing

Compilation

To build the workspace with all targets and specified features:

The CI pipeline validates builds across ubuntu-latest, macos-latest, and windows-latest .github/workflows/rust-tests.yml:17-22

Running Tests

The project includes extensive integration tests. The parallel feature flag enables parallel iteration tests via Rayon Cargo.toml75

The CI also ensures that benchmarks compile using the --no-run flag .github/workflows/rust-tests.yml:65-66

For Python developers, the python-net-release.yml workflow demonstrates how to run integration tests for the WebSocket client using uv and a dedicated shell script .github/workflows/python-net-release.yml:35-45

Sources: .github/workflows/rust-tests.yml:17-66 .github/workflows/python-net-release.yml:35-45 Cargo.toml75