Native Rust Client

Relevant source files

• Cargo.lock
• experiments/simd-r-drive-muxio-service-definition/Cargo.toml
• experiments/simd-r-drive-ws-client/Cargo.toml
• experiments/simd-r-drive-ws-server/Cargo.toml

Purpose and Scope

The simd-r-drive-ws-client crate provides a native Rust client library for remote access to the SIMD R Drive storage engine via WebSocket connections. This client enables Rust applications to interact with a remote DataStore instance through the Muxio RPC framework with bitcode serialization.

This document covers the native Rust client implementation. For the WebSocket server that this client connects to, see WebSocket Server. For the Muxio RPC protocol details, see Muxio RPC Framework. For the Python bindings that wrap this client, see Python WebSocket Client API.

Sources: experiments/simd-r-drive-ws-client/Cargo.toml:1-22

Architecture Overview

The native Rust client is structured as a thin wrapper around the Muxio RPC client infrastructure, providing type-safe access to remote DataStore operations.

Key Components:

graph TB
    subgraph "Application Layer"
        UserApp["User Application\nRust Code"]
end
    
    subgraph "simd-r-drive-ws-client Crate"
        ClientAPI["Client API\nDataStoreReader/Writer Traits"]
WsClient["WebSocket Client\nConnection Management"]
end
    
    subgraph "RPC Infrastructure"
        ServiceCaller["muxio-rpc-service-caller\nMethod Invocation"]
TokioRpcClient["muxio-tokio-rpc-client\nTransport Layer"]
end
    
    subgraph "Shared Contract"
        ServiceDef["simd-r-drive-muxio-service-definition\nRPC Interface"]
Bitcode["bitcode\nSerialization"]
end
    
    subgraph "Network"
        WsConnection["WebSocket Connection\ntokio-tungstenite"]
end
    
 
   UserApp --> ClientAPI
 
   ClientAPI --> WsClient
 
   WsClient --> ServiceCaller
 
   ServiceCaller --> TokioRpcClient
 
   ServiceCaller --> ServiceDef
 
   TokioRpcClient --> Bitcode
 
   TokioRpcClient --> WsConnection
    
    style ClientAPI fill:#f9f9f9,stroke:#333,stroke-width:2px
    style ServiceDef fill:#f9f9f9,stroke:#333,stroke-width:2px

Sources: experiments/simd-r-drive-ws-client/Cargo.toml:13-21 Cargo.lock:1302-1318

Client API Structure

The client implements the same DataStoreReader and DataStoreWriter traits as the local DataStore, enabling transparent remote access with minimal API differences.

Core Traits:

graph LR
    subgraph "Trait Implementations"
        Reader["DataStoreReader\nread()\nexists()\nbatch_read()"]
Writer["DataStoreWriter\nwrite()\ndelete()\nbatch_write()"]
end
    
    subgraph "Client Implementation"
        WsClient["WebSocket Client\nAsync Methods"]
ConnState["Connection State\nURL, Options"]
end
    
    subgraph "RPC Layer"
        Serializer["Request Serialization\nbitcode"]
Caller["Service Caller\nCall Routing"]
Deserializer["Response Deserialization\nbitcode"]
end
    
 
   Reader --> WsClient
 
   Writer --> WsClient
 
   WsClient --> ConnState
 
   WsClient --> Serializer
 
   Serializer --> Caller
 
   Caller --> Deserializer
    
    style Reader fill:#f9f9f9,stroke:#333,stroke-width:2px
    style Writer fill:#f9f9f9,stroke:#333,stroke-width:2px

• DataStoreReader : Read-only operations (read, exists, batch_read, iteration)
• DataStoreWriter : Write operations (write, delete, batch_write)
• async-trait : All methods are asynchronous, requiring a Tokio runtime

Sources: experiments/simd-r-drive-ws-client/Cargo.toml:14-21 experiments/simd-r-drive-muxio-service-definition/Cargo.toml:1-16

Connection Management

The client manages persistent WebSocket connections to the remote server with automatic reconnection and error handling.

Connection Lifecycle:

sequenceDiagram
    participant App as "Application"
    participant Client as "WebSocket Client"
    participant Transport as "muxio-tokio-rpc-client"
    participant Server as "Remote Server"
    
    Note over App,Server: Connection Establishment
    App->>Client: connect(url)
    Client->>Transport: create WebSocket connection
    Transport->>Server: WebSocket handshake
    Server-->>Transport: connection established
    Transport-->>Client: client ready
    Client-->>App: connected client
    
    Note over App,Server: Normal Operation
    App->>Client: read(key)
    Client->>Transport: serialize request
    Transport->>Server: send via WebSocket
    Server-->>Transport: response data
    Transport-->>Client: deserialize response
    Client-->>App: return result
    
    Note over App,Server: Error Handling
    Server-->>Transport: connection lost
    Transport-->>Client: connection error
    Client->>Transport: reconnection attempt
    Transport->>Server: reconnect

1. Initialization : Client connects to server URL with connection options
2. Authentication : Optional authentication via Muxio RPC mechanisms
3. Active State : Client maintains persistent WebSocket connection
4. Error Recovery : Automatic reconnection on transient failures
5. Shutdown : Graceful connection termination

Sources: Cargo.lock:1302-1318 experiments/simd-r-drive-ws-client/Cargo.toml:16-19

graph TB
    subgraph "Client Side"
        Method["Client Method Call\nread/write/delete"]
ReqBuilder["Request Builder\nCreate RPC Request"]
Serializer["bitcode Serialization\nBinary Encoding"]
Sender["WebSocket Send\nBinary Frame"]
end
    
    subgraph "Network"
        WsFrame["WebSocket Frame\nBinary Message"]
end
    
    subgraph "Server Side"
        Receiver["WebSocket Receive\nBinary Frame"]
Deserializer["bitcode Deserialization\nBinary Decoding"]
Handler["Request Handler\nExecute DataStore Operation"]
Response["Response Builder\nCreate RPC Response"]
end
    
 
   Method --> ReqBuilder
 
   ReqBuilder --> Serializer
 
   Serializer --> Sender
 
   Sender --> WsFrame
 
   WsFrame --> Receiver
 
   Receiver --> Deserializer
 
   Deserializer --> Handler
 
   Handler --> Response
 
   Response --> Serializer
    
    style Method fill:#f9f9f9,stroke:#333,stroke-width:2px
    style Handler fill:#f9f9f9,stroke:#333,stroke-width:2px

Request-Response Flow

All client operations follow a standardized request-response pattern through the Muxio RPC framework.

Request Structure:

Response Structure:

Sources: experiments/simd-r-drive-muxio-service-definition/Cargo.toml:14-15 Cargo.lock:392-402

Async Runtime Requirements

The client requires a Tokio async runtime for all operations. The async-trait crate enables async methods in trait implementations.

Runtime Configuration:

graph TB
    subgraph "Application"
        Main["#[tokio::main]\nasync fn main()"]
UserCode["User Code\nawait client.read()"]
end
    
    subgraph "Client"
        AsyncMethods["async-trait Methods\nDataStoreReader/Writer"]
TokioTasks["Tokio Tasks\nNetwork I/O"]
end
    
    subgraph "Tokio Runtime"
        Executor["Task Executor\nWork Stealing Scheduler"]
Reactor["I/O Reactor\nepoll/kqueue/IOCP"]
end
    
 
   Main --> UserCode
 
   UserCode --> AsyncMethods
 
   AsyncMethods --> TokioTasks
 
   TokioTasks --> Executor
 
   TokioTasks --> Reactor
    
    style AsyncMethods fill:#f9f9f9,stroke:#333,stroke-width:2px
    style Executor fill:#f9f9f9,stroke:#333,stroke-width:2px

• Multi-threaded Runtime : Default for concurrent operations
• Current-thread Runtime : Available for single-threaded use cases
• Feature Flags : Requires tokio with rt-multi-thread and net features

Sources: experiments/simd-r-drive-ws-client/Cargo.toml:19-21 Cargo.lock:279-287

Error Handling

The client propagates errors from multiple layers of the stack, providing detailed error information for debugging and recovery.

Error Types:

Error Propagation Flow:

Sources: experiments/simd-r-drive-ws-client/Cargo.toml:17-18 Cargo.lock:1261-1271

Usage Patterns

Basic Connection and Operations

The client follows standard Rust async patterns for initialization and operation:

Concurrent Operations

The client supports concurrent operations through standard Tokio concurrency primitives:

Sources: experiments/simd-r-drive-ws-client/Cargo.toml:19-21

graph TB
    subgraph "Service Definition"
        Methods["Method Definitions\nread, write, delete, etc."]
Types["Request/Response Types\nbitcode derive macros"]
MethodHash["Method ID Hashing\nXXH3 of method names"]
end
    
    subgraph "Client Usage"
        ClientImpl["Client Implementation\nUses defined methods"]
TypeSafety["Type Safety\nCompile-time checking"]
end
    
    subgraph "Server Usage"
        ServerImpl["Server Implementation\nHandles defined methods"]
Routing["Request Routing\nHash-based dispatch"]
end
    
 
   Methods --> ClientImpl
 
   Methods --> ServerImpl
 
   Types --> ClientImpl
 
   Types --> ServerImpl
 
   MethodHash --> Routing
 
   ClientImpl --> TypeSafety
    
    style Methods fill:#f9f9f9,stroke:#333,stroke-width:2px
    style Types fill:#f9f9f9,stroke:#333,stroke-width:2px

Integration with Service Definition

The client relies on the shared service definition crate for type-safe RPC communication.

Shared Contract Benefits:

• Type Safety : Compile-time verification of request/response types
• Version Compatibility : Client and server must use compatible service definitions
• Method Resolution : XXH3 hash-based method identification
• Serialization Schema : Consistent bitcode encoding across client and server

Sources: experiments/simd-r-drive-ws-client/Cargo.toml15 experiments/simd-r-drive-muxio-service-definition/Cargo.toml:1-16

Performance Considerations

The native Rust client provides several performance advantages over alternative approaches:

Performance Characteristics:

Memory Efficiency:

• Zero-copy where possible through bitcode and WebSocket frames
• Efficient buffer reuse in Tokio's I/O layer
• Minimal allocation overhead compared to HTTP-based protocols

Throughput:

• Supports request pipelining for high-throughput workloads
• Concurrent operations through Tokio's work-stealing scheduler
• Batch operations reduce round-trip overhead

Sources: Cargo.lock:392-402 Cargo.lock:1302-1318

Comparison with Direct Access

The WebSocket client provides remote access with different tradeoffs compared to direct DataStore usage:

When to Use the Client:

• Remote access to centralized storage
• Microservice architectures requiring shared state
• Language interoperability (via Python bindings)
• Isolation of storage from compute workloads

When to Use Direct Access:

• Single-machine deployments
• Latency-critical applications
• Maximum throughput requirements
• Zero-copy read performance needs

Sources: experiments/simd-r-drive-ws-client/Cargo.toml14

Logging and Debugging

The client uses the tracing crate for structured logging and diagnostics.

Logging Levels:

• TRACE : Detailed RPC message contents and serialization
• DEBUG : Connection state changes, request/response flow
• INFO : Connection establishment, disconnection events
• WARN : Recoverable errors, retry attempts
• ERROR : Unrecoverable errors, connection failures

Diagnostic Information:

• Request/response timing
• Serialized message sizes
• Connection state transitions
• Error context and stack traces

Sources: experiments/simd-r-drive-ws-client/Cargo.toml20 Cargo.lock:279-287