WebSocket Server

Loading…

WebSocket Server

Relevant source files

• experiments/simd-r-drive-ws-server/README.md
• experiments/simd-r-drive-ws-server/src/cli/cli_parser.rs
• experiments/simd-r-drive-ws-server/src/cli/help_template.rs
• experiments/simd-r-drive-ws-server/src/main.rs
• tests/compaction_tests.rs

The simd-r-drive-ws-server is an experimental RPC wrapper around the core DataStore engine. It enables remote access to the append-only storage via a multiplexed WebSocket transport, leveraging the muxio framework for high-concurrency request/response handling.

Overview and Architecture

The server acts as a bridge between the physical storage on disk and remote clients. It encapsulates a DataStore instance within an Arc<RwLock<DataStore>> to allow safe, concurrent access across multiple asynchronous WebSocket connections handled by tokio experiments/simd-r-drive-ws-server/src/main.rs:31-39

Request Lifecycle

1. Transport : The server initializes an RpcServer which manages the underlying WebSocket transport experiments/simd-r-drive-ws-server/src/main.rs:42-43
2. Multiplexing : The muxio protocol allows multiple concurrent RPC calls over a single WebSocket connection without head-of-line blocking.
3. Dispatch : The RpcServiceEndpointInterface receives a frame, identifies the METHOD_ID, and dispatches it to the registered prebuffered handler experiments/simd-r-drive-ws-server/src/main.rs:55-57
4. Execution : Handlers use task::spawn_blocking to move heavy I/O and locking operations off the async executor experiments/simd-r-drive-ws-server/src/main.rs:60-67 They acquire a read or write lock on the DataStore (e.g., blocking_write or blocking_read) and execute the corresponding operation experiments/simd-r-drive-ws-server/src/main.rs:62-63
5. Response : Results are serialized using bitcode (via the service definition) and sent back through the multiplexer experiments/simd-r-drive-ws-server/src/main.rs:64-65

Data Flow Diagram

The following diagram illustrates how the server wraps the core library and exposes it over the network.

WebSocket Server Data Flow

graph TD
    subgraph "NetworkLayer"
        "RemoteClient" -- "WebSocket (muxio)" --> "RpcServer[muxio-tokio-rpc-server]"
    end

    subgraph "ServerProcess(simd-r-drive-ws-server)"
        "RpcServer" -- "Dispatch" --> "Endpoint[RpcServiceEndpointInterface]"
        "Endpoint" -- "spawn_blocking" --> "Handlers[RPC Handlers]"
        "Handlers" -- "blocking_write()" --> "SharedStore[Arc<RwLock<DataStore>>]"
        "Handlers" -- "blocking_read()" --> "SharedStore"
        
        subgraph "CoreEngine"
            "SharedStore" -- "I/O" --> "DataStore[simd_r_drive::DataStore]"
        end
    end

    subgraph "Storage"
        "DataStore" -- "Append/Mmap" --> "StorageFile[.bin]"
    end

Sources: experiments/simd-r-drive-ws-server/src/main.rs:36-67 experiments/simd-r-drive-ws-server/src/cli/cli_parser.rs:16-42

RPC Endpoints and Service Registration

The server registers a suite of endpoints that mirror the DataStoreReader and DataStoreWriter traits experiments/simd-r-drive-ws-server/src/main.rs:55-163 These are defined in the simd-r-drive-muxio-service-definition crate to ensure type safety between the server and the client.

Code Entity Mapping

Sources: experiments/simd-r-drive-ws-server/src/main.rs:14-19 experiments/simd-r-drive-ws-server/src/main.rs:55-163 experiments/simd-r-drive-ws-server/src/main.rs:172-223

CLI Configuration

The server is started via a CLI that configures the storage path and network binding. The CLI is built using clap experiments/simd-r-drive-ws-server/src/cli/cli_parser.rs:8-16

Arguments and Flags

• storage : The positional argument specifying the path to the storage file. If the file does not exist, the server initializes a new one experiments/simd-r-drive-ws-server/src/cli/cli_parser.rs:17-22
• --host : The IP address to bind to. Defaults to 127.0.0.1 experiments/simd-r-drive-ws-server/src/cli/cli_parser.rs:25-31
• --port : The TCP port for the WebSocket server. If set to 0 (default), the OS assigns a random free port experiments/simd-r-drive-ws-server/src/cli/cli_parser.rs:33-41

Error Handling

The CLI includes a custom error handler in Cli::parse_args() that detects MissingRequiredArgument errors and displays a detailed help template instead of a generic error message experiments/simd-r-drive-ws-server/src/cli/cli_parser.rs:45-61

Sources: experiments/simd-r-drive-ws-server/src/cli/cli_parser.rs:1-62 experiments/simd-r-drive-ws-server/src/cli/help_template.rs:4-14

Implementation Details

Concurrency Model

The server uses a multi-layered concurrency approach:

1. Outer Lock : An Arc<RwLock<DataStore>> protects the DataStore instance across multiple tokio tasks experiments/simd-r-drive-ws-server/src/main.rs:36-39
2. Blocking Tasks : Because DataStore operations involve synchronous file I/O and CPU-intensive SIMD/hashing, handlers use task::spawn_blocking to prevent blocking the tokio worker threads experiments/simd-r-drive-ws-server/src/main.rs:60-67
3. Read/Write Separation : Handlers use blocking_read() for operations like read, len, and exists to allow parallel read access, while using blocking_write() for write and delete to ensure exclusive access during appends experiments/simd-r-drive-ws-server/src/main.rs62 experiments/simd-r-drive-ws-server/src/main.rs103

Serialization

Request and response parameters are serialized using bitcode. The server decodes incoming requests (e.g., Write::decode_request(&bytes)) and encodes outgoing responses (e.g., Write::encode_response(...)) using methods provided by the RpcMethodPrebuffered trait implementation experiments/simd-r-drive-ws-server/src/main.rs:61-65

Sources: experiments/simd-r-drive-ws-server/src/main.rs:31-40 experiments/simd-r-drive-ws-server/src/main.rs:55-115