This documentation is part of the "Projects with Books" initiative at zenOSmosis.

The source code for this project is available on GitHub.

Python WebSocket Client Binding

Loading…

Python WebSocket Client Binding

Relevant source files

The simd_r_drive_ws_client Python package provides a high-performance bridge to the Rust WsClient experiments/bindings/python-ws-client/README.md:7-15 it allows Python applications to interact with a simd-r-drive-ws-server using a familiar, dictionary-like API while leveraging the multiplexed I/O and SIMD-optimized serialization of the underlying Rust implementation.

Architecture and Data Flow

The binding is implemented using PyO3 and maturin experiments/bindings/python-ws-client/README.md:14-15 It wraps a Rust WsClient and manages network operations transparently for the Python caller.

Component Hierarchy

The following diagram illustrates how Python calls are dispatched through the Rust shim to the WebSocket transport.

Client Dispatch Architecture

graph TD
    subgraph "Python_Space"
        PY_APP["Python Application"]
DS_WS_CLIENT["DataStoreWsClient (Python)"]
end

    subgraph "Rust_Shim_(PyO3_Module)"
        BASE_WS_PY["BaseDataStoreWsClient (Rust Struct)"]
TOKIO_RT["Tokio Runtime"]
end

    subgraph "Rust_Core_(simd-r-drive-ws-client)"
        WS_CLIENT["WsClient"]
TRAITS["AsyncDataStoreReader / AsyncDataStoreWriter"]
end

 
   PY_APP -->|calls| DS_WS_CLIENT
 
   DS_WS_CLIENT -->|inherits| BASE_WS_PY
 
   BASE_WS_PY -->|executes_via| TOKIO_RT
 
   TOKIO_RT -->|manages| WS_CLIENT
 
   WS_CLIENT -->|implements| TRAITS

Sources: experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.py11 experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi8 experiments/bindings/python-ws-client/README.md:14-15

Data Flow Mapping

This diagram maps Python method calls to their corresponding internal definitions and logic.

Method Binding Map

graph LR
    subgraph "Python_API_(data_store_ws_client.pyi)"
        P_WRITE["write(key, data)"]
P_READ["read(key)"]
P_LEN["__len__()"]
P_CONTAINS["__contains__(key)"]
end

    subgraph "PyO3_Rust_Shim_(BaseDataStoreWsClient)"
        R_WRITE["py_write"]
R_READ["py_read"]
R_LEN["py_len"]
R_EXISTS["py_exists"]
end

    subgraph "Python_Logic_(data_store_ws_client.py)"
        P_STRUCT["batch_read_structured()"]
end

 
   P_WRITE --> R_WRITE
 
   P_READ --> R_READ
 
   P_LEN --> R_LEN
 
   P_CONTAINS --> R_EXISTS
 
   P_STRUCT --> R_READ

Sources: experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi:27-150 experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.py:11-62

Core API: DataStoreWsClient

The primary interface is the DataStoreWsClient class, which inherits from BaseDataStoreWsClient experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.py11

Key Methods and Signatures

Method	Description	Implementation Detail
`write(key, data)`	Appends a KV pair.	Appends key-value pair to storage experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi:27-38
`read(key)`	Retrieves a value.	Performs a memory copy into a new `bytes` object experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi:79-94
`batch_read(keys)`	Reads multiple keys.	Fetches a flat list of keys in one operation experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi:96-107
`exists(key)`	Check key presence.	Checks if key exists and is active experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi:53-63
`__len__()`	Active key count.	Returns the total number of active entries experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi:143-150
`file_size()`	Disk usage.	Returns total file size on server experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi:161-168

Structured Batch Reads

The Python implementation includes batch_read_structured, which accepts a dictionary or list of dictionaries where values are keys to be fetched experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.py:12-31

Decompile: Flattens all dictionary values into a single keys_to_fetch list while mapping original keys experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.py:35-46
Fetch: Executes one high-performance batch_read call experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.py49
Reconstruct: Rebuilds the original nested structure with the fetched results experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.py:51-62

Sources: experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.py:11-62

Key Utilities: NamespaceHasher

To prevent collisions across different logical domains, the NamespaceHasher utility generates 16-byte deterministic keys using XXH3 experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi:171-186

Format: [8 bytes Namespace Hash] || [8 bytes Key Hash] experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi:201-216
Initialization: The prefix is hashed once upon instantiation to serve as a unique identifier experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi:188-199

Sources: experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi:171-216

Test Environment and Lifecycle

The project uses uv for environment management and an integrated shell script for end-to-end testing.

Integration Test Lifecycle (`integration_test.sh`)

The integration_test.sh script automates the full stack testing experiments/bindings/python-ws-client/integration_test.sh:1-5:

Server Startup: Starts simd-r-drive-ws-server in the background via cargo run with temporary storage experiments/bindings/python-ws-client/integration_test.sh:50-56
Environment Setup: Uses uv venv and uv pip install to prepare the Python environment with pytest and maturin experiments/bindings/python-ws-client/integration_test.sh:70-78
Doc-Test Extraction: Runs extract_readme_tests.py to convert README examples into executable pytest functions experiments/bindings/python-ws-client/integration_test.sh:79-80
Execution: Runs pytest against the live server using exported TEST_SERVER_HOST and TEST_SERVER_PORT experiments/bindings/python-ws-client/integration_test.sh:82-87
Cleanup: Uses a trap to kill the server process group and remove temporary storage files experiments/bindings/python-ws-client/integration_test.sh:17-34

README Test Extraction

The extract_readme_tests.py script uses regular expressions to find python fenced code blocks in README.md experiments/bindings/python-ws-client/extract_readme_tests.py:21-24 It wraps these snippets into isolated test_readme_block_{i} functions and writes them to tests/test_readme_blocks.py experiments/bindings/python-ws-client/extract_readme_tests.py:30-42

Sources: experiments/bindings/python-ws-client/integration_test.sh:1-90 experiments/bindings/python-ws-client/extract_readme_tests.py:1-45

Development and Installation

The package is built with maturin and targets Python 3.10+ experiments/bindings/python-ws-client/README.md:14-21

Build Command:

experiments/bindings/python-ws-client/README.md:34-35

Sources: experiments/bindings/python-ws-client/README.md:1-59 experiments/bindings/python-ws-client/pyproject.toml:1-46 experiments/bindings/python-ws-client/uv.lock:1-120

Keyboard shortcuts

rust-simd-r-drive Documentation