Building Python Bindings

Relevant source files

• experiments/bindings/python-ws-client/README.md
• experiments/bindings/python-ws-client/extract_readme_tests.py
• experiments/bindings/python-ws-client/integration_test.sh
• experiments/bindings/python-ws-client/pyproject.toml
• experiments/bindings/python-ws-client/simd_r_drive_ws_client/init.py
• experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.py
• experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi
• experiments/bindings/python-ws-client/uv.lock

Purpose and Scope

This page documents the build system and tooling required to compile the Python bindings for SIMD R Drive. It covers PyO3 integration, the Maturin build backend, installation procedures, dependency management with uv, and supported Python versions. For information about using the Python client API, see Python WebSocket Client API. For integration testing procedures, see Integration Testing.

Overview of the Build Architecture

The Python bindings are implemented in Rust using PyO3, which provides Foreign Function Interface (FFI) between Rust and Python. The build process transforms Rust code into platform-specific Python wheels using Maturin, a specialized build backend designed for Rust-Python projects.

graph TB
    subgraph "Source Code"
        RustSrc["Rust Implementation\nBaseDataStoreWsClient"]
PyO3Attrs["PyO3 Attributes\n#[pyclass], #[pymethods]"]
CargoToml["Cargo.toml\ncrate-type = cdylib\npyo3 dependency"]
end
    
    subgraph "Build Configuration"
        PyProject["pyproject.toml\nbuild-backend = maturin\nbindings = pyo3"]
Manifest["Package Metadata\nversion, requires-python\nclassifiers"]
end
    
    subgraph "Build Process"
        Maturin["maturin build\nor\nmaturin develop"]
RustCompiler["rustc + cargo\nCompile to .so/.dylib/.dll"]
FFILayer["PyO3 FFI Bridge\nAuto-generated C bindings"]
end
    
    subgraph "Output Artifacts"
        Wheel["Python Wheel\n.whl package"]
SharedLib["Native Library\nsimd_r_drive_ws_client.so"]
PyStub["Type Stubs\ndata_store_ws_client.pyi"]
end
    
 
   RustSrc --> Maturin
 
   PyO3Attrs --> Maturin
 
   CargoToml --> Maturin
 
   PyProject --> Maturin
 
   Manifest --> Maturin
    
 
   Maturin --> RustCompiler
 
   RustCompiler --> FFILayer
 
   FFILayer --> SharedLib
 
   SharedLib --> Wheel
 
   PyStub --> Wheel
    
 
   Wheel -->|pip install| UserEnv["User Python Environment"]

Build Pipeline Architecture

Sources: experiments/bindings/python-ws-client/pyproject.toml:1-46 experiments/bindings/python-ws-client/README.md:26-38

PyO3 Integration

PyO3 is the Rust library that enables writing Python native modules in Rust. It provides macros and traits for exposing Rust structs and functions to Python.

PyO3 Configuration

The PyO3 bindings type is declared in the Maturin configuration:

Sources: experiments/bindings/python-ws-client/pyproject.toml:29-35

Python Module Structure

The binding layer exposes Rust types as Python classes:

Sources: experiments/bindings/python-ws-client/simd_r_drive_ws_client/init.py:1-13 experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.py:1-63 experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi:1-219

Maturin Build System

Maturin is a build tool for creating Python wheels from Rust projects. It handles cross-compilation, ABI compatibility, and packaging.

Build Commands

Maturin Configuration Details

Sources: experiments/bindings/python-ws-client/pyproject.toml:29-35 experiments/bindings/python-ws-client/README.md:31-38

Installation Methods

Method 1: Install from PyPI (Production)

This installs a pre-built wheel. No Rust toolchain required.

Sources: experiments/bindings/python-ws-client/README.md:26-29

Method 2: Build from Source (Development)

Requires Rust toolchain and Maturin:

The -m flag specifies the manifest path when building from outside the package directory.

Sources: experiments/bindings/python-ws-client/README.md:31-36

Method 3: Development Installation with uv

Using uv for dependency management:

This installs the package in editable mode with development dependencies.

Sources: experiments/bindings/python-ws-client/integration_test.sh:70-77

Installation Flow Comparison

Sources: experiments/bindings/python-ws-client/README.md:25-38 experiments/bindings/python-ws-client/integration_test.sh:70-77

Dependency Management with uv

The project uses uv, a fast Python package installer and resolver, for development dependencies. Unlike traditional requirements.txt or setup.py, dependencies are declared in pyproject.toml using PEP 735 dependency groups.

Dependency Group Structure

Sources: experiments/bindings/python-ws-client/pyproject.toml:37-46

uv Lockfile

The uv.lock file pins exact versions and hashes for reproducible builds. This lockfile includes:

• Direct dependencies (e.g., maturin>=1.8.7)
• Transitive dependencies (e.g., certifi, httpcore)
• Platform-specific wheels
• Python version markers

Sources: experiments/bindings/python-ws-client/uv.lock:1-1036

uv Workflow in CI/Testing

The integration test script demonstrates the uv workflow:

Sources: experiments/bindings/python-ws-client/integration_test.sh:62-87

Supported Python Versions and Platforms

Python Version Matrix

The bindings support Python 3.10 through 3.13 (CPython only):

Sources: experiments/bindings/python-ws-client/pyproject.toml:7-24 experiments/bindings/python-ws-client/README.md:17-24

Platform Support

Sources: experiments/bindings/python-ws-client/pyproject.toml:25-27 experiments/bindings/python-ws-client/uv.lock:117-129

Version Constraints in pyproject.toml

The project metadata declares supported versions for PyPI classifiers:

Programming Language :: Python :: 3.10
Programming Language :: Python :: 3.11
Programming Language :: Python :: 3.12
Programming Language :: Python :: 3.13

Sources: experiments/bindings/python-ws-client/pyproject.toml:21-24

Build Process Deep Dive

File Transformation Pipeline

Sources: experiments/bindings/python-ws-client/pyproject.toml:1-46

Build Artifacts Location

After maturin develop, the installed package structure looks like:

site-packages/
├── simd_r_drive_ws_client/
│   ├── __init__.py                       # Python wrapper
│   ├── data_store_ws_client.py           # Extended API
│   ├── data_store_ws_client.pyi          # Type stubs
│   ├── simd_r_drive_ws_client.so         # Native library (Linux)
│   └── simd_r_drive_ws_client.cpython-*.so
└── simd_r_drive_ws_client-0.11.1.dist-info/
    ├── METADATA                           # Package metadata
    ├── RECORD                             # File integrity
    └── WHEEL                              # Wheel format version

Sources: experiments/bindings/python-ws-client/simd_r_drive_ws_client/init.py:1-13 experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.py:1-63 experiments/bindings/python-ws-client/simd_r_drive_ws_client/data_store_ws_client.pyi:1-219

CI/CD Integration

The CI build recipe referenced in the README demonstrates automated wheel building. The GitHub Actions workflow typically:

1. Sets up multiple Python versions (3.10-3.13)
2. Installs Rust toolchain
3. Installs Maturin
4. Builds wheels for all platforms using maturin build --release
5. Uploads wheels as artifacts

For full details, see the CI/CD Pipeline documentation.

Sources: experiments/bindings/python-ws-client/README.md38

Troubleshooting Common Build Issues

Sources: experiments/bindings/python-ws-client/integration_test.sh:62-68