This documentation is part of the "Projects with Books" initiative at zenOSmosis.

The source code for this project is available on GitHub.

Integration Testing

Loading…

Integration Testing

Relevant source files

Purpose and Scope

This document describes the integration testing infrastructure for the Python WebSocket client bindings. The integration test suite validates that the Python client (simd-r-drive-ws-client) can successfully communicate with the WebSocket server (simd-r-drive-ws-server) over a real network connection. The tests automatically extract and validate code examples from the README documentation to ensure documentation accuracy.

For information about building Python bindings, see Building Python Bindings. For details on the Python WebSocket Client API itself, see Python WebSocket Client API.

Test Architecture Overview

Sources: experiments/bindings/python-ws-client/integration_test.sh:1-91 experiments/bindings/python-ws-client/extract_readme_tests.py:1-46

Test Orchestration Script

The integration test suite is orchestrated by the integration_test.sh Bash script, which manages the complete test lifecycle including server startup, environment setup, test execution, and cleanup.

Script Configuration

Configuration Variable	Default Value	Purpose
`EXPERIMENTS_DIR_REL_PATH`	`../../`	Relative path to experiments directory
`SERVER_PACKAGE_NAME`	`simd-r-drive-ws-server`	Cargo package name for server
`STORAGE_FILE`	`/tmp/simd-r-drive-pytest-storage.bin`	Temporary storage file path
`SERVER_HOST`	`127.0.0.1`	Server bind address
`SERVER_PORT`	`34129`	Server listen port

Sources: experiments/bindings/python-ws-client/integration_test.sh:8-14

Execution Flow

Sources: experiments/bindings/python-ws-client/integration_test.sh:35-90

Cleanup Mechanism

The script registers a cleanup() function with trap cleanup EXIT to ensure resources are released regardless of how the script terminates:

Process Group Termination : Uses kill -9 "-$SERVER_PID" to kill the entire process group, ensuring the server and any child processes are stopped
Storage File Removal : Deletes the temporary storage file at /tmp/simd-r-drive-pytest-storage.bin
Error Suppression : Uses || true to prevent cleanup failures from failing the script

Sources: experiments/bindings/python-ws-client/integration_test.sh:18-30 experiments/bindings/python-ws-client/integration_test.sh:32-33

README Example Extraction

The extract_readme_tests.py script automatically generates pytest test functions from Python code blocks in the README documentation, ensuring that documented examples remain functional.

graph LR
    subgraph "Input"
        README[README.md\nPython code blocks]
    end
    
    subgraph "Extraction Functions"
        EXTRACT[extract_python_blocks]
        REGEX["re.compile pattern\n```python...```"]
STRIP[strip_non_ascii]
        WRAP[wrap_as_test_fn]
    end
    
    subgraph "Output"
        TEST_FN["def test_readme_block_{i}"]
TEST_FILE[tests/test_readme_blocks.py]
    end
    
 
   README --> EXTRACT
 
   EXTRACT --> REGEX
 
   REGEX --> STRIP
 
   STRIP --> WRAP
 
   WRAP --> TEST_FN
 
   TEST_FN --> TEST_FILE

Extraction Process

Sources: experiments/bindings/python-ws-client/extract_readme_tests.py:15-42

Key Functions

Function	Input	Output	Purpose
`extract_python_blocks`	README text	`list[str]`	Uses regex r"```python\n(.*?)```" to extract code blocks
`strip_non_ascii`	Code string	ASCII string	Removes non-ASCII characters using `encode("ascii", errors="ignore")`
`wrap_as_test_fn`	Code string, index	Test function string	Wraps code in `def test_readme_block_{idx}():` with proper indentation

Sources: experiments/bindings/python-ws-client/extract_readme_tests.py:21-34

File Paths and Constants

The script uses fixed file paths defined at module level:

README = Path("README.md") - Source documentation file
TEST_FILE = Path("tests/test_readme_blocks.py") - Generated test file

The generated test file includes a header comment indicating it is auto-generated and imports pytest.

Sources: experiments/bindings/python-ws-client/extract_readme_tests.py:18-19 experiments/bindings/python-ws-client/extract_readme_tests.py:40-41

Test Execution with pytest

The test suite uses pytest as the test runner, executed through the uv run command to ensure correct virtual environment activation.

pytest Configuration

The test execution command uses specific flags:

Flag	Purpose
`-v`	Verbose output showing individual test names
`-s`	Disable output capture (show print statements)

Sources: experiments/bindings/python-ws-client/integration_test.sh87

Environment Variables

The test suite relies on environment variables to locate the running server:

Variable	Source	Usage
`TEST_SERVER_HOST`	`$SERVER_HOST` from script	Server IP address for client connection
`TEST_SERVER_PORT`	`$SERVER_PORT` from script	Server port for client connection

These variables are exported before pytest execution so test code can access the server endpoint.

Sources: experiments/bindings/python-ws-client/integration_test.sh:84-85

Test Function Generation

Each Python code block from README.md becomes an isolated test function following the pattern:

def test_readme_block_0():
    <indented code from README>

def test_readme_block_1():
    <indented code from README>

The test functions are numbered sequentially starting from 0. Each test runs independently with its own test context.

Sources: experiments/bindings/python-ws-client/extract_readme_tests.py:30-34

graph TB
    subgraph "Server Startup Sequence"
        CD[cd to experiments dir]
        SET_M["set -m\nEnable job control"]
CARGO["cargo run --package simd-r-drive-ws-server"]
ARGS["-- $STORAGE_FILE --host $HOST --port $PORT"]
BG["& (background)"]
CAPTURE["SERVER_PID=$!"]
UNSET_M["set +m\nDisable job control"]
end
    
 
   CD --> SET_M
 
   SET_M --> CARGO
 
   CARGO --> ARGS
 
   ARGS --> BG
 
   BG --> CAPTURE
 
   CAPTURE --> UNSET_M
    
    style CARGO fill:#f9f9f9
    style CAPTURE fill:#f9f9f9

Server Lifecycle Management

The integration test script manages the WebSocket server lifecycle to provide a clean test environment for each run.

Server Startup

The script uses set -m to enable job control before starting the server, which allows proper PID capture of background processes. After capturing the PID, job control is disabled with set +m.

Sources: experiments/bindings/python-ws-client/integration_test.sh:47-56

Server Configuration

The server is started with the following arguments:

Storage File : Positional argument specifying the data file path (/tmp/simd-r-drive-pytest-storage.bin)
--host : Bind address (127.0.0.1 for localhost-only access)
--port : Listen port (34129 for test isolation)

These arguments are passed after the -- separator to distinguish cargo arguments from application arguments.

Sources: experiments/bindings/python-ws-client/integration_test.sh53

Server Termination

The cleanup function terminates the server using process group kill:

-9 : SIGKILL signal for forceful termination
"-$SERVER_PID" : Negative PID to kill entire process group
2>/dev/null: Suppress error messages
|| true : Prevent script failure if process already exited

Sources: experiments/bindings/python-ws-client/integration_test.sh25

graph TB
    subgraph "uv Environment Setup"
        CHECK["command -v uv"]
VENV["uv venv"]
INSTALL_BASE["uv pip install pytest maturin"]
INSTALL_DEV["uv pip install -e . --group dev"]
end
    
    subgraph "Python Dependencies"
        PYTEST[pytest]
        MATURIN[maturin]
        DEV_DEPS[Development dependencies\nfrom pyproject.toml]
    end
    
    subgraph "Lock File"
        UV_LOCK[uv.lock]
        ANYIO[anyio]
        HTTPX[httpx]
        HTTPCORE[httpcore]
        NUMPY[numpy]
        MYPY[mypy]
    end
    
 
   CHECK --> VENV
 
   VENV --> INSTALL_BASE
 
   INSTALL_BASE --> PYTEST
 
   INSTALL_BASE --> MATURIN
 
   INSTALL_BASE --> INSTALL_DEV
 
   INSTALL_DEV --> DEV_DEPS
    
    UV_LOCK -.resolves.-> ANYIO
    UV_LOCK -.resolves.-> HTTPX
    UV_LOCK -.resolves.-> HTTPCORE
    UV_LOCK -.resolves.-> NUMPY
    UV_LOCK -.resolves.-> MYPY
    
    style CHECK fill:#f9f9f9
    style VENV fill:#f9f9f9
    style UV_LOCK fill:#f9f9f9

Command	Purpose
`uv venv`	Creates a virtual environment in `.venv` directory
`uv pip install --quiet pytest maturin`	Installs test runner and build tool
`uv pip install -e . --group dev`	Installs package in editable mode with dev dependencies
`uv run <command>`	Executes command in virtual environment context

The --quiet flag suppresses installation progress output for cleaner logs.

Sources: experiments/bindings/python-ws-client/integration_test.sh:70-80

Dependency Lock File

The uv.lock file pins exact versions and hashes for all dependencies:

Package	Version	Purpose
`pytest`	Latest	Test framework
`maturin`	1.8.7+	PyO3 build system
`anyio`	4.9.0+	Async I/O foundation
`httpx`	0.28.1+	HTTP client (WebSocket support)
`mypy`	1.16.1+	Static type checking
`numpy`	2.2.6+/2.3.0+	Numerical computing (conditional)

The lock file uses resolution markers to handle different Python versions (e.g., python_full_version >= '3.11').

Sources: experiments/bindings/python-ws-client/uv.lock:1-7 experiments/bindings/python-ws-client/uv.lock:110-130 experiments/bindings/python-ws-client/uv.lock:133-169

uv Availability Check

Before proceeding with environment setup, the script validates that uv is installed:

This check ensures clear error messages if the prerequisite tool is missing.

Sources: experiments/bindings/python-ws-client/integration_test.sh:62-68

graph TB
    START["integration_test.sh start"]
subgraph "Phase 1: Setup"
        CD_EXPERIMENTS[cd experiments/]
        BUILD_SERVER["cargo run --package simd-r-drive-ws-server &"]
CAPTURE_PID[Capture SERVER_PID]
    end
    
    subgraph "Phase 2: Environment"
        CD_CLIENT[cd bindings/python-ws-client]
        CHECK_UV[Check uv availability]
        CREATE_VENV[uv venv]
        INSTALL_DEPS[uv pip install pytest maturin]
        INSTALL_EDITABLE["uv pip install -e . --group dev"]
end
    
    subgraph "Phase 3: Test Generation"
        RUN_EXTRACT[uv run extract_readme_tests.py]
        PARSE_README[Parse README.md]
        GENERATE_TESTS[Generate tests/test_readme_blocks.py]
    end
    
    subgraph "Phase 4: Test Execution"
        EXPORT_ENV[Export TEST_SERVER_HOST/PORT]
        RUN_PYTEST["uv run pytest -v -s"]
EXECUTE_TESTS[Execute test_readme_block_* functions]
    end
    
    subgraph "Phase 5: Cleanup"
        TRAP_EXIT[trap EXIT triggers]
        KILL_SERVER["kill -9 -$SERVER_PID"]
REMOVE_FILE["rm -f $STORAGE_FILE"]
end
    
 
   START --> CD_EXPERIMENTS
 
   CD_EXPERIMENTS --> BUILD_SERVER
 
   BUILD_SERVER --> CAPTURE_PID
 
   CAPTURE_PID --> CD_CLIENT
    
 
   CD_CLIENT --> CHECK_UV
 
   CHECK_UV --> CREATE_VENV
 
   CREATE_VENV --> INSTALL_DEPS
 
   INSTALL_DEPS --> INSTALL_EDITABLE
    
 
   INSTALL_EDITABLE --> RUN_EXTRACT
 
   RUN_EXTRACT --> PARSE_README
 
   PARSE_README --> GENERATE_TESTS
    
 
   GENERATE_TESTS --> EXPORT_ENV
 
   EXPORT_ENV --> RUN_PYTEST
 
   RUN_PYTEST --> EXECUTE_TESTS
    
 
   EXECUTE_TESTS --> TRAP_EXIT
 
   TRAP_EXIT --> KILL_SERVER
 
   KILL_SERVER --> REMOVE_FILE
    
    style START fill:#f9f9f9
    style TRAP_EXIT fill:#f9f9f9

Test Execution Workflow Summary

The complete integration test workflow coordinates multiple tools and processes:

Sources: experiments/bindings/python-ws-client/integration_test.sh:1-91 experiments/bindings/python-ws-client/extract_readme_tests.py:1-46

Dismiss

Refresh this wiki

Enter email to refresh

rust-simd-r-drive Documentation

Integration Testing

Integration Testing

Purpose and Scope

Test Architecture Overview

Test Orchestration Script

Script Configuration

Execution Flow

Cleanup Mechanism

README Example Extraction

Extraction Process

Key Functions

File Paths and Constants

Test Execution with pytest

pytest Configuration

Environment Variables

Test Function Generation

Server Lifecycle Management

Server Startup

Server Configuration

Server Termination

Environment Setup with uv

Dependency Resolution

uv Commands

Dependency Lock File

uv Availability Check

Test Execution Workflow Summary

Keyboard shortcuts

rust-simd-r-drive Documentation