pyphi.parallel

Provides an interface for parallel computation.

This module provides map_reduce(), a single parallel map-reduce entry point over the Scheduler Protocol with pluggable backends:

• local / process: single-machine process pool (default, ~1-5ms overhead per task)

• thread: thread pool (shared state; the auto choice on free-threaded runtimes)

• dask: distributed cluster scheduler (optional dependency)

• auto: resolves per runtime

The backend is chosen by pyphi.config.parallel_backend (default "local"), or overridden per call by passing backend= to map_reduce().

Examples

>>> import pyphi
>>> with pyphi.config.override(parallel=True):
...     pyphi.config.infrastructure.parallel
True

Functions

false(*args, **kwargs)	Default short-circuit function that never short-circuits.
get(items[, shortcircuit_func, ...])	Iterate over results, optionally returning early if a value is found.
get_num_processes()	Return the number of worker processes to use, from config.
map_reduce(fn, items, *more_items[, ...])	Map fn over items (zipped with more_items) and reduce.
shortcircuit(items[, shortcircuit_func, ...])	Yield items, stopping after the first for which a predicate holds.

Submodules

backends	Backend implementations for parallel computation.
chunking	Pure index-partition helpers for parallel chunking.
sampling	Cost-sampling chunksize calculation for the Scheduler Protocol.
scheduler	Scheduler Protocol and policy types for the parallelization layer.