compute.parallel

Utilities for parallel computation.

pyphi.compute.parallel.get_num_processes()

Return the number of processes to use in parallel.

pyphi.compute.parallel.init(*args, **kwargs)

Initialize Ray if not already initialized.

pyphi.compute.parallel.false(*args, **kwargs)
pyphi.compute.parallel.shortcircuit(items, shortcircuit_func=<function false>, shortcircuit_callback=None, shortcircuit_callback_args=None)

Yield from an iterable, stopping early if a certain value is found.

pyphi.compute.parallel.as_completed(object_refs: List[ObjectRef], num_returns: int = 1)

Yield remote results in order of completion.

pyphi.compute.parallel.get(items, remote=False, ordered=False, shortcircuit_func=<function false>, shortcircuit_callback=None, shortcircuit_callback_args=None)

Get (potentially) remote results.

Optionally return early if a particular value is found.

NOTE: If ordered is True, all items will be computed regardless of shortcircuiting, though the shortcircuiting logic will still be applied.

pyphi.compute.parallel.backpressure(func, *argslist, inflight_limit=1000, **kwargs)
pyphi.compute.parallel.progress_hook(progress_bar)
class pyphi.compute.parallel.MapReduce(map_func: ~typing.Callable, iterable: ~typing.Iterable, *iterables, reduce_func: ~typing.Callable | None = None, reduce_kwargs: dict | None = None, parallel: bool = True, ordered: bool = False, total: int | None = None, chunksize: int | None = None, sequential_threshold: int = 1, max_depth: int | None = None, max_size: int | None = None, max_leaves: int | None = None, branch_factor: int = 2, shortcircuit_func: ~typing.Callable = <function false>, shortcircuit_callback: ~typing.Callable | None = None, shortcircuit_callback_args: ~typing.Any | None = None, inflight_limit: int = 1000, progress: bool | None = None, desc: str | None = None, map_kwargs: dict | None = None)

Map-reduce engine.

Parallelized using Ray remote functions.

Specifying tree size: order of precedence:

chunksize, max_depth, max_size, max_leaves

run()

Perform the computation.