pyphi.parallel.map_reduce

pyphi.parallel.map_reduce(fn, items, *more_items, reduce_func=<function _flatten>, reduce_kwargs=None, parallel=True, ordered=False, total=None, chunksize=None, sequential_threshold=1, shortcircuit_func=<function false>, shortcircuit_callback=None, shortcircuit_callback_args=None, progress=None, desc=None, map_kwargs=None, size_func=None, backend='auto')

Map fn over items (zipped with more_items) and reduce.

Runs in parallel through the scheduler selected by backend (or config.infrastructure.parallel_backend). With parallel=False it runs serially in-process. reduce_func defaults to flattening the per-item results into a list. size_func returns a relative per-item cost estimate used to pack cost-balanced chunks (parent-side, so it must be cheap); None packs equal-count chunks.

When shortcircuit_func fires, shortcircuit_callback (if given) is invoked with shortcircuit_callback_args, or — when those are not given — with the list of results collected so far, ending with the triggering result. The payload is the same on every backend and dispatch path.

Raises:

ValueError – If size_func is given together with ordered=True: cost balancing reorders items across chunks, so it cannot preserve input order.

Parameters:

• fn (Callable[[...], Any])

• items (Iterable[Any])

• more_items (Iterable[Any])

• reduce_func (Callable[[...], Any])

• reduce_kwargs (dict[str, Any] | None)

• parallel (bool)

• ordered (bool)

• total (int | None)

• chunksize (int | None)

• sequential_threshold (int)

• shortcircuit_func (Callable[[...], Any])

• shortcircuit_callback (Callable[[...], Any] | None)

• shortcircuit_callback_args (Any)

• progress (bool | None)

• desc (str | None)

• map_kwargs (dict[str, Any] | None)

• size_func (Callable[[...], float] | None)

• backend (str)

Return type:

Any