misc

Miscellaneous helpers for parallel mapping and nested attribute access.

Functions:

parallel_map –

Apply func to every item in iterable, optionally in parallel.
deep_getattr –

Resolve a dotted path through nested Mapping keys and object attributes, else default.

parallel_map ¶

parallel_map(
    func: Callable[..., T],
    iterable: Iterable[Any],
    num_workers: Optional[int] = None,
    *,
    total: Optional[int] = None,
    desc: Optional[str] = None,
    show_progress: bool = False,
) -> List[T]

Apply func to every item in iterable, optionally in parallel.

When num_workers is None or 0, items are processed sequentially in the current process. Otherwise joblib spawns num_workers workers and results are streamed back as each task completes.

Parameters:

func (Callable[..., T]) –

Callable invoked on each item.
iterable (Iterable[Any]) –

Items to map over.
num_workers (Optional[int], default: None ) –

Number of worker processes. None or 0 disables parallelism.
total (Optional[int], default: None ) –

Item count for the progress bar. Inferred from len(iterable) when not provided.
desc (Optional[str], default: None ) –

Progress bar description.
show_progress (bool, default: False ) –

Whether to display a tqdm progress bar. The bar advances on task completion (not dispatch), so it stays accurate under parallel execution.

deep_getattr ¶

deep_getattr(
    obj: Any, path: str, default: Any = None
) -> Any

Resolve a dotted path through nested Mapping keys and object attributes, else default.

At each step a Mapping (e.g. a batch dict) is indexed by key and any other object is read by attribute, so one call resolves both deep_getattr(batch, "octree.depth") (dict then attribute) and deep_getattr(module, "criterion.ignore_index") (attribute then attribute). A missing key or attribute (or a None reached mid-path) returns default rather than raising.

Parameters:

obj (Any) –

Root object: a Mapping, an arbitrary object, or any nesting of the two.
path (str) –

Dotted access path, e.g. "criterion.ignore_index".
default (Any, default: None ) –

Value returned when a segment cannot be resolved.

Returns:

Any –

The resolved value, or default if the path is not fully traversable.