optim

Optimizer utilities: parameter-group construction.

generate_param_groups mirrors monai.optimizers.generate_param_groups: parallel sequences of matcher callables, match strategies, and per-group LR values, with a trailing "others" group for unmatched parameters. Scalar match_types or lr_values are broadcast to the length of layer_matches.

"filter" matchers take the parameter name (a str) and return a bool, so fnmatch.fnmatchcase and friends drop in directly:

import fnmatch
from functools import partial

layer_matches = [partial(fnmatch.fnmatchcase, pat="*block*")]

Functions:

generate_param_groups –

Split a network's parameters into optimizer parameter groups with per-group LRs.

generate_param_groups ¶

generate_param_groups(
    network: Module,
    layer_matches: Sequence[Callable[..., Any]],
    match_types: Union[MatchType, Sequence[MatchType]],
    lr_values: Union[float, Sequence[float]],
    include_others: bool = True,
) -> List[Dict[str, Any]]

Split a network's parameters into optimizer parameter groups with per-group LRs.

For each \((\text{layer\_matches}[i], \text{match\_types}[i], \text{lr\_values}[i])\):

"select": layer_matches[i](network) returns a Module; its parameters form group \(i\).
"filter": layer_matches[i](name) is called for every parameter name and returns a bool; matching parameters join group \(i\).

Scalar match_types / lr_values are broadcast to the length of layer_matches. Groups may overlap (a parameter can appear in multiple groups). When include_others=True, a final group collects parameters not matched by any group, with no LR override.

Parameters:

network (Module) –

Source network.
layer_matches (Sequence[Callable[..., Any]]) –

Matcher callables, one per group.
match_types (Union[MatchType, Sequence[MatchType]]) –

"select" or "filter" (per group, or a single value broadcast).
lr_values (Union[float, Sequence[float]]) –

Per-group LR (per group, or a single value broadcast).
include_others (bool, default: True ) –

Append a final group with the unmatched parameters.

Returns:

List[Dict[str, Any]] –

Optimizer parameter-group dicts: matched groups in order, then "others"
List[Dict[str, Any]] –

(when included).