Skip to content

PointMLP

PointMLP classification and segmentation models.

First page of Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework

2202.07123 · February 2022

Classes:

  • PointMLPIntermediate –

    Input features and point cloud of one encoder block, recorded before it downsamples.

  • ResidualLinearBlock –

    A residual linear block consisting of two linear layers, normalization and activation.

  • PointMLPEncoderBlock –

    One encoder stage: optional FPS downsampling, then a geometric affine \(k\)-NN aggregation.

  • ResidualFeaturePropagation –

    Feature propagation block: \(k\)-NN interpolation to the skip resolution, concatenation with the

  • PointMLPEncoder –

    Stack of PointMLPEncoderBlock stages that progressively decimate the cloud with FPS.

  • PointMLPDecoder –

    Stack of ResidualFeaturePropagation blocks that walk the encoder intermediates back to full resolution.

  • PointMLPClassification –

    PointMLP classification model from

  • PointMLPSegmentation –

    PointMLP segmentation model from

PointMLPIntermediate

Bases: NamedTuple

Input features and point cloud of one encoder block, recorded before it downsamples.

ResidualLinearBlock 

ResidualLinearBlock(
    channels: int,
    expansion: float = 1.0,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = True,
)

Bases: Module

A residual linear block consisting of two linear layers, normalization and activation.

The default flow is:

x -> Lin1 -> Norm1 -> Act -> Lin2 -> Norm2 -> Act -> y
|                                          ^
+------------------------------------------+

Parameters:

  • channels (int) –

    The number of input and output channels.

  • expansion (float, default: 1.0 ) –

    The expansion factor for the hidden channels.

  • act (Union[str, Callable, None], default: 'relu' ) –

    The activation function to use. If None, no activation is applied.

  • act_kwargs (Optional[Dict[str, Any]], default: None ) –

    Keyword arguments for the activation function.

  • act_first (bool, default: False ) –

    Whether to apply the activation function before the normalization.

  • norm (Union[str, Callable, None], default: 'batch_norm' ) –

    The normalization function to use. If None, no normalization is applied.

  • norm_kwargs (Optional[Dict[str, Any]], default: None ) –

    Keyword arguments for the normalization function.

  • bias (bool, default: True ) –

    Whether to use a bias for the linear layers.

Examples:

>>> import torch
>>> from torch_pointcloud.models.pointmlp import ResidualLinearBlock
>>> block = ResidualLinearBlock(64, expansion=2, act="relu", norm="batch_norm", bias=False)
>>> x = torch.randn(32, 64)
>>> y = block(x)
>>> print(y.shape)
torch.Size([32, 64])

PointMLPEncoderBlock 

PointMLPEncoderBlock(
    in_channels: int,
    out_channels: int,
    k: int,
    spatial_dim: int = 3,
    num_pre_blocks: int = 2,
    num_pos_blocks: int = 2,
    normalize: Literal["center", "anchor"] = "center",
    std_mode: Literal["graph", "batch"] = "graph",
    res_expansion: float = 1.0,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = False,
    add_self_loops: bool = False,
    use_pos: bool = True,
    downsample: Optional[Module] = None,
)

Bases: Module

One encoder stage: optional FPS downsampling, then a geometric affine \(k\)-NN aggregation.

A pre-aggregation MLP of num_pre_blocks residual blocks lifts the grouped features to out_channels, and a post-aggregation MLP of num_pos_blocks residual blocks refines the max-pooled result.

ResidualFeaturePropagation 

ResidualFeaturePropagation(
    in_channels: int,
    out_channel: int,
    *,
    num_layers: int = 1,
    k: int,
    expansion: float = 1.0,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = True,
)

Bases: Module

Feature propagation block: \(k\)-NN interpolation to the skip resolution, concatenation with the skip features, then a LinearBlock followed by num_layers ResidualLinearBlock units.

PointMLPEncoder 

PointMLPEncoder(
    *,
    channels: Sequence[int],
    spatial_dim: int = 3,
    num_neighbors: Union[int, Sequence[int]],
    ratios: Union[float, Sequence[float]],
    num_pre_blocks: Union[int, Sequence[int]] = 2,
    num_pos_blocks: Union[int, Sequence[int]] = 2,
    normalize: Literal["center", "anchor"] = "center",
    std_mode: Literal["graph", "batch"] = "graph",
    res_expansion: float = 1.0,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = True,
    add_self_loops: bool = False,
    use_pos: bool = True,
    fps_random_start: Optional[bool] = None,
    aggr: str = "max",
)

Bases: Module

Stack of PointMLPEncoderBlock stages that progressively decimate the cloud with FPS.

A stage with a ratio of 0 keeps every point and only transforms features. When return_intermediates=True is passed to forward, the pre-downsampling features of every stage are returned in coarse-to-fine order, ready to be consumed as decoder skips.

Methods:

  • configure_block –

    Build the PointMLPEncoderBlock for stage index, with an FPS sampler when its ratio is non-zero.

configure_block 

configure_block(index: int) -> Module

Build the PointMLPEncoderBlock for stage index, with an FPS sampler when its ratio is non-zero.

PointMLPDecoder 

PointMLPDecoder(
    channels: Sequence[int],
    skip_channels: Sequence[int],
    depths: Sequence[int],
    *,
    spatial_dim: int = 3,
    dropout: float = 0.0,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = True,
)

Bases: Module

Stack of ResidualFeaturePropagation blocks that walk the encoder intermediates back to full resolution.

Methods:

  • configure_block –

    Build the ResidualFeaturePropagation block for stage index.

configure_block 

configure_block(index: int) -> Module

Build the ResidualFeaturePropagation block for stage index.

PointMLPClassification 

PointMLPClassification(
    in_channels: int,
    num_classes: int,
    *,
    spatial_dim: int = 3,
    encoder_channels: Sequence[int],
    num_neighbors: Union[int, Sequence[int]],
    ratios: Union[float, Sequence[float]],
    num_pre_blocks: Union[int, Sequence[int]] = 2,
    num_pos_blocks: Union[int, Sequence[int]] = 2,
    normalize: Literal["center", "anchor"] = "center",
    std_mode: Literal["graph", "batch"] = "graph",
    res_expansion: float = 1.0,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = True,
    add_self_loops: bool = False,
    use_pos: bool = True,
    dropout: float = 0.0,
    head_channels: Optional[Sequence[int]] = None,
    head_dropout: float = 0.0,
    global_pool: PoolLike = "max",
)

Bases: ClassificationModel

PointMLP classification model from Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework by Xu Ma, Can Qin, Haoxuan You, Haoxi Ran, Yun Fu.

A pure residual MLP network: each encoder stage samples centroids with farthest point sampling, normalizes each \(k\)-NN neighborhood with a geometric affine module, and applies residual MLP blocks before and after the neighborhood aggregation. Point features are pooled globally after the encoder for classification.

Methods:

  • configure_stem –

    Build the linear stem lifting the input features to the first encoder channel.

  • configure_encoder –

    Build the PointMLPEncoder backbone.

Attributes:

  • num_features (int) –

    Feature dimension \(C\) of the encoder output.

num_features property 

num_features: int

Feature dimension \(C\) of the encoder output.

configure_stem 

configure_stem() -> Module

Build the linear stem lifting the input features to the first encoder channel.

configure_encoder 

configure_encoder() -> PointMLPEncoder

Build the PointMLPEncoder backbone.

PointMLPSegmentation 

PointMLPSegmentation(
    in_channels: int,
    num_classes: int,
    *,
    spatial_dim: int = 3,
    encoder_channels: Sequence[int],
    num_neighbors: Union[int, Sequence[int]],
    ratios: Union[float, Sequence[float]],
    num_pre_blocks: Union[int, Sequence[int]] = 2,
    num_pos_blocks: Union[int, Sequence[int]] = 2,
    decoder_channels: Sequence[int],
    decoder_blocks: Sequence[Module],
    normalize: Literal["center", "anchor"] = "center",
    std_mode: Literal["graph", "batch"] = "graph",
    res_expansion: float = 1.0,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = True,
    add_self_loops: bool = False,
    use_pos: bool = True,
    dropout: float = 0.0,
)

Bases: SegmentationModel

PointMLP segmentation model from Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework by Xu Ma, Can Qin, Haoxuan You, Haoxi Ran, Yun Fu.

The PointMLP encoder is followed by a decoder of residual feature-propagation blocks with skip connections and a per-point linear head.

Methods:

  • configure_stem –

    Build the linear stem lifting the input features to the first encoder channel.

  • configure_encoder –

    Build the PointMLPEncoder backbone.

  • configure_decoder –

    Build the PointMLPDecoder, mirroring the encoder channels in reverse.

Attributes:

  • num_features (int) –

    Feature dimension \(C\) of the decoder output.

num_features property 

num_features: int

Feature dimension \(C\) of the decoder output.

configure_stem 

configure_stem() -> Module

Build the linear stem lifting the input features to the first encoder channel.

configure_encoder 

configure_encoder() -> PointMLPEncoder

Build the PointMLPEncoder backbone.

configure_decoder 

configure_decoder() -> PointMLPDecoder

Build the PointMLPDecoder, mirroring the encoder channels in reverse.