kpconv

KPConv classification and segmentation models.

First page of KPConv: Flexible and Deformable Convolution for Point Clouds

1904.08889 · April 2019

Classes:

KPFCNNEncoderBlock –

One encoder stage: an optional grid subsampling followed by depth KPResidualBlock blocks.
KPFCNNEncoder –

KP-FCNN encoder: KPFCNNEncoderBlock stages from finest to coarsest, every stage but the first preceded by a
KPFCNNClassification –

KPConv Network for classification tasks as described in the paper
KPFCNNSegmentation –

KPConv Network for segmentation tasks as described in the paper

KPFCNNEncoderBlock [source] ¶

KPFCNNEncoderBlock(
    *,
    depth: int,
    radius: float,
    pool_radius: Optional[float] = None,
    max_num_neighbors: int,
    spatial_dim: int,
    in_channels: int,
    out_channels: int,
    kernel_size: int,
    kp_radius: Union[float, Sequence[float]],
    kp_sigma: Union[float, Sequence[float]],
    kp_influence: str = "linear",
    fixed_position: Literal[
        "none", "center", "vertical"
    ] = "center",
    aggregation_mode: str = "sum",
    deformable: Union[bool, Sequence[bool]] = False,
    modulated: Union[bool, Sequence[bool]] = False,
    bias: bool = False,
    act: Union[str, Callable, None] = "leaky_relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    downsample: Optional[VoxelGridPool] = None,
)

Bases: Module

One encoder stage: an optional grid subsampling followed by depth KPResidualBlock blocks.

Neighborhoods are recomputed once at the stage entry, using pool_radius for the strided first block and radius for the remaining ones.

Parameters:

downsample (Optional[VoxelGridPool], default: None ) –

Grid pooling applied before the first block. Makes that block strided.

KPFCNNEncoder [source] ¶

KPFCNNEncoder(
    in_channels: int,
    *,
    depths: Sequence[int],
    grid_sizes: Sequence[float],
    radii: Sequence[float],
    channels: Sequence[int],
    max_num_neighbors: Sequence[int],
    kernel_size: int,
    kp_sigma: Union[float, Sequence[float]],
    kp_radius: Union[float, Sequence[float]],
    kp_influence: str = "linear",
    fixed_position: Literal[
        "none", "center", "vertical"
    ] = "center",
    aggregation_mode: str = "sum",
    deformable: Union[bool, Sequence] = False,
    modulated: Union[bool, Sequence] = False,
    act: Union[str, Callable, None] = "leaky_relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = False,
    spatial_dim: int = 3,
)

Bases: Module

KP-FCNN encoder: KPFCNNEncoderBlock stages from finest to coarsest, every stage but the first preceded by a VoxelGridPool subsampling.

Parameters:

in_channels (int) –

Number of channels entering the first stage.
depths (Sequence[int]) –

Number of residual blocks in each stage.
grid_sizes (Sequence[float]) –

Grid size of each downsampling step, one per stage transition.
radii (Sequence[float]) –

Neighborhood search radius of each stage.
channels (Sequence[int]) –

Output channels of each stage.
max_num_neighbors (Sequence[int]) –

Maximum number of neighbors queried at each stage.
kernel_size (int) –

Number of kernel points of every KPConv.
kp_sigma (Union[float, Sequence[float]]) –

Kernel extent of each stage.
kp_radius (Union[float, Sequence[float]]) –

Kernel point radius of each stage.
kp_influence (str, default: 'linear' ) –

Influence function: "constant", "linear", or "gaussian".
fixed_position (Literal['none', 'center', 'vertical'], default: 'center' ) –

Which kernel point is pinned: "none", "center", or "vertical".
aggregation_mode (str, default: 'sum' ) –

"sum" over all kernel points, or "closest" to keep only the nearest one.
deformable (Union[bool, Sequence], default: False ) –

Whether each stage uses deformable kernels.
modulated (Union[bool, Sequence], default: False ) –

Whether each stage uses modulated deformable kernels.
act (Union[str, Callable, None], default: 'leaky_relu' ) –

Activation function.
act_kwargs (Optional[Dict[str, Any]], default: None ) –

Optional keyword arguments for the activation factory.
act_first (bool, default: False ) –

Whether the activation comes before the normalization in the MLPs.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization layer.
norm_kwargs (Optional[Dict[str, Any]], default: None ) –

Optional keyword arguments for the normalization factory.
bias (bool, default: False ) –

Whether the convolutions and MLPs use a bias.
spatial_dim (int, default: 3 ) –

Spatial dimension of the input point cloud.

Inputs

x: Point features of shape \((N, \text{in\_channels})\). pos: Point coordinates of shape \((N, D)\). batch: Batch indices of shape \((N,)\).

Outputs

Features, coordinates and batch indices at the coarsest stage. With return_intermediates=True, also one skip per downsampled stage: the features, coordinates, batch indices and pooling inverse entering that stage.

KPFCNNClassification [source] ¶

KPFCNNClassification(
    in_channels: int,
    num_classes: int,
    *,
    spatial_dim: int = 3,
    stem_channels: Optional[int] = None,
    stem_type: Literal["linear", "kpconv"] = "kpconv",
    encoder_depths: Sequence[int],
    encoder_channels: Sequence[int],
    encoder_num_neighbors: Sequence[int],
    grid_sizes: Sequence[float],
    radii: Sequence[float],
    kernel_size: int,
    kp_radius: Union[float, Sequence[float]],
    kp_sigma: Union[float, Sequence[float]],
    kp_influence: str = "linear",
    fixed_position: Literal[
        "none", "center", "vertical"
    ] = "center",
    aggregation_mode: str = "sum",
    deformable: Union[bool, Sequence] = False,
    modulated: Union[bool, Sequence] = False,
    act: Union[str, Callable, None] = "leaky_relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = False,
    dropout: float = 0.0,
    global_pool: PoolLike = "max",
)

Bases: ClassificationModel

KPConv Network for classification tasks as described in the paper KPConv: Flexible and Efficient Convolution for Point Clouds by Hugues Thomas, Charles R. Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, Leonidas J. Guibas.

KPConv introduces a novel point convolution operator that uses kernel points to define the spatial extent and weights of the convolution. The kernel points are arranged in space to define the convolution pattern, with weights determined by their spatial correlation with input points. This allows for flexible and efficient convolution on irregular point clouds while maintaining permutation invariance and translation invariance. The network uses a hierarchical architecture with strided convolutions for spatial pooling and feature aggregation.

Note

The implementation is based on the original paper and the authors' code KPConv-PyTorch.

Important

This implementation was completely rewritten to be compatible with torch-geometric library.

Parameters:

in_channels (int) –

Number of input channels.
num_classes (int) –

Number of output classes.
spatial_dim (int, default: 3 ) –

Spatial dimension of the input point cloud.
stem_channels (Optional[int], default: None ) –

Number of channels in the stem layer.
stem_type (Literal['linear', 'kpconv'], default: 'kpconv' ) –

Type of stem layer to use.
encoder_depths (Sequence[int]) –

List of depths for each encoder block, i.e. corresponds to the number of residual blocks at each level.
encoder_channels (Sequence[int]) –

List of channels for each encoder block.
encoder_num_neighbors (Sequence[int]) –

List of maximum number of neighbors for each encoder block.
grid_sizes (Sequence[float]) –

List of grid sizes for each downsampling block.
radii (Sequence[float]) –

Search radius for each downsampling block.
kernel_size (int) –

Size of the kernel for each KPConv block.
kp_radius (Union[float, Sequence[float]]) –

List of kernel radius for KPConv blocks, at each level.
kp_sigma (Union[float, Sequence[float]]) –

List of kernel extent for KPConv blocks, at each level.
kp_influence (str, default: 'linear' ) –

Influence function to use for KPConv blocks. Options are "constant", "linear", "gaussian".
fixed_position (Literal['none', 'center', 'vertical'], default: 'center' ) –

Whether to fix the position of the kernel points in KPConv blocks. Options are "none", "center", "vertical".
aggregation_mode (str, default: 'sum' ) –

Aggregation mode to use for the KPConv blocks. Options are "sum", "mean", "max".
deformable (Union[bool, Sequence], default: False ) –

Whether to use a deformable kernel for the KPConv blocks.
modulated (Union[bool, Sequence], default: False ) –

Whether to use a modulated kernel in KPConv operation.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization to use for the KPConv blocks.
act (Union[str, Callable, None], default: 'leaky_relu' ) –

Activation function to use for the KPConv blocks.
bias (bool, default: False ) –

Whether to use a bias for the KPConv blocks.
dropout (float, default: 0.0 ) –

Dropout rate before the classification head.
global_pool (PoolLike, default: 'max' ) –

Global pooling method to use before the classification head. Options are "max", "mean".

Methods:

configure_stem –

Build the stem lifting the input features to stem_channels, or None when stem_channels is unset.
configure_encoder –

Build the KPFCNNEncoder backbone.
reset_classifier –

Resets the classification head with new parameters.
forward_features –

Forward pass of the encoder, returning pre-pooling features.
forward_head –

Forward pass of the classification head from pre-pooling features.
forward –

Forward pass of the classification model.

Attributes:

num_features (int) –

Feature dimension \(C\) of the encoder output.

num_features `property` ¶

num_features: int

Feature dimension \(C\) of the encoder output.

configure_stem [source] ¶

configure_stem() -> Optional[Module]

Build the stem lifting the input features to stem_channels, or None when stem_channels is unset.

configure_encoder [source] ¶

configure_encoder() -> KPFCNNEncoder

Build the KPFCNNEncoder backbone.

reset_classifier [source] ¶

reset_classifier(
    num_classes: int,
    global_pool: Optional[PoolLike] = None,
    **kwargs: Any,
) -> None

Resets the classification head with new parameters.

Note

To set an empty classification head, use num_classes=0.

Parameters:

num_classes (int) –

Number of output classes.
global_pool (Optional[PoolLike], default: None ) –

Pooling method to aggregate point features ("max" or "mean"). If None, keeps the current pooling.
**kwargs (Any, default: {} ) –

Additional keyword arguments to pass to the classification head.

forward_features [source] ¶

forward_features(
    x: OptTensor,
    pos: Tensor,
    batch: Tensor,
    return_intermediates: Literal[True],
) -> Tuple[
    Tensor, Tensor, Tensor, List[PooledFeaturesDict]
]

forward_features(
    x: OptTensor,
    pos: Tensor,
    batch: Tensor,
    return_intermediates: Literal[False] = False,
) -> Tuple[Tensor, Tensor, Tensor]

forward_features(
    x: OptTensor,
    pos: Tensor,
    batch: Tensor,
    return_intermediates: bool = False,
) -> Any

Forward pass of the encoder, returning pre-pooling features.

Parameters:

x (OptTensor) –

Point features of shape \((N, C)\). If None, pos is used as features, which requires in_channels to match the dimension of pos.
pos (Tensor) –

Point coordinates of shape \((N, D)\).
batch (Tensor) –

Batch indices for each point of shape \((N,)\).
return_intermediates (bool, default: False ) –

Whether to also return the per-stage intermediates.

Returns:

Any –

A tuple (x, pos, batch) at the coarsest level, where x has shape \((N', \text{encoder\_channels}[-1])\) and \(N'\) is the number of downsampled points. If return_intermediates=True, the per-stage intermediates are appended to the tuple.

forward_head [source] ¶

forward_head(
    x: Tensor, batch: Tensor, pre_logits: bool = False
) -> Tensor

Forward pass of the classification head from pre-pooling features.

Parameters:

x (Tensor) –

Pre-pooling features of shape \((N', \text{encoder\_channels}[-1])\) where \(N'\) is the number of downsampled points.
batch (Tensor) –

Batch indices for each downsampled point of shape \((N',)\).
pre_logits (bool, default: False ) –

Whether to return pre-logits. Defaults to False.

Returns:

Tensor –

Classification logits of shape \((B, \text{num\_classes})\).

forward [source] ¶

forward(x: OptTensor, pos: Tensor, batch: Tensor) -> Tensor

Forward pass of the classification model.

Parameters:

x (OptTensor) –

Point features of shape \((N, C)\). If None, pos is used as features, which requires in_channels to match the dimension of pos.
pos (Tensor) –

Point coordinates of shape \((N, D)\).
batch (Tensor) –

Batch indices for each point of shape \((N,)\).

Returns:

Tensor –

Classification logits of shape \((B, \text{num\_classes})\).

KPFCNNSegmentation [source] ¶

KPFCNNSegmentation(
    in_channels: int,
    num_classes: int,
    *,
    spatial_dim: int = 3,
    stem_channels: Optional[int] = None,
    stem_type: Literal["linear", "kpconv"] = "kpconv",
    encoder_depths: Sequence[int],
    encoder_channels: Sequence[int],
    encoder_num_neighbors: Sequence[int],
    fp_channels: Sequence[Sequence[int]],
    grid_sizes: Sequence[float],
    radii: Sequence[float],
    kernel_size: int,
    kp_radius: Union[float, Sequence[float]],
    kp_sigma: Union[float, Sequence[float]],
    kp_influence: str = "linear",
    fixed_position: Literal[
        "none", "center", "vertical"
    ] = "center",
    aggregation_mode: str = "sum",
    deformable: Union[bool, Sequence] = False,
    modulated: Union[bool, Sequence] = False,
    act: Union[str, Callable, None] = "leaky_relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = False,
    dropout: float = 0.0,
    head_channels: Optional[Sequence[int]] = None,
)

Bases: SemanticSegmentationModel

KPConv Network for segmentation tasks as described in the paper KPConv: Flexible and Efficient Convolution for Point Clouds by Hugues Thomas, Charles R. Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, Leonidas J. Guibas.

KPConv introduces a novel point convolution operator that uses kernel points to define the spatial extent and weights of the convolution. The kernel points are arranged in space to define the convolution pattern, with weights determined by their spatial correlation with input points. This allows for flexible and efficient convolution on irregular point clouds while maintaining permutation invariance and translation invariance. The network uses a hierarchical architecture with strided convolutions for spatial pooling and feature aggregation.

Note

The implementation is based on the original paper and the authors' code KPConv-PyTorch.

Important

This implementation was completely rewritten to be compatible with torch-geometric library.

Parameters:

in_channels (int) –

Number of input channels.
num_classes (int) –

Number of output classes.
spatial_dim (int, default: 3 ) –

Spatial dimension of the input point cloud.
stem_channels (Optional[int], default: None ) –

Number of channels in the stem layer.
stem_type (Literal['linear', 'kpconv'], default: 'kpconv' ) –

Type of stem layer to use.
encoder_depths (Sequence[int]) –

List of depths for each encoder block, i.e. corresponds to the number of residual blocks at each level.
encoder_channels (Sequence[int]) –

List of channels for each encoder block.
encoder_num_neighbors (Sequence[int]) –

List of maximum number of neighbors for each encoder block.
fp_channels (Sequence[Sequence[int]]) –

List of channels for each feature propagation block.
grid_sizes (Sequence[float]) –

List of grid sizes for each downsampling block.
radii (Sequence[float]) –

Search radius for each downsampling block.
kernel_size (int) –

Size of the kernel for each KPConv block.
kp_radius (Union[float, Sequence[float]]) –

List of kernel radius for KPConv blocks, at each level.
kp_sigma (Union[float, Sequence[float]]) –

List of kernel extent for KPConv blocks, at each level.
kp_influence (str, default: 'linear' ) –

Influence function to use for KPConv blocks. Options are "constant", "linear", "gaussian".
fixed_position (Literal['none', 'center', 'vertical'], default: 'center' ) –

Whether to fix the position of the kernel points in KPConv blocks. Options are "none", "center", "vertical".
aggregation_mode (str, default: 'sum' ) –

Aggregation mode to use for the KPConv blocks. Options are "sum", "mean", "max".
deformable (Union[bool, Sequence], default: False ) –

Whether to use a deformable kernel for the KPConv blocks.
modulated (Union[bool, Sequence], default: False ) –

Whether to use a modulated kernel in KPConv operation.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization to use for the KPConv blocks.
act (Union[str, Callable, None], default: 'leaky_relu' ) –

Activation function to use for the KPConv blocks.
bias (bool, default: False ) –

Whether to use a bias for the KPConv blocks.
dropout (float, default: 0.0 ) –

Dropout rate before the classification head.

Methods:

configure_stem –

Build the stem lifting the input features to stem_channels, or None when stem_channels is unset.
configure_encoder –

Build the KPFCNNEncoder backbone.
configure_decoder –

Build the PointNet2Decoder upsampling the coarsest features back through the encoder skips.

Attributes:

num_features (int) –

Feature dimension \(C\) of the decoder output.

num_features `property` ¶

num_features: int

Feature dimension \(C\) of the decoder output.

configure_stem [source] ¶

configure_stem() -> Optional[Module]

Build the stem lifting the input features to stem_channels, or None when stem_channels is unset.

configure_encoder [source] ¶

configure_encoder() -> KPFCNNEncoder

Build the KPFCNNEncoder backbone.

configure_decoder [source] ¶

configure_decoder() -> PointNet2Decoder

Build the PointNet2Decoder upsampling the coarsest features back through the encoder skips.

kpconv

KPFCNNEncoderBlock [source] ¶

KPFCNNEncoder [source] ¶

KPFCNNClassification [source] ¶

num_features property ¶

configure_stem [source] ¶

configure_encoder [source] ¶

reset_classifier [source] ¶

forward_features [source] ¶

forward_head [source] ¶

forward [source] ¶

KPFCNNSegmentation [source] ¶

num_features property ¶

configure_stem [source] ¶

configure_encoder [source] ¶

configure_decoder [source] ¶

num_features `property` ¶

num_features `property` ¶