KPConv

KPConv classification and segmentation models.

First page of KPConv: Flexible and Deformable Convolution for Point Clouds

1904.08889 · April 2019

Classes:

KPConv –

Kernel point convolution over a neighborhood graph.
KPConvBlock –

Kernel point convolution followed by normalization and activation.
KPResidualBlock –

Bottleneck residual block: a channel-reducing MLP, a KPConvBlock, and a channel-restoring MLP.
KPFCNNGridPool –

Voxel grid subsampling: points falling in the same cell are reduced to a single point.
KPFCNNEncoderBlock –

One encoder stage: an optional grid subsampling followed by depth KPResidualBlock blocks.
KPFCNNEncoder –

KP-FCNN encoder: KPFCNNEncoderBlock stages from finest to coarsest, every stage but the first preceded by a
KPFCNNClassification –

KPConv Network for classification tasks as described in the paper
KPFCNNSegmentation –

KPConv Network for segmentation tasks as described in the paper

Functions:

create_kernel_points –

Builds the kernel point positions of a KPConv kernel, randomly rotated and jittered.

KPConv ¶

KPConv(
    spatial_dim: int,
    in_channels: int,
    out_channels: int,
    kernel_size: int,
    kp_radius: float,
    kp_sigma: float,
    kp_influence: str = "linear",
    fixed_position: Literal[
        "none", "center", "vertical"
    ] = "center",
    aggregation_mode: str = "sum",
    deformable: bool = False,
    modulated: bool = False,
    bias: bool = False,
    track_running_stats: bool = True,
)

Bases: Module

Kernel point convolution over a neighborhood graph.

Each of the \(K\) kernel points carries its own weight matrix, and a neighbor contributes to a kernel point with a weight given by their distance through the influence function. When deformable is set, a nested KPConv predicts a per-point offset for every kernel point.

Parameters:

spatial_dim (int) –

Spatial dimension of the input point cloud.
in_channels (int) –

Number of input channels.
out_channels (int) –

Number of output channels.
kernel_size (int) –

Number of kernel points \(K\).
kp_radius (float) –

Radius of the sphere the kernel points are placed on.
kp_sigma (float) –

Kernel extent, the distance over which a kernel point still influences a neighbor.
kp_influence (str, default: 'linear' ) –

Influence function: "constant", "linear", or "gaussian".
fixed_position (Literal['none', 'center', 'vertical'], default: 'center' ) –

Which kernel point is pinned: "none", "center", or "vertical".
aggregation_mode (str, default: 'sum' ) –

"sum" over all kernel points, or "closest" to keep only the nearest one.
deformable (bool, default: False ) –

Whether to predict per-point kernel offsets.
modulated (bool, default: False ) –

Whether the deformable branch also predicts a per-kernel-point modulation.
bias (bool, default: False ) –

Whether to add a bias to the output.
track_running_stats (bool, default: True ) –

Whether to keep the deformable activations of the last forward pass, for regularization.

Methods:

configure_offsets –

Builds the rigid KPConv and the bias predicting the deformable offsets (and modulations).
configure_kernel –

Builds the fixed kernel point positions registered as the kernel buffer.

Attributes:

deformable (bool) –

Whether the kernel points are shifted by predicted per-point offsets.

deformable `property` ¶

deformable: bool

Whether the kernel points are shifted by predicted per-point offsets.

configure_offsets ¶

configure_offsets() -> Tuple[Module, Parameter]

Builds the rigid KPConv and the bias predicting the deformable offsets (and modulations).

configure_kernel ¶

configure_kernel() -> Tensor

Builds the fixed kernel point positions registered as the kernel buffer.

KPConvBlock ¶

KPConvBlock(
    spatial_dim: int,
    in_channels: int,
    out_channels: int,
    kernel_size: int,
    kp_radius: float,
    kp_sigma: float,
    kp_influence: str = "linear",
    fixed_position: Literal[
        "none", "center", "vertical"
    ] = "center",
    aggregation_mode: str = "sum",
    deformable: bool = False,
    modulated: bool = False,
    act: Union[str, Callable, None] = "leaky_relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = False,
)

Bases: Module

Kernel point convolution followed by normalization and activation.

KPResidualBlock ¶

KPResidualBlock(
    spatial_dim: int,
    in_channels: int,
    out_channels: int,
    kernel_size: int,
    kp_radius: float,
    kp_sigma: float,
    kp_influence: str = "linear",
    fixed_position: Literal[
        "none", "center", "vertical"
    ] = "center",
    aggregation_mode: str = "sum",
    deformable: bool = False,
    modulated: bool = False,
    strided: bool = False,
    act: Union[str, Callable, None] = "leaky_relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = False,
)

Bases: Module

Bottleneck residual block: a channel-reducing MLP, a KPConvBlock, and a channel-restoring MLP.

Set strided=True when the block maps a support cloud onto a coarser query cloud, so that the skip connection is max-pooled over the same neighborhoods.

KPFCNNGridPool ¶

KPFCNNGridPool(grid_size: float, reduce: str = 'max')

Bases: Module

Voxel grid subsampling: points falling in the same cell are reduced to a single point.

Positions are averaged within a cell while features use the reduce operation.

KPFCNNEncoderBlock ¶

KPFCNNEncoderBlock(
    *,
    depth: int,
    radius: float,
    pool_radius: Optional[float] = None,
    max_num_neighbors: int,
    spatial_dim: int,
    in_channels: int,
    out_channels: int,
    kernel_size: int,
    kp_radius: Union[float, Sequence[float]],
    kp_sigma: Union[float, Sequence[float]],
    kp_influence: str = "linear",
    fixed_position: Literal[
        "none", "center", "vertical"
    ] = "center",
    aggregation_mode: str = "sum",
    deformable: Union[bool, Sequence[bool]] = False,
    modulated: Union[bool, Sequence[bool]] = False,
    bias: bool = False,
    act: Union[str, Callable, None] = "leaky_relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    downsample: Optional[KPFCNNGridPool] = None,
)

Bases: Module

One encoder stage: an optional grid subsampling followed by depth KPResidualBlock blocks.

Neighborhoods are recomputed once at the stage entry, using pool_radius for the strided first block and radius for the remaining ones.

Parameters:

downsample (Optional[KPFCNNGridPool], default: None ) –

Grid pooling applied before the first block. Makes that block strided.

KPFCNNEncoder ¶

KPFCNNEncoder(
    in_channels: int,
    *,
    depths: Sequence[int],
    grid_sizes: Sequence[float],
    radii: Sequence[float],
    channels: Sequence[int],
    max_num_neighbors: Sequence[int],
    kernel_size: int,
    kp_sigma: Union[float, Sequence[float]],
    kp_radius: Union[float, Sequence[float]],
    kp_influence: str = "linear",
    fixed_position: Literal[
        "none", "center", "vertical"
    ] = "center",
    aggregation_mode: str = "sum",
    deformable: Union[bool, Sequence] = False,
    modulated: Union[bool, Sequence] = False,
    act: Union[str, Callable, None] = "leaky_relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = False,
    spatial_dim: int = 3,
)

Bases: Module

KP-FCNN encoder: KPFCNNEncoderBlock stages from finest to coarsest, every stage but the first preceded by a KPFCNNGridPool subsampling.

Parameters:

in_channels (int) –

Number of channels entering the first stage.
depths (Sequence[int]) –

Number of residual blocks in each stage.
grid_sizes (Sequence[float]) –

Grid size of each downsampling step, one per stage transition.
radii (Sequence[float]) –

Neighborhood search radius of each stage.
channels (Sequence[int]) –

Output channels of each stage.
max_num_neighbors (Sequence[int]) –

Maximum number of neighbors queried at each stage.
kernel_size (int) –

Number of kernel points of every KPConv.
kp_sigma (Union[float, Sequence[float]]) –

Kernel extent of each stage.
kp_radius (Union[float, Sequence[float]]) –

Kernel point radius of each stage.
kp_influence (str, default: 'linear' ) –

Influence function: "constant", "linear", or "gaussian".
fixed_position (Literal['none', 'center', 'vertical'], default: 'center' ) –

Which kernel point is pinned: "none", "center", or "vertical".
aggregation_mode (str, default: 'sum' ) –

"sum" over all kernel points, or "closest" to keep only the nearest one.
deformable (Union[bool, Sequence], default: False ) –

Whether each stage uses deformable kernels.
modulated (Union[bool, Sequence], default: False ) –

Whether each stage uses modulated deformable kernels.
act (Union[str, Callable, None], default: 'leaky_relu' ) –

Activation function.
act_kwargs (Optional[Dict[str, Any]], default: None ) –

Optional keyword arguments for the activation factory.
act_first (bool, default: False ) –

Whether the activation comes before the normalization in the MLPs.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization layer.
norm_kwargs (Optional[Dict[str, Any]], default: None ) –

Optional keyword arguments for the normalization factory.
bias (bool, default: False ) –

Whether the convolutions and MLPs use a bias.
spatial_dim (int, default: 3 ) –

Spatial dimension of the input point cloud.

Inputs

x: Point features of shape \((N, \text{in\_channels})\). pos: Point coordinates of shape \((N, D)\). batch: Batch indices of shape \((N,)\).

Outputs

Features, coordinates and batch indices at the coarsest stage. With return_intermediates=True, also one skip per downsampled stage: the features, coordinates, batch indices and pooling inverse entering that stage.

KPFCNNClassification ¶

KPFCNNClassification(
    in_channels: int,
    num_classes: int,
    *,
    spatial_dim: int = 3,
    stem_channels: Optional[int] = None,
    stem_type: Literal["linear", "kpconv"] = "kpconv",
    encoder_depths: Sequence[int],
    encoder_channels: Sequence[int],
    encoder_num_neighbors: Sequence[int],
    grid_sizes: Sequence[float],
    radii: Sequence[float],
    kernel_size: int,
    kp_radius: Union[float, Sequence[float]],
    kp_sigma: Union[float, Sequence[float]],
    kp_influence: str = "linear",
    fixed_position: Literal[
        "none", "center", "vertical"
    ] = "center",
    aggregation_mode: str = "sum",
    deformable: Union[bool, Sequence] = False,
    modulated: Union[bool, Sequence] = False,
    act: Union[str, Callable, None] = "leaky_relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = False,
    dropout: float = 0.0,
    global_pool: PoolLike = "max",
)

Bases: ClassificationModel

KPConv Network for classification tasks as described in the paper KPConv: Flexible and Efficient Convolution for Point Clouds by Hugues Thomas, Charles R. Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, Leonidas J. Guibas.

KPConv introduces a novel point convolution operator that uses kernel points to define the spatial extent and weights of the convolution. The kernel points are arranged in space to define the convolution pattern, with weights determined by their spatial correlation with input points. This allows for flexible and efficient convolution on irregular point clouds while maintaining permutation invariance and translation invariance. The network uses a hierarchical architecture with strided convolutions for spatial pooling and feature aggregation.

Note

The implementation is based on the original paper and the authors' code KPConv-PyTorch.

Important

This implementation was completely rewritten to be compatible with torch-geometric library.

Parameters:

in_channels (int) –

Number of input channels.
num_classes (int) –

Number of output classes.
spatial_dim (int, default: 3 ) –

Spatial dimension of the input point cloud.
stem_channels (Optional[int], default: None ) –

Number of channels in the stem layer.
stem_type (Literal['linear', 'kpconv'], default: 'kpconv' ) –

Type of stem layer to use.
encoder_depths (Sequence[int]) –

List of depths for each encoder block, i.e. corresponds to the number of residual blocks at each level.
encoder_channels (Sequence[int]) –

List of channels for each encoder block.
encoder_num_neighbors (Sequence[int]) –

List of maximum number of neighbors for each encoder block.
grid_sizes (Sequence[float]) –

List of grid sizes for each downsampling block.
radii (Sequence[float]) –

Search radius for each downsampling block.
kernel_size (int) –

Size of the kernel for each KPConv block.
kp_radius (Union[float, Sequence[float]]) –

List of kernel radius for KPConv blocks, at each level.
kp_sigma (Union[float, Sequence[float]]) –

List of kernel extent for KPConv blocks, at each level.
kp_influence (str, default: 'linear' ) –

Influence function to use for KPConv blocks. Options are "constant", "linear", "gaussian".
fixed_position (Literal['none', 'center', 'vertical'], default: 'center' ) –

Whether to fix the position of the kernel points in KPConv blocks. Options are "none", "center", "vertical".
aggregation_mode (str, default: 'sum' ) –

Aggregation mode to use for the KPConv blocks. Options are "sum", "mean", "max".
deformable (Union[bool, Sequence], default: False ) –

Whether to use a deformable kernel for the KPConv blocks.
modulated (Union[bool, Sequence], default: False ) –

Whether to use a modulated kernel in KPConv operation.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization to use for the KPConv blocks.
act (Union[str, Callable, None], default: 'leaky_relu' ) –

Activation function to use for the KPConv blocks.
bias (bool, default: False ) –

Whether to use a bias for the KPConv blocks.
dropout (float, default: 0.0 ) –

Dropout rate before the classification head.
global_pool (PoolLike, default: 'max' ) –

Global pooling method to use before the classification head. Options are "max", "mean".

Methods:

configure_stem –

Build the stem lifting the input features to stem_channels, or None when stem_channels is unset.
configure_encoder –

Build the KPFCNNEncoder backbone.
reset_classifier –

Resets the classification head with new parameters.
forward_features –

Forward pass of the encoder, returning pre-pooling features.
forward_head –

Forward pass of the classification head from pre-pooling features.
forward –

Forward pass of the classification model.

Attributes:

num_features (int) –

Feature dimension \(C\) of the encoder output.

num_features `property` ¶

num_features: int

Feature dimension \(C\) of the encoder output.

configure_stem ¶

configure_stem() -> Optional[Module]

Build the stem lifting the input features to stem_channels, or None when stem_channels is unset.

configure_encoder ¶

configure_encoder() -> KPFCNNEncoder

Build the KPFCNNEncoder backbone.

reset_classifier ¶

reset_classifier(
    num_classes: int,
    global_pool: Optional[PoolLike] = None,
    **kwargs: Any,
) -> None

Resets the classification head with new parameters.

Note

To set an empty classification head, use num_classes=0.

Parameters:

num_classes (int) –

Number of output classes.
global_pool (Optional[PoolLike], default: None ) –

Pooling method to aggregate point features ("max" or "mean"). If None, keeps the current pooling.
**kwargs (Any, default: {} ) –

Additional keyword arguments to pass to the classification head.

forward_features ¶

forward_features(
    x: OptTensor,
    pos: Tensor,
    batch: Tensor,
    return_intermediates: Literal[True],
) -> Tuple[Tensor, Tensor, Tensor, List[Dict[str, Tensor]]]

forward_features(
    x: OptTensor,
    pos: Tensor,
    batch: Tensor,
    return_intermediates: Literal[False] = False,
) -> Tuple[Tensor, Tensor, Tensor]

forward_features(
    x: OptTensor,
    pos: Tensor,
    batch: Tensor,
    return_intermediates: bool = False,
) -> Any

Forward pass of the encoder, returning pre-pooling features.

Parameters:

x (OptTensor) –

Point features of shape \((N, C)\). If None, pos is used as features, which requires in_channels to match the dimension of pos.
pos (Tensor) –

Point coordinates of shape \((N, D)\).
batch (Tensor) –

Batch indices for each point of shape \((N,)\).
return_intermediates (bool, default: False ) –

Whether to also return the per-stage intermediates.

Returns:

Any –

A tuple (x, pos, batch) at the coarsest level, where x has shape \((N', \text{encoder\_channels}[-1])\) and \(N'\) is the number of downsampled points. If return_intermediates=True, the per-stage intermediates are appended to the tuple.

forward_head ¶

forward_head(
    x: Tensor, batch: Tensor, pre_logits: bool = False
) -> Tensor

Forward pass of the classification head from pre-pooling features.

Parameters:

x (Tensor) –

Pre-pooling features of shape \((N', \text{encoder\_channels}[-1])\) where \(N'\) is the number of downsampled points.
batch (Tensor) –

Batch indices for each downsampled point of shape \((N',)\).
pre_logits (bool, default: False ) –

Whether to return pre-logits. Defaults to False.

Returns:

Tensor –

Classification logits of shape \((B, \text{num\_classes})\).

forward ¶

forward(x: OptTensor, pos: Tensor, batch: Tensor) -> Tensor

Forward pass of the classification model.

Parameters:

x (OptTensor) –

Point features of shape \((N, C)\). If None, pos is used as features, which requires in_channels to match the dimension of pos.
pos (Tensor) –

Point coordinates of shape \((N, D)\).
batch (Tensor) –

Batch indices for each point of shape \((N,)\).

Returns:

Tensor –

Classification logits of shape \((B, \text{num\_classes})\).

KPFCNNSegmentation ¶

KPFCNNSegmentation(
    in_channels: int,
    num_classes: int,
    *,
    spatial_dim: int = 3,
    stem_channels: Optional[int] = None,
    stem_type: Literal["linear", "kpconv"] = "kpconv",
    encoder_depths: Sequence[int],
    encoder_channels: Sequence[int],
    encoder_num_neighbors: Sequence[int],
    fp_channels: Sequence[Sequence[int]],
    grid_sizes: Sequence[float],
    radii: Sequence[float],
    kernel_size: int,
    kp_radius: Union[float, Sequence[float]],
    kp_sigma: Union[float, Sequence[float]],
    kp_influence: str = "linear",
    fixed_position: Literal[
        "none", "center", "vertical"
    ] = "center",
    aggregation_mode: str = "sum",
    deformable: Union[bool, Sequence] = False,
    modulated: Union[bool, Sequence] = False,
    act: Union[str, Callable, None] = "leaky_relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    act_first: bool = False,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
    bias: bool = False,
    dropout: float = 0.0,
    head_channels: Optional[Sequence[int]] = None,
)

Bases: SegmentationModel

KPConv Network for segmentation tasks as described in the paper KPConv: Flexible and Efficient Convolution for Point Clouds by Hugues Thomas, Charles R. Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, Leonidas J. Guibas.

KPConv introduces a novel point convolution operator that uses kernel points to define the spatial extent and weights of the convolution. The kernel points are arranged in space to define the convolution pattern, with weights determined by their spatial correlation with input points. This allows for flexible and efficient convolution on irregular point clouds while maintaining permutation invariance and translation invariance. The network uses a hierarchical architecture with strided convolutions for spatial pooling and feature aggregation.

Note

The implementation is based on the original paper and the authors' code KPConv-PyTorch.

Important

This implementation was completely rewritten to be compatible with torch-geometric library.

Parameters:

in_channels (int) –

Number of input channels.
num_classes (int) –

Number of output classes.
spatial_dim (int, default: 3 ) –

Spatial dimension of the input point cloud.
stem_channels (Optional[int], default: None ) –

Number of channels in the stem layer.
stem_type (Literal['linear', 'kpconv'], default: 'kpconv' ) –

Type of stem layer to use.
encoder_depths (Sequence[int]) –

List of depths for each encoder block, i.e. corresponds to the number of residual blocks at each level.
encoder_channels (Sequence[int]) –

List of channels for each encoder block.
encoder_num_neighbors (Sequence[int]) –

List of maximum number of neighbors for each encoder block.
fp_channels (Sequence[Sequence[int]]) –

List of channels for each feature propagation block.
grid_sizes (Sequence[float]) –

List of grid sizes for each downsampling block.
radii (Sequence[float]) –

Search radius for each downsampling block.
kernel_size (int) –

Size of the kernel for each KPConv block.
kp_radius (Union[float, Sequence[float]]) –

List of kernel radius for KPConv blocks, at each level.
kp_sigma (Union[float, Sequence[float]]) –

List of kernel extent for KPConv blocks, at each level.
kp_influence (str, default: 'linear' ) –

Influence function to use for KPConv blocks. Options are "constant", "linear", "gaussian".
fixed_position (Literal['none', 'center', 'vertical'], default: 'center' ) –

Whether to fix the position of the kernel points in KPConv blocks. Options are "none", "center", "vertical".
aggregation_mode (str, default: 'sum' ) –

Aggregation mode to use for the KPConv blocks. Options are "sum", "mean", "max".
deformable (Union[bool, Sequence], default: False ) –

Whether to use a deformable kernel for the KPConv blocks.
modulated (Union[bool, Sequence], default: False ) –

Whether to use a modulated kernel in KPConv operation.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization to use for the KPConv blocks.
act (Union[str, Callable, None], default: 'leaky_relu' ) –

Activation function to use for the KPConv blocks.
bias (bool, default: False ) –

Whether to use a bias for the KPConv blocks.
dropout (float, default: 0.0 ) –

Dropout rate before the classification head.

Methods:

configure_stem –

Build the stem lifting the input features to stem_channels, or None when stem_channels is unset.
configure_encoder –

Build the KPFCNNEncoder backbone.
configure_decoder –

Build the PointNet2Decoder upsampling the coarsest features back through the encoder skips.

Attributes:

num_features (int) –

Feature dimension \(C\) of the decoder output.

num_features `property` ¶

num_features: int

Feature dimension \(C\) of the decoder output.

configure_stem ¶

configure_stem() -> Optional[Module]

Build the stem lifting the input features to stem_channels, or None when stem_channels is unset.

configure_encoder ¶

configure_encoder() -> KPFCNNEncoder

Build the KPFCNNEncoder backbone.

configure_decoder ¶

configure_decoder() -> PointNet2Decoder

Build the PointNet2Decoder upsampling the coarsest features back through the encoder skips.

create_kernel_points ¶

create_kernel_points(
    radius: float,
    num_points: int,
    fixed_position: Literal[
        "none", "center", "vertical"
    ] = "center",
    method: Literal["lloyd", "gradient"] = "lloyd",
) -> Tensor

Builds the kernel point positions of a KPConv kernel, randomly rotated and jittered.

Positions are optimized on the unit sphere, cached under CACHE_DIR and reused across calls.

Parameters:

radius (float) –

Radius of the sphere the kernel points are scaled to.
num_points (int) –

Number of kernel points \(K\).
fixed_position (Literal['none', 'center', 'vertical'], default: 'center' ) –

Which kernel point is pinned: "none", "center", or "vertical".
method (Literal['lloyd', 'gradient'], default: 'lloyd' ) –

Optimization used to spread the points, either "lloyd" or "gradient".

Returns:

Tensor –

Kernel point positions of shape \((K, 3)\).

KPConv

KPConv ¶

deformable property ¶

configure_offsets ¶

configure_kernel ¶

KPConvBlock ¶

KPResidualBlock ¶

KPFCNNGridPool ¶

KPFCNNEncoderBlock ¶

KPFCNNEncoder ¶

KPFCNNClassification ¶

num_features property ¶

configure_stem ¶

configure_encoder ¶

reset_classifier ¶

forward_features ¶

forward_head ¶

forward ¶

KPFCNNSegmentation ¶

num_features property ¶

configure_stem ¶

configure_encoder ¶

configure_decoder ¶

create_kernel_points ¶

deformable `property` ¶

num_features `property` ¶

num_features `property` ¶